Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Showcase / [WIP] Tiled importer v.2

Author
Message
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 3rd Mar 2017 19:34 Edited at: 24th Mar 2017 16:20
My old thread has long since been locked, https://forum.thegamecreators.com/thread/210176

Clonkex and I have gone back and forth over the years implementing this. I'm revisiting the project to update it so it works with AppGameKit 2. With UDTs now supporting dynamic arrays, I can contain the entire map and it's data into a single variable. This helps keep things cleaner and also should theoretically allow you load multiple maps at a time (if for some reason you needed to).

I'm almost caught up with everything that the old version had, which I should finish up today. I did add support for external tilesets finally. My biggest concern right now is speed. CSV appears the best method right now. The Base64 is extremely slow with large maps. It took close to 4 minutes to decode the string from my zelda map, which was a 120k character encoded string. I've tried using memblocks and gained practically no speed increase. I have one more idea to try though.

Image and Object layers are trivial and will be added shortly. I also plan to implement properties (finally) and animations.

Sorry, no pics yet, wouldn't be much of anything to show really. If anyone has suggestions or a request for something that was missing in my earlier versions, shout it out.


Features:
Supports encoding: xml, csv, base64(uncompressed)
Supported map types: orthogonal, isometric, staggered isometric, hexagonal
Supports tileset images which use margin/spacing
Supports image layers
Layer alpha and visibility
Set the map viewport area (Define area of screen where map can only be visible)
Supports tile flipping
Support for TSX tilesets (external tileset xml)
Compression (gzip, zlib)
Objects
Properties
Animations
Terrain
Image Collections
Tile render order

Update: 24 Mar 2017
* Objects fully supported

Update: 23 Mar 2017
* Objects is partially supported (rectangle, ellipse, polygon)
* Fixed a bug in the xml parser
* worked out some issues with layer offsets

Update: 20 Mar 2017
* Add support for image collections (which I didn't know existed until today)

Update: 19 Mar 2017
* Added animated tiles

Update: 18 Mar 2017
* Added properties
* Restructured how layers are handled
* Added image layers

Update: 15 Mar 2017
* Added hexagonal support, but still need to work out rendering order for some of the various configurations

Update: 4 Mar 2017
* Base64 has been redesigned and now is significantly faster (about 1000x faster)
* After much headache, the XML parser has been rewritten from scratch for a huge speed boost (up to 50x faster)

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
RickV
TGC Development Director
14
Years of Service
User Offline
Joined: 27th Aug 2002
Location: United Kingdom
Posted: 5th Mar 2017 08:33
Hi Phaelax,

This is great. Let me know when you have it working, I'd like to try it out.

Rick
Development Director
TGC Team
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 10th Mar 2017 18:34 Edited at: 10th Mar 2017 18:34
Here's a download you can test with for now. The video shows a quick demo map I made, 32x32 tiles with 3 layers. That's 3072 tiles, loaded in under 200ms. It's encoded with base64, no compression. For large maps, I highly discourage the raw plain xml format.




"I like offending people, because I think people who get offended should be offended." - Linus Torvalds

Attachments

Login to view attachments
SpecTre
Developer
14
Years of Service
User Offline
Joined: 24th Feb 2003
Location: UK
Posted: 10th Mar 2017 19:17
Phaelax this is very good work, the loading looks very fast too. Cool
The Amiga and Amos were great!
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 15th Mar 2017 22:59
I updated Tiled the other day and boy was I out of date! When did they add hexagonal maps? Well alrighty then, I've got that added now!

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds

Attachments

Login to view attachments
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 18th Mar 2017 18:49 Edited at: 18th Mar 2017 18:55
So unless someone can give me a good reason to implement terrain types, I see no reason for it. As far as I can tell, it's nothing that would be used in the display of the render map and more to do so with the editing in Tiled. But I could be wrong about that.

I've finished adding image layers and have restructured the layer format slightly. Tiled layers, image layers, and eventually object layers, are all of one 'type'. For the various functions a user made need to call on a map, the type of map loaded does not change the command needed. A single function (such as positionMap) encapsulates the headaches for you.

Properties has also been added for map, layers, and tilesets. A note about how properties are stored, they are stored in a simple key-value array with values stored as strings. Though the TMX format has a type attribute to define what type of data is being represented, I found it easier to leave them as strings and let the user decide what/how to convert them. Since the user likely won't be accessing property values in a map without already knowing what they are. And I can't store different data types into the same array.

Object layers will be next on my to-do list and then I'll tackle animations.

Here's what I find to be the best format to use in your Tiled maps, from fastest to sloweset:
1. Base64 (uncompressed)
2. CSV
3. XML

XML is about twice as slow as any of the others, and grows exponentially as the map size increases. CSV is ok for smaller maps, but after you get to a certain size, and with multiple layers, it too can become slow to load. The slowdowns are due to the fact that XML and CSV are all stored textually. Even though Base64 is a string, it's a single string that represents an entire layer of data. A simple command can very quickly throw this string into a memblock, thus all calculations at that point are just bytes. This eliminates the slowest part of the loading process, which is string parsing.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 20th Mar 2017 02:08
Ok, so I skipped object layers and went right to animation. Grabbed an animated hexagonal sprite I found in some random forum and made a quick sprite sheet and built an animation sequence. My initial tests, as seen in the video, are working perfectly. I have not yet tested it on a large map with multiple animations. This test was a small 24x24 map with 2 layers, 2 tilesets, and 2 animations. I've structured it as efficiently as I could see how to. Only those tiles with animations will be in the animation loops. The tradeoff? A separate array contains a list of those sprites that need to be managed, which means more memory used. When I'm finished with this, I'd like to get some folks to test this out on mobile devices because memory usage means almost nothing on a PC.



"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
pjsmith67
1
Years of Service
Recently Online
Joined: 2nd Feb 2016
Location: Nebraska, USA
Posted: 20th Mar 2017 15:47
I tried using the TMX Importer last night and kept getting an error. I created a simple orthogonal map, 43x24 and filled it with one tile.

The error I'm getting is "Array index out of bounds, index: -1, Array Length 10 in xml_dom_parser.agc at line 163,"

Any idea off the top of your head? I've attached the tmx file I'm using. Your sample tmx file works, though.
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 20th Mar 2017 18:46
I don't see any attachment. Did the file use compression? I've been making small corrections to my xml code as I go along, those array errors being the most common. It's possible I've already fixed it. I'll try to post an updated code tonight.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
pjsmith67
1
Years of Service
Recently Online
Joined: 2nd Feb 2016
Location: Nebraska, USA
Posted: 20th Mar 2017 20:16
oops. lets try this again. Here's the attachment. Thanks!

Attachments

Login to view attachments
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 20th Mar 2017 20:48
I don't think this is related to the error you got, but the image path might cause an issue. I don't think AppGameKit can access a file directly from outside of it's project folder. I would change that so the TMX and image file are in the same directory (the media folder).

And could you post the panel.png image please.

Your format is slightly different that what I've worked with. Generally, I'd see an image tag as a direct descendant of tileset. In yours, I see TILE inside the TILESET, which then includes IMAGE. Generally, I had only animations inside a tile within a tileset, so I'm going to have to check the documentation.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
pjsmith67
1
Years of Service
Recently Online
Joined: 2nd Feb 2016
Location: Nebraska, USA
Posted: 20th Mar 2017 21:13
Here's panel.png. Sorry, forgot about the image file.

Hmmm... ok, didn't think about that when i created the file. Let me start over with a fresh map and see what happens.

Attachments

Login to view attachments
pjsmith67
1
Years of Service
Recently Online
Joined: 2nd Feb 2016
Location: Nebraska, USA
Posted: 20th Mar 2017 21:27
Ok, I took your grassland_tiles example, added a collection of tiles and put a singe image in there. Saved it and tried tmx_importer and got the the same error.

So, it doesn't like it when you us a collection of files for a tile set.
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 20th Mar 2017 22:11 Edited at: 20th Mar 2017 23:29
Apparently, Tiled has a feature I was unaware of. The format documentation really doesn't say anything about image collections either. http://doc.mapeditor.org/reference/tmx-map-format/
I think I understand how they work, but I'll have to think about how to implement it.

Googling it has only shown me posts from several development forums (for various projects) of people asking how to add support for image collections. It must be something they threw in and never told anyone about because it appears to me that many implementations are missing it.

The most frustrating thing about Tiled's XML format is the fact that some things start at 1 while others are based at 0. It's very infuriating!

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 21st Mar 2017 01:44
pjsmith67, try this code. Your map *should* work now. And it was a pain in the butt. The only thing that might appear off is the depth of tiles. I haven't gone through and set up the different render orders yet.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds

Attachments

Login to view attachments
pjsmith67
1
Years of Service
Recently Online
Joined: 2nd Feb 2016
Location: Nebraska, USA
Posted: 21st Mar 2017 02:43
Yep, that worked. Thanks!

Now, about object layers...

Just kidding... you're doing great work so far and it is appreciated.

Phil
Phaelax
DBPro Master
13
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Agartha
Posted: 24th Mar 2017 16:18
Objects have now been added

The shapes are drawn using 2D commands, so they'll always draw on top regardless of layer order. My plan is to hopefully draw the shape once to an image and turn them into sprites, but that's not an immediate concern of mine as I don't think objects were ever intended to actually be part of the map visually.

I have a few more functions to write, mainly get/set functions, then I'll release the final code for everyone to test.

I still haven't tackled terrain types yet.

"I like offending people, because I think people who get offended should be offended." - Linus Torvalds
FPSGuy21
User Offline
Joined: 30th Sep 2016
Location:
Posted: 30th Mar 2017 22:55
Hey Phaelax,

what about releasing your code on github so everyone can contribute? looking forward to the final code version. Thanks for your work I really appreciate it!

Login to post a reply

Server time is: 2017-03-31 01:32:30
Your offset time is: 2017-03-31 01:32:30