Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Paul or Daniel - can you explain the long load times?

Author
Message
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 30th Jun 2012 03:38
Here's the situation - I'm working on a project that reads in ten files of words. Lot of words. About 800k worth of data.

When I broadcast to my devices (iPad, iPod Touch, HTC Incredible phone, Kindle Fire) after everything is transferred and the program starts, it takes, oh, less than a second, to read in the data.

When I compile and build the project (both xCode for iOs and Eclipse for the Android devices) the load time for the ten files is really really long. About 10 seconds for the iPad, 20 seconds for the iPod, and almost 3 minutes for the Android devices.

So... why the difference? Why does the broadcast version load the data so much more quickly?

Conventional wisdom would say that a compiled version should work just as fast or faster than the broadcast version.

But no.

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
Hodgey
14
Years of Service
User Offline
Joined: 10th Oct 2009
Location: Australia
Posted: 30th Jun 2012 03:59
If you like, I can test the Xcode project out on the device simulators to give you a second opinion. Just upload the project (or the 10 files) or email me.

bjadams
AGK Backer
16
Years of Service
User Offline
Joined: 29th Mar 2008
Location:
Posted: 30th Jun 2012 11:21
The ios Simulator is going to take 6-8 times longer to load the data files than a real ios device!
JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 30th Jun 2012 13:00
Are you building with debug on? That can add a lot of time.

-- Jim
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 30th Jun 2012 13:22 Edited at: 1st Jul 2012 22:27
Jim, I'm not sure how to check if I'm building with the debugger on. Do you know how I can check, both in xCode and in Eclipse?

EDIT: as far as I can tell, I'm not using the debug build for either platform.

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 3rd Jul 2012 18:02
No Paul? No Daniel?

I'm thinking that Android has to unpack the apk every time the program runs, but that the AppGameKit player transfers all the files to the device, so one solution could be to load the data, then write the data back to the storage area, but only the first time the program runs. I'll try that when I get home from work tonight. Hey, it might just decrease the load times on iOS as well!

One thing... reading the words data as x ReadLine() calls is very slow, but reading the data as 1 ReadString() call is VERY fast. However, parsing the single string (either into an array or just searching the string for a single word) is VERY slow. It would be good to have an AppGameKit command like FindSubString() that was quicker than looping through each character and checking against a mid() call.

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
Ancient Lady
Valued Member
20
Years of Service
User Offline
Joined: 17th Mar 2004
Location: Anchorage, Alaska, USA
Posted: 3rd Jul 2012 18:19
Rich, are the words in your files alphabetically sorted?

If you load all the words in a sorted order into an array, you could create another array with indices pointing to the first word starting with each letter. Then you would be able to select the words to check faster.

Further, if you added another level of indices to indicate the location for each letter in the second position, associated with the first position letter, your search would be even faster.

In T2, a tree could be created to quickly find a target match, or failure.

Doing the same in T1 is a bit more difficult, but probably not impossible. You would need an array of integers dimmed to 26 to the power of whatever level of indexing you want. E.g. for only the first letter, it would be 26^1. For first two it would be 26^2.

The array would then be structured something like this:


The search might do something like:


This is a very quickly put together bit, not cleaned up.

Cheers,
Ancient Lady
AGK Community Tester
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 3rd Jul 2012 22:29 Edited at: 3rd Jul 2012 22:30
Ancient Lady, my word lists are indeed alphabetically sorted, and I've considered indexing each list.

But... I'm using ten arrays, one for 1-letter words, one for 2-letter words, and so on. When all the words are loaded into their arrays, verifying any particular word is done so quickly that the user experiences no delay between hitting "ENTER" and the word being accepted or rejected. Even searching the longest array (19,458 eight-letter words) for an invalid entry, has no perceptible delay. That's on windows, iPad, iPod, Android phone and Kindle Fire. I was surprised how fast AppGameKit really is.

The slow part is loading all the words into the arrays at the start of the program. As stated above, I believe the delay is due to the files being unpacked from the .apk (and whatever iOS uses) every time the program is run. I'll test my theory when I get home tonight. I plan to read the files in and then write them back out, so that the next time the program runs, the files exist in the sandboxed storage location, and hopefully load in very quickly. I'll do the read/write only the first time the program runs after installation.

Wish me luck!

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
Ancient Lady
Valued Member
20
Years of Service
User Offline
Joined: 17th Mar 2004
Location: Anchorage, Alaska, USA
Posted: 3rd Jul 2012 22:35
Good Luck and Happy Programming!

Cheers,
Ancient Lady
AGK Community Tester
Paul Johnston
TGC Developer
21
Years of Service
User Offline
Joined: 16th Nov 2002
Location: United Kingdom
Posted: 3rd Jul 2012 22:51
I can't think why it would be fast using the player but slow when compiled natively. You may be on the right track with the assets folder verses the write folder causing a difference in read speed, but I've not experienced anything similar. Good luck tracking it down!
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 3rd Jul 2012 23:24
Paul, thanks for the reply. As far as you know, is an assets folder supposed to be created when the Android apk is installed? With all my files in it? Because as far as I can tell, there is only a folder called AGK/Unknown/ created, with the usual joystick pngs, etc. but NOT with the files in my media folder.

However, when the AppGameKit Player runs the program, the correct folder is created, with all my files in it.

So, either I'm doing something wrong with the Android compile, or there's a problem with the compiled version install.

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 4th Jul 2012 00:09
I'm having an "interesting" time constructing a test suite for AppGameKit for Pascal. What works or doesn't work on Android appears to be a bit unpredictable - both within AppGameKit and using API calls.

As far as the words go, there are far more efficient algorithms for doing this. For sorted word lists, a binary search is the best thing to do, since it is [Log2 N] performance. If the data is static (you have provided the word list) then a bit of pre-processing could give you the start index of words beginning with "A" and its end.

This would be, I think, faster than bothering with word lengths.

-- Jim
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 4th Jul 2012 00:25 Edited at: 4th Jul 2012 00:29
Quote: "As far as the words go, there are far more efficient algorithms for doing this. For sorted word lists, a binary search is the best thing to do, since it is [Log2 N] performance. If the data is static (you have provided the word list) then a bit of pre-processing could give you the start index of words beginning with "A" and its end.

This would be, I think, faster than bothering with word lengths."


Jim, I'm sure you and Ancient Lady are right about the indexing, but I'm happy with my searching code. There is no noticeable difference in searching through ALL the eight-letters words, and searching all the eight-letter that start with "A" for example.

It's the loading code that is really slow.

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
JimHawkins
14
Years of Service
User Offline
Joined: 26th Jul 2009
Location: Hull - UK
Posted: 4th Jul 2012 01:01
Rich - I'm sure you're right. Android appears to me to be the worst-case scenario for quite a lot of things!

The loading issue is actually very important because it affects so much.

-- Jim
Kevin Picone
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th Jul 2012 01:35
Rich,

How are you loading the text ? as baching it into chunks, or perhaps loading it all into a single string is likely to be quicker than the file interface.

Paul Johnston
TGC Developer
21
Years of Service
User Offline
Joined: 16th Nov 2002
Location: United Kingdom
Posted: 4th Jul 2012 03:25
Quote: "As far as you know, is an assets folder supposed to be created when the Android apk is installed? With all my files in it?"


The asset files are zipped up into the apk, but I don't know if it unzips them somewhere first or just reads them straight out of the zip file on demand. There are special commands for accessing files from the assets folder so I suspect that latter.
Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 4th Jul 2012 03:47
Kevin, you are correct, loading it as single string is VERY fast, but then I have to parse that really long string, either during game play (unacceptable - it takes too long to find a sub-string by hand) or during the load process, which again takes about the amount of time as loading each word as a string.

I gotta stress that the problem DOES NOT occur in the windows version, or using the Android/iOS players, only in the compiled iOS (semi-long load time) and the compiled Android version (extremely long load).

But I'm home now and shall start testing!

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!
Kevin Picone
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th Jul 2012 04:45 Edited at: 4th Jul 2012 04:46
Quote: "
I have to parse that really long string, either during game play
"


Nah, if the words in each block are the same length, so you can run through with mid$() and slap the chunks in the your chosen array.



or alternatively you could try inserting a delimiter into the list and use the CountStringTokens

Rich Dersheimer
AGK Developer
14
Years of Service
User Offline
Joined: 1st Jul 2009
Location: Inside the box
Posted: 4th Jul 2012 12:14 Edited at: 4th Jul 2012 12:15
PROBLEM SOLVED!

Here's what I did...

I check to see if this is the first time the program has run. If so, as I load in the word lists, I write them back out again.

The next time the program runs, it finds the files in the sandboxed storage folder, and does not need to unpack them from the media folder in the apk file (this is the Android build).

Load time for the dictionary files in the "sandbox" is under 1 second!

I now feel like I've got a product I can release on iOS and Android, which is the reason I bought AppGameKit "Write Once, Deploy Everywhere" in the first place. (Well, one of the reasons )

EDIT: Thanks everyone for all the great suggestions, you people ROCK!

Some days, the bear will eat you.
Some days, you'll eat the bear.
Today, I'm eating the bear!

Login to post a reply

Server time is: 2024-05-04 14:09:51
Your offset time is: 2024-05-04 14:09:51