Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / FPS slowdowns with lots of assets loaded

Author
Message
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 19:26
Link
I'm not sure if I'm overlooking something simple here, but here are my symptoms:

When I run my game with only a few images loaded into memory (whether I'm displaying them or not), I get a full 60 frames per second.

When I load a large number of resources (images, hundreds of them), my framerate drops to just 10 frames per second, even though I'm not displaying them.

To clarify, there is no difference in the code, or in what is displayed on the screen, between the two versions. In one, I load a small number of assets, and in the other, I load a large number of assets, for a total memory usage difference of about 30MB. Nothing is done with the additional assets - just loading them slows my program to a crawl.

Task manager reports that my program is eating about 230MB of system memory when I load only a few of the images, and about 260MB of system memory when I load them all.

In both cases the program commands about 30% of my CPU power (there's no change in CPU usage between the two versions).

What gives?
Back to top
Profile PM Email
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 21st Mar 2011 19:36 Edited at: 21st Mar 2011 19:37
Link
Is your memory low? Sounds like you have low system memory and the program is forcing the computer to page file to the hdd which is way slower than paging directly to memory. not much of an issue these days generally but even the best system can run out of system memory and rely on virtual memory. that is the main reason upgrading your memory speeds up the system, because it no longer needs to page to virtual ram.
Other than that not sure, apart from I have noticed DB can slow down more when using lots of objects, even simple ones than a few high detail ones..

http://s6.bitefight.org/c.php?uid=103081
Back to top
Profile PM Website
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 19:50 Edited at: 21st Mar 2011 20:34
Link
I'm not even close to using up my system memory. The slowdowns seem to have nothing to do with memory usage or CPU capability. I thought at first that I was cycling through all of the assets somewhere in my code whether they were being used or not, but I don't remember doing so and I can't find any code that does so - and even if this was the case, my CPU load would have to hit 100% before framerates would suffer, so logic follows that this isn't the case.

Does anyone consider it possible that this is some sort of bug in DBP? The actual amount of memory being used by the extra assets isn't substantial (it's less than 30MB extra) and it's a couple of hundred extra images being loaded. This doesn't seem like a lot, but I thought I'd throw it out there: Has anybody else had any sort of trouble with loading a large quantity of assets? Is it possible that the image id numbers themselves are relevant?

I just don't understand why merely loading the assets (and doing nothing at all with them) would cost me frame rates.
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 20:41
Link
Further testing shows the following statistics:

SYSTEM IDLE
===========
CPU Usage 11%
Pagefile Usage 834MB
Physical Memory Available 1344/2096


GAME RUNNING FEW ASSETS
=======================
CPU Usage 46%
Pagefile Usage 1200MB
Physical Memory Available 1136/2096
FPS 32
Cycletime 22


GAME RUNNING MANY ASSETS
========================
CPU Usage 56%
Pagefile Usage 1240MB
Physical Memory Available 1139/2096
FPS 11
Cycletime 81



...

None of my system resources are even close to being saturated. My page file size is 2046, so its not approaching capacity. I just don't get it. What could cause something like this? I'm not doing anything with the extra assets that are loaded, they're merely occupying memory that is not strained for capacity.

What gives?
Back to top
Profile PM Email
Quel
15
Years of Service
User Offline
Joined: 13th Mar 2009
Location:
Posted: 21st Mar 2011 21:09
Link
Welcome to the world of DBPro...

...a miraculous plain of existence where nothing is what it seems to be, and nothing works as intended.

I'm sorry, but i can't have a smiley face on when there is a problem since the beginning of TGC, and no improvement has been done yet. The very slow loading of media, and slowdowns for those just "being there" is unnacceptable for something 'Professional'.
Back to top
Profile PM
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 21:27
Link
Well, to be fair, I'm not yet prepared to conclude that I've encountered a bug in DBP. It still seems possible that I've overlooked something even though this would not be the first bug I've found in DBP, or even the first astoundingly major bug that should never have made it past the first beta of DBP.

Someone must have some idea of either something I might try, do differently, or has seen this problem before to confirm it isn't my fault (this would actually be the worst possible outcome as from my point of view it's unfixable). I just want to know if I'm doing something wrong or if DBP itself is at fault. Has anybody encountered this problem before? Know of a workaround? Aware of a bugfix?
Back to top
Profile PM Email
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 21st Mar 2011 21:30
Link
Had a look at a program myself and got odd results from it

As you can see the numbers are very odd. I ran the tests out of order, starting with idle. then max objects, then knocking it down to just 10. Don't think you will have much look using task manager as a guide here. looks like the memory isn't being reclaimed after each use.

http://s6.bitefight.org/c.php?uid=103081
Back to top
Profile PM Website
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 21:57 Edited at: 21st Mar 2011 21:59
Link
Thanks for trying to help, but I'd need the Viewer Speed/fps for the first two tests for this information to be useful. Also, I'm not using 3D objects but 2D images, and the internal process used to store such assets is very likely to be different and not subject to the same bug, if such a bug exists.

It is common knowledge that DBPro does not clean up assets very well when they are "deleted" at code level during runtime. However, I am not deleting any of these assets; they are supposed to be present in memory, even though they not being used just yet.

Furthermore, this problem is a recent development. At one stage, perhaps a month ago, I was loading all assets all the time, and wasn't experiencing any difficulties whatsoever. Then I reduced the number of assets being loaded at initialisation to speed up load times for testing purposes, as I was doing some rapidfire adjustments to other parts of the code and compiling to test repeatedly. I've left the lower asset load for some time, and today I bumped it back up to normal for a full test and discovered the slowdown. This seems to indicate that the fault is at code level, but the same code runs no matter whether the additional assets are loaded or not, and that code doesn't use the additional assets (which is why I cut back on them in the first place for quicker load times during testing - I don't need them yet anyway).

Beyond that, the fact remains that while there is a high quantity of assets, their actual size is tiny. Several hundred images, altogether representing about 30MB in size. I am wondering whether the numerical quantity of assets can become an issue, in any circumstance?

It just doesn't make sense. Any other thoughts?
Back to top
Profile PM Email
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 21st Mar 2011 22:14 Edited at: 21st Mar 2011 22:33
Link
I seem to remember a similar issue with a fighting game I started many years ago, I think it started to slow at about 400 images if I remember correctly. Perhaps you could post some code for us to test? See if we get the same issue. Perhaps loading just one image repeatedly, instead of having to post a huge amount of images?

Edit - I have made a little test program. I am only loading the same image over and over, but I get no issues even with 10000 images loaded in. the only thing that effects my overall speed is how many sprites I paste down.



I get around 50+ fps with the settings there, lower amounts and I get over 3000+. The test image is a small 128x128 2k image.

http://s6.bitefight.org/c.php?uid=103081
Back to top
Profile PM Website
Kevin Picone
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 21st Mar 2011 22:27
Link
Quote: "Several hundred images, altogether representing about 30MB in size. "


might I ask where are you're getting this figure from ? Is this 30mb on disc ?


Hmm, I wonder what's coming in 2011?
Back to top
Profile PM Website
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 21st Mar 2011 22:37
Link
I remember my first 2d game almost died due to sprites... are you using sprites or just loading images? The problem I had was that you load a sprite and get no problems, but you change the sprite priority to some really large number and the game grinds to a halt

Back to top
Profile PM Email Website
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 21st Mar 2011 22:48 Edited at: 21st Mar 2011 22:52
Link
When all the assets are loaded into memory, total memory usage is 30MB greater than when only 10% of the assets are loaded, according to the task manager. The figure of 30MB is only relevant to demonstrate that I'm not loading huge, complicated assets here, just large numbers of them. They are all 32x32 pixel BMP images, roughly 4kb each in size. There are seven and a half thousand of them. If I load only a few hundred, the game runs at full speed. If I load them all, I hit slowdowns.

Dustin: I'm loading the images and then doing nothing with them. They are never displayed, never accessed, never utilised for any purpose. If I don't load them at all, the game runs at full speed. If I do load them, the game runs at one third the framerate it should.

I do intend to use them in future, which is why I want to load them, but the slowdown occurs when all I do is load them. There is no code yet, anywhere in the program, that does anything with the images besides load them in the first place.
Back to top
Profile PM Email
DVader
20
Years of Service
User Offline
Joined: 28th Jan 2004
Location:
Posted: 22nd Mar 2011 00:06
Link
Well for one I would use png files, way smaller for one thing. Superior to bmp imo.

http://s6.bitefight.org/c.php?uid=103081
Back to top
Profile PM Website
GIDustin
15
Years of Service
User Offline
Joined: 30th May 2008
Location:
Posted: 22nd Mar 2011 00:11 Edited at: 22nd Mar 2011 00:21
Link
Quote: "They are all 32x32 pixel BMP images, roughly 4kb each in size. There are seven and a half thousand of them."


Even at the highest possible resolution my monitor supports, if every 32x32 tile was different I would still only need ~2000 to cover the whole screen...

There may be an underlying problem with the amount of images you have loaded, but you should really consider a "load on demand" type option...


Edit: Ran my own test with 7,000 32x32 pngs and it ran fine. The fps never slowed. I even loaded them 1,000 at a time a required a keypress to load the next batch.

I assume you keep all the image IDs in some sort of array. Is there anywhere where you loop through them all?

Back to top
Profile PM Email Website
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 00:35
Link
I get no slowdown in the main loop on my old laptop with the following code. I think we need more details.

Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 01:16 Edited at: 22nd Mar 2011 01:26
Link
Gandalf: I increase your number of images to 10,000 and I still get in excess of 3500fps on my desktop. This code doesn't seem to reproduce the problem. Still, in my game code it looks like just having the images there in memory kills framerates. Since the only thing that is ever done with the images is to load them, I can simply reduce the number of iterations in an ordinary FOR...NEXT loop to load fewer images, and when I do that I get an improvement in framerate. The program doesn't miss the images because it doesn't use them anyway - but I want it to soon, so I need to load those images.

The image ID's are not held in an array, but the areas of the code that access the images decide which one to use by the values in an array.

Ok, let me give specifics. We are talking about an RPG game. I have 7500-odd images that are 32x32 pixel bmp files of no more than 5k in filesize each. The images themselves are of RPG characters and NPC's and, importantly, a large number of various equipment pieces that they can wear. We take one of the images, a nude human male form, and we draw it to the screen. Then we take another image, one that has his clothes in it, and draw that over the top, then his armor, then his helmet, and his cape, etc, until we have a composite image made up of five to ten separate images that, together, represent the character and all of his wearable equipment.

Now, I don't have a large database of items in my game yet (only about 30 items) so I have only been loading a very small number of these images because most of them are never actually used.

However, altogether all of these individual "component" images number seven and a half thousand, and as my database of ingame items expands I need to load more and more images to accomodate them. I decided to just load them all into memory and access them at will, as required. They do not take up much memory (as I said, only about 30MB) and take only a couple of seconds to load. They are not a large resource hog.

When I commenced development on the game, I started out by loading all of these assets - all of them - and I did not have any slowdown issues from having them in memory. To speed load times during rapidfire adjust-compile-test sessions, I reduced the number of assets that were loaded, and I left it that way for more than a month - until today, when I decided to start work on my item database and wanted to load all of my assets again. When I changed the numbers back to their full values, I got the slowdown problem.

Now, I feel the need to reiterate that these assets are not a resource hog. Increasing from small number of assets to large number of assets consumes 30MB of memory and results in no discernable increase in CPU load or pagefile access.

I also need to restate that there is no problem displaying large numbers of images on the screen. I must mention again that most of these images are never used or accessed in any way - they are just stored in memory. I want to know why storing these images is reducing my framerate, even though I am never displaying them or accessing them in any way and even though the rest of the code is identical.

I don't need suggestions on efficient ways to display graphics. What I need to know is why loading images to memory - just loading them - impacts performance when nothing is ever done with them and especially when there are free CPU cycles allocated to the program (that isn't doing anything more than it was when I loaded fewer resources).

For reference, here is an example of the resource load code:



The value of numitems affects the framerate of the game, which begins executing down the track after this code fragment and several others like it. If I reduce the value of numitems, I get better framerates. If I increase it, framerate drops. At no stage in the code are any of the images actually used, referenced, accessed, or any other verb you can think of. They are never touched again after they are loaded. Just being there in memory is affecting framerates.
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 01:25 Edited at: 22nd Mar 2011 01:26
Link
I'm not displaying all the images. I'm creating lots of images and only using 100 of them.

Quote: "I want to know why storing these images is reducing my framerate, even though I am never displaying them or accessing them in any way and even though the rest of the code is identical."


Quote: "What I need to know is why loading images to memory - just loading them - impacts performance when nothing is ever done with them"


It doesn't. Look at the code again. You are probably doing something else which is slowing it up. Without the code it's hard to know what exactly.
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 01:27 Edited at: 22nd Mar 2011 01:28
Link
Sorry Gandalf, I was editing my post as you posted yours. I misunderstood you the first time around but I've got a handle on it now. My code does exactly what yours does - loads thousands of images into memory and then only uses a very very small number of them. My edited post might explain better than the original one. Note that merely changing the value of numitems before that code fragment is executed affects framerates, even though the variable numitems is never used again anywhere else in the code.
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 01:38
Link
Quote: "Note that merely changing the value of numitems before that code fragment is executed affects framerates"


That's the whole point, it doesn't. The framerate is about 170 whether I create 100 images or 25600 - on my old laptop at least. What fps values do you get with my snippet for different values of nImages? Perhaps something is special about your machine?
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 02:14
Link
Using your snippet of code, values of ten thousand and above yield greater than 3500fps.

It's easy to say that it doesn't, but it's a fact that in my code I never use the extra images, and just by not loading them I increase my framerate.
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 02:53
Link
Remember that I'm talking about numitems in my code here, not nImages in yours. No matter what value I use for nImages, I receive superfluous framerates. Changing nImages in your code has no apparent effect, but changing numitems in mine has a dramatic effect.
Back to top
Profile PM Email
tiresius
21
Years of Service
User Offline
Joined: 13th Nov 2002
Location: MA USA
Posted: 22nd Mar 2011 04:16
Link
Quote: "When I commenced development on the game, I started out by loading all of these assets - all of them - and I did not have any slowdown issues from having them in memory. To speed load times during rapidfire adjust-compile-test sessions, I reduced the number of assets that were loaded, and I left it that way for more than a month - until today, when I decided to start work on my item database and wanted to load all of my assets again. When I changed the numbers back to their full values, I got the slowdown problem."

It sounds like you need to determine what changed between your initial development (when all the images were loaded with no problems) and the code as it is now. All our snippets and test code is going to look like what you did before you saw the problem.

I thought for sure it was the high number you use for image IDs, but my tests do not show much conclusive evidence for a difference. I also changed it to paste image because I presume that's what you are doing, not sprite commands. What are you using exactly to put the images on the screen?

Put a Perf timer before and after your "sync" command, and you'll know for sure if it is the DBPro render engine causing the slowdown and not something in your code you are unaware of.


A 3D marble platformer using Newton physics.
Back to top
Profile PM Email
Wyldhunt
14
Years of Service
User Offline
Joined: 27th Sep 2009
Location: The Dark Side
Posted: 22nd Mar 2011 11:07
Link
You load a lot of images in to memory and your program slows down. Reproducing the issue has been difficult.
So, thinking of how RPG's are generally laid out, here's my question:

Once you have loaded a lot of images, you must be tracking them somehow.
Quote: "The image ID's are not held in an array, but the areas of the code that access the images decide which one to use by the values in an array."

The array to track 100 images and 7,500 images may be very different beasts. I'd check any code that uses your array in any way and see if you are cycling through the entire array every loop or something similar on accident. If you fat-fingered something and your program is checking 7,500 entries every loop to check an IF statement or something, that could do it.
I can't seem to replicate your issue by simply loading images either.
I know that showing all of the code for an RPG isn't realistic, but unfortunately it's very hard to help debug this sort of issue blind.
Back to top
Profile PM
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 11:08
Link
I thought it might be high image ID values as well, but I can't find any evidence to support it either. I'm actually using PASTE SPRITE for the most part, as I like to adjust dither, alpha, rotation etc. There's only one sprite - I do what I want with it, then paste it, then replace it with the next thing I want to draw, adjust it and paste it, repeatedly. The number of images loaded doesn't affect the number of images drawn, however, so I don't think this draw system has anything to do with the slowdown issue.

I will try timing the sync command just to get that data, it sounds like an intelligent thing to look at. Thanks for the suggestion.
Back to top
Profile PM Email
Wyldhunt
14
Years of Service
User Offline
Joined: 27th Sep 2009
Location: The Dark Side
Posted: 22nd Mar 2011 11:16
Link
If you're using sprites (Or, a sprite), you may want to experiment with the Advanced Sprites DLL. Everyone who used it said that their FPS shot through the roof when they used it. I doubt that the sprite is your issue here... Drawing a sprite shouldn't change dependent upon how many images you have loaded and not used, I don't think.
But, it does appear to improve frame rate when using sprites.
Link to download and thread can be found here:
http://forum.thegamecreators.com/?m=forum_view&t=92836&b=5
Back to top
Profile PM
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 12:27 Edited at: 22nd Mar 2011 12:30
Link
Yeah, I don't think the sprite is the issue either. Bugtracking is nasty business! When gets me is that I'm not new to this (I've been coding for 24 years, 7 of them in DBPro) and I'm no slouch at coding or bugtracking. With each hour a bug escapes my detection it becomes easier for me to believe that it's caused by an anomoly in DBPro's command set. If I'd never encountered such an anomoly before I'd be sure I'd done something stupid somewhere, but infortunately I have and I am forced to wonder whether I'm doing anything wrong at all.

I have implemented a sync timer and it's returning an expectedly small value (two milliseconds). I also have a mainloop timer which covers the sync as well as all screen updates, and all calculations that take place every iteration, which is returning 80 milliseconds per loop (it's only 21 when I load fewer images). This seems to preclude the SYNC command from the cause, though I never truly suspected it to begin with.

It's a fresh day for me, just got out of bed, so I'm looking at the problem with a hopefully new perspective. I'm going to start throwing thoughts out there as I think them, just in case someone adds two and two where I've missed it:

I use a set of variables to determine how many images are loaded:


Then I call the resource loading function. Note that this is the only place in the entire code that the above declared values are referenced. I will include here the entire resource loading function even though only part of it seems relevant:


The defineitems() function merely clears the array. The loadscreentext() function isn't important either; it just displays the gold 'Solodor' logo with the "Loading Entities..." text that you see at boot in the public alpha.

Here is the loadaccessory() function, called from the loadresources() function above:


After this code is executed, all of the assets needed by my game are loaded. Just how many images are loaded will be determined by the variable set declared in the first code snippet. If I set usetestnumbers to 0, I get a drastic reduction in framerates. If I set it to 1, I get full speed, and if I set it to 2, I get a somewhat reduced framerate.

Next is the routine that draws from the 7500 images that seem to be in question. It bears some explanation. Each entity is represented by a composite image: first we draw a nude, bald body, over which we draw his hair style, then we draw his clothes, his helmet, then his cape, then his wings, etc, until we have a composite image representing our character. Each entity has a set of arrays (I probably should have used a UDT for this, but for some reason I can't remember, I didn't) that contains item ID numbers (not image ID numbers). The set of instructions below that assign the contents of the WID() array convert the item ID numbers into image ID numbers, and populates the WID() array with the image numbers that will be drawn. The big complex IF statements containing checks against camx, camy, screen width etc are just checks to make sure the object is actually within the bounds of the viewable area of the map before we draw it - ie, we don't draw stuff that is off-camera:


I believe that this is all of the relevant code in the entire program. I have started running some more tests, isolating parts of my code (ie, not calling entire functions in the mainloop that perform draw updates or critical calculations). This is resulting in a malfunctioning game, but I am testing my framerates with each exclusion to see if excluding any one function restores framerate. This might point to some culprit code. I feel sure I've either done something noobish or DBPro itself is at fault.

Then again, I suppose one of those things is always to blame for a bug. Can anybody see anything obviously dumb I've done above?
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 12:44
Link
Ok I'm just going to post test results as I encounter them, conclusive or otherwise:

When I de-limit the framerate (SYNC RATE 0) and exclude all functions (yes, ALL of them) and do nothing at all in my main loop besides drawing the HUD to the screen and a sync, I am getting a framerate disparity. If I have loaded only a few images, the fps is 250. If I have loaded them all, the framerate is 200. I can't see any possible way my code can be responsible for this.

Here is some code - Mainloop looks like this right now:


As you can see, all functions except drawhud are excluded. Drawhud draws a chat and system console, a few status bars and a set of debug information, including the mainloop timer, the sync timer and SCREENFPS(). This is the only thing being done in the mainloop right now - everything else is commented out.

Why the fps disparity, then?
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 12:49 Edited at: 22nd Mar 2011 13:12
Link
Quote: "Can anybody see anything obviously dumb I've done above? "


Not yet. The first thing I'd test though is this:



I can't see anything obviously wrong that. However, my concern is that when you are using only a few images their ids will be widely separated whereas in my snippet the used ids are always the first few. I'll test that to see if separating the used ids makes a difference. I have a vague feeling that something behind the scenes may be searching or sorting and that may be slowing things up. That's a pure guess at the moment though.

I'll edit this post when I've done a bit more testing.

Edit Just run the following on my desktop with widely separated image ids and saw no obvious reduction in framerate. I threw in an extra level (102400 images!) for good measure and there might be a slight reduction - but the framerate varies in the range 860-910 anyway so it's hard to tell.

Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 12:59
Link
Hey Gandalf.

I thought so too, but as in my last post, the one immediately above yours, I'm getting a frame rate disparity when I do nothing in my mainloop other than display FPS, between loading a few images and loading lots of images. I don't even have to reference any images at all to cause a slowdown - I just load more of them.
Back to top
Profile PM Email
Wyldhunt
14
Years of Service
User Offline
Joined: 27th Sep 2009
Location: The Dark Side
Posted: 22nd Mar 2011 13:15
Link


If I'm looking at that correctly, you're getting a few issues with your image numbers. 33000+7000=40000. Your final tileset image(40000) is using the first ability icon slot(40000).
I'm at work right now, but I'll try to take a closer look when I get home.
Back to top
Profile PM
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 13:17 Edited at: 22nd Mar 2011 13:19
Link
Just edited previous post. I was posting at the same time as you before so didn't see your post about the HUD till my post appeared.

That is strange. I don't know what to suggest now. What does drawhud() do? I'm also unfamiliar with the sync sleep command. What happens if you exclude that?

Edit Just seen Wyldhunt's post. Seems worth looking into.
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 13:53 Edited at: 22nd Mar 2011 13:54
Link
Wyldhunt, thanks for pointing that out. It'll only be an issue if I actually have 7000 tileset images. There's only about 400 of them at the moment. Since file #7000 doesn't exist, image number 40000 will never be used. Even if it were, the program would crash when I tried to load a second image into an occupied slot. Just the same, I've changed the FOR...NEXT loop for the tileset load code to iterate 6999 times instead of 7000 times. No change. This isn't causing the slowdowns.

I've actually had some interesting test results that don't make sense on the face of it and bear further scrutiny. I implemented a system that times each and every function that I call from the mainloop, in the same way that I timed the SYNC command in my mainloop:



I am bracketing each and every function call in my mainloop with timing code to see how long each function takes, and dumping the results to screen at runtime.

Regardless of how many images I load, most of the functions require the same amount of time to execute. If I load all the images rather than some, two functions are taking up substantially more execution time. The two functions are the ones that draw the map to the screen - *very* strange, since the same number of tiles are loaded no matter what value I set for USETESTNUMBERS.

This will bear further investigation. If anyone thinks of anything in the meantime, keep posting. This bug has got to go.

EDIT: Forgot to answer a couple of questions. SLEEP SYNC 1 gives a single millisecond of CPU time back to the OS for its own operations. It makes the program CPU friendly and causes the program not to eat 100% of the cycles from a core. I tested omitting the command and it has no effect on framerates but increases the program's CPU load.
Back to top
Profile PM Email
Wyldhunt
14
Years of Service
User Offline
Joined: 27th Sep 2009
Location: The Dark Side
Posted: 22nd Mar 2011 14:21 Edited at: 22nd Mar 2011 14:30
Link
What I'd recommend at this point is to code a simple "While File Exists && FilesLoaded < X" that loads every image regardless of name or number and then a basic game loop that does nothing beyond show the FPS and sync.
You can change the value of X to test FPS with different numbers of images loaded.
That way, there's no chance of issues with any of your functions or numbering systems or display or anything else.
Assuming that you maintain FPS with the basic 'Load Everything' loop, then you can start to experiment by adding in a few sprite commands and see if the draw sprite commands are lagging when a lot of images are loaded or something.
Trying to determine if it's DBP from inside of a full games worth of code is a PITA. Sometimes, it's best to make a new Test project that you can butcher to your liking. If that passes, then you know that it's not DBP... Or, if it is DBP, you can figure out exactly which command is lagging with high numbers of images loaded.

I'll be home in a couple hours and I'll see if I can come up with something like that to test with on my side and see what my results are. I've a ton of small icons I can experiment with.

EDIT:
You mentioned that this didn't seem to be a problem in older versions of DBP, so I checked the Codebase:
http://code.google.com/p/darkbasicpro/updates/list
Looks like they did play with the image code lately, r75 & r80 caught my eye. I haven't the time now to see what they changed, exactly. I'm no good with C++ anyway.
Back to top
Profile PM
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 14:31
Link
Quote: "two functions are taking up substantially more execution time. The two functions are the ones that draw the map to the screen - *very* strange, since the same number of tiles are loaded no matter what value I set for USETESTNUMBERS"


That at least has to be progress even if the precise problem hasn't been identified.

Any chance of posting one or both of those two functions?

Thanks for the info about sync sleep. I don't see why it doesn't have an effect on fps - in fact it has a predictably big effect on my machine but still no dependence on number of images loaded. What screen mode are you using?
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 14:37
Link
I may have figured this out... I think IMAGE EXIST() might be the culprit. Every time I display an image, I am error checking to make sure it exists before I try to display it, and skipping the paste if it doesn't. This means I am running the IMAGE EXIST() command hundreds of times every mainloop iteration. If there are lots of images loaded, this command seems to take MUCH longer to execute. Because of this, I might be slowing down my mainloop substantially just by loading more images into memory.

Does this logic follow for anyone? Comments? Thoughts?

I am compiling a test for my theory, and I need someone to do the same thing independently to corroborate my results.
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 14:38
Link
Quote: "Does this logic follow for anyone?"


Yes. I'll do a quick test now.
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 14:47
Link
My test results show that IMAGE EXIST() is indeed the culprit. I am holding my breath waiting for you to corroborate. This could be the death of the most annoying bug ever to plague this 8-month project...
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 14:55
Link
I've implemented an alternative means of determining whether an image exists. I have added this code to the end of my loadresources() function:



This adds about three seconds to the boot time, but the tradeoff is that I can now skip using IMAGE EXIST() everywhere in my code, and simply check the value of imageexists().

Implementing this into my map draw code shows a stupid improvement in efficiency. What used to require 45ms to process now requires 2ms and my framerates have been restored to maximum.

This result squashes the bug and I am very pleased we have identified it. It had threatened to cripple my entire project.
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 15:04
Link
I've just tested the following code:



and the results are a bit odd.

1. With 100 images the fps is noticeably higher - with or without the image check.
2. In all the other cases the fps soon settled down to 333 - but starting from a much lower value.
3. With more images it took noticeably longer for the FPS to settle down - several seconds in fact when using 102400 images starting from about 160 fps.

I'd guess you're on to something here.
Back to top
Profile PM Email
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 15:05
Link
As usual I took too long to finish my post.

Sounds like good news. Useful to know.
Back to top
Profile PM Email
IanM
Retired Moderator
21
Years of Service
User Offline
Joined: 11th Sep 2002
Location: In my moon base
Posted: 22nd Mar 2011 15:37
Link
What version of DBPro?

One of the fixes I got in under the wire for the latest beta was a different storage mechanism for images - rather than a linear search for images, it's now logarithmic, so much faster for larger numbers of images.

Quote: "sync sleep 1"

Only needed once at setup, like SYNC RATE is.

Get the latest Matrix1Utility plug-ins (23-DEC-2010)
Back to top
Profile PM Email Website
Green Gandalf
VIP Member
19
Years of Service
User Offline
Joined: 3rd Jan 2005
Playing: Malevolence:Sword of Ahkranox, Skyrim, Civ6.
Posted: 22nd Mar 2011 15:46
Link
I was using U77RC6.

Perhaps that's why I found it hard to reproduce the problem. Would the changes explain why the fps took a while to settle down in my last test? Does the search/lookup procedure build up a table of some sort in the background as images are accessed?
Back to top
Profile PM Email
Agent
19
Years of Service
User Offline
Joined: 7th Sep 2004
Location: Sydney, Australia
Posted: 22nd Mar 2011 15:48
Link
Oh! That's nice, I thought you had to call it every iteration. Thanks for that.

A logarithmic search makes a lot more sense and would have solved my problem Fortunately I was able to manage a workaround, but just working out what the problem was in the first place was a nightmare!

I'm using version 1.073.
Back to top
Profile PM Email
tiresius
21
Years of Service
User Offline
Joined: 13th Nov 2002
Location: MA USA
Posted: 22nd Mar 2011 17:11
Link
Glad you found the culprit. I have timer checks (using IanM's hitimer() ) around all my major game functions inside the loop logic so I can monitor if something spikes or gets ugly for some reason. Too much time spent in Newton when X happens, Zone logic processing, etc.

I'm all for protective programming (my delete_X() type functions always check for the existence of an asset before doing the deed). But for the game itself wouldn't you want it to crash on an "image does not exist at line 13234" as opposed to... not displaying something? Or do you at least put it to a text file somewhere logging the problem?

Otherwise you'll never find those missing images unless someone happens to realize that when NPC A talks to NPC B, NPC A's hair disappears under their helm.


A 3D marble platformer using Newton physics.
Back to top
Profile PM Email
Wyldhunt
14
Years of Service
User Offline
Joined: 27th Sep 2009
Location: The Dark Side
Posted: 22nd Mar 2011 17:35
Link
Heh. Glad to hear you got it fixed.
In fact, I'm very glad. The game I'm working on does something very similar to what you were doing...
<Goes to change some code>
Back to top
Profile PM
Back to top

Login to post a reply

Server time is: 2024-05-18 23:03:26
Your offset time is: 2024-05-18 23:03:26