Sorry your browser is not supported!

Hi guys, I have run into a problem with my fighting game and I will need an advice.
There have been several threads describing the same or similar problems, but I decided to make a new one, since no actual solutions were provided so I wanted to try my luck, with hopefully some more fresh ideas.

Here's the problem. I recently implemented in my game Enhanced Animations and Ninja Mat's normal map shader. I use a large amount of animations and some textures are in very high resolution, so I suppose that my game now is using a Lot of memory (ok ,it is not a guess, it does).

So this is when the problem started. While the game is running now, DBpro will randomly choose not to load a media file of any type, and in most cases the program crushes.

Ok, some people will probably speculate that it is a problem related to bad media management in my code, so let me offer some details.

First thing I did is to make sure that I am providing the correct file paths in my code. I moved ALL media files in 1 folder. Every load command uses an absolute path "C:\...". Also this is a problem that happens randomly, sometimes it will happen and sometimes it will not. Also it happens in pre-textured objects, sometimes the object will load without their texture (both objects and textures are in the same folder). So this is not a problem related to bad media management.

This is how my game works. At the beginning of a match, 2 fighters and 1 background will be loaded. When the fight is over, the objects loaded will be deleted and the next ones will be loaded for the next match, and so on...
I am doing this because at the final version of the game I will be using a large amount of fighters and backgrounds so I will not be able to keep everything loaded in memory at the same time. And this has worked well so far, until this problem started.

Here is an example. Yesterday I tried the game for more than 30 minutes. All game media files were loaded successfully, I had other programs running at the same time too, no problems happened. But today I booted up my PC, started my game (nothing else was running) and at the second fight all background textures did not show up. Only the normal maps.

See the snowy look?
I closed the program, started it again, chose the exact same fighters and backgrounds in the same order, and this time everything was loaded successfully.


So I think that it happens randomly. When DBpro fails to load an image or an object the game crushes. If a texture from a pre-textured object is not loaded the object appears white. When an animation is not loaded the game crushes and an Enh. Animations error report is created.

Now the problem starts to happen after I delete the first loaded objects. So I tried something. I loaded all available fighters and backgrounds at the beginning of my program. Everything was loaded successfully every time that I restarted the game, no problems, so this is not a problem related to memory limitations. But this is not a very efficient solution. Now I have 7 fighters and 5 backgrounds available but the final version will have more than 20 fighters and 10 backgrounds all of them with many texture variations. I can't have all media files Loaded at the beginning of the program, I am afraid that I will hit a memory limit.

So I am stuck now, any advice will be very appreciated guys.

Thanks in advance.

Additional info. Character's texture sizes are 1024x1024. All characters have 2 textures(Primary and alternative). Background texture sizes vary from 1024x1024 to 2048x2048.
I am loading 250MB of animation data.

If any of that info helps...

I'm probably not skilled enough to help here, but have you tried to manually re-texture the object after loading just to see?

No I didn't tried that. But again the "white" object is not the same every time, it is always a random object, so I can't texture it manually. Of course I can have all objects textured manually, this could work but again I would have to load all textures at the start of the program. This will be the worst case scenario solution, because I will probably have to reduce texture sizes.

DBPro programs are 32-bit, which means they can only access 2Gb of RAM. If you exceed that, memory allocations will start failing and you'll either get a crash or the effect that your media for example isn't loaded if the function loading it has implemented such as an error handler, which most of the standard resorce loading in DBPro seems to do.

Edit: Really nice looking graphics by the way!

@Rudolpho
I mentioned that when I load all my media files at the start of the program they all load successfully and the game runs smoothly. I am not exceeding the 2GB memory limit at this point of development.

This problem happens when I load media files after I delete the first objects. As I can tell from the Windows resource monitor deleting objects does release memory.

Perhaps you could check to see if your objects' textures exist upon loading with LIMB TEXTURE(n,0) and then manually texture if a zero is returned...?

How much memory is your program using and how much is free on your system when you run it?

@ Derek Darkly
I could do that, but the problem is not limited to pre-textured objects, it happens to any media file. I can have every object loaded and textured manually but DBpro might fail to load an image or even the object itself and that will crush the program. That prevents error handling too. I simply receive the "could not load..." message and the game stops running.

@Rudolpho
Just tested. I only have my browser opened and the game running. The game uses close to 700MB of RAM and I have almost 4GB RAM available.

Double posting and quoting my own words.

This is definitely not a memory issue, it does happen randomly. I just thought now, could that be a case of a hardware problem? The truth is that it has been a long time since I tested on another PC.

If you have a memory leak it could be.
OR if some other program in the background allocates more memory at some time. It does not seem to be the case (at least on my machine) that all 32-bit applications may only access the first 4Gb or memory on a 64-bit machine, but this is a rumour that pops up every now and then here and there. You could try making a program that allocates say 1GB of memory and run 5 instances of it at the same time; if one of them eventually fails that may be your problem.

a leak is one possible cause and fragmentation is another. When a program allocates and frees memory often, an unfortunate by product of this is memory fragmentation. This can cause future allocations to fail when the requested chunk size of continuous memory is no longer available within the pool, even if the pool has much more than memory than needed.

I think FPSC shares the same source code as DBPro. And FPSC has the same problems...

http://forum.thegamecreators.com/?m=forum_view&t=208910&b=28&msg=2500566#m2500566

This sounds like a resource management issue when it occurs after
deleting loaded objects.

It could be shader related double check your the shaders for errors.

Also if you are using Matrix1's dll make sure you are using the latest version.
We had come across a bug in an old version which caused blank textures. It was affecting image numbers that were being used by shaders.
Here is the link to that post.

http://forum.thegamecreators.com/?m=forum_view&t=195737&b=1




[img][/img]

Thank you guys, for all your replies.

First, it is not shader related issue, I removed the effect and this happened again, it failed to load a large texture(2048x2048). And I am using the latest Matrix1's dll.

I believe that it is related to memory, perhaps it it fragmentation that causes it. Although, in the above example with the screenshots, I had only loaded and deleted the first 2 characters and 1 background, they were only some MB, how could this fragment the available memory that much, to create an issue. And more important I should have mentioned that after this background appeared white, the next two backgrounds were load successfully, although they use the same texture sizes. I will also look if there is a memory leak.

But if it is memory fragmentation issue, then I wonder how FPSC games work? A complete FPSC game is larger than 2GB. I suppose that it does not load all media at start, right? Once a player enters a new stage, all previously loaded background media are deleted and the next ones are loaded. So, how does it avoid memory fragmentation problems? And generally how do all professional 32bit large games work? They do some kind of real time memory defragment or something like that?
Then again there is the "flush video memory" command that is useless now. In the documentation it says:

Maybe this could fix those issues, if it was working.

Well it seems that this will not have an easy fix. If I can't find a solution then I will have to reduce my media files sizes, and have them loaded at start, maybe remove the normal maps, and use compressed files. Anyway time for a small holiday break, got to get away from my PC for a while.

If your memory allocations are of about the same size you wouldn't encounter much fragmentation issues (ie. you delete a model of 512Kb, then load another of the same size - it will fit into the newly deallocated memory space).
Professional games make use of custom memory management to reduce these kinds of issues; one simple thing that is easy to implement is lazy deletion of resources - basically you allocate a large block of memory at startup (like an array) and fill it out as you go. When you "delete" an element from it you don't actually free the memory but rather just mark the element as deleted. You also maintain a list of such free slots in the memory block so that when you need to create a new element you just overwrite an old unused one. In this way you won't fragment memory. Of course it doesn't work all that well for elements of dynamic size (you could determine an overly large element size that "should fit" anything you throw at it, but that will of course end up wasting lots of memory instead).
Another idea can be to use linked lists instead of contiguous memory; this allows you to allocate smaller memory blocks for each piece of information - this of course increases fragmentation, but it is also more likely that the smaller allocations will succeed in finding a contiguous memory block than larger ones. It is also quite slow to constantly allocate / free memory so you have to assess whether this approach would actually serve to improve your application.
More involved solutions can of course make use of both of those techniques as well as others to provide a more general purpose-esque solution.

And yes, you only load what you need - per scene loading is easy and often good enough, but commercial games of late often are able to unload media that isn't currently visible in the scene on the fly and load in others that is if the available memory is starting to get low.

Also, video memory flushing will - as suggested - flush your video memory. It won't to anything to your main work memory where these issues occur. That said, that function may still be broken; I've never used it.

To what Rudolfo said, I wrote a memory management class (in c++) for my own programming library some time ago. And the latest version can allocate even complex array structures as a single allocation. So it's ~very~ gentle with the windows memory manager. It also automatically does aligned allocs so the allocations function more smoothly with the CPU, and especially they're friendly to SSE2 instructions (which require aligned allocs to function most efficiently). All the little details you have to think of are already handled by the class, so you don't have to worry about it.

Also with linked lists, I'm rewriting a linked list class now actually, that I originally wrote 15 years ago in plain C while working for a software company. Generally I've had to optimize my code for performance, so I use arrays. And in all this time I've had no real need for linked lists until now. Anyway, I use a single allocation for the entire list as well. No need to allocate each element separately. But a linked list, as you point out, would have the flexibility that if a large allocation was unavailable (to expand the list), it could allocate another smaller chunk, for in my present implementation, I use array pointers to link it (my original version linked it with array indices, which would perform a bit slower), so adding separate allocations would be seamless in its ongoing performance. In general, I'm allocating in chunks, and when I need to expand my allocation, I resize it, and update the pointers it contains. So unless I run into fragmentation, it's one gentle allocation with the windows memory manager, no matter the size.

The main reason I wrote my MemoryClass is because in my earlier programming years, I found and observed in others that memory leaks and oversights are the most common bug/confusion to surface while writing a program - and they can be nasty to track down (at least with the tools I had at the time). So I wrote the class to handle all my memory management automatically, so there's no need to have to repeat it in my programs or worry about overlooking something in my code. Nearly program I write instantiates that MemoryClass - it makes a BIG difference! It's very rare I run into memory problems, and when I do they're easy to spot and correct.

As a principle fragmentation comes from ongoing allocations and deletions. So if I know I'm going to have a need for the same size memory over and over again, I'll allocate it once, then reuse it. I won't allocate and free it repeatedly. I do, at times, have allocations in a function that I free at the end of the function, but those are only for functions that run once or at most twice in the program. For any functions that may be called repeated, I'll allocate once a class-wide temp in the constructor, and then reuse that same allocation over and over as needed, and let the MemoryClass's destructor free it.

So I just share here as ideas for other programmers. It really pays to have your own well-written memory management module. Having proper foundations in place (such as good memory management) is really a major key to keeping programs simple, easy, and well-functional, as well as not needing to re-invent the same thing project after project after project. And hence you can spend your time focusing on the fun stuff that's directly relevant to the current project!