Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Dark GDK / Multi-threaded loading

Author
Message
SunDawg
19
Years of Service
User Offline
Joined: 21st Dec 2004
Location: Massachusetts
Posted: 12th Jul 2010 07:18 Edited at: 12th Jul 2010 07:33
A while back I had made an example program in the GDK showing how some operations could be safely moved out of the drawing thread. Someone suggested, though I can't remember who off the top of my head, that a separate thread was used for loading media so that the program wouldn't hang while you did that. A year or so later, I finally decided to try it!



In order to run the example you'll have to copy all of the media from the included textures from ...\The Game Creators\Dark GDK\Media\Textures to your project directory (or drop the compiled EXE into that directory and run). When the load is finished the process safely terminates and the images can be used in your program! Watch the framerate carefully as it loads and you'll see that the rendering is being done at full speed despite loading that pile of pictures. And that's just 24 MB of information, imagine what could be done with more! You could provide your players with a mini-game while the main one loads (This is seen in retail games like Assassin's Creed 2, which has you fool around in a construct like level while the actual level loads).
_Pauli_
AGK Developer
14
Years of Service
User Offline
Joined: 13th Aug 2009
Location: Germany
Posted: 12th Jul 2010 10:41
Didn't test this yet, but Wow that's awesome!
This opens the door for streaming worlds and stuff!
Great work.

Now the plot thickens, the fps decreases, and the awesomeness goes through the roof.
Hassan
15
Years of Service
User Offline
Joined: 4th May 2009
Location: <script> alert(1); </script>
Posted: 12th Jul 2010 14:54 Edited at: 12th Jul 2010 15:08
Did you try it? does it work? it doesn't seem to, well, in some stage of dbLoadImage, the IDirect3D9Device interface is used to load the image, and dbSync also uses this device multiple times, so, if dbLoadImage was called while dbSync is being executed, an unhandled exception will happen, because the device is not multithread-safe

maybe this program work, but it's just about luck, you could try loading infinite number of images, and wait a few seconds, you would probably get the exception

what you can do is the following:
first off, what part makes loading images/objects somewhat slow? reading the data from the file of course! but actually putting it in the game takes some RALLY microscopic time

And thus, you should read the file data in one thread manually, and put it in a queue for the main thread, then the main thread uses the data given and signs it in the device, this won't affect the main thread's FPS by a great amount at all

now for reading, i'd suggest you to make your own file format, they're actually simple, e.g, for an image format, what is an image? a couple of pixels, so you just store each pixel's color in the file and tada, you're done
what is the object? a couple of vertices and indices, so you just store them and tada! it's done, well i havn't really experienced with animations as i don't know how they work, but probably something simple.

not to mention the great advantage of having your own format, for example, you can have a position in the file for the object, and alot of properties, how does this help? ok, you could create a program that reads all files in one folder, so everytime you make a new a file in the folder it will be read without any changes on the exe, so, everytime you apply changes to the map (e.g. a patch for the game), you dont need to re-distribute the whole game, just a patch file that creates some new objects, and you dont need to know the positions in the exe's, as they will be written in the file itself

Hope it made sense

SunDawg
19
Years of Service
User Offline
Joined: 21st Dec 2004
Location: Massachusetts
Posted: 12th Jul 2010 16:19 Edited at: 12th Jul 2010 16:20
Streaming wouldn't be a bad thing to test, though I think it'd be tough to manage loading of data as it's difficult to tell what has been loaded.

@Hassan: I get roughly 6000 frames per second when running this demo, which means 6k calls to sync every second. If that exception was going to be a problem I would think it would manifest immediately.
Hassan
15
Years of Service
User Offline
Joined: 4th May 2009
Location: <script> alert(1); </script>
Posted: 12th Jul 2010 17:56 Edited at: 12th Jul 2010 18:01


this used to trigger the error, after a couple of minutes

also,
Quote: "Original post by Dark Coder
GDK isn't thread safe."

direct3d 9 is not thread safe either
Quote: "Original post by MSDN
Direct3D 9 does not default to thread safe. However, when you call CreateDevice or CreateDeviceEx to create a device, you can specify the D3DCREATE_MULTITHREADED flag to make the Direct3D 9 API thread safe. This causes significant synchronization overhead. Therefore, making the Direct3D 9 API thread safe is not recommended because performance can be degraded."


KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 12th Jul 2010 19:01
Quote: "this used to trigger the error, after a couple of minutes"


I'm thinking maybe the key there is the dbPositionObject command in the thread. If you're manipulating a GDK object when you hit dbSync, then I would think it more likely to complain.

Loading an object in a thread may indeed be safe, you just have to manipulate the objects in the main thread after you are sure they've already been loaded.

Hassan
15
Years of Service
User Offline
Joined: 4th May 2009
Location: <script> alert(1); </script>
Posted: 12th Jul 2010 20:04
you can try, i dont really think that it has anything to do with it, because all what position object do is to loop through objects and find the specified id then change a variable there, and possibly re-arrange objects according to the distance, anyway you could try it out, but im pretty sure it wont work

KISTech
16
Years of Service
User Offline
Joined: 8th Feb 2008
Location: Aloha, Oregon
Posted: 12th Jul 2010 20:22
The way it was described to me for multithreading in DBPro is you can use a thread to do pretty much anything you want to, but you cannot manipulate or access an object in either the thread or the main at the same time.

So I'm thinking that dbLoadObject is fine in a thread, because until the object is done loading, the main thread knows nothing about it's existence yet. However, once it's loaded in the thread, the main knows it exists, and changing it's position in the thread while the main thread is allowed to do a dbSync would certainly be a cause for a conflict.

At least that's the way I understood it at the time.

All we can do is try it and see.

SunDawg
19
Years of Service
User Offline
Joined: 21st Dec 2004
Location: Massachusetts
Posted: 12th Jul 2010 21:18
Why are you passing the address of your function to _beginthread? It seems to work without the &.
Hassan
15
Years of Service
User Offline
Joined: 4th May 2009
Location: <script> alert(1); </script>
Posted: 12th Jul 2010 23:17
my bad, i wrote that code here, anyway, what do you mean by "It seems to work without the &."? it compiles, or it loads without problems? if it loads for some minutes without problems then it's weird, because i have tried it several times with alot of ways and none of them worked

im still sure that neither GDK nor Direct3D 9 are thread safe, which means this direct way of loading will not work, there are many threads on the board discussing this subject, and ALL of them agreed that this way wouldn't work with darkGDK

Triumph Daytona
14
Years of Service
User Offline
Joined: 18th Apr 2010
Location: UK
Posted: 13th Jul 2010 01:01
Interesting topic.

I also use multi-threaded processes however, always good to remember the mutual exclusion problems e.g. writing to and reading back from variables/shared resources 'at the same time'thus causing race conditions (i.e. crashing at random points).

Before calling threads I would initialise critical sections, eg.



Then call the thread:



And the thread handler (example)



Another twist -- if possible -- try to download resources from a server that way you're keeping the resources lite.

Login to post a reply

Server time is: 2024-07-04 09:35:28
Your offset time is: 2024-07-04 09:35:28