Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / [STICKY] DBPro 9Ex

Author
Message
Chris Tate
DBPro Master
10
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 14th Dec 2017 03:24
This is a perspective from usage of the purchased vanilla installation of the compiler (not the open source one). I have not yet migrated to DBPro 9Ex because of having near completed my DBP projects.

It appears to me that this function limit error is hardware dependant.

On my old system, with about 2500 functions, it would crash. Lots of functions were created for setting and getting values via function pointer.

The following code generates a variable amount of empty functions. On my old system, I would get the error during linking of about 2,500 functions. My new system managed to compile 8,000 with no problem.



A number of variables such as the number of call stacks per function or sub routine could also cause some kind of memory leak in the compiler, or something along those lines.

You may have already tried this a million times, but to be sure, it might be worth monitoring Compile.exe during the compilation of your project in its current state as it is about to silent crash using a performance monitor and debugger on seperate machines to see any differences and signs of memory leaks or something else which could cause it to shut down prematurely without any error dialog.

To reiterate, I've seen this limit occur consistently on one machine, but not another; using the original version of the language.
Green Gandalf
13
Years of Service
User Offline
Joined: 3rd Jan 2005
Location: Cornwall UK
Posted: 17th Dec 2017 14:57
Rudolpho,

Just reporting an oddity with your version which has cropped up in another thread. The following simple code doesn't display anything unless an extra sync is added:



I'm using the fulldesktop screen mode option in case that's relevant.
Rudolpho
12
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 24th Dec 2017 13:06 Edited at: 24th Dec 2017 13:08
New compiler build available
Click here to download it.

This build fixes the issue CumQuaT was describing, where very large amounts of interconnected source code would cause a stack overflow in the compiler.
The reason for this was that the compiler would use recursion to compile adjacent tokens in blocks. I'm not sure why this was ever done as iteration works perfectly fine for what was done, without the added wasted memory (and stack overflows when compiling complex enough code).

Chris Tate wrote: "To reiterate, I've seen this limit occur consistently on one machine, but not another; using the original version of the language."

This may be because the default stack size is set higher on one machine than the other. It should be 1MB by default on more recent versions of Windows.
Take note that the issue was not caused by functions alone, but also labels, variables, user types and data blocks.

@Green Gandalf: I can confirm that I also get this issue on Windows 10 (1703). As Microsoft decided to remove compatibility with all D3D9-related debugging tools I'm sort of stuck using Windows 7 to troubleshoot, which makes this difficult to pin down. One possible reason could be that newer Windows versions (ie. Windows 10) uses DX to draw everything in the OS. Thus if a program requests manual DX control, it may be that its window will not be drawn unless it flips its display buffer either. However were this the case, this problem shouldn't be specific to DBP9Ex but occur with the official versions too. From what you've shown it seems easy enough to just add the extra sync call, or does this cause more severe issues in other circumstances?


Edit: Also, merry Christmas to those who celebrate it!
CumQuaT
AGK Master
8
Years of Service
User Offline
Joined: 28th Apr 2010
Location: Brisbane, Australia
Posted: 24th Dec 2017 13:28
It's a Christmas miracle
Help out a fellow dev! Download and rate my games! They're free!

Wizzed Off! - (Android) (iOS)
God of Thunder - (Android) (iOS)
Extreme Lawn Mowing - (Android) (iOS)
Drunken Heroes - (Android) (iOS)

And you can get Fluffy Knuckleduster merchandise at our new store!
Chris Tate
DBPro Master
10
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 25th Dec 2017 06:59
Quote: "This build fixes the issue CumQuaT was describing"


Fantastic job finding the solution; a notable milestone for the community. As suspected, it really was something to do with memory.

Have a merry christmas!
Green Gandalf
13
Years of Service
User Offline
Joined: 3rd Jan 2005
Location: Cornwall UK
Posted: 25th Dec 2017 15:18
Yes seconded, Merry Christmas Rudolpho and thanks for continuing to work on this, your efforts are really appreciated.

Thanks also for confirming the symptoms I described. I guessed it was a MS W10 "improvement" that was to blame. If there isn't a simple solution it doesn't really matter since the fix is simple as you've noted. It can cause a new user to wonder what on earth is happening though (I'm not new but was also similarly baffled when I tried that simple snippet) - since it isn't immediately clear that anything is actually running - or more importantly what to do about it. I suppose one option is to trigger an automatic sync whenever the command "sync on" is called? I can see that could cause problems sometimes so a better option might be an optional flag for the command so that when the user doesn't want the extra sync they can use the flag - a bit like the optional DoNotGenerateExtraData flag on the load effect command. Just a thought.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 25th Jan 2018 04:11 Edited at: 25th Jan 2018 14:06
Are you still trying to improve this?

Look at this post I made 15 years ago. It's about the speed of various loops.

https://forum.thegamecreators.com/thread/830

It is still true today, FOR/NEXT is the fastest loop

(i use it instead of DO/LOOP for my main loop)

No one from TGS ever commented on why DO/LOOP is slower than a FOR/NEXT.

some example code (might have to increase the "count" variable if you have a beefy machine)

Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 25th Jan 2018 14:05
Not asking you to fix it or anything, I'm just wondering why.

I mean a DO/LOOP is basically

label:

'code here

goto label:

is it not? with an optional EXIT DO....

I just can't see how that should be slower than a FOR/NEXT where you actually have to do a compare ,too.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 04:13
Using ALMOST any command, I get the following error:

---------------------------
Test 1: Test 1.exe - Entry Point Not Found
---------------------------
The procedure entry point Direct3DCreate9Ex could not be located in the dynamic link library d3d9.dll.
---------------------------
OK
---------------------------

sample code:
//get the crash
sync on
sync rate 0

do
Print str$(timer())
sync
loop

//do not get the crash
sync on
sync rate 0

do
Print "Hello"
sync
loop

Any idea what's going on? I have the 2010 DX installed.

Works fine on standard 1.0761 with Synergy June 2010

It's like the 9Ex compiler (on my machine) doesn't understand anything other than the very simplest of DBP commands....

I made a copy of my install folder, copied over the 9Ex folders as directed, edited the .cfg files to point to the new paths.

But I'm not seeing 9Ex work at all with anything useful.




Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 07:38
Tried a few more things.

Make 2 copies of my original DBP folder.

Edit the configs in both copies to set the correct new paths.

Both copies compile my projects, run as expected, no errors.

Copy the 9Ex Editor folder into copy 1.

Still runs, no errors.

Copy the 9Ex Compiler folder into copy 1

Same error as before, only the simplest of commands will produce a working EXE. anything else errors out with:

---------------------------
Test 1: Test 1.exe - Entry Point Not Found
---------------------------
The procedure entry point Direct3DCreate9Ex could not be located in the dynamic link library d3d9.dll.
---------------------------
OK
---------------------------


Any ideas from anyone welcome.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 07:47
Oh, and I'm using Windows XP Home Edition, SP3 if that matters...
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 07:55
My limited internet research on this problem seems to tell me that DBP 9Ex will only work on Windows Vista and above if I get that error.



Guess I won't be able to use this.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 11:56
I guess I read the original post wrong to begin with

As the name hints, it has also had it's DirectX version updated from 9.0c to 9Ex, which offers better interoperability with the Windows Desktop Manager on Windows Vista and later (while still working the same as 9.0c on Windows XP)

I took that to mean "Compiler Works on Windows XP"

Any chance of a fork with these enhancements with regular DX9C? Or is that not possible?
Ortu
DBPro Master
10
Years of Service
User Offline
Joined: 21st Nov 2007
Location: Austin, TX
Posted: 26th Jan 2018 13:59
I know this isn't exactly helpful, but man its time to upgrade Windows, at least to 7. XP was great but you can hardly even get it to run properly on current hardware now and it is becoming more vulnerable every year.
http://games.joshkirklin.com/sulium

A single player RPG featuring a branching, player driven storyline of meaningful choices and multiple endings alongside challenging active combat and intelligent AI.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 26th Jan 2018 23:48
"I know this isn't exactly helpful, but man its time to upgrade Windows, at least to 7. XP was great but you can hardly even get it to run properly on current hardware now and it is becoming more vulnerable every year."

Oh yeah, agreed. The laptop I'm currently working on is 15 years old an not upgradeable. A Whopping 768mb in memory

I'm currently in a foreign land teaching English to kids and adults (also do some C# teaching on the side of that).

When I get back to the States in some months, I get back to my beefy desktop.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 27th Jan 2018 00:05
Yesterday I managed to get DX 10 installed on XP using some hacks (dxdiag reports it as DX10), still no joy with 9Ex. But I just had a thought, now that I'm reporting DX10, if I reinstall DX9...will it see I have DX 10 and give me the 9Ex dll.

I'll give that whirl today when I have a bit of time.
Rudolpho
12
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 27th Jan 2018 10:46
@Zep: The error message would seem to indicate you do in fact not have D3D9Ex installed. I admit I've never tried running this on XP, it is possible that 9Ex is only available for Vista and above. It is possible that this could be worked around by detecting the presence of 9Ex and use the regular Direct3DCreate9 call if not found. At this point there may well be things that require 9Ex built-in though; unfortunately I never expected this to be a problem.
Try to download the latest D3D9 version you can find (should be from the latter part of 2010, I think October). As the files are different depending on your OS I cannot say whether this will work with XP, but it would be worth a try in any case. Downloading DX10 will not help.

About the loop thing, DBP performs internal updates, message pump work etc. for each loop cycle. Not so in a for..next loop however.
The code to do this is basically injected right before your endwhile / loop / repeat command.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 27th Jan 2018 23:35
Quote: "About the loop thing, DBP performs internal updates, message pump work etc. for each loop cycle. Not so in a for..next loop however.
The code to do this is basically injected right before your endwhile / loop / repeat command."


Ah, interesting...wonder if I could possibly be hosing myself down the line by using a FOR/NEXT for my main loop.

Could you expound a bit on what gets updated/injected? It is really important stuff?
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 16th Feb 2018 14:06
Ortu found a nasty memory leak with Make Camera, Delete Camera

When a camera is deleted, memory is leaked.

You can see it in this little snippet:

sync on
sync rate 0
backdrop off 0

make object cube 1, 10

do
make camera 1
delete camera 1
sync
loop


I have no idea if you are still working on this or not Rudolpho. When I get back to the USA in a few months to my beefy machine, I could look at some of this stuff and help.

Is there a fork with your changes?

Though, I'd still appreciate a XP compatible version in the meantime.
Green Gandalf
13
Years of Service
User Offline
Joined: 3rd Jan 2005
Location: Cornwall UK
Posted: 8th Mar 2018 18:17 Edited at: 8th Mar 2018 18:19
Rudolpho,

I've been using your 9Ex version of DBPro almost exclusively recently without significant problems - till yesterday .

There seems to be an issue with animating certain types of skinned mesh. I've attached a simple demo containing a simple crude model of a snake's head. The model animates correctly when I compile this using DBPU77RC7 but not with your 9Ex version. I've included the whole project plus two exe files, one compiled using 9Ex the other using U77RC7. I've no idea how widespread this problem is though. This model has animated correctly with every version since it was created in 2008 - till now .

Any ideas what has gone wrong?

Edit: Sorry!! Forgot zip file. Here it is:

Attachments

Login to view attachments
Rudolpho
12
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 9th Mar 2018 23:20
Zep wrote: "Ortu found a nasty memory leak with Make Camera, Delete Camera

When a camera is deleted, memory is leaked..."

The program generated by the D3D9Ex compiler from your posted code does not appear to leak any memory on my machine. Care to elaborate a bit on how you came to this conclusion?
It isn't impossible that your OS (or D3D runtime) may cache some data structure used by the camera as long as you have plenty of free memory for example, and only reclaim it when it deems such appropriate. Though I don't think that is likely to be the case.


Zep wrote: "I have no idea if you are still working on this or not Rudolpho."

I am quite busy with work and a personal game project at the moment, as well as some home repairs. Work on this project is further hampered by the fact that I have to boot up Windows 7 to even work on it as Microsoft decided to deprecate most D3D9 debugging tools to such an extent that they flat out won't work on later Windows versions.
That said, if you find an issue feel free to drop a post describing it as well as possible (Green Gandalf's post above here with an attached mini project replicating the issue is a great example of something that is very helpful for me to have when addressing bugs) and I will try to get to it whenever I am able.

Zep wrote: "Is there a fork with your changes?"

There is not, this started as a request job by StarWraith 3D Games a couple of years ago and is, for the time being at least, not public source'd. Not that there's anything to hide


@Green Gandalf: Hm, that is indeed curious.
If you add the bounds flag and set it to true (one) at the end of the ROTATE LIMB command it does work as intended for me:

Can you try to manually provide that parameter and set it to zero when using U77RC7 and see if that works?
That flag denotes whether the bounds of the object has to be updated. I haven't had time to properly look through the function that accomplishes that but it is quite possible that it sets some flag that causes the new mesh data to be updated to the GPU and that isn't set by the ROTATE LIMB command itself. If so it may be that this bounds updating was enabled by default in U77RC7. Otherwise something else may be at play here.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 10th Mar 2018 00:26
Quote: "The program generated by the D3D9Ex compiler from your posted code does not appear to leak any memory on my machine. Care to elaborate a bit on how you came to this conclusion?"


Turns out it depends on what version of DBProCameraDebug you are using. I switched to the one on github and the leak went away.

Quote: "That said, if you find an issue feel free to drop a post describing it as well as possible"


Wish I could, but can't. As I mentioned before, I am on Win XP and get a D3D9Ex error when I try to run anything.
Green Gandalf
13
Years of Service
User Offline
Joined: 3rd Jan 2005
Location: Cornwall UK
Posted: 10th Mar 2018 00:31
Rudolpho,

Thanks for that suggestion. I'll try that tomorrow and report back.
Rudolpho
12
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 10th Mar 2018 13:15
@Zep: Hm, does that mean you're trying to use DBP9Ex dll's with the 9.0c compiler and other dll's?
That's bound to fail in various ways so that may very well leak memory as a function in my CameraDebug.dll may allocate a structure of say 280 bytes when creating a camera, which is then released via DBProCore.dll which thinks the camera struct is only 260 bytes large. You're also likely to encounter crashes sooner or later if you're mixing the dll's.
For the record I've seen no memory leaks using DBP9Ex's DBProCameraDebug.dll with the rest of the 9Ex runtime.
Green Gandalf
13
Years of Service
User Offline
Joined: 3rd Jan 2005
Location: Cornwall UK
Posted: 10th Mar 2018 15:57
Quote: "Can you try to manually provide that parameter and set it to zero when using U77RC7 and see if that works?
That flag denotes whether the bounds of the object has to be updated. I haven't had time to properly look through the function that accomplishes that but it is quite possible that it sets some flag that causes the new mesh data to be updated to the GPU and that isn't set by the ROTATE LIMB command itself. If so it may be that this bounds updating was enabled by default in U77RC7. Otherwise something else may be at play here."


Haven't had a chance to check U77RC7 yet but that flag works for that particular case. However, in another demo, a skinned model provided in the MS DX9 SDK won't animate in one program but does in another which is a bit weird. I'll try to identify the essential difference.
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 11th Mar 2018 00:47
Quote: "@Zep: Hm, does that mean you're trying to use DBP9Ex dll's with the 9.0c compiler and other dll's?
"


No, I'm using the DLLs from git hub in a RC7 updated DB Pro.

The D3D9Ex erros only occur when I try to use your build. I mentioned that before, this is because I am on WIN XP and it doesn't support D3D9Ex.

Rudolpho
12
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 11th Mar 2018 09:26 Edited at: 11th Mar 2018 09:26
True, but then how did you encounter that memory leak? Wrong forum thread?
Zep
16
Years of Service
User Offline
Joined: 31st Aug 2002
Location: From PA, USA. Currently reside in Hanoi, Vietnam
Posted: 11th Mar 2018 10:03
Quote: "True, but then how did you encounter that memory leak? Wrong forum thread?
"


The DBPRO I originally downloaded was using other DLLs. The Old (or new...who knows, all the DLLs show the same Version #)

When I switched to the DLL on github, that make camera/delete camera leak went away.
A Long Time Ago
2
Years of Service
User Offline
Joined: 23rd Nov 2015
Location:
Posted: 23rd Mar 2018 21:12
I haven't looked at the dbpro forum for quite a while (I thought dbpro was pretty much dead thanks to the all new shiny stuff tgc are upto these days) and stumbled across this a few days ago and I have to say that this is a fantastic piece of work by Rudolpho, congrats!

The compile time of my project has fallen from about 8-9 minutes to about 60 seconds, which is amazing.

Of course, there's always a problem, and the one I have is that I use IanM's function pointer calls heavily in my code due to lua scripting (so the scripts can call internal functions and you know, actually do stuff), which just causes crashes presumably because the function table has moved or something?

Does anyone know if there's another way to get a pointer to a function other than IanM's utilities please? As I'm guessing, IanM isn't likely to update his utilities again anytime soon, which is a pity as they are brilliant too.

Bored of the Rings
13
Years of Service
User Offline
Joined: 25th Feb 2005
Location: Middle Earth
Posted: 24th Mar 2018 20:49
yes it's really simple. I'll add code here soon.
Professional Programmer, languages: SAS, C++, SQL, PL-SQL, DBPro, Purebasic, JavaScript, others
Kuper
10
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 20th Jun 2018 01:25
Hi everyone!
Find another bug:

I have clear screen ( empty checklist ).Original compiler works fine.
Kuper
10
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 10th Aug 2018 12:25
PS.
For whom who have troubles with raycasting using DarkDynamix.
I have unexpectable crashes when getting ray cast hit data or try to change raycast max distance during main loop.
There are no such troubles with original compiler.Anyway there is no much issue - just use the code below
right after setting on your DarkDynamix.

Login to post a reply

Server time is: 2018-09-21 21:07:04
Your offset time is: 2018-09-21 21:07:04