Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Work in Progress / DBLibWrapper

Author
Message
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 3rd Dec 2013 21:30 Edited at: 10th Dec 2013 21:30
Update 2013-12-09:
The WrapperBuilder has been updated and can now extract string tables from, and thus generate wrapper functions for, licensed plugins as well. It also has two new configuration files (Ignore Exports.cfg and Stringtable Exceptions; see post 8 for some more information about how to use these).
The following plugins have been tested and seem to work (if you can test any more and report whether they indeed work or not I would be thankful!):

All core plugins of DBPro
Enhanced Animations
DarkNet
DarkClouds
DarkDynamix *
DarkPhysics
DarkShader
BlueGui
Matrix1Utilities

*dynStart() causes a null pointer dereference, essentially making the whole library unusable. I have thus far been unable to determine the cause for this.
------------------------
Original posts with some edits follow
------------------------
This is mostly a proof of concept.
I've been telling people over on the DarkGDK board that the basic workings of DarkGDK 2 / PureGDK supposedly wouldn't be all that hard to duplicate on a few occasions. Since the replies to these posts were... well, pretty much non-existant, I felt the need to prove my point and so I have built my own small C++ library for interfacing with the DBPro dll's, as well as a really user-unfriendly application for automatically generating wrapper functions for all discovered functions in the string tables of those dll's.
Admittedly this wasn't as easy as I had imagined but this core part was still implemented in about three days.
Well, enough back story.


Features:
Wraps the standard DarkBASIC Professional libraries as well as any (of the ones tested so far at least) third party libraries, licensed or public. Initiation as well as termination of the DBPro engine appears to be working as intended. All tested wrapped functions work (of course I've only tested like maybe 5% so far, but it seems to be holding up).

Can also wrap any third party dll with proper setup routines carried out. I've only tried a few so far though.
Update 2013-12-09: Licensed plugins are now working as well as any "user" (free) plugins I've tested so far.

Single function call setup and shutdown of the engine.

Supports the same display modes as DBPro (windowed, full desktop, fullscreen window, fullscreen exclusive and hidden).

All string arguments and return values are autoboxed to std::string for convenience - this also removes the need to manage string memory manually.

The WrapperBuilder application can be configured to use different prefixes for function names (such as 'db') for different libraries. It can also be set to automatically add #include <your-pch-file> to all generated source files if you use precompiled headers in your project.

Rudimentary error handling - you can get an error code of the last runtime error using the gdk::GetLastError() function. There is no enumeration of the different error codes included as of yet though and errors cannot be popped - it will keep reporting the last error until another one occurs. Since DBPro's standard error handling is to show a message box and terminate this may not be a very big issue, but it definitively could be handled better.

Easy to set up - see the instructions for Visual Studio 2010 below.

Since the job of figuring out what should get called in what order is already done and Windows dll calling is the same everywhere, it should be relatively easy to port this wrapper library to other languages such as Java, C# or VisualBasic.


Known issues
All used plugins have to be included with your application if you decide to deploy it.
This can be an inconvenience as well as look rather ugly. Perhaps they can be merged into a single file later on.

Does TGC approve?
This is probably bordering on being disallowed work since all it is really good for is that classic line from license agreements - "you should not re-release this work". However, since DBPro is open source now the only potential problems would be with third party libraries. The user of course must have these himself in order to use them, but in order to release a program made with this wrapper library you would have to include all the dll's as well. That of course happens all the time anyway, since they're included in the compiled DBPro exe's.





Instructions
Generating the wrapper files:

All wrapper files are generated by parsing string tables of dll files.
This is done by the WrapperBuilder application included in the attached archive. You specify the libraries you want included in the «build.cfg» file in the same directory as WrapperBuilder.exe. This one accepts a single input per line, however an asterisk (*) can be used as a wildcard symbol to match several dll's per line. You can optionally add a comma followed by a prefix to the end of each line in this file. An example:

The above will make WrapperBuilder.exe scan the DBPro installation's core plugins folder for .dll files; since we did not specify any prefix "db" will be used by default - this will then generate all standard wrapper functions like dbSync(), dbMakeObjectCube(), etc. After this is done we scan the plugins-user folder for all .dll files whose names begin with Matrix1Util. Here we also specify a prefix. Thus the generated functions from here will get names like m1uClamp(), m1uNiceWait(), etc.
Note that if a prefix is specified it will carry over to all subsequent lines - the default "db" prefix is only valid until another one is set.
Update 2013-12-09: A function name prefix for a certain library can now start with "::", in which case all wrapped functions will be put in a namespace of the "prefix" name instead. You can also supply "~" as the prefix, which means no prefix will be used - useful for libraries that already use prefixes in their DB function names (like DarkDynamix ("dyn") and Enhanced Animations ("EnAn_"). Apart from this any function name that starts with the same letters as its prefix will have the prefix letters stripped from its function name. This will for example let you strip the "mn" from the beginning of all DarkNet functions and instead put them in a namespace "mn".
------------
After preparing the build.cfg file you would simply run WrapperBuilder.exe. It will ask you to specify a folder to which the generated files will be saved and then it will do its thing. Some logs, including warnings, will be output to a console window. After this is done you would copy your files (or use them directly from where you had them generated) to your C++ project. You also have to include the attached "include/gdk_internal.h" and "include/gdk_internal.cpp" files with your project, but that's it.

Setting up a Visual Studio 2010 project:
Create a new project; it can be a console project or a win32 project. It is easiest to click the "Create empty project" checkbox, but each to his own. If you use a setup where you have a precompiled header you can tell WrapperBuilder.exe to #include that from all generated cpp files by supplying the command line argument "pch:your-header-file-here", for example "pch:stdafx.h".
As mentioned above you need to include the gdk_internal.h and gdk_internal.cpp files. I prefer to make a filter (in Visual Studio) and a corresponding real Windows folder named something like "gdk" and drop them there. You then put all your generated wrapper files in this same folder.
You also have to put the DBManager.dll file in your main project folder so that it is accessible from your program's CWD during startup (DBManagerDebug.dll is needed for debug builds, see more info below).
Next you would go to Project Properties -> Configuration Properties -> General and set Character Set to Not set. This is necessary since DBPro uses single byte ASCII character encoding.
Thereafter go to Project Properties -> Configuration Properties -> C++ -> Code Generation and set Runtime Library to Multi-threaded DLL.
Important: If you want you can also use Multi-threaded Debug DLL. If you do this you will have to replace the default "DBManager.dll" file with the "DBManagerDebug.dll" file or you'll get runtime issues. Pay close attention to this; each DBManager library will only run in either debug or release mode, not both. There is currently no code in the wrapper library that switches the loaded library based on your build configuration (that's easily added though, just check #ifdef NDEBUG ... #else ... #endif).
Note that unless you change the DBManager dll for the debug one you must build your program in release mode or it will crash on startup!
Just thought it better to point that out since Visual Studio projects tend to start out in debug mode by default.
Update 2013-12-09: gdk_internal.cpp now uses a conditional precompiler statement based on whether NDEBUG is defined or not to use either DBManager.dll or DBManagerDebug.dll.

That's it for setup. Just include "FullInclude.h" from the folder where you put all your wrapper files and you should be good to go.

Here's an example program that uses the standard "db" prefix and no precompiled headers during wrapper generation:


Setup in any other development environment should reflect these settings and it should theoretically just work - the main part of setting this up is just copying / importing files.
You also need to have the Windows SDK installed, but who doesn't?

The core functions of the system reside in the gdk namespace and are as follows:
gdk:: LoadLib(const std::string&) - Loads a DBPro dll file and registers it with the DBManager; it will go through appropriate construction and destruction routines when the engine is started. This setup is done automatically in the wrapper files.
gdk:: Start(const sSetupData&) - Starts the DBPro engine, initializing all previously registered dll's. The sSetupData argument can be filled out with various data that is used when starting the engine such as display mode and sync rate
gdk:: GetLastError() - Returns a DWORD error code corresponding to the last raised internal runtime error of the engine. You should probably quit if this is non-zero.
gdk:: DisplayAboutDialog() - Displays a message box with some brief information about DBLibWrapper such as release date. Probably not something you would want to use a lot
gdk:: CreateDeleteString(char**, DWORD) - Used internally for DBPro core string management. You should never have to call this manually.
gdk:: Shutdown() - Shuts down the DBPro engine and releases all loaded library handles. Should be called before closing your program.
-----------------------------------------------------


Well, feel free to try it out and let me know what you think / if you encounter any issues


"Why do programmers get Halloween and Christmas mixed up?"

Attachments

Login to view attachments
Chris Tate
DBPro Master
15
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 5th Dec 2013 20:06
Good luck with this project; I am sure there must be loads of indie developers interested in something like this.

BatVink
Moderator
20
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 5th Dec 2013 23:20
Pre DarkGDK there was a wrapper written by IanM. He came to some kind of agreement with TGC that his version would disappear. But it was around for a long time and got a lot more people into DBP.

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 5th Dec 2013 23:21
Thanks.
I've made some progress in that I can now parse the exported functions of a dll file, provided it uses name mangling that contains information about the datatypes of arguments and return types. This makes it possible to wrap functions from libraries that doesn't have readily readable string tables; however there are some limitations - since there is no information about what the DBPro version of the function name is, the exported name must be used (which is usually OK, but sometimes the exported name and the DBPro version name differ quite some). There are also only type info for arguments so they are generically named things like "iArg1", "fArg2" when generated in this way.
What's more bothersome is that most licensed plugins (as well as some user third party ones, Matrix1Utils being a big source, but less of a problem since it also features string tables) export unmangled names. I have no way of making out what kinds of parameters or return types these use and as such cannot automatically wrap them.
I am thus wondering if these libraries do in fact have string tables detailing how they should be called, but that are hidden / encrypted somehow, or if the DBPro compiler is actually manually updated so that it knows these things without them being on file somewhere? If anybody happens to know (or just have some ideas about this) I would be very happy to hear it.

I've furthermore discovered that certain licensed plugins like AdvAnim.dll use some custom setup step where it needs to be passed a key to a function OptionalSecurityCode(int). Reverse engineering have proven that the key sent by DBPro to this function changes from one run to another so that makes it quite difficult to replicate from my custom dll startup system.


"Why do programmers get Halloween and Christmas mixed up?"
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 6th Dec 2013 00:58
Sneaky ninja-posting bats...
Hm, is that so, I never heard of it. Well... maybe I should better just give up on this now then Thanks for pointing it out.

On a side note, figured the security code part out and got Enhanced Animations to work. Oh well.


"Why do programmers get Halloween and Christmas mixed up?"
BatVink
Moderator
20
Years of Service
User Offline
Joined: 4th Apr 2003
Location: Gods own County, UK
Posted: 6th Dec 2013 09:36
I was mentioning it to encourage you
Back then, DB Pro wasn't free, but now it is.

Here is a taster of what IanM produced, it may mean something to you, it's way beyond me.

Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 6th Dec 2013 18:38
Ah, well that is true... Thanks.
Unfortunately the "time machine" didn't seem to store any of his actual source files (which I guess is for the best since he chose to remove them), but I did find some interesting lines of text on there, so thanks for that

I recently found something that resembles the standard string table in MikeNet.dll (a licensed plugin that doesn't have any string resources accessible through the standard LoadString function) so maybe things are looking up a bit with this.


"Why do programmers get Halloween and Christmas mixed up?"
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 9th Dec 2013 11:42
An updated version of WrapperBuilder.exe can now be downloaded from the opening post or here.

This one is capable of exporting licensed plugin functions as well (the previous version did not); all such plugins I have tried have yielded working wrapper functions so hopefully this is a universal solution.
Besides the updated usage information in the top post there are two new configuration files for use with the WrapperBuilder;
Stringtable Exceptions.cfg defines certain plugins that do not have a "normal" readable set of string table entries. The WrapperBuilder scans for function declarations in three passes; first it looks for normal string resources, failing that it looks for the "hidden" string table used by licensed plugins and if that also fails (which it shouldn't; this is mostly a remnant of my old approach) it will enumerate the exported functions of the dll in question and try to generate function call prototypes from any exported functions that follow the name mangling scheme of Visual Studio.
Now there is a slight problem; reading the standard string resources from some licensed plugins will cause access violations. If that happens you should add the offending dll's on a new line to this file and the builder will ignore the first pass and look for the hidden string table directly.

Ignore Exports.cfg defines libraries which should be included for internal use by the engine, but not have any exported functions wrapped. This is used for some of the core libraries like ConvX.dll which are used internally by the DBPro engine, however their functions should not be used from outside. Also most of these will cause name clashes with functions exported from other DBPro plugins (ConvX exports LoadMesh, as does DBProBasic3DDebug.dll, etc.).


"Why do programmers get Halloween and Christmas mixed up?"
Rudolpho
18
Years of Service
User Offline
Joined: 28th Dec 2005
Location: Sweden
Posted: 13th Dec 2013 21:30
Unfortunately it seems TGC are not quite OK with this type of project since it can be used to circumvent the licensed validation for third party DBPro plugins.
As such the work on this will be suspended for the time being.


"Why do programmers get Halloween and Christmas mixed up?"

Login to post a reply

Server time is: 2024-03-29 11:53:07
Your offset time is: 2024-03-29 11:53:07