Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Speed up the compiler

Author
Message
Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 13th Feb 2015 04:47
How can I speed up the compiler?
My project is getting bigger everyday, now it's almost 35k lines of code, everytime i need to test it, it takes like 30-40 seconds to compile (only a few seconds for the loading though), which is annoying, is there a way to reduce this waiting time?
Sasuke
19
Years of Service
User Offline
Joined: 2nd Dec 2005
Location: Milton Keynes UK
Posted: 13th Feb 2015 04:59 Edited at: 13th Feb 2015 05:00
Because DBP re-compiles everything there's no real way to speed the process up. You can only make your code more efficient. For instance eliminating hard coding and putting all that into scripts or other types of files that you'd feed into your engine, which is what you'd do anyway with large projects. An engine just processes saved game data.

"Get in the Van!" - Van B
Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 13th Feb 2015 05:02
what do you mean putting all that into scripts? what kind of scripts?
Sasuke
19
Years of Service
User Offline
Joined: 2nd Dec 2005
Location: Milton Keynes UK
Posted: 13th Feb 2015 05:14
Well the idea is the Engine contains no actual game data (this only applies to big engines or engines that you'll re-use for many projects).

The engine will process config, data or script files containing everything from level data, script data, animation data and the list goes on. The compiled final EXE will read whatever script that tell the engine what data is needed to be loaded into the engine, how it will run it and so on.

If your creating a game where you've coded the game into the exe (well into dbp) then there's no way round the re compiler speed but when the compiler has built the exe and you don't have to recomplie and just run the game you won't have the speed issue.

"Get in the Van!" - Van B
Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 01:55
Sometimes I feel that having an i7 quadcore it's useless, I mean, with Windows 8.1 opening the pc main folder has some delays, compiling now 35k lines of code take 30-40 seconds, I don't see a huge difference between this one and my old dual core amd, not a huge difference with an ancient computer I had with Pentium 2 with 350 mhz.
Come on, we are in 2015 and I still have a delay when opening the pc folder (sometimes even seconds, normally half second), or this Windows 8.1 is trash, or 3 of the 4 cores I have are sleeping, but I have tested this also in other 2 pcs of my family both quad core both Windows 8.1 (one of the 2 is the same as mine) and the result was the same, so I suppose is the OS.
Windows 8.1 gave me several annoyance, and maybe if the compiler takes so much it's really because only 1 core is working, no way an i7 processor quad core turbo etc etc can takes so long to load 35k lines of code
MrValentine
AGK Backer
14
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 14th Feb 2015 06:16
Cescano, are you using HDD or SSD drives? a Laptop or a Desktop? if a laptop ensure you have the power settings set to High Performance Mode

I use a Surface Pro 2 running Windows 8.1 Pro and my large projects take a few seconds to compile... so I am guessing you are using a HDD as this is more than likely what affects your compile time, I went from HDD to SSD a few years back and the difference was very noticeable, also some SSD drives are actually slow due to component types and quality...

If you have not yet made the move to SSD, now would be a good time...

You can keep your OS as it is if you are using a desktop or if your laptop has a spare bay then this should work for you [If it has a DVD Drive there are Bay Converters out there, Why anybody needs an internal DVD drive on a laptop these days beats me ], simply add the SSD drive onto your system and install DBPro to it and place your project files on it too...

Even getting a smaller but faster SSD drive just as a swap or compile drive is probably a good idea too... Have done this in the past...

Good luck...



Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 06:30
I am using a notebook, a lenovo z500, i7 quadcore 2.2 GHz up to 3.2 with turbo if I am not wrong, 8 GB of ram, intel hd4000 for power saving and nvidia gt635m for games, the HDD it's big enough, 1 TB, but is not a SSD so it's not the best for loading times.

I don't know if my notebook is compatible for a SSD and I don't even know how much it costs to install one
MrValentine
AGK Backer
14
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 14th Feb 2015 07:07
Yup, If I were you I would look into it, do you have any USB 3.0 ports? you could try a fast USB 3.0 pen drive say 16/32/64GB...

How much of your Internal 1TB are you actually using? you could go for a 240GB~ SSD and place your Internal HDD into a USB case...

Costs nothing to install, you do it yourself and you can find videos on the web, pretty sure there might even be a video about opening your laptop model too... if you have never done this before, I suggest you watch a video in your native language and hopefully they will include Dos and Don'ts pointers...

Your laptop should be SATA II at minimum... so, the overall gain from an SSD would be the Cap of SATA II... which is why I would suggest USB3.0 as it can go up to 5Gbits using SuperSpeed

This one? [href]http://shop.lenovo.com/gb/en/laptops/lenovo/z-series/z500-touch/[href]

If you are not using your DVD/BD Drive, you could look for something like this [href]http://www.amazon.com/Generic-Drive-Caddy-Lenovo-Ideapad/dp/B00IGVF9F6[href] but again this will likely be SATA II

I could not quickly find out which SATA port the main hard disk had but for a cheap trial, you could try a cheap but decent speed/spec USB 3.0 pen drive around 16/32/64GB in size, or convert an SSD drive to a USB 3.0 case, this will give you peak 5Gbits versus 3Gbits

Some useful links relating to SATA II/III and USB 3.0

http://en.wikipedia.org/wiki/Serial_ATA#SATA_revision_2.0_.283_Gbit.2Fs.2C_300_MB.2Fs.29

http://en.wikipedia.org/wiki/USB_3.0

And an example of what I mean, maybe you can find faster models?

http://www.amazon.co.uk/SanDisk-Ultra-Flash-Drive-SDCZ48-064G-U46/dp/B00DQGBYDC/ref=sr_1_3?ie=UTF8&qid=1423893826&sr=8-3&keywords=fast+usb+3.0+64gb

MrValentine
AGK Backer
14
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 14th Feb 2015 07:08
Oh, but run some speed testers on your current Hard Drive to see its peak performance, there might even be a tool in your BIOS for this...

Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 07:39
I still have left 640 GB, HDD is SATA 3 6 Gbits, i have 3 USB ports but I don't know actually which version they are
MrValentine
AGK Backer
14
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 14th Feb 2015 08:14
Look at http://en.wikipedia.org/wiki/USB_3.0 above, it shows what USB 3.0 looks like, you should have 1 on your laptop...

Burning Feet Man
17
Years of Service
User Offline
Joined: 4th Jan 2008
Location: Sydney, Australia
Posted: 14th Feb 2015 10:08
You should configure your AV to ignore DBPro and your DBPro project folders, as you may find that it's your anti-virus scanner that's holding up the compilation of your project.

Help build an online DarkBASIC Professional help archive.
DarkBasic Help Wikia
basjak
15
Years of Service
User Offline
Joined: 16th Apr 2010
Location: feel like signing up for mars
Posted: 14th Feb 2015 12:34
I work in huge DBpro projects all the time. the best option to speed up the compile time:

- don't make the application as media application (unless in your last compile)
- turn all your constants into global values.
- put all necessary information in .dat file(or any format u choose) and then define your variables to read the information at loading time.
- don't put a lot of code in the program main story (do - loop), instead make the main program is for writing you software application story so you'll see more functions within the do -loop.
- create better logic for your application:
example:
instead of:
Quote: "if right key
move object right value#
else
move object left value#
endif"


you can use:
Quote: "move object right value# * (rightkey() - leftkey())"


- try to use more bitewise operators in your logic than the classic ones and/or .....

if possible use matrix utility to help with functions pointers and arrays, where u can call functions using pointers (I know how to implement it but never used it yet as am still Ok without it.)

you will see significant improvement in compile and runtime speed.

Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 13:24 Edited at: 14th Feb 2015 13:26
I already do most of the things you mentioned, I actually didn't know about the bitwise thing and probably never used.
Probably there is still something I can do to lower the lines but I have already made functions to avoid duplicate code on most of it.
Can you explain the bitwise thing? Everytime I attempted to learn it I didn't understand it very well, even if I am not a noob, is something which my mind refuses to learn lol

Also please explain the variables into a .dat file

As of now I have a lot of functions in my code, basically is all under functions except globals declaration at the top of the code and a few other things, what I hate of the editor is that everytime I open it I have to minimize all the functions manually, while I wish to have them already minimized at start and extend just the one I need to edit, I do not use the #include commands 1st because I think the code would be more hackerable opening the include files, even if saved in a .pak block I don't feel safe, and 2nd because when I need to do a quick search for a word, Having the Whole code on the same file let me search for that thing in 1 search only
TheComet
17
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 14th Feb 2015 13:55
Quote: "Sometimes I feel that having an i7 quadcore it's useless, I mean, with Windows 8.1 opening the pc main folder has some delays, compiling now 35k lines of code take 30-40 seconds, I don't see a huge difference between this one and my old dual core amd, not a huge difference with an ancient computer I had with Pentium 2 with 350 mhz."


That's because CPUs have stopped getting faster. Instead, they're getting more cores, and they're becoming a lot more power efficient.

The DBPro compiler is single-threaded, so you will definitely not experience any performance boost when compiling with modern CPUs.

The only way to do great work is to love what you do -- Steve Jobs
Chris Tate
DBPro Master
16
Years of Service
User Offline
Joined: 29th Aug 2008
Location: London, England
Posted: 14th Feb 2015 15:02
Quote: "The DBPro compiler is single-threaded, so you will definitely not experience any performance boost when compiling with modern CPUs."


Very true

Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 19:12
So my game will run on 1 core only?
basjak
15
Years of Service
User Offline
Joined: 16th Apr 2010
Location: feel like signing up for mars
Posted: 14th Feb 2015 20:44
not necessary

Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 14th Feb 2015 20:57
What do you mean? It can uses more cores? If yes, how?
basjak
15
Years of Service
User Offline
Joined: 16th Apr 2010
Location: feel like signing up for mars
Posted: 14th Feb 2015 21:16
of course, the compiler is 32bit system but as DBpro relies on Dlls, that means it depends on the Dll power.

I think Dark Occlusion/imposters is multi threaded dlls.

u can even create DX11 Dlls and embed it in DBpro or even open GL. etc....

TheComet
17
Years of Service
User Offline
Joined: 18th Oct 2007
Location: I`m under ur bridge eating ur goatz.
Posted: 15th Feb 2015 00:20
Quote: "What do you mean? It can uses more cores? If yes, how?"


It's possible that some plugins may be multi-threaded, but DBPro is not thread safe by design. You cannot multi-thread DBP code safely.

The only way to do great work is to love what you do -- Steve Jobs
MrValentine
AGK Backer
14
Years of Service
User Offline
Joined: 5th Dec 2010
Playing: FFVII
Posted: 16th Feb 2015 02:28
You could also look at Intervals, References and Callbacks found in STYX or maybe also in M1Utils.

Also things like LUA and Config file type formats such as xml/ini/cfg/dat files... [Flat Files]

Those files allow you to keep variables outside of your program, thus not requiring a recompile every time you want to adjust a variable... among other things...

For Flat Files [Common for Web files such as xml, not sure if used for programming speak though] .dat for example is simply a structured data file used line by line, whereas xml and ini allow you to read certain values and set them as well [Instead of the whole file], there is even a method to allow two programs to use the same text file at the same time [Enhancement pack: File Mapping, not sure if this is in M1Utils]... I will be showcasing this for Windows 8.1+ users soon... still not experienced with File Mapping, just tried to get it working, it works, just not sure how it is structured at the moment, just about to doze off...

Good Luck!

Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 16th Feb 2015 03:18 Edited at: 16th Feb 2015 04:46
Quote: "My project is getting bigger everyday, now it's almost 35k lines of code"


It'd be beneficial if you could post some excerpts (not the whole thing) of your program. Perhaps others can suggest alternative structures to further reduce down the programs foot print. The Dbpro compiler is known to be slow, which I suspect a by product of it's parsers internal string handling. So apart from throwing more and more cpu power at it, the only way to increase build time is give it less code to deal with up front.

What somebody out there should think of doing is writing a simple pre-processing tool that applies #constants, script comments through to Rename functions and other do-dads, from the input code up front. The resulting source is passed through to DBpro.


EDIT:

I've tested this before (years ago), but couldn't find the original examples, so these will have to do.

The following mock up code is create by a program, there\'s two versions of it. The human written version and a simulation of what a pre processor could do with it. Namely stripping comments, removing null lines & tabs and replacing long variable names with shorter one. Which is where I think the core problem with speed is, it\'s doing string compares when searching for variables, so the bigger the strings and or larger the keyword list becomes, more performance degrades.

The free Dbpro builds in the 11K lines version in about 80 seconds consistently, where the 3K line version builds in 55 seconds. Not earth shattering, but food for thought. (Test system 10 year old AlthonFX 3000. Attempting to build 30K lines took over 5 minutes, where i ended up stopping the process.

Sample code attached...

Attachments

Login to view attachments
Kuper
17
Years of Service
User Offline
Joined: 25th Feb 2008
Playing: Planescape:Torment
Posted: 28th Feb 2015 11:36
Ive tested compiler speed both with SSD and HDD and there is no great difference between them.But one thing that i found give some speed encrease. You have the main source file and some include files.But you have two ways to use them: you can include them in dbpro project window or just write #include "file name" in the main code file. So I get somу speed when do it the last way.
P.S. The Dbpro compiler is very old thing so doesnt support multiple cores.All you can do - buy new core , like last model of i7 which give you some more speed.

Cescano
10
Years of Service
User Offline
Joined: 4th Sep 2014
Location: Italy
Posted: 28th Feb 2015 13:24
I already have an i7 quad core, and it takes around 40 or more seconds to compile around 38k lines, which is very annoying actually, for the include thing, I actually prefer to include the files directly in the project than using the #include command, I found it easier to manage as my project is getting bigger and bigger and I often need to swap between one file to the other.
I didn't notice any speed compiling difference by having all the source in a file or having it split in like 15 different files (which it became a must because of the size of the project)

Login to post a reply

Server time is: 2025-05-12 23:18:15
Your offset time is: 2025-05-12 23:18:15