Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

DarkBASIC Professional Discussion / Question- Would anyone be interested in a DBPro/Unity3D Integration?

Author
Message
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 18th Jul 2023 22:31
So, I love Dark Basic Pro. It was my first love. Years have past, and eventually I moved away from DBP and went the way of XNA, which lead me into Monogame and then into the dreaded Unity3D. I'm working on a game at the moment, and the game itself is about making games... I've started re-writing Dark Basic Pro so that the language can be parsed and then interpreted inside of net6, and Unity3D. My plan is to implement a _subset_ of the original DBPro commands, specifically around 2D text and sprites and such. I'm curious, would anyone be interested in this, outside of this specific indie game?

I've thought about trying to build out the concept as a standalone asset in Unity, maybe even call it "darkUnity", or "unityBasic", idk... Anyway... Just asking the question

As always, Dark Basic Pro is the best!
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 22nd Jul 2023 06:06 Edited at: 22nd Jul 2023 06:08
Well, I've decided I'm doing this anyway, haha!

I'm taking a slightly more difficult route than I need to, and I think it is because I may have a vague notion that I want to do it "well"... So I'm starting totally from scratch. I have these forums, some random online materials people have made, the wonderful open sourced DBPro, and the Hands On DarkBasic v1&2 from years ago that are still on my shelf... Eventually, I'll go dig up my first ever 20 line challenge submission, and shudder at the horror the whole way.

I've started by creating a lexer for the language that tokenizes source code. I feed that into a parser, and it gives me an abstract syntax tree of the code. Then, I'm compiling that tree into a home grown byte-code. Finally, the byte code can be run in a little in-process/memory VM that has access to virtual registers and a memory heap. So far, I've been focusing on getting a vertical slice done, and I think I've just done it. I've got some basic maths done, limited declaration and assignment support, limited array support, and the ability to invoke C# functions and pass/return primitives.

So for example, I have this program,


And that gets transformed into an AST that looks like this, (excuse my random string-ification)


And that then gets compiled down to some jank-hankin byte code that I dare not post- but will link it if you dare

Anyway, it runs! I have some tests that inspect the registers and heap at the end of the execution. I have 117 unit tests to be precise at the moment.
I've run the execution through Benchmarker Dotnet, and that code runs with this performance... I know without anything to compare it to, this benchmark is a bit useless. At some point, I'll try and get LUA or something and see what that scripting performance looks like for a similar program.


---

I'm going to keep on plunking away at this. I'm really really looking forward to the moment when I get structs (TYPE) , control-flows (FOR), and functions done. I guess I still need IF too
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 23rd Jul 2023 15:45
I got multi-dimensional arrays working tonight, which hurt my brain a little bit. I'll need to go back and add re-dim, and undim, but... I don't want to do that right now

But the way more compelling thing is that I downloaded Moonsharp (a LUA interpreter for C# that seems to be popular for Unity3D), and did a little contest between comparable programs... I used BenchmarkDotnet to do a stress test, and uh, so far my DBPro compiler & execution is much faster.

This is the DBPro program, its not God's gift to DBP



And here is the LUA program,




It is a semi unfair comparison I guess, because I cannot find a way to split apart the compilation from the execution in LUA. That gives my DBPro variant a huge edge, because game developers can preload their scripts ahead of time.
Honestly, I'm just happy that my stuff isn't way behind LUA. I feel like this project is growing some legs to stand on.
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 27th Jul 2023 13:16
I got custom types working this morning before work.
Now I can write stuff like this,



And I can make assertions that 72 bytes are allocated for the x array; because there are 3*4 elements of `egg`, and the `egg` type is 6 bytes big. It is 6 bytes big because it has a word and an integer.

The full Csharp test code looks like this, although I understand it is a bit cryptic...


At this point, I think I am done with the basic memory stuff... Not 100%, but good enough to move on and prove out some other stuff in the language.
I can
1. use primitives, including int, word, byte, float, etc
2. use arrays, and multi-dim arrays
2. use strings (they are arrays under the hood )
3. use types (or records, or as I'm calling them, structs)
4. nested structs
5. arrays of structs

So I think I'm going to move onto some of the control flow work. I think I'll start with GOSUB and GOTO style commands to work out how to manage the jump instructions.

---

And because I think its fun, these are the current INSTRUCTION codes that my little virtual machine evaluator can handle...
These are obviously still WIP

flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 28th Jul 2023 14:09
This morning I got GOTO/GOSUB/RETURN all working. That made me figure out how to do Instruction-jumps in the VM, so I'm primed to tackle control flow this weekend.
Kevin Picone
21
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 29th Jul 2023 16:51
So you're writing a compiler / vm in what ?

I'd recommended thinking more along the lines of transpiler and translate at source level.


PlayBASIC To HTML5/WEB - Convert PlayBASIC To Machine Code
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 30th Jul 2023 04:10
I'm writing the compiler in vm in Dotnet (specifically, net standard 2.0)
The goal is to be able to pick this up and run it in Unity3d. The vm is not a real VM, its just a vm-esque piece of software that has a giant byte array as a heap, and some bytecode.

What do you mean about a transpiler? Like, from what to what? DBP to... executing in Unity? I think that is what I'm doing, but I need to build up the language in Dotnet in the first place.

flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 30th Jul 2023 23:10
Ah, implementing FOR loops really broke my brain. But, they're finally passing all my tests! I have while loops, and if/if-else/if-then statements working as well now. This week, I plan on finishing up the control-flow statements. I think I just have Do/Loop, Repeat/Until, and the Select statement left.

To get a sense of what I'm doing, this is the function that compiles the ForStatement down into byte-code.
https://gist.github.com/cdhanna/fff9b3fbe3bbce268b9c7aae06e4ffa6

I'm going from a tree-like format of the language into a home-brew byte code, and then I'm evaluating that byte-code in what I'm calling a "vm", although its really not a vm at all.
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 1st Aug 2023 12:42 Edited at: 1st Aug 2023 12:43
This morning before work, I finished up do/loop, repeat/until, and the mighty select statement!
The select statement turned out to be really interesting to implement. I didn't want to just compile it as though it were a bunch of if statements... I know there is a common question, "what is the difference between a select statement and a bunch of if statements?" and the answer is usually some vague sense of "performance is better, trust me!"

Well! Since I just implemented these, I figured I'd run a benchmark on the execution of 2 similar intent programs, but each implemented differently...

So here is a program that uses select...


And here essentially the same intent, but with if-statements


And for kicks, if you're interested, here is the entire Benchmark class in dotnet


The results are interesting!

They're both plenty fast, but, the select/switch statement is relatively faster by a wide margin. This is because the select statement is building a jump table in byte code, instead of actually processing every if statement.
So the performance gain is proportional to the number of cases there are... I did the same test again, but instead of 8 cases and 8 ifs, I made it 4 cases and 4 ifs, and as expected, the performance edge drops.




So, I'm excited! because now, all the memory layouts, and now all the control flows are DONE! Hoozah! The next big giant scary thing to tackle are functions, a critical part of any complete DBP meal.
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 5th Aug 2023 06:53 Edited at: 5th Aug 2023 07:22
Cool beans yo- functions are _mostly_ done... Heh... But I'm pretty jazzed up! There are a few crusty places in the core language support (void return functions, static analysis, declaration and combined assignments, extended primitive support, etc), but for the most part, things are working! I know its a bit meaningless without context, but I have 360 unit tests that are checking the core language, which includes expressions, arrays, types, functions, gosubs, if statements, selects, for loops, etc...

I took this opportunity to try getting stuff hooked up in Unity. The highlight is that I have a text file that looks like this,



And it can exist inside Unity, and some handy dandy code will import it and compile it as part of Unity's asset import phase.


In that image,
#1 - the asset on disk, this is the text file itself; but I threw a chunk DBP icon on it for fun (will def change that). The arrow on #1 is going to the imported object, and you can see there are no errors, and there is a lot of byte code.
#2 - a Unity GameObject sits in the scene, and its arrow is pointing to the inspector for that GameObject, where it has a `Dark Basic Runner` component. That component needs to load up some specific DBP script...
#3 - I've pointed the script reference to the asset from step 1.


And then when I run the game, it actually runs! The `debug` command is a fake standin command that prints stuff to the Unity console.
I've implemented the `wait key` command by suspending execution of the virtual machine, and resuming it after Unity receives some keyboard input.

I'm really excited, I think this is a huge milestone! I'm sure I have lots of bugs and things to fix... But I think I have enough to push ahead and try to implement a bunch of the core DBP commands, and then the I'd love to get to the Sprite based commands... I dug up a lot of my old DBP source code from like, 10+ years ago... I have a dream to just plug it into this program and have it work. Obviously it won't, because of the asset differences... But, I can always dream.

Here is a quick video showing some of the Unity stuff.
https://youtu.be/XXMpnIWw8Rk
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 15th Aug 2023 14:26
quick update- I've been going after performance wins for the last week or so... Before I forget how this code works, I want to make sure I'm keeping it snappy. My original benchmark tests were incorrect, actually, the real execution is in the order of hundreds of nanoseconds. I dropped the runtimes down by about 55%, and now I'm looking into replacing my C# method calls with Reflection out for Source Generated calls. Its going well, but just deep in the weeds
Mage
17
Years of Service
User Offline
Joined: 3rd Feb 2007
Location: Canada
Posted: 15th Aug 2023 21:03
Good luck with the project. Very interesting. I am sure there will be more interest when it's practical to use it.

Mage's Modular Code - Get awesome UI controls and powerful Bitmap Fonts!
Screen Device Recovery - Stop your apps from crashing when minimized/tabbed/screen locked.
CPU Friendly Frame Limiter - Make your app generate less computer heat + noise, and use less battery life!
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 22nd Aug 2023 13:40 Edited at: 22nd Aug 2023 13:41
Thanks, Mage!

I hit a pretty major milestone this morning before work. Back in 2012 (ish), I wrote some jank-hat code to play a silly card game called "Idiot's Delight". There is no user choice in the game whatsoever. It is a simple card "game" where the player just filters the deck in a particular way. Its a way to waste time. Anyway, I wrote a version of it in DBP at some point, and I was just able to copy/paste that code and have it work!


Here is a screenshot of it working in Unity


And for the record, here is the DBP code that is running


I'm slowly building out the command palette from ye olde DBP land. This first test is really really promising. I think I have an old text adventure engine on a shelf that uses some more commands, so I'll take that out for a spin next. It uses the text properties more, the file suite, and some mouse input.

The insanity of this is not lost on me. I wrote some code in 2012, and in order to get it run on my mac machine, I ... wrote... a freaking compiler...
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 24th Aug 2023 05:17 Edited at: 24th Aug 2023 05:19
I'm on page 60 of the Hands On Dark Basic Pro volume 1 book, for implementing commands and stuff... I've jumped around a little bit, but i have
- print,
- text
- cls
- ink
- rgb
- input
- center text
- set text font
- wait key

and a few other commands all working. I'm getting pretty confident now that the command language I've built up is capable. I just need to handle a bit more complex "optional" scheme to support commands like "INPUT" that take an optional parameter as their _first_ parameter... Eh...
To a give a sense of what some of these commands look like in C#, I thought I'd share a few...



If you take a look, you'll see that you write C# methods fairly normally, with a few interesting caveats... You can get access to the underlying VM using the `[FromVm]` attribute. And you can use params, and optionals, and ref parameters...
However, because I'm running this inside Unity, I want Unity to be able to keep running while the DBP emulator is stalled waiting for something, such as the INPUT command.
The input implementation is the most complicated so far, and gets access to the value parameter using a special `RawArg` style. The `RawArg` is treated as a ref parameter, but allows the C# developer to manually jump into the VM and set the value of the parameter whenever they want. Chaos would reign if this was a multi-threaded system, luckily it isn't!!!



Like I said, the INPUT command still isn't done, because technically, the 'label' arg is optional... I was considering just supporting method overloading, but I think that is going to be harder to achieve at the parsing layer for my system. Instead, I plan on just adding support for mid-arg optionals using something like the system interop `[Default]` attribute.

And once that is done, I plan to press on in the text book that is Hands On Dark Basic Pro, and just... implement....


EDIT:
oh, and like I said a few posts back, I'm NOT using Reflection to call these methods. I've created a C# Source Generator, such that when you compile those functions, I auto-matically generate a calling function that has a cached delegate pointer available for all the stages of compiling. The augo-gen'd code for one of those methods looks like this. Excuse the horrible formatting, this isn't meant to be human editable code. At some point, I want to see if I can support C# doc strings to automatically build up a help library from the target methods.
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 5th Sep 2023 19:35
Quick update- I'm still working on this!
I've made some good progress on getting bug fixes and tidying up some issues with my language implementation. Originally, I'd misunderstood why some DBP commands required the paren characters, and others didn't. Erroneously, I'd decided to make my re-creation accept parens as optional in all commands. Eventually, I hit an issuing in the parser... And I realized that the commands that RETURN something require parens, and actually, when I added that assumption to the parser, my issue totally went away, haha...

I've also been working on sprite commands and making things work nicely with the `sync` commands.

I am taking a short break this week, but I hope to be back at it soon <3
flashing snall
18
Years of Service
User Offline
Joined: 8th Oct 2005
Location: Boston
Posted: 12th Sep 2023 16:52
Today Unity announced that they're charging a 'runtime' fee. I think that invalidates my desire to use Unity for the original side game this entire DBP re-creation was based on.
I may very well give up on the Unity side of it, and re-create the evalulator in monogame/godot.

Rage.
Dark_ITheI _Angel
21
Years of Service
User Offline
Joined: 3rd Sep 2002
Location:
Posted: 22nd Sep 2023 00:19
Bro, GODOT is the way to go. But i believe that is a very big task you have taken and to mantain it...dont waste your time.
Animals

Login to post a reply

Server time is: 2024-05-01 06:28:23
Your offset time is: 2024-05-01 06:28:23