Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Programming Talk / Play Basic - BASIC Code Conversions

Author
Message
Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 6th Apr 2013 03:42 Edited at: 6th Apr 2013 03:46
Code Conversion Tools (March 28, 2013)

Programming has enough little annoying problems as it is, let alone trying to move code around between languages. Bizarrely the problem hasn't really improved that much since when I started programming back in 1982, at least then there was a good reason, like.. you know... assembly - Today, there's lots of conversion tools between main stream languages, but there's just so many languages out there, lots and lots of legacy code just ends up sitting around doing nothing.

I dunno about anybody else, but I try to hang onto my legacy source codes in one big retro collection. The oldest stuff dates back to the mid 90's (don't really have any my 8bit stuff ;( ).. Anyway, it's mostly AMOS and 68k Assembly from the Amiga days. The asm isn't much use, but a lot of legacy BASIC code can be. The trouble with languages like Amos, is that the source code files are tokenized, so you either need AMOS to view them, or a conversion tool.



Converting AMOS to PlayBASIC

Those with a long memories might recall I've had a few tinkers with writing translation programs over the years, even getting a 'mock up' translator to produce actually working conversion of some simple AMOS programs (and a few other dialects) programs. While the Amos convertor worked (to a point) it's basically a brute force solution, since the translator is working in string level making it potentially very hit and miss.

Normally when parsing code, you'd translate the text stream into a tokenized state, then decipher what's going on from the tokens. But for AMOS programs, it makes sense to load the tokenized source code and decode that directly. I'd looked at doing this before and got a fair way into it, only to find a tool that already did most of it. Today though, i've dragged out my old source and cut'n'pasted a new project from it's ashes. Only took a few hours to get it a reasonable translation of a real world binary Amos program.

What you have to remember though, is that AMOS is huge language and I have absolutely NO intention of building a one stop shop convertor. Rather, the objective is make the tool able to spit out a reasonable translation that a programmer can then use as a basis for the PlayBASIC version. Meaning the program LOGIC should be in tact, but the of course there's no easy way of mapping the AMOS command set to PlayBASIC without writing an abstraction layer.. Which is possible for simple programs (since that's how the previous test worked) but pretty unrealistic for medium to large AMOS programs.


What's Working?

The loader supports about 90% of low level primary token types that AMOS uses, what's missing is the command set declarations. Basically each command name has a numeric token, in some situations that have two or more variations. When you combine that with programs can use external command sets that plug into it, then the dark clouds of doom are soon circling over head. To combat this, i've make declaration tables external.. So yes, [red]you[/red] can improve the quality of the translation by working out what token is what...

At this point the program is basically just spiting out a text file that represents the original AMOS source code (it only knows about 50 commands). While I'll probably leave that in, What I want to do is post process this into something that's more PlayBASIC Friendly.




Catch Up With Work In Progress Blog

There's number of posts in the WIP subject on UnderwareDesign.com that i've skipped here. You can read them bellow.

AMOS BASIC Code Conversionp




Amos To PlayBASIC - Operator Remapping (April 6th,2013)

Each time i fire it up, the solution keeps improving, and only really big stumbling block has been the expression order remapping stuff, but that's now up and running. So far, it's only used when dealing with the ROL/ROR operators. Which present another little challenge, since PlayBASIC has these are functions, so the parameters need remapping completely into expressions to get something meaningful in the output. But it all seems to eb fitting together nicely for once. The remapping pass leaves us with the tokens run that represent each parameter, so just needed to insert the requires param back into the expression in front of the current pointer..

Which just means that when it sees AMOS code like this

Ror.l 1, Variable

we get this fragment back..

Variable=Ror32(Variable,1)

Provided the parameters make sense, the output should make sense.. should, but it's probably possible they won't either



Sample Conversion

Bellow we have today's translation of the standard AMOS sample code. What's missing is mapping of the String function to native versions. But you can wrap (make functions in PB that do what the Amos version does), which works fine..




Becomes this in PlayBASIC,




Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 17th Apr 2013 08:57 Edited at: 17th Apr 2013 09:00
Amos To PlayBASIC - Cont..

After a few days away from the project, I'm back adding support for loading extensions declarations into the Amos decoder. The only extension I appear to have installed, is something called Jwindows which is a way to use Amos to create OS legal apps with AGA chipset support. The main program that uses it is the G3D editor, which was useful when testing the command set mapping support. The G3d code is around 400K of Amos code for the GUI frontend and the conversion seems pretty accurate now. You can't actually port that particular program, since all the rendering code is actually done in assembly. But you can pull all the logic out it and that's the point.

Leading up to last weekend, i'd been looking at decoding the extension command tables directly from the Amos Extensions themselves, with some pretty mixed results. Found a few resources online about the file format, but there seems to be some sketchy data sections in the few that i've found anyway. Where it's getting some of the commands names out of a couple, but gets completely lost of others. I'm sure it'd doable, but beyond the scope of the project as it stands now.

Another thing i've messing around with this afternoon is batching the conversion. If all your projects use the same extensions (or none of them), then getting it to just convert an entire folder of old Amos projects seems a lot easier to me. This works, but you get the odd project where it hangs the parser. Not too sure what the cause of that is at the moment, but it will happen on some files for sure (Amos files can have machine code embedded within them, the convertor doesn't have a clue about that). So if the decoder hits something oddly unique in your Amos source code file and there's no decoder to handle it built in, then it'll more than likely die..




Conversion Quality ?

The conversion passes haven't really changed since the last update. For me, I tended to use AMOS to build tables / data conversion/ algorithm proofs mainly. Which tend to be pretty small Amos programs. These seem to to convert really well. Where as programs that are bound to the behavior of Amos command sets won't, without some serious abstraction layer work from You! - It's possible to doing something like that, but in a throw away freebie.. that's never going to happen !


Read -> Amos To PlayBASiC - Work In Progress

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 25th Apr 2013 23:47
Amos To PlayBASIC - Expression Remapping Cont..

Been away on a cycling weekend since late last week and still sore some days later, but ya get that. Firing this up tonight and been dropping in some generalize expression remapping. Picking through some of the example program in AMOS and there seems to be handful of situations where the input code has to be rearranged to make some useful in the output. The only ones that need radically changes to made them work are implicit statements such as the roll operators, or the Read/Input statements. This type of syntax is common in many traditional dialects of BASIC.

Sometime last week I added a set of functions to help with rewriting expressions at token level, but even though they're fairly high level, these have been wrapped into even simpler interfaces. The new interfaces makes it easier to create a behavior tables. So when the convert pass hits a command/function name token it searches the table and reacts accordingly. So there's handle for mapping operators to function calls, standard function and command calls. Only thing left to wedge in is support for expression where the remapper has to remove the parameter.

Ie. Statements like this,

Input #1, Variable$


Would need to be rearranged into,

Variable$=ReadString$(1)


This shouldn't be too much trouble, since I've already got a solver for converting the math operators to function calls, here it just needs to remove the inserted result field. The coolest thing about the convertor thus far is that once the Amos file is loader, the process is largely generic. Which has been one of my personal bug bears with writing convertors over the years.



Batching Performance..

Dropped batch support into the program last week also, but was running into the odd program that would hang the conversion. One program that would hang it, wasn't actually an Amos encoded source code file, it was some Amos code saved out as text with the extension .AMOS. The other one turned out to have machine code chunks embedded in the binary, which obvious can't be decoded. Unless you're willing to write a 68K disassembler..

Performance wise the convertor is actually pretty quick, averaging around 250 milliseconds for the 480K G3D source code. Where the program is loaded/decoded, dumped to straight text, converted to PB and dumped out as text again. The text folder has about 180 programs in it ATM, and conversion averages at about 40 seconds.

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th May 2013 00:32 Edited at: 4th May 2013 00:33
Amos To PlayBASIC - Front End

The conversion engine has progressed to a point where it's more than a little useful, it's not perfect, but it's able to handle a good percentage of my personal AMOS programs, and thus good of Amos code on the internet (there's lots & lots of it BTW). In my programs there's still the odd Amos opcode that it doesn't understand, but those can be tweaked in the external token declarations. Meaning the community will be able to improve the conversion results by tweaking the keywords. I wouldn't bother asking me to do it for you, that won't be happening !

Tonight's little task has been to jam the conversion engine includes into the standard tool template. The template is a just a project with same old menu / scrolling console system that's used in the last few throw away apps. For this one, I think we can get away with just dumping the output to the console. Which is how I've been testing it previously but from from debugger console. So only real difference is the conversion is the #print statements turn to console prints.

The tool only has two modes at this point, those being convert file and convert path. it does need some switches to toggle stuff on/off like recursive conversion so you can point it an a folder and it'll do all of them. So the only GUI stuff it needs is a setting dialogs, but should be able to cut'n'paste most, if not all it from some of the other tools. Was tempted to build in complete in Flexi, but figured with the existing stock pile of code it'd be less effort to jam everything together.

Only drama i've been having so far is a strange crash when loading the token def's, not exactly sure what the cause is, so will have to suss it out later. Anyway, bellow we can see the standard (in not familiar) shot of the tool.. I can't imagine it looking much different in the final, although unlike some of the other PB tools this one actually requires PlayBASIC installation to work.



Amos To PlayBASIC - Extension Mapping

Things are back up and rolling again after the crashing dramas of last week. Which seemed fairly trivial initially but turned out to be a right pain to track down, ultimately turning out to be an overflow with fragment searcher. It's funny how such a simple issue would create all sorts of odd behaviors in the program afterwards. The overflow was writing data into/over other structures, making it die in all sorts of strange ways. But since that's been fixed it's all running perfectly.

Prior to the problems, I'd been wedging the conversion routines into the stock template, so tonight i've continued hooking the various parts of the interface up to conversion engine. So far there's only going to be two settings pages, one with general properties such as 'recursive folder scanning' & output to console flags. The other is to allow the user to create extension mappings.

The Extension remapping window is a bit more work, but seemed like nothing a good old cut'n' paste session couldn't fix. The bulk of the gui code was initially cut from the 'Play-Pic2Map', I'd figured making a new window would just be a cut'n'paste with some replaces, but it turns out that the gadget code was all hooked back to the parent structures, meaning to make a second/third windows, you end up with double/triple the code. Not a fan of that, so converted all the common stuff like the gadget drawing routines into generic functions. It's still a bit of a mess, but it's a lot less code than it was and things can be reused between windows..

Amos programs can have something like 26 extensions libraries plugged in. I'm pretty limited in terms of what gadgets I actually render though, so i've gone with display the list as an array of Text messages with a 'Load" button to replace any one of them. There's only 10 per page, so added a tree of option buttons to page through them. Pretty simple, but easy to use. When the user clicks the LOAD button they'll be prompted with a windows dialog, where hopefully they select the extension they want. As with anything GUI related, the back end has to idiot proof then users selection. In this case scanning the file to see if it actually is a extension mapping file. Not a huge task, but one of those little things that users never think twice about, that programmers can't avoid..

Anyway, here's a bit of a piccy of current progress. Will probably let the user load/save their mappings as will.



Amos To PlayBASIC - V0.21 - Cleaning up

Pretty much completed the GUI stuff yesterday, which always takes a bit of time hooked up all the little 'nothing' things behind the scene from firing events to saving/loading preferences, which could thankfully cut'n paste over. The end program is simple, but works pretty well, could always work better I guess, but I'm pretty happy with the result really. So this is another little legacy tool I can cross off the list.

This morning I've been skimming through more AMOS programs tweaking the tables to improve the output. Can't do everything with tables though. The PRINT statement is a good example of this. Print is one areas where PB and traditional BASIC go different ways. The compiler in PB treats print as any other function, where BASIC generally incorporates support for using [red];[/red] and [red],[/red] as separators. Which allows different data types to be joined (or printed separately).. So it feels more like an implicit statement. While this can be handy, in order to covert these expressions to PlayBASIC we need to remap the separators into string additions.

Remapping the expression is a bit tricky when don't actually know the data types of the items in the expression. Some things are known such as Variables and Arrays, but AMOS functions are a bit of a lottery, but I think with some educated guessing it should be able to give a understandable result more often than not.

Bellow are some AMOS examples,




and the PlayBASIC conversion




yeah, it's not 100%, but far from a bad result when the remapping routine is use applying some assumptions to the fragments. That's the nice thing about converting code that is know to work, once you know the syntax of the input language it's easy to trap the most common circumstances.

Anyway, will probably have a few more tweak sessions over the conversion and release it.




For pictures/downloads etc visit
Amos To PlayBASIC (WIP)

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 8th May 2013 00:16 Edited at: 8th May 2013 00:18
Amos Bank Viewer (May 06, 2013)

Amos source codes are really binary files with mixed hunks in them. There's the leading code hunk, which is what the conversion stuff above deals with, but then there's the options for the various graphics,sound & data hunks. I didn't think i'd ever used them in my programs, but to my surprise found a few programs with Sprite banks in them. The the art in them isn't particularly interesting, but that got me thinking that for those moving old AMOS games to the PC then they're bound to have better art. It's not that I see that as an all in one solution, but being able to export the original art can be a handy references for updating the originals. I said references since these titles are running in much lower resolutions than modern day games.

The same could be said for sound and data hunks. Found one program that's literally only 50 odd lines long, that builds these large assembly includes. Initially thought the conversion was broken, since the amos file is much bigger than the output code, but there seems to be attached data fragments. The program isn't of any use to me today, but writing a chunk from memory to disc isn't difficult, so might as well dump them all out..

The Amiga used bitplane display modes, so it's no great surprise to find that Amos Sprite and Icon banks are stored in raw planar, although it's a little surprising they're not interleaved planar, but anyway. To convert these to something we can use, first we convert the planar back to chunk pixels, then render out a picture using the palette which is thankfully stored with the sprites. Otherwise, you'd have to work out the palette yourself.. I.e. You'd be fat out of luck.

Bellow we see the results of the convertor running over a bunch of sprite banks I found with a little googling. While the code seems to work pretty well, there's the odd bank that it has an issue with.. But ya get that. While searching around found a few programs that basically do the same thing, so I suggest trying those if Amos To PlayBASIC won't export the media correctly.




Export Amos Sounds Bank To Wave Files

Been have a playing with exporting more of the Amos bank types, focusing upon pulling the raw samples out from the bank and writing them to separate wave files, which turned out to be pretty easy. Now since Amos (due to Amiga's sound hardware) only has 8bit sample resolution, the exported sounds are still 8bit. But, they actually sound fairly decent.

You can find an attached example which includes the original data block and exported samples in it.. The actual samples seem to be mostly ripped from movies from what I can tell.. But it works..



Amos Bank Viewer - Pacic exporter

When looking at the picture format yesterday I'd assumed it was RLE encoded in horizontal spans, like the IFF image files, but it turns out that they actually use vertical spans. After a bit of googling found a good document on the Picac Format, but even with that it's still taken most of the afternoon to get it working. The routine as it stands and pull the picture data as a 8bit chunky.

The legacy Amiga chipset includes HAM6 (Hold and Modify modes) which are a type of special picture format, boosting the on screen colour count from 64 colours (Half bright mode) to 4096 colours. Later Amiga models include HAM8 which gives you an approximately an 18bit image using only 8bits of colour data per pixels. Variations of the same core concept are used in modern texture compression today.

So far the test program can converting folder full of pictures (decompress, planar to chunky and & render) in about 180->190 milliseconds. Which is not too bad for an 8 year old system.


For pictures/downloads etc visit
Amos To PlayBASIC (WIP)

Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 14th May 2013 05:01 Edited at: 14th May 2013 05:05
Amos Bank Viewer - Exports All Banks (May 08, 2013)

The test exporter now supports all the known banks (there's probably some others), the main types such as SPRITES, ICONS, SAMPLES & PICTURE are converted into either bitmap or wav files. The only exception is the MUSIC banks which are in a custom format. These banks are exported to disc, but you'll have to convert them yourself. There's a few tools that do this on the AMIGA[color=red][/color] ( Abk2Mod-II & NoiseConverter ), but i'm unsure if there's something on the PC version. But there probably is.

All banks are exported regardless, ones that are unknown are just dumped as raw data. When data banks are stored within the main source code, they're given that source code name as the root file, with the bank index and media name in the output file name. So they all end up together in the folder. One thing that would be handy when dropping sprites, is dropping a little table with all the Hot spot data in it. Which would just be text file.

While looking around I found some power packer depack source code/dll. Power Packer was popular file compression program on the Amiga, and I would seem AMOS supports packed banks.. So have slapped together a little wrapper to decompress such files. Most of my files on A1200 HD image are all PP'd so it should come in very handy.



Amos To PlayBASIC - V0.24 - Sicking it all together ( May 10, 2013)

Dropped the various bank exporters in the main project yesterday and they all seem to work ok thus far. They don't have error detection in them though, so if the file is damaged in some way or just uses some rare feature then they'll more than likely die. All the test AMOS source codes seem to decode fine though, so hopefully that's a really rare situation.

It's not all rosie though , as after finding some public domain examples that use encrypted procedures (something I never used) it gets a bit crash happy. I'd assumed this was going to be none issue, since the decoding routine is posted all over the place (like here ) but for some reason the resulting code just won't decode in any sensible manner, leading to a crash. (Also tried Unlocker but no go) Getting more than little annoyed, decided to track down the AmosPRO source code. It's written in 68K, so it's just matter of pulling out the decrypt routine. Not hard tracking it down and initially there does seem like there might some differences. But we'll see.

CODE REMOVED:

Anyway, hoping to get this out there over the weekend, even if the procedure decryption stuff is a bit iffy, it's still working pretty well.


EDIT: - Ported the above routine and it all works as expected, using the new version Amos To PlayBASIC successfully decodes all the decrypted procedures in the example programs with out issue.


For pictures/downloads etc visit
Amos To PlayBASIC (WIP)

Phaelax
DBPro Master
23
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 15th May 2013 11:50
Quote: "I dunno about anybody else, but I try to hang onto my legacy source codes in one big retro collection"

As do I. I have tons of QB and Pascal programs. But AMOS goes a bit further back than I do.

Though I have no need for these tools, this sounds like a project that would be fun challenge to work on.

"You're all wrong. You're all idiots." ~Fluffy Rabbit
Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 16th May 2013 04:45
Quote: "As do I. I have tons of QB and Pascal programs. But AMOS goes a bit further back than I do."


Well yeah, but the weird thing is that due to the emulation scene, there's a number of people still happily hacking away at AMOS Pro. Even I fire it once in a blue moon ( AGE Development Blog )

Are legacy QB files tokenized ?

Phaelax
DBPro Master
23
Years of Service
User Offline
Joined: 16th Apr 2003
Location: Metropia
Posted: 18th May 2013 13:15 Edited at: 18th May 2013 13:18
Quote: "Are legacy QB files tokenized ?"


If you're referring to how the source files were saved, it was just plain text as far as I can remember. I still have the compilers, but I can't seem to launch them even in compatibility mode.

I might look into learning some AMOS to see what its like. I still have amiga, and even have many of the original boxes the disks came in up until last summer when my brother destroyed most of them while I was away.

"You're all wrong. You're all idiots." ~Fluffy Rabbit
Kevin Picone
23
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 19th May 2013 05:27
Quote: "If you're referring to how the source files were saved, it was just plain text as far as I can remember."


Had a bit of look around and apparently there's a couple of formats, text and binary. Which can be pretty annoying when you have old working code are unable to access it.


Quote: "
I might look into learning some AMOS to see what its like. I still have amiga, and even have many of the original boxes the disks came in up until last summer when my brother destroyed most of them while I was away.
"


AMOS is ok. Was never a huge fan, just wrote lots of tools etc in it. Still have A1200/A500 systems but neither are really functional. Just use WINUAE these days. Once it's setup it's pretty easy, but setting it up can be pain.

Login to post a reply

Server time is: 2026-06-11 16:16:16
Your offset time is: 2026-06-11 16:16:16