Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

Geek Culture / Does anyone know any good resources for programming compilers

Author
Message
Gen
20
Years of Service
User Offline
Joined: 6th Jul 2004
Location: Oklahoma, USA
Posted: 3rd Dec 2004 15:30
I'd like to start looking into making compilers, do anyone know of any free ebooks, websites, etc.

Gen

Soon to come... Dark IDE
New IDE for DarkBASIC Pro!
Jeku
Moderator
21
Years of Service
User Offline
Joined: 4th Jul 2003
Location: Vancouver, British Columbia, Canada
Posted: 3rd Dec 2004 15:31
There's a post about this not far below this one:

http://forum.thegamecreators.com/?m=forum_view&t=24693&b=2


--[GameBasic - Coming Soon]-- ^^^ banner generously designed by TheBigBabou
MikeyP
21
Years of Service
User Offline
Joined: 21st Aug 2003
Location: Manchester, UK
Posted: 3rd Dec 2004 23:45
Jeku, I think you mis-understood. I think he wants to make a compiler himself, to like create machine code...

Gen: I found a site once, it explained a bit but I have lost it now... Search google for a bit.
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 4th Dec 2004 02:32
Yes.

http://compilers.iecc.com/crenshaw/

Build a compiler in Pascal, so it'd be easy to go to DBPro. Outputs machine code.

Get 15 new commands, all the date / time commands left out of DBPro for free!
DOWNLOAD PLUGINS HERE: http://www.davidtattersall.me.uk/ and select "DarkBasic"
Neophyte
21
Years of Service
User Offline
Joined: 23rd Feb 2003
Location: United States
Posted: 4th Dec 2004 03:50
@Phreeky Mike

Jeku, I think you mis-understood. I think he wants to make a compiler himself, to like create machine code...

Jeku didn't misunderstand. That post contains plenty of information on how to build a compiler. You just have to scroll down to where we start talking about it. After that, thread is basically me posting tons and tons of links regarding compiler creation and optimization as well as a whole bunch of other tangently related issues.
Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th Dec 2004 04:07
Quote: "
Build a compiler in Pascal, so it'd be easy to go to DBPro. Outputs machine code.
"


Well there ya go, basially an entire tut on it. Although the kids should know it spits out 680x0 machine code. So it must be good

Kevin Picone
[url]www.underwaredesign.com[/url]
Play Nice! Play Basic - Next Generation Basic (Release V1.03 Out Now)
Shadow Robert
22
Years of Service
User Offline
Joined: 22nd Sep 2002
Location: Hertfordshire, England
Posted: 4th Dec 2004 05:08
heh, personally I found taking apart other compilers the best learning experience.

GCC, NASM, Cg are all open source, and designed for different tasks and complexities.

I started with Cg, which through the community at NVIDIA I actually had a big hand in developing the (cough:buggy:cough) 1.12 Compiler. After that though it became alot clearer on the basics of what to do, even though it didn't really teach me the actual machine side of thing it was very useful because of it's simplicity.

Kinda gone from there.

Quote: "Although the kids should know it spits out 680x0 machine code. So it must be good"


Heh, true.. but not as much fun as PPC


TKF15H
21
Years of Service
User Offline
Joined: 20th Jul 2003
Location: Rio de Janeiro
Posted: 4th Dec 2004 05:11
PPC? Aw, c'mon, the most fun is 6502!

MikeyP
21
Years of Service
User Offline
Joined: 21st Aug 2003
Location: Manchester, UK
Posted: 4th Dec 2004 06:14
Ahh ok, my apologies
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 4th Dec 2004 06:55
Quote: "Well there ya go, basially an entire tut on it. Although the kids should know it spits out 680x0 machine code. So it must be good "


I'm reinstalling Delphi right now to have a go

Kevin, is that how you made PlayBasic? That go to machine code, or just interpreted?

Get 15 new commands, all the date / time commands left out of DBPro for free!
DOWNLOAD PLUGINS HERE: http://www.davidtattersall.me.uk/ and select "DarkBasic"
Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th Dec 2004 07:49
Quote: "I'm reinstalling Delphi right now to have a go "


Good luck..

Quote: " Kevin, is that how you made PlayBasic?"


From the bit i was reading before, yeah basically.. it's describing a lexical scanner based compiler, i.e Source to -> tokens -> generate code. Which is how PB works.

it's sad, today it all seems blatantly obvious today, but I really wish i'd twigged onto the 'token' concept a decade ago during my first feeble attempt to write a language on the Amiga. Although, it was written in pure assembly, but elegant it wasn't. Totally string and situation based..

Quote: " That go to machine code, or just interpreted? "


It's generates byte code. So programs run from optimized Byte Code version of the program. Aka.. like Java/Flash/VB etc.. Older versions worked a little more like the simple (chumpie) 680x0 Vcpu model I wrote for kyruss (http://www.kyruss.underwaredesign.com).

The current VM has a vastly reduced instruction set of mostly smart instructions (around 40->50 core opcodes). Each smart op-code represents a common collective process, rather than interpreting 'each' step of the process like it first did. I.e Read/Write arrays etc

While there was a time when i though generating binaries directly from the compiler was a smart/clever approach (hence the above).. But, lets just a say, today I know better. . Don't get me wrong, that's a the direction we're heading also, but the source->bytecode-> binary gives a greater level of abstraction and flexibility.

Kevin Picone
[url]www.underwaredesign.com[/url]
Play Nice! Play Basic - Next Generation Basic (Release V1.03 Out Now)
empty
22
Years of Service
User Offline
Joined: 26th Aug 2002
Location: 3 boats down from the candy
Posted: 4th Dec 2004 08:14
Quote: "I'm reinstalling Delphi"

A good decision! It was a mistake to uninstall in the first place.


Quote: "it's sad, today it all seems blatantly obvious today, but I really wish i'd twigged onto the 'token' concept a decade ago during my first feeble attempt to write a language on the Amiga. Although, it was written in pure assembly, but elegant it wasn't. Totally string and situation based.. "

Reminds me on my poor script engine attempts on the ST. Ah memories... hunting me again.

Play Nice! Play Basic!
Version 1.02 available now!
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 4th Dec 2004 16:57 Edited at: 4th Dec 2004 16:59
Quote: "A good decision! It was a mistake to uninstall in the first place. "


Didn't uninstall it, it broke Could find 'System.pas'. It wasn't in one of the Source/Rtl/ folders so I think somewhere along the line that file went missing.

Thanks for the explanation Kevin. As I unserstand it PB's compiler exports to a kind of bytecode - how do you get all of PB's functions into that code - do you write them as some sort of library then call it? (sorry if that was a bit innacurate - the edit message page doesn't allow you to see above posts :p)

Get 15 new commands, all the date / time commands left out of DBPro for free!
DOWNLOAD PLUGINS HERE: http://www.davidtattersall.me.uk/ and select "DarkBasic"
Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 4th Dec 2004 22:14
Quote: " As I unserstand it PB's compiler exports to a kind of bytecode "


Yep.

Quote: "- how do you get all of PB's functions into that code"


You don't have to, The compiler generates byte code for the 'language constructs' not the command set. So the compiler only generates the structural framework code, the commands/functions are called from language function library.

A easy example of this is how Dbpro works. While the compiler generates 'machine code' for the core, when it comes to the command sets, it's merely dumping a 'call procedure' to call the function from the library. This is defined in the command definition table. So the parser just matches the command and dumps it's known functions inputs/outputs and name.

So something like

Position object 1,xpos,ypos

in pseudo code ends up looking something like this.

Push Ypos
Push Xpos
Push 1
Call Dll Function, "PositionObject"

Now structural code is dumped directly into the machine/byte code.
So something like this,

B =99
A=B
C=A+5
D=A+B+C

in pseudo code might end up looking something like this (2 register operations).

; B=99
Move B, #99 ; B=99

; A=B
Move A, B ; A=B


; C=A+5
Move Temp,A ; Protect A, moving it's value to a temp register
Add Temp, #5 ; Add the 5 to the temp register
Move C, Temp ; Move the temp into the result


; D=A+B+C
Move Temp, A ; Pull A into a temp register
Add Temp,B ; Add B to the temp
Add Temp,C ; Add C to temp
Move D,Temp ; Move tallied Temp into result

The Byte Code the Virtual Machine in PB uses, is basically a 2 / 3 register version of the above. The VM is a sort of processor simulation. Where the simulated cpu is 'pretend', but obeys much the same rules as a real cpu. Obviously being a 680x0 fan, PB opcodes resemble a dumbed down 68K somewhat, just without most, if not all, of the instruction modes.

So since the PB Vm supports a subset, the opcodes are smarter than a real cpu's opcodes.. (meaning they can deal with mixed register 'types' through a common opcode, rather than through many varied opcodes.

The PB VM instruction set has only a handful of central instructions. For example..

Move (various modes), AddInt, SubInt, MultInt, DivInt, Add, Sub, Mult, Div, Cmp (various modes), And, Or, Xor, Not, Branch, Jump, Jsr , Push, Pop and a bunch of hybrid op-codes to replace 'common' sequences of instructions, mostly for loop structures and scope changes..

So all the Structural code is broken down and output as these op-codes. Which the VM just interprets to execute a program. Obviously if the compiler generates a standard set of tokens for the core stuff, this opens up the door to doing some post processing upon the tokens, and even translation of the byte code op-codes into other languages, and native cpu's..

Kevin Picone
[url]www.underwaredesign.com[/url]
Play Nice! Play Basic - Next Generation Basic (Release V1.05 Out Now)
David T
Retired Moderator
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: England
Posted: 5th Dec 2004 03:21
Ah I see - thanks

I've started going through that tutorial I posted earlier and here's what I've got. A translator that parses expressions such as

variable = -5 + 9 * (x-1)

and exports machine code. I haven't tested the machine code yet, mind



Full delphi code attached

Get 15 new commands, all the date / time commands left out of DBPro for free!
DOWNLOAD PLUGINS HERE: http://www.davidtattersall.me.uk/ and select "DarkBasic"
Kevin Picone
22
Years of Service
User Offline
Joined: 27th Aug 2002
Location: Australia
Posted: 6th Dec 2004 01:31
It seems like it'd work, although you'd be wise to cast the operation width. While most 68k assemblers will have a default to longs, some do default to word, or choose word when assembling with optimizations enabled if it's not explict.

I.e
Move d0,-(sp)

So to ensure a consistence results assembler to assembler, this would require the width for the move/add/sub etc statements, so it's explicitly declared to be dealing with 32bit values, rather than the assemblers default.

Move.l d0,-(sp)

if it's all 32bit integer only, this is ok, but the 68000 and 010 only support a 16 bit signed multiply (sourc / dest are 16bit, resulty is 32bit) at 52 cycles a pop, it's full 32bit in 68020 and above though.

There's seems to be some addressing mode/flag anomalies which i can't recall being issues off hand, but may well have good cause. I can't really remember

I'm thinking mainly here about the negatation of values, it may well be required to do it like that, but my gut feeling is you can just push them directly on the stack negated.

; Clear D0
CLR D0
; Push the Cleared D0 to the Stack
MOVE D0,-(SP)
; Move positive 5 to the d0
MOVE #5,D0
; pop + subtract the previous value from the stack (zero) from D0 .. i.e. d0 = d0-0
SUB (SP)+,D0
; invert sign bit of d0
NEG D0
; push it back to the stack
MOVE D0,-(SP)


You should be able to simply

move.l #-5,-(sp)

or at worst

Moveq.l #5,d0
Neg.l d0
Move.l d0,-(sp)

but, it's been a while

Kevin Picone
[url]www.underwaredesign.com[/url]
Play Nice! Play Basic - Next Generation Basic (Release V1.05 Out Now)

Login to post a reply

Server time is: 2024-09-23 02:27:52
Your offset time is: 2024-09-23 02:27:52