Sorry your browser is not supported!

Just for fun, I'm trying to make a syntax highligher. Here's a picture of what I go so far after loading a simple file with it.



The current problem is how I'm parsing the text. Right now, I'm using a string tokenizer I wrote and use a space(" ") as the delimiter. So as you can see, formatting is lost in the code when it is outputted. What's a better way to go about this?

u can also change the colors in DBP to yours

I used space,tab plus anything else that wasn't a capital or lowercase letter to signal a word. Take that word and compare to a keyword list.

Now in pro you have these multiple word length commands. So you look for words that may go with your first word before highlighting.

Like sprite commands are a good example. Your first word is "Sprite". Now you call your word lookup function to parse out the next word. If they go together to make a command then you highlight them.

But you already knew this.

Looks good Phaelax but why are you removing the spaces. You can use them as the delimiter and still add them to the current token.

What I do is read in every character and apply the following rules:

if we're not reading in a token (default):
- if char is numeric, change a flag indicating we are reading a number, and store token start position
- if char is alphanumerica, change a flag indicating we are reading a text string, and store token start position

if we are currently reading in a token:
- if we are reading NUMBER and current char is NOT numeric and is NOT a "." then grab the value of the token we have been reading and store it, then set the reading flag to false

- if we are reading TEXT and current char is not alphanumeric, then grab token value and store. Reset reading flag

it may be easier to understand in pseudocode:

isreading = 0

for i = 0 to len(string$)

char$ = mid$(string$,i)

if isreading = 0

if IsNumeric(char$)
tokenstart = i
isreading = 1
tokentype = "num"
endif

else

if tokentype = "num" and char$<>"." and IsNumeric($char)=false
tokenval$ = mid(string$,tokenstart,i-tokenstart
isreading = 0
store token
endif

endif

Same rules can be applied for:

STRING:
starts with: "
ends with: "

TEXT:
starts with: [a-z]
ends with: [a-z,0-9]

OPERATORS:
starts with [=,-,*,/,+] and is stored straight away

ARRAY:
starts with: [a-z]
ends with: [ (as in php's array[key])

It's a pretty effective way to parse, my c# parser can get vary complex

If you want to preserve spaces I suppose you could have a "space" token that's parsed every time you hit a space, and when you go through your tokens afterwards you just skip the space.

Lost in Thought,
I've added the spaces back in. My original tokenizer code removed the delimiters.

Thx David, I'll play with that.