Hi
I'm currently creating my own container file format for my
game engine.
However, I ran into a problem.
The containers should contain textures, objects, sounds, and so on.
Currently the textures are saved from a memblock to the container.
While the jpg file has 19Kb the resulting container had 1,6Mb.
This increase of something like 8531% was just unaccaptable.
So, I created a small prototype of a compression and got some nice results.
It's crashing before the file is completely stored in the
container, but the intermediate results are pretty promising.
Code:
SET WINDOW ON
HIDE WINDOW
OPEN CONSOLE
PRINT TO CONSOLE
GLOBAL compress AS BOOLEAN = 1
GLOBAL File$ AS STRING = "test.jpg"
GLOBAL VIA$ AS STRING = "compression.via"
GLOBAL SizeBeforeVIA AS DOUBLE INTEGER
GLOBAL MemblockPtr AS INTEGER
GLOBAL MemblockSize AS DOUBLE INTEGER
GLOBAL VIASize AS DOUBLE INTEGER
GLOBAL Percent AS DOUBLE FLOAT
`Check Image Size
IF OPEN DATAFILE TO READ(1, File$) `Open file
SizeBeforeVIA = DATAFILE SIZE(1) `Get size
PRINT "Filesize = '";SizeBeforeVIA;"'"
CLOSE DATAFILE 1 `Close file
ENDIF
LOAD IMAGE File$, 1, 1 `Load Image
MAKE MEMBLOCK FROM IMAGE 1, 1 `Make Memblock from image
MemblockPtr = GET MEMBLOCK PTR(1) `Memblock pointer
MemblockSize = GET MEMBLOCK SIZE(1) `Memblock size
PRINT "Memblock size = '";MemblockSize;"'"
DELETE IMAGE 1
`Open Datafile to write
IF OPEN DATAFILE TO WRITE(1, VIA$)
DATAFILE STRING TYPE 1, 6
DATAFILE FLUSH 1, 1
IF NOT compress
`Write to Datafile without compression
SET DATAFILE SIZE 1, 0 `Null file size
WRITE DATAFILE MEMORY 1, MemblockPtr, MemblockSize `Write Memory TO file
VIASize = DATAFILE SIZE(1) `Get size
PRINT "VIA size = '";VIASize;"'"
CLOSE DATAFILE 1 `Close file
ELSE
`Write to Datafile with compression
LOCAL currentbyte AS BYTE
LOCAL adr AS DOUBLE INTEGER
LOCAL adr2 AS DOUBLE INTEGER
LOCAL Count AS DOUBLE INTEGER
LOCAL MemblockEnd AS DOUBLE INTEGER
LOCAL Written AS DOUBLE INTEGER
LOCAL bytesneeded AS INTEGER : bytesneeded = 16 `Bytes needed to write the patern
LOCAL doubleinteger_max AS DOUBLE INTEGER : doubleinteger_max = 9223372036854775807 `Double integer max
MemblockEnd = MemblockPtr + MemblockSize
adr = MemblockPtr `Current Adress
SET DATAFILE SIZE 1, 0 `Null file size
REPEAT
currentbyte = PEEK BYTE(adr) `Get current byte
Count = 1 `Set count TO 1 initially each LOOP
Percent = adr-MemblockPtr : Percent = (Percent/MemblockSize) * 100 `Get finished%
`Get count of following bytes of the same value
FOR adr2 = adr+1 TO MemblockEnd `Start with the next byte -> adr+1
IF Count < doubleinteger_max `Even if it sounds ridiculous -> Keep the count below double integer maximum
IF PEEK BYTE(adr2) = currentbyte `Check for the current byte
INC Count `Increase the count
ELSE
EXIT `Exit for loop
ENDIF
ELSE
EXIT `Exit the loop
ENDIF
NEXT adr2
IF Count > bytesneeded `If count of replacable bytes is bigger than the patern requirement
`Here takes the main part of the compression place
`For big chunks of identical bytes put a smaller
`chunk of data
`Example: Patern:
`'wwwwwwwwwwwwwwwwwwwwww'
` = '<-|' + Count + 'x' + Byte + '|->'
` '<-|22xw|->'
WRITE DATAFILE STRING 1, "<-|" `Begin the patern
WRITE DATAFILE DOUBLE INTEGER 1, Count `Write the count
WRITE DATAFILE STRING 1, "x" `Patern
WRITE DATAFILE BYTE 1, currentbyte `Write the byte value
WRITE DATAFILE STRING 1, "|->" `End Patern
INC Written, bytesneeded `Increase the written bytes
PRINT Percent;"% finished. Bytes left:'";MemblockEnd-adr;"' Done:'";adr-MemblockPtr;"' Written:'";Written;"'"
INC adr, Count `Increase the current adress by the count
ELSE
FOR db = 1 TO Count `Loop through the count
WRITE DATAFILE BYTE 1, currentbyte `Write the current byte
NEXT db
INC Written, Count `Increase written bytes
INC adr, Count `Increase current adress
ENDIF
UNTIL adr > MemblockEnd
PRINT "VIA size = '";DATAFILE SIZE(1);"'"
CLOSE DATAFILE 1 `Close file
ENDIF
ENDIF
DELETE MEMBLOCK 1
WAIT KEY
END
You will need the
Matrix1Utils to compile this.
The process always crashes around 70%-80% leaving the
unfinished file. I can't figure out why it's crashing XD.
It doesn't even give me a reason or an error message.
It just "doesn't work anymore".
A little bit of help would be appreciated XD, I got the feeling
somthing fishy is going on.
Just put any image file into the project folder and adjust the
globals.
EDIT: Turns out the problem was "INC" command in
combination with double integer variables -_-".
"INC" is expecting a integer value.