Memblock save game data (dynamic string length)

Author

Message

jd_zoo

5

Years of Service

User Offline

Joined: 12th May 2018

Location: Nova Scotia

Posted: 22nd May 2022 20:59

Link

So I am growing frustrated with my save game CSV structure, and found a way to convert over to binary and use memblocks to directly load integers and floats without converting from strings. My save game data is only ~15% strings, and a save can be 1000 tokens in size.

This working example requires you to write and read back the data in the exact same order to take advantage of the built-in indexing. Integers and floats get 4 bytes. When inserting strings first an integer with the ByteLen() + 1 gets added followed by the string, this is all done in the background and keeps track of the offset as you read and write. The memory block size is maintained as a multiple of 4.

I have yet to implement into Automation, this was the first step as a proof of concept. I will post back with loading times but it won't be a direct comparison as I plan on re-doing the way the level gets generated after loading the data. It can't get any worse than reading back from strings, can it?

Anyway if this concept is of use to anyone out here it is a cleaned up copy pasted below.

JD

/*
 *	Memory Block Save Game Data Example
 *
 *		May 2022 One Tree Hill Studios
 *
 *		- Build (write) and Read order must match for the offset to work
 *		- Build the memory block with filemb_addInteger(), filemb_addFloat(), and filemb_addString()
 *		- Read back the memory block with filemb_getInteger(), filemb_getFloat(), and filemb_getString()
 *		- Uses custom data type filemb_dataValueType to keep track of the offset while writing/reading
 *		- Write and read directly to disk with CreateFileFromMemblock() and CreateMemblockFromFile()
 */

SetErrorMode(2)

// set window properties
SetWindowTitle( "memblock_fileSave" )
SetWindowSize( 1024, 768, 0 )
SetWindowAllowResize( 1 ) // allow the user to resize the window

// set display properties
SetVirtualResolution( 1024, 768 ) // doesn't have to match the window
SetOrientationAllowed( 1, 1, 1, 1 ) // allow both portrait and landscape on mobile devices
SetSyncRate( 30, 0 ) // 30fps instead of 60 to save battery
SetScissor( 0,0,0,0 ) // use the maximum available screen space, no black borders
UseNewDefaultFonts( 1 )
SetPrintSize(16)

type filemb_dataValueType	//This is a required type for the memory block build/read functions
	mb as integer			//Memory Block
	offset as integer		//Offset that gets automatically incremented
	inter as integer			//Byte size 4
	floater as float			//Byte size 4
	stringer as string		//Byte size = ByteLen() + 1
endtype

/*
 *	This variable will be used to build and read the memory block.
 */
global filemb_dataValue as filemb_dataValueType

/*
 *	The following type and variables are for the working demo
 *		(the game data would match your save game stucture)
 */
type gameDataType
	screen as integer
	name as string
	date as string
	time as string
	version as float
endtype
global gameData as gameDataType		//Randomly generated game data placed here
global displayDataTxt as integer		//Display text for the generated game data
global displayMBTxt as integer		//Display text for reading back the memblock data

controlsLoadBtn()					//Load the control buttons (text buttons)
		
gameDataLoad()						//Loads randomly generated game data to gameData (gameDataType)
displayGeneratedData()				//Display the randomly generated game data

do
    Print( ScreenFPS() )
    
    controlsChkBtn()					//Control (text) button controls
    
    
    Sync()
loop

/*
 *	This function is an example of how to build up the memory block, the offset is automatically incremented.
 *	
 *		All integers and floats use 4 bytes, strings are dynamic and will automatically add a
 *		preceeding integer with the ByteLen() + 1 of the string
 */
function mb_buildData()
	
	if (GetMemblockExists(filemb_dataValue.mb) = 1)  then  DeleteMemblock(filemb_dataValue.mb)	
	filemb_dataValue.mb = CreateMemblock(0)		//Memory block sizes are auto incremented as data is added
	filemb_dataValue.offset = 0					//Remember to reset this to zero when reading back
	
	filemb_dataValue = filemb_addInteger(filemb_dataValue, gameData.screen)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.name)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.date)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.time)
	filemb_dataValue = filemb_addFloat(filemb_dataValue, gameData.version)

endfunction

/*
 *	This function is an exmaple of how to read back data from the memory block.
 *	
 *		Like the add functions, the offset is automatically incremented.
 */
function mb_readData()
	gameData.screen = -1
	gameData.name = ""
	gameData.date = ""
	gameData.time = ""
	gameData.version = 0.0
	
	filemb_dataValue.offset = 0				//Reset offset to zero for reading back values
	
	//Read the data back in the exact same order as it was created
	gameData.screen = filemb_getInteger(filemb_dataValue)
	gameData.name = filemb_getString(filemb_dataValue)
	gameData.date = filemb_getString(filemb_dataValue)
	gameData.time = filemb_getString(filemb_dataValue)
	gameData.version = filemb_getFloat(filemb_dataValue)

endfunction

/*
 *	WORKING EXAMPLE FUNCTIONS
 *
 */

//Randomly generat some game data
function gameDataLoad()
	
	gameData.screen = Random(1, 200)
	gameData.name = "scrn_" + str(Random(1, 2000))
	gameData.date = GetCurrentDate()
	gameData.time = GetCurrentTime()
	gameData.version = Random(100, 999) / 10.0
	
endfunction

//Load the text button controls
function controlsLoadBtn()
	
	global btn_loadRandom as integer = 0
	btn_loadRandom = CreateText("Generate Random Game Data")
		SetTextSize(btn_loadRandom, 20)
		SetTextColor(btn_loadRandom, 255, 255, 255, 255)
		SetTextAlignment(btn_loadRandom, 2)
		SetTextPosition(btn_loadRandom, 260, 50)
	
	global btn_buildMB as integer = 0
	btn_buildMB = CreateText("Build Memory Block")
		SetTextSize(btn_buildMB, 20)
		SetTextColor(btn_buildMB, 255, 255, 255, 255)
		SetTextAlignment(btn_buildMB, 2)
		SetTextPosition(btn_buildMB, 260, 300)
	
endfunction

function controlsChkBtn()
	
	//Button control for generating random game dat
	if (GetTextHitTest(btn_loadRandom, GetPointerX(), GetPointerY()) = 1)	
		SetTextColor(btn_loadRandom, 0, 255, 0, 255)
		if (GetPointerPressed() = 1)
			gameDataLoad()
			displayGeneratedData()
		endif
	else
		SetTextColor(btn_loadRandom, 255, 255, 255, 255)	
	endif
	
	//Button control for generating memblock and reading back data
	if (GetTextHitTest(btn_buildMB, GetPointerX(), GetPointerY()) = 1)
		SetTextColor(btn_buildMB, 0, 255, 0, 255)
		if (GetPointerPressed() = 1)
			mb_buildData()
			mb_readData()
			displayBuiltMB()
		endif
	else
		SetTextColor(btn_buildMB, 255, 255, 255, 255)	
	endif

endfunction

//Displays the randomly generated game data
function displayGeneratedData()
	dispDataTxt as string = ""
	dispDataTxt = "Generated Game Data:" + chr(10)
	dispDataTxt = dispDataTxt + "gameData.screen: " + str(gameData.screen) + chr(10)
	dispDataTxt = dispDataTxt + "gameData.name: " + gameData.name + chr(10)
	dispDataTxt = dispDataTxt + "gameData.date: " + gameData.date + chr(10)
	dispDataTxt = dispDataTxt + "gameData.time: " + gameData.time + chr(10)
	dispDataTxt = dispDataTxt + "gameData.version: " + str(gameData.version) + chr(10)
	
	if (GetTextExists(displayDataTxt) = 1)  then  DeleteText(displayDataTxt)
	displayDataTxt = CreateText(dispDataTxt)
		SetTextSize(displayDataTxt, 20)
		SetTextColor(displayDataTxt, 255, 255, 255, 255)
		SetTextAlignment(displayDataTxt, 0)
		SetTextPosition(displayDataTxt, 300, 25)
	
endfunction

//Displays the game data read back from memory block
function displayBuiltMB()
	dispMBTxt as string = ""
	dispMBTxt = dispMBTxt + "Built Memory Block:" + chr(10)
	dispMBTxt = dispMBTxt + "Size = " + str(GetMemblockSize(filemb_dataValue.mb)) + chr(10)
	
	dispMBTxt = dispMBTxt + "Screen = " + str(gameData.screen) + chr(10)
	dispMBTxt = dispMBTxt + "Name = " + gameData.name + chr(10)
	dispMBTxt = dispMBTxt + "Date = " + gameData.date + chr(10)
	dispMBTxt = dispMBTxt + "Time = " + gameData.time + chr(10)
	dispMBTxt = dispMBTxt + "Version = " + str(gameData.version) + chr(10)
		
	if (GetTextExists(displayMBTxt) = 1)  then  DeleteText(displayMBTxt)
	displayMBTxt = CreateText(dispMBTxt)
		SetTextSize(displayMBTxt, 20)
		SetTextColor(displayMBTxt, 255, 255, 255, 255)
		SetTextAlignment(displayMBTxt, 0)
		SetTextPosition(displayMBTxt, 300, 250)

endfunction

/*
 *	MAIN MEMEORY BLOCK FUNCTIONS
 *
 *  Memory Block Game Save
 *	May 2022 by John Douglas
 *	www.onetreehillstudios.com
 *
 */

//Writes an integer to a memory block
function filemb_addInteger(getMBint as filemb_dataValueType, getData as integer)
	makemb_tempMB_int as integer = 0
	makemb_tempMB_int = CreateMemblock(GetMemblockSize(getMBint.mb) + 4)				//Create temporary memory block with size for integer
	CopyMemblock(getMBint.mb, makemb_tempMB_int, 0, 0, GetMemblockSize(getMBint.mb))
	
	SetMemblockInt(makemb_tempMB_int, getMBint.offset, getData)						//Set integer data
	getMBint.offset = getMBint.offset + 4											//Increment offset for filemb_dataValueType
	
	if (GetMemblockExists(getMBint.mb) = 1)  then  DeleteMemblock(getMBint.mb)

getMBint.mb = makemb_tempMB_int													//Copy temporary memory block over to filemb_dataValueType
	
endfunction getMBint

//Writes a float to a memory block
function filemb_addFloat(getMBfloat as filemb_dataValueType, getData as float)
	makemb_tempMB as integer = 0
	makemb_tempMB = CreateMemblock(GetMemblockSize(getMBfloat.mb) + 4)
	CopyMemblock(getMBfloat.mb, makemb_tempMB, 0, 0, GetMemblockSize(getMBfloat.mb))
	
	SetMemblockFloat(makemb_tempMB, getMBfloat.offset, getData)						//Set float data
	getMBfloat.offset = getMBfloat.offset + 4										//Increment offset for filemb_dataValueType
	
	if (GetMemblockExists(getMBfloat.mb) = 1)  then  DeleteMemblock(getMBfloat.mb)

getMBfloat.mb = makemb_tempMB													//Copy temporary memory block over to filemb_dataValueType
	
endfunction getMBfloat

//Writes a string to a memory block including the preceeding integer containing length of string
function filemb_addString(getMBstr as filemb_dataValueType, getDataString as string)
	makemb_len as integer = 0														//Each string is preceeded by its length as an integer
	makemb_len = ByteLen(getDataString) + 1											//Length of string plus 1 for end return character
	
	getMBstr = filemb_addInteger(getMBstr, makemb_len)								//Add string length (this will automatically increment the offset)
	
	makemb_tempMB_str as integer = 0
	makemb_tempMB_str = CreateMemblock(GetMemblockSize(getMBstr.mb) + makemb_len)
	CopyMemblock(getMBstr.mb, makemb_tempMB_str, 0, 0, GetMemblockSize(getMBstr.mb))
	
	SetMemblockString(makemb_tempMB_str, getMBstr.offset, getDataString)				//Set string data
	getMBstr.offset = getMBstr.offset + makemb_len									//Increment offset for filemb_dataValueType based on string size
	
	if (GetMemblockExists(getMBstr.mb) = 1)  then  DeleteMemblock(getMBstr.mb)
	
	getMBstr.mb = makemb_tempMB_str													//Copy temporary memory block over to filemb_dataValueType
	
	getMBstr = checkDataMod4(getMBstr)												//Check that the offset and memblock size are a multiple of 4
	
endfunction getMBstr

//Reads an integer from a memory block
function filemb_getInteger(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns Integer value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	getData.inter = GetMemblockInt(getData.mb, getData.offset)
	getData.offset = getData.offset + 4												//Byte offset for integer size 4
	
endfunction getData.inter

//Reads a float from a memory block
function filemb_getFloat(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns Float value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	getData.floater = GetMemblockFloat(getData.mb, getData.offset)
	
	getData.offset = getData.offset + 4												//Byte offset for float size 4
endfunction getData.floater

//Reads a string from a memory block including the preceeding integer containing length of string
function filemb_getString(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns String value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	readmb_stringLen as integer = 0													//Temporary value for length of string to return
	readmb_stringLen = GetMemblockInt(getData.mb, getData.offset)					//Get length of string
	getData.offset = getData.offset + 4												//Byte offset for string length value (integer)
	readmb_checkMBsize(getData, readmb_stringLen)
	
	if (readmb_stringLen <= 0)
		Message("Found Empty String" + chr(10) + "getData.mb = " + str(getData.mb) + chr(10) + "readmb_stringLen = " + str(readmb_stringLen) + chr(10) + "getData.offset = " + str(getData.offset))
	endif
	
	getData.stringer = GetMemblockString(getData.mb, getData.offset, readmb_stringLen)
	getData.offset = getData.offset + readmb_stringLen								//Byte offset for string (based on string length)
	
	if (Mod(getData.offset, 4) <> 0)													//Offsets need to be a multiple of 4, check for remainder here
		while (Mod(getData.offset, 4) <> 0)
			getData.offset = getData.offset + 1										//Keep adding 1 until multiple of 4
		endwhile
	endif
	
endfunction getData.stringer

//Clears the generic data type used to retrieve save game
function filemb_dataValueTypeClear(getData ref as filemb_dataValueType)
	getData.inter = 0
	getData.floater = 0.0
	getData.stringer = ""
	
endfunction

//Checks if MB is big enough to accomodate offset
function readmb_checkMBsize(getData ref as filemb_dataValueType, getSize as integer)
	readmb_size_mb as integer
	if (GetMemblockSize(getData.mb) < (getData.offset + 4))							//Memblock is smaller than current position plus size of data
		readmb_size_errtxt as string = ""
		readmb_size_errtxt = "Error: Memblock (" + str(getData.mb) + ") to small" + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "MB size = " + str(GetMemblockSize(getData.mb)) + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "Current Offset = " + str(getData.offset) + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "Requested bytes = " + str(getSize)
		Message(readmb_size_errtxt)
	endif
	
endfunction

//Keep all data indexs as multiples of 4
function checkDataMod4(getData as filemb_dataValueType)
	
	//Fix offset
	if (Mod(getData.offset, 4) <> 0)
		getData.offset = makeMod4(getData.offset)
	endif
	
	//Mix memory block size
	if (Mod(GetMemblockSize(getData.mb), 4) <> 0)
		getData.mb = makeMod4MB(getData.mb)
	endif

endfunction getData

//Fixes offset so multiple of 4
function makeMod4(getNumber as integer)
	if (Mod(getNumber, 4) <> 0)
		while (Mod(getNumber, 4) <> 0)
			getNumber = getNumber + 1
		endwhile	
	endif

endfunction getNumber

//Fixes memblock size so multiple of 4
function makeMod4MB(getMB as integer)
	mm4mb_tempSize as integer = 0
	mm4mb_tempSize = GetMemblockSize(getMB)

if (Mod(mm4mb_tempSize, 4) <> 0)
		while (Mod(mm4mb_tempSize, 4) <> 0)
			mm4mb_tempSize = mm4mb_tempSize + 1
		endwhile	
	endif
	
	mm4mb_tempMB as integer = 0
	mm4mb_tempMB = CreateMemblock(mm4mb_tempSize)
	CopyMemblock(getMB, mm4mb_tempMB, 0, 0, GetMemblockSize(getMB))
	DeleteMemblock(getMB)

endfunction mm4mb_tempMB

+ Code Snippet



/*
 *	Memory Block Save Game Data Example
 *
 *		May 2022 One Tree Hill Studios
 *
 *		- Build (write) and Read order must match for the offset to work
 *		- Build the memory block with filemb_addInteger(), filemb_addFloat(), and filemb_addString()
 *		- Read back the memory block with filemb_getInteger(), filemb_getFloat(), and filemb_getString()
 *		- Uses custom data type filemb_dataValueType to keep track of the offset while writing/reading
 *		- Write and read directly to disk with CreateFileFromMemblock() and CreateMemblockFromFile()
 */




SetErrorMode(2)

// set window properties
SetWindowTitle( "memblock_fileSave" )
SetWindowSize( 1024, 768, 0 )
SetWindowAllowResize( 1 ) // allow the user to resize the window

// set display properties
SetVirtualResolution( 1024, 768 ) // doesn't have to match the window
SetOrientationAllowed( 1, 1, 1, 1 ) // allow both portrait and landscape on mobile devices
SetSyncRate( 30, 0 ) // 30fps instead of 60 to save battery
SetScissor( 0,0,0,0 ) // use the maximum available screen space, no black borders
UseNewDefaultFonts( 1 )
SetPrintSize(16)


type filemb_dataValueType	//This is a required type for the memory block build/read functions
	mb as integer			//Memory Block
	offset as integer		//Offset that gets automatically incremented
	inter as integer			//Byte size 4
	floater as float			//Byte size 4
	stringer as string		//Byte size = ByteLen() + 1
endtype


/*
 *	This variable will be used to build and read the memory block.
 */
global filemb_dataValue as filemb_dataValueType



/*
 *	The following type and variables are for the working demo
 *		(the game data would match your save game stucture)
 */
type gameDataType
	screen as integer
	name as string
	date as string
	time as string
	version as float
endtype
global gameData as gameDataType		//Randomly generated game data placed here
global displayDataTxt as integer		//Display text for the generated game data
global displayMBTxt as integer		//Display text for reading back the memblock data

controlsLoadBtn()					//Load the control buttons (text buttons)
		
gameDataLoad()						//Loads randomly generated game data to gameData (gameDataType)
displayGeneratedData()				//Display the randomly generated game data


do
    Print( ScreenFPS() )
    
    controlsChkBtn()					//Control (text) button controls
    
    
    Sync()
loop




/*
 *	This function is an example of how to build up the memory block, the offset is automatically incremented.
 *	
 *		All integers and floats use 4 bytes, strings are dynamic and will automatically add a
 *		preceeding integer with the ByteLen() + 1 of the string
 */
function mb_buildData()
	
	if (GetMemblockExists(filemb_dataValue.mb) = 1)  then  DeleteMemblock(filemb_dataValue.mb)	
	filemb_dataValue.mb = CreateMemblock(0)		//Memory block sizes are auto incremented as data is added
	filemb_dataValue.offset = 0					//Remember to reset this to zero when reading back
	
	filemb_dataValue = filemb_addInteger(filemb_dataValue, gameData.screen)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.name)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.date)
	filemb_dataValue = filemb_addString(filemb_dataValue, gameData.time)
	filemb_dataValue = filemb_addFloat(filemb_dataValue, gameData.version)

endfunction




/*
 *	This function is an exmaple of how to read back data from the memory block.
 *	
 *		Like the add functions, the offset is automatically incremented.
 */
function mb_readData()
	gameData.screen = -1
	gameData.name = ""
	gameData.date = ""
	gameData.time = ""
	gameData.version = 0.0
	
	filemb_dataValue.offset = 0				//Reset offset to zero for reading back values
	
	//Read the data back in the exact same order as it was created
	gameData.screen = filemb_getInteger(filemb_dataValue)
	gameData.name = filemb_getString(filemb_dataValue)
	gameData.date = filemb_getString(filemb_dataValue)
	gameData.time = filemb_getString(filemb_dataValue)
	gameData.version = filemb_getFloat(filemb_dataValue)

endfunction






/*
 *	WORKING EXAMPLE FUNCTIONS
 *
 */

//Randomly generat some game data
function gameDataLoad()
	
	gameData.screen = Random(1, 200)
	gameData.name = "scrn_" + str(Random(1, 2000))
	gameData.date = GetCurrentDate()
	gameData.time = GetCurrentTime()
	gameData.version = Random(100, 999) / 10.0
	
endfunction



//Load the text button controls
function controlsLoadBtn()
	
	global btn_loadRandom as integer = 0
	btn_loadRandom = CreateText("Generate Random Game Data")
		SetTextSize(btn_loadRandom, 20)
		SetTextColor(btn_loadRandom, 255, 255, 255, 255)
		SetTextAlignment(btn_loadRandom, 2)
		SetTextPosition(btn_loadRandom, 260, 50)
	
	global btn_buildMB as integer = 0
	btn_buildMB = CreateText("Build Memory Block")
		SetTextSize(btn_buildMB, 20)
		SetTextColor(btn_buildMB, 255, 255, 255, 255)
		SetTextAlignment(btn_buildMB, 2)
		SetTextPosition(btn_buildMB, 260, 300)
	
endfunction




function controlsChkBtn()
	
	//Button control for generating random game dat
	if (GetTextHitTest(btn_loadRandom, GetPointerX(), GetPointerY()) = 1)	
		SetTextColor(btn_loadRandom, 0, 255, 0, 255)
		if (GetPointerPressed() = 1)
			gameDataLoad()
			displayGeneratedData()
		endif
	else
		SetTextColor(btn_loadRandom, 255, 255, 255, 255)	
	endif
	
	//Button control for generating memblock and reading back data
	if (GetTextHitTest(btn_buildMB, GetPointerX(), GetPointerY()) = 1)
		SetTextColor(btn_buildMB, 0, 255, 0, 255)
		if (GetPointerPressed() = 1)
			mb_buildData()
			mb_readData()
			displayBuiltMB()
		endif
	else
		SetTextColor(btn_buildMB, 255, 255, 255, 255)	
	endif

endfunction



//Displays the randomly generated game data
function displayGeneratedData()
	dispDataTxt as string = ""
	dispDataTxt = "Generated Game Data:" + chr(10)
	dispDataTxt = dispDataTxt + "gameData.screen: " + str(gameData.screen) + chr(10)
	dispDataTxt = dispDataTxt + "gameData.name: " + gameData.name + chr(10)
	dispDataTxt = dispDataTxt + "gameData.date: " + gameData.date + chr(10)
	dispDataTxt = dispDataTxt + "gameData.time: " + gameData.time + chr(10)
	dispDataTxt = dispDataTxt + "gameData.version: " + str(gameData.version) + chr(10)
	
	if (GetTextExists(displayDataTxt) = 1)  then  DeleteText(displayDataTxt)
	displayDataTxt = CreateText(dispDataTxt)
		SetTextSize(displayDataTxt, 20)
		SetTextColor(displayDataTxt, 255, 255, 255, 255)
		SetTextAlignment(displayDataTxt, 0)
		SetTextPosition(displayDataTxt, 300, 25)
	
endfunction


//Displays the game data read back from memory block
function displayBuiltMB()
	dispMBTxt as string = ""
	dispMBTxt = dispMBTxt + "Built Memory Block:" + chr(10)
	dispMBTxt = dispMBTxt + "Size = " + str(GetMemblockSize(filemb_dataValue.mb)) + chr(10)
	
	dispMBTxt = dispMBTxt + "Screen = " + str(gameData.screen) + chr(10)
	dispMBTxt = dispMBTxt + "Name = " + gameData.name + chr(10)
	dispMBTxt = dispMBTxt + "Date = " + gameData.date + chr(10)
	dispMBTxt = dispMBTxt + "Time = " + gameData.time + chr(10)
	dispMBTxt = dispMBTxt + "Version = " + str(gameData.version) + chr(10)
		
	if (GetTextExists(displayMBTxt) = 1)  then  DeleteText(displayMBTxt)
	displayMBTxt = CreateText(dispMBTxt)
		SetTextSize(displayMBTxt, 20)
		SetTextColor(displayMBTxt, 255, 255, 255, 255)
		SetTextAlignment(displayMBTxt, 0)
		SetTextPosition(displayMBTxt, 300, 250)

endfunction








/*
 *	MAIN MEMEORY BLOCK FUNCTIONS
 *
 *  Memory Block Game Save
 *	May 2022 by John Douglas
 *	www.onetreehillstudios.com
 *
 */


//Writes an integer to a memory block
function filemb_addInteger(getMBint as filemb_dataValueType, getData as integer)
	makemb_tempMB_int as integer = 0
	makemb_tempMB_int = CreateMemblock(GetMemblockSize(getMBint.mb) + 4)				//Create temporary memory block with size for integer
	CopyMemblock(getMBint.mb, makemb_tempMB_int, 0, 0, GetMemblockSize(getMBint.mb))
	
	SetMemblockInt(makemb_tempMB_int, getMBint.offset, getData)						//Set integer data
	getMBint.offset = getMBint.offset + 4											//Increment offset for filemb_dataValueType
	
	if (GetMemblockExists(getMBint.mb) = 1)  then  DeleteMemblock(getMBint.mb)

	getMBint.mb = makemb_tempMB_int													//Copy temporary memory block over to filemb_dataValueType
	
endfunction getMBint



//Writes a float to a memory block
function filemb_addFloat(getMBfloat as filemb_dataValueType, getData as float)
	makemb_tempMB as integer = 0
	makemb_tempMB = CreateMemblock(GetMemblockSize(getMBfloat.mb) + 4)
	CopyMemblock(getMBfloat.mb, makemb_tempMB, 0, 0, GetMemblockSize(getMBfloat.mb))
	
	SetMemblockFloat(makemb_tempMB, getMBfloat.offset, getData)						//Set float data
	getMBfloat.offset = getMBfloat.offset + 4										//Increment offset for filemb_dataValueType
	
	if (GetMemblockExists(getMBfloat.mb) = 1)  then  DeleteMemblock(getMBfloat.mb)

	getMBfloat.mb = makemb_tempMB													//Copy temporary memory block over to filemb_dataValueType
	
endfunction getMBfloat



//Writes a string to a memory block including the preceeding integer containing length of string
function filemb_addString(getMBstr as filemb_dataValueType, getDataString as string)
	makemb_len as integer = 0														//Each string is preceeded by its length as an integer
	makemb_len = ByteLen(getDataString) + 1											//Length of string plus 1 for end return character
	
	getMBstr = filemb_addInteger(getMBstr, makemb_len)								//Add string length (this will automatically increment the offset)
	
	makemb_tempMB_str as integer = 0
	makemb_tempMB_str = CreateMemblock(GetMemblockSize(getMBstr.mb) + makemb_len)
	CopyMemblock(getMBstr.mb, makemb_tempMB_str, 0, 0, GetMemblockSize(getMBstr.mb))
	
	SetMemblockString(makemb_tempMB_str, getMBstr.offset, getDataString)				//Set string data
	getMBstr.offset = getMBstr.offset + makemb_len									//Increment offset for filemb_dataValueType based on string size
	
	if (GetMemblockExists(getMBstr.mb) = 1)  then  DeleteMemblock(getMBstr.mb)
	
	getMBstr.mb = makemb_tempMB_str													//Copy temporary memory block over to filemb_dataValueType
	
	getMBstr = checkDataMod4(getMBstr)												//Check that the offset and memblock size are a multiple of 4
	
endfunction getMBstr




//Reads an integer from a memory block
function filemb_getInteger(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns Integer value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	getData.inter = GetMemblockInt(getData.mb, getData.offset)
	getData.offset = getData.offset + 4												//Byte offset for integer size 4
	
endfunction getData.inter



//Reads a float from a memory block
function filemb_getFloat(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns Float value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	getData.floater = GetMemblockFloat(getData.mb, getData.offset)
	
	getData.offset = getData.offset + 4												//Byte offset for float size 4
endfunction getData.floater



//Reads a string from a memory block including the preceeding integer containing length of string
function filemb_getString(getData ref as filemb_dataValueType)						//Save Game Read Memblock - Returns String value
	filemb_dataValueTypeClear(getData)
	readmb_checkMBsize(getData, 4)
	readmb_stringLen as integer = 0													//Temporary value for length of string to return
	readmb_stringLen = GetMemblockInt(getData.mb, getData.offset)					//Get length of string
	getData.offset = getData.offset + 4												//Byte offset for string length value (integer)
	readmb_checkMBsize(getData, readmb_stringLen)
	
	if (readmb_stringLen <= 0)
		Message("Found Empty String" + chr(10) + "getData.mb = " + str(getData.mb) + chr(10) + "readmb_stringLen = " + str(readmb_stringLen) + chr(10) + "getData.offset = " + str(getData.offset))
	endif
	
	getData.stringer = GetMemblockString(getData.mb, getData.offset, readmb_stringLen)
	getData.offset = getData.offset + readmb_stringLen								//Byte offset for string (based on string length)
	
	if (Mod(getData.offset, 4) <> 0)													//Offsets need to be a multiple of 4, check for remainder here
		while (Mod(getData.offset, 4) <> 0)
			getData.offset = getData.offset + 1										//Keep adding 1 until multiple of 4
		endwhile
	endif
	
endfunction getData.stringer



//Clears the generic data type used to retrieve save game
function filemb_dataValueTypeClear(getData ref as filemb_dataValueType)
	getData.inter = 0
	getData.floater = 0.0
	getData.stringer = ""
	
endfunction


//Checks if MB is big enough to accomodate offset
function readmb_checkMBsize(getData ref as filemb_dataValueType, getSize as integer)
	readmb_size_mb as integer
	if (GetMemblockSize(getData.mb) < (getData.offset + 4))							//Memblock is smaller than current position plus size of data
		readmb_size_errtxt as string = ""
		readmb_size_errtxt = "Error: Memblock (" + str(getData.mb) + ") to small" + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "MB size = " + str(GetMemblockSize(getData.mb)) + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "Current Offset = " + str(getData.offset) + chr(10)
		readmb_size_errtxt = readmb_size_errtxt + "Requested bytes = " + str(getSize)
		Message(readmb_size_errtxt)
	endif
	
endfunction



//Keep all data indexs as multiples of 4
function checkDataMod4(getData as filemb_dataValueType)
	
	//Fix offset
	if (Mod(getData.offset, 4) <> 0)
		getData.offset = makeMod4(getData.offset)
	endif
	
	//Mix memory block size
	if (Mod(GetMemblockSize(getData.mb), 4) <> 0)
		getData.mb = makeMod4MB(getData.mb)
	endif

endfunction getData


//Fixes offset so multiple of 4
function makeMod4(getNumber as integer)
	if (Mod(getNumber, 4) <> 0)
		while (Mod(getNumber, 4) <> 0)
			getNumber = getNumber + 1
		endwhile	
	endif

endfunction getNumber


//Fixes memblock size so multiple of 4
function makeMod4MB(getMB as integer)
	mm4mb_tempSize as integer = 0
	mm4mb_tempSize = GetMemblockSize(getMB)

	if (Mod(mm4mb_tempSize, 4) <> 0)
		while (Mod(mm4mb_tempSize, 4) <> 0)
			mm4mb_tempSize = mm4mb_tempSize + 1
		endwhile	
	endif
	
	mm4mb_tempMB as integer = 0
	mm4mb_tempMB = CreateMemblock(mm4mb_tempSize)
	CopyMemblock(getMB, mm4mb_tempMB, 0, 0, GetMemblockSize(getMB))
	DeleteMemblock(getMB)

endfunction mm4mb_tempMB

Check out Automation on Steam now
https://store.steampowered.com/app/1698690/Automation/

Back to top

Profile PM Email Website

Kevin Picone

21

Years of Service

User Offline

Joined: 27th Aug 2002

Location: Australia

Posted: 23rd May 2022 15:42 Edited at: 23rd May 2022 15:47

Link

The concept is sound. You could just wrap the file IO functions and have the wrapper read/write to the current memory block. So the user calls some type of Output/write function that reset the current pointer and or recreate the block to flush out and legacy data. Ideally the library would handle the size and resizing on the users behalf, making sure the buffer is sized and only expands only when an overflow is likely, reducing the memory thrashing.

I know the code is proof of concept, wrote a quick replacement for Makemod4() function but it would seem AND isn't behaving in AGK.
.

+ Code Snippet


function MakeMod4(getNumber as integer)
		Bytes_To_Next_Mult_of_4 = ((GetNumber and 3)<>0) *4
		getNumber=(GetNumber and $fffffffc) + Bytes_To_Next_Mult_of_4
endfunction getNumber

This one seems to work.

+ Code Snippet


function MakeMod4(getNumber as integer)
		RoundedValue = (GetNumber /4) * 4
		Bytes_To_Next_Mult_of_4 = ((GetNumber - RoundedValue )<>0) *4
		getNumber=RoundedValue + Bytes_To_Next_Mult_of_4
endfunction getNumber

PlayBASIC To HTML5/WEB - Convert PlayBASIC To Machine Code

Back to top

Profile PM Website

jd_zoo

5

Years of Service

User Offline

Joined: 12th May 2018

Location: Nova Scotia

Posted: 23rd May 2022 22:16

Link

Yep I agree this simplified version could get packaged or easily re-purposed, and yeah I have already modified the mod4 so that it is caught before creating the string's memblock. My full working demo has multiple memblocks for different data sets of the game, then creates a header with all their sizes before combining into a single memblock which gets written to disk. Another layer of complexion but possible to include as a full packaged set of save game tools.

One thing is that this results in a new file for each save game, as opposed to a new line in a single CSV file.

Check out Automation on Steam now
https://store.steampowered.com/app/1698690/Automation/

Back to top

Profile PM Email Website

Kevin Picone

21

Years of Service

User Offline

Joined: 27th Aug 2002

Location: Australia

Posted: 26th May 2022 03:07 Edited at: 29th May 2022 02:58

Link

Hi jd_zoo;

Quote: "
My full working demo has multiple memblocks for different data sets of the game, then creates a header with all their sizes before combining into a single memblock which gets written to disk. Another layer of complexion but possible to include as a full packaged set of save game tools.
"

That's the way to go, only thing to be careful about would be the amount of number crunching.. That's not really a strength of the AGKS runtime; if there's a function to write a chunk of memblock data directly to disk; you could do the composite to the target rather than combine in memory-> then save it all.

Something I noticed the other day and I get the above code is mostly cut'n'paste from useage situation, but the AddInteger etc functions have a to create the buffer then copy old data across to the stream.

+ Code Snippet

 
//Writes an integer to a memory block
function filemb_addInteger(getMBint as filemb_dataValueType, getData as integer)
    makemb_tempMB_int as integer = 0
    makemb_tempMB_int = CreateMemblock(GetMemblockSize(getMBint.mb) + 4)                //Create temporary memory block with size for integer
    CopyMemblock(getMBint.mb, makemb_tempMB_int, 0, 0, GetMemblockSize(getMBint.mb))
     
    SetMemblockInt(makemb_tempMB_int, getMBint.offset, getData)                     //Set integer data
    getMBint.offset = getMBint.offset + 4                                           //Increment offset for filemb_dataValueType
     
    if (GetMemblockExists(getMBint.mb) = 1)  then  DeleteMemblock(getMBint.mb)
 
    getMBint.mb = makemb_tempMB_int                                                 //Copy temporary memory block over to filemb_dataValueType
     
endfunction getMBint

The issue with this is that if anyone ever needs to store a large block of Integers / floats or whatever; then for every push; there's an memory allocation combined with a buffer copy. So if the mem block was a megabyte say; They'd only have to push a 1000 values and it's thrashing a gigabyte of memory.

Easiest way to reduce or remove this would be to pre-size the output mem block up front. Then each time we push data; the function compares the current output address+size of data to be written against it's current size of the mem block. if the write will overflows the mem block; resize it.. otherwise; just drop the data to output address and bump the output address.

Example: ( Untested written in browser

)

function filemb_addInteger(getMBint as filemb_dataValueType, getData as integer)

//  Check our current write address to the block size + size of data    (4 for integer)
   //  I would pre-compute it's size and store that in our type; to avoid having to call that's effectively a constant function.  Given the size won't be changing much; if at all
  //  so when the block was created at store in this structure_  it's store it size then.      (ie getMBint.mb_Currrent_Size   =  GetMemblockSize(getMBint.mb) )
    if  (     getMBint.offset + 4   =>  getMBint.mb_Currrent_Size )
          //  call function to resie the buffer and clean up old buffer
                filemb_Resize_Buffer(getMBint)
    endif 
     
    SetMemblockInt(getMBint.mb, getMBint.offset, getData)                     //Set integer data
    getMBint.offset = getMBint.offset + 4                                           //Increment offset for filemb_dataValueType
     
endfunction getMBint

// NOTE:   Assuming getMBint will be passed by ref..  but you could equally refer to the current output in a global.

function filemb_Resize_Buffer(getMBint as filemb_dataValueType, Resize_Pad_Size as integer )

makemb_tempMB_int = 0

OldMB    = getMBint.mb
   OldMB_Size  =getMBint.mb_size

NewMB_Size = (OldMB_Size  * 2) + Resize_Pad_Size
   NewMB = CreateMemblock(NewMB_Size)

//  copy the old buffer into the newly created bigger one
    CopyMemblock(OldMB, NewMB, 0, 0,OldMB_SIZE )

//  Store the new block index and it's new max size
     getMBint.mb = NewMB
     getMBint.mb_Size = NewMB_SIZE

//  delete the old memory block to avoid leaks
     DeleteMemblock(OldMB)

EndFunction

+ Code Snippet

function filemb_addInteger(getMBint as filemb_dataValueType, getData as integer)

   //  Check our current write address to the block size + size of data    (4 for integer)
   //  I would pre-compute it's size and store that in our type; to avoid having to call that's effectively a constant function.  Given the size won't be changing much; if at all
  //  so when the block was created at store in this structure_  it's store it size then.      (ie getMBint.mb_Currrent_Size   =  GetMemblockSize(getMBint.mb) )
    if  (     getMBint.offset + 4   =>  getMBint.mb_Currrent_Size )
          //  call function to resie the buffer and clean up old buffer
                filemb_Resize_Buffer(getMBint)
    endif 
     
    SetMemblockInt(getMBint.mb, getMBint.offset, getData)                     //Set integer data
    getMBint.offset = getMBint.offset + 4                                           //Increment offset for filemb_dataValueType
     
endfunction getMBint
 

  // NOTE:   Assuming getMBint will be passed by ref..  but you could equally refer to the current output in a global.  

function filemb_Resize_Buffer(getMBint as filemb_dataValueType, Resize_Pad_Size as integer )

    makemb_tempMB_int = 0

   OldMB    = getMBint.mb
   OldMB_Size  =getMBint.mb_size

   NewMB_Size = (OldMB_Size  * 2) + Resize_Pad_Size
   NewMB = CreateMemblock(NewMB_Size)

  //  copy the old buffer into the newly created bigger one
    CopyMemblock(OldMB, NewMB, 0, 0,OldMB_SIZE )

   //  Store the new block index and it's new max size
     getMBint.mb = NewMB
     getMBint.mb_Size = NewMB_SIZE

    //  delete the old memory block to avoid leaks
     DeleteMemblock(OldMB)

EndFunction

PlayBASIC To HTML5/WEB - Convert PlayBASIC To Machine Code

Back to top

Profile PM Website

jd_zoo

5

Years of Service

User Offline

Joined: 12th May 2018

Location: Nova Scotia

Posted: 27th May 2022 01:44

Link

Ahhhh ok yes thank you, my mind doesn't appreciate the back end workings! I was thinking one step at a time, but in the end the sync could be pretty crushing.

I really like the idea of incrementing it in preset sizes as you need more space... I have definitely learned a lot about this over last few weeks!

Check out Automation on Steam now
https://store.steampowered.com/app/1698690/Automation/

Back to top

Profile PM Email Website

jd_zoo

5

Years of Service

User Offline

Joined: 12th May 2018

Location: Nova Scotia

Posted: 19th Jun 2022 15:40

Link

Not a fair comparison, my old string looping through the CSV file was not an overly efficient approach in hindsight. But for my situation the flexibility with a binary setup is the first reason and speed is a bonus...

Check out Automation on Steam now
https://store.steampowered.com/app/1698690/Automation/

Back to top

Profile PM Email Website

Sorry your browser is not supported!

AppGameKit Studio Chat / Memblock save game data (dynamic string length)