Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / Bug - CreateFileFromMemblock

Author
Message
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 18th Nov 2021 15:47
I have created a server which periodacally saves registered user's data.

After about 8 hours of running, the server.exe stops with an error dialog box with literally no text, no data writing on it.

When I remove backup code it runs more than 8 hours, well I couldn't test more then 12 hours yet, but I know last line of my code before it produces that wired error with no error data.

It uses same memblock for gaining speed.

It fills the memblock with thw appopriate data which is 'bit precised'.

It then deletes the outdated file and immediately creates new user profile file.

At this point after deleting the file, it throws an error with empty dialog box, the file is deleted but the new one is not there.

So, createfilefrommemblock() fails internally.

Attachments

Login to view attachments
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 18th Nov 2021 16:29 Edited at: 18th Nov 2021 16:41
Yeah it sure sounds like an IMA.

if there is a memory corruption they maybe AppGameKit fails to "Format" the error message as it also uses memory for Unicode string conversion in its uString class, that would explain the empty dialog but it does not explain the initial corruption, are you 100% sure the buffer is large enough for the data and all offsets remain correct.

It sounds like you know exactly what you are doing here so you will understand what I am saying, createfilefrommemblock simply accesses the memory pointer and writes the data with fwrite() and immediately closes the file, along the way 2 error checks are performed, the 1st checks that the memblock exists and the 2nd checks that the file was created and is open to write, so it could also be a file lock issue and not memory, does anything else access the file?

Edit: But that still would not explain the empty dialog, my bet is on a memory overrun causing an IMA and curropting the AppGameKit heap allocation before it can format the error message, double check your offsets and maybe add some padding.
Open Source plugins
Cl - DnD Plugin
Buy Me A Coffee
Game_Code_here
3
Years of Service
User Offline
Joined: 2nd Jun 2020
Location:
Posted: 18th Nov 2021 22:48
Yes this sounds like a memory problem.

What you can do is save on each hour to a new file then if the files are more then so many do whatever.

Could also be a firewall problem.

The empty data is caused by there being no saved file at the end, I'm sure, that is the only thing that makes sense.

But Just to let you know I have no experience with app game kits file write commands.

I'm just saying what makes sense.
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 19th Nov 2021 07:15 Edited at: 19th Nov 2021 07:39
Thanks for the answers...

No memory leaks, no #gp fault exceptions...
The server has two real players and only 32 bots for now, and we can use it properly.
I open it at midnight and when I wake up I get this error, so no interactions made with the server during the night.

This morning it failed again.

I created a mini test;



The error message;




I guess, AppGameKit pushes a structure data into a vector array for internal file handling and not popping it back from that array when using createfilefrommemblock??
type ftype
index as uint
path as tchar * 260
name as tchar * 260
size as ulongint
pos as ulongint
....
..
.

??

I repeat, the memblock is globally created once at the start and works for about 8 hours fine, so I'm pretty sure it is not the memblock fault.

This server must run like forever...

I've already created a winsock library in FreeBasic and I can switch to it which I don't want to do.

I can stop using createfilefrommemblock() and use writeinteger() but this will be so slow, yet I'll try this and see if it creates an error this night.
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 19th Nov 2021 08:27 Edited at: 19th Nov 2021 08:35
No AppGameKit does not pop anything from the memblock with calling createfilefrommemblock, it leaves the buffer intact, AppGameKit does use an internal list to keep track of the memblocks but this merely holds the pointer address and gives you an unsigned int for ease of reference, the same list class is used everywhere in AppGameKit (for sprites, objects, text, edit etc etc) so I can say with certainly that this is not the problem or it would be bugging out in every project.

This:


Is allocating 67MB per frame??, I would expect it to crash! lol

when freeing the memory, it does not crash


The problem is not the memblock its the way its being used.

I am 99.99% sure this is an overrun issue causing an IMA (Invalid Memory Access)

try freeing the buffer once in a while and reallocating, also some things to bear in mind

the max buffer size AppGameKit will allow is < 100000000 bytes

agk allocates the memory as:


so if you have a signed char in your data you will get overrun? (as I understand it?)
Quote: "signed char, which gives you at least the -127 to 127 range. (-128 to 127 is common)
unsigned char, which gives you at least the 0 to 255 range."


Edit: but I could be wrong on that, not great on memory stuff myself, I think the *sign* takes an extra byte or 2, maybe someone with more knowledge/experience can chime in here?
Open Source plugins
Cl - DnD Plugin
Buy Me A Coffee
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 19th Nov 2021 11:24
Yes, excatly, the memblock list should have elements like pointer, size and index of an memblock. As I imagine a file structure just like that list. So, the pushing and popping is not from the memblock but an internal array which holds all the file data structures, but just a guess.

Yea, the test code allocates 64mb/frame to test what happens and what error code is given, the error code gives something at least.

The buffer is 1024 bytes only.

Yes, I am also thinking to deallocate the buffer and reallocate it after all player's datas are saved. (it backups 1 player data in 1 frames for speed.)

I am also thinking to end the server.exe after let's say 4 hours and restart it, so every 4 hours there will be some delays. To achive this, another exe will run the server.exe and if server.exe ends it will re-run the server.exe...

Nothing to do with the sign.

The sign takes only 1 bit.

Thanks.
PartTimeCoder
AGK Tool Maker
9
Years of Service
User Offline
Joined: 9th Mar 2015
Location: London UK
Posted: 19th Nov 2021 12:24
Well I hope you find a working solution, sorry I could not be more help, let us know how you get on.



Open Source plugins
Cl - DnD Plugin
Buy Me A Coffee
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 24th Nov 2021 14:52
I cahnged


To


The server is alive more than a day now.

So, the issue is fixed for me but the bug remains.
Createfilefrommemblock is innocent by the way.
adambiser
AGK Developer
8
Years of Service
User Offline
Joined: 16th Sep 2015
Location: US
Posted: 27th Nov 2021 05:45
On Windows, DeleteFile calls "DeleteFileW". The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED."
It's possible that CreateFileFromMemblock is happening before the OS has a chance to actually process the delete. It's a race condition. Related

Doesn't explain the blank error message though.
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 29th Nov 2021 15:43
yeah It came to my mind too and I tried

deletfile(file)
do
if getfileexist(file) = 0 then exit
loop
createfilefrommemblock(file)

it crushed again after 7 - 8 hours with blank error message again.

it is about a forgetten file handle to be released i guess...?

luckily 'createfilefrommemblock()' changed the whole file even it exists before, without checking file existence and then deleting it.
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 29th Nov 2021 22:35
Ok, it is official all CreateFileFromMemblock(), GetFileExists() and DeleteFile() are buggy!



run this code, open task manager, go to details tab and enjoy how this mini code consumes the ram.
Then add GetFileExists() and see how the consuming speeds up!
Then add the DeleteFile() and you will clearly see, they are memory consuming, at some point some kind of handles are not released i guess...

It took so long now, someone in charge must fix these already!
Game_Code_here
3
Years of Service
User Offline
Joined: 2nd Jun 2020
Location:
Posted: 29th Nov 2021 23:27
i = Creatememblock(1024)

Creating a memory block with 1024 mg of memory
CreateMemblock( memID, size )

Creating a file with the size of 1024 mg of memory
CreateFileFromMemblock("somefile.txt", i)


I can see how this would slow down any ones program.

As I read it you are creating a large memory block and then creating a file with that memory.
Arch-Ok
AGK Developer
4
Years of Service
User Offline
Joined: 11th Jul 2019
Location: Bursa/TÜRKIYE
Posted: 2nd Dec 2021 21:57
Update;
creatememblockfromfile() does the same thing and uses more and more memory after every call.

try this code and open task manager and go to details tab and watch for yourselves;

you need to create a "a.txt" file in the project's media folder.

Scraggle
Moderator
20
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 2nd Dec 2021 22:04 Edited at: 2nd Dec 2021 22:24
That one is obviously the expected result.
If you continually create a memblock (set aside an area of memory to write to) then if course it's going to continually increase the amount of memory it uses. That's exactly what you are asking it to do.

Edit
Sorry, I didn't see the DeleteMemblock() call. That changes things

Login to post a reply

Server time is: 2024-03-28 23:57:04
Your offset time is: 2024-03-28 23:57:04