Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Studio Chat / Denied access to external storage on some Android devices but not others.

Author
Message
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 25th Jul 2020 00:49
Link

I have run into an issue that is preventing read/write access using a raw: path.

I used the following code to request external write permission:



and attempted to open a file to write in the following way:



Obviously myfolder and myfile.txt are not the actual names (although I probably waste more time naming variables and files than I do actual coding)
The code was written in AppGameKit Studio version 2020.04.16 and compiled with External Write permission ticked. The .apk installs and runs fine on a Samsung Galaxy S7 Edge running Android 8 and a Samsung Tab S4 running Android 9.
I ran into the issue when I tested the app on a Samsung Galaxy S9 running Android 10. The usual permission requester appeared, was confirmed to allow but this was immediately followed by a message displaying:

Message

Error: Failed to create folder "emulated" in path "/storage/emulated/0/myfolder/myfile.txt", the app may not have permission to create it in main.agc at line 48

Other than device and OS version, the S7 Edge and Tab S4 both have physical sd cards, the S9 does not, however I don't think that should make any difference at all.
I've tried various permutations and paths without any success at all. Writing to the default write path in the apps own protected storage is the only place that I can save to, but means I can't access the file from outside of the app itself which I need to do.
Googling around hasn't revealed anything I can use and what I came across refers only to coding in Android Studio.

I'm beginning to feel this app is doomed what with this and it's GPS tracking having issues due to Android caching the process once the app is minimised...but that's another issue.

Any help with the folder access issue would be greatly appreciated.


Back to top
Profile PM
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 25th Jul 2020 01:30
Link
What is the actual path to the external storage path on the S9?
Sounds like Samsung/android have changed their naming convention between version 9 and 10
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 25th Jul 2020 01:39
Link
What you encountered is part of Google's planned migration away from external storage in Android (beginning with version 10, solidified in 11 with no legacy option). They are moving to a limited access system (scoped storage) that only allows file access under specific limited conditions and locations. As an AppGameKit developer, you'll need to accommodate the transition if you want to support Android 10 and newer versions.

In my opinion, the easiest way to currently do that while still continuing to support older versions of Android is to move to the SharedVariable command set. Take what you would normally store in the files line by line and slot those values into SharedVariable indexes by named prefix, then value. I like to use what would be the file name followed by the description of the value for the prefix, then the value after the comma. Makes retrieval and managing pretty simple.

You can store all kinds of things this way, including a huge data set, all with good performance and varied Android version compatibility. You will still need to prompt for external write storage permission with your sample code above, but once set, you can store and read whatever you want in the SharedVariable indexes.

The commands are pretty simple:

SaveSharedVariable(Prefix$,Value$)
LoadSharedVariable(Prefix$,DefaultValue$)

Instructions are included with AGK.
SW3DG | Evochron
Back to top
Profile PM Website
blink0k
Moderator
11
Years of Service
User Offline
Joined: 22nd Feb 2013
Location: the land of oz
Posted: 25th Jul 2020 02:02
Link
very interesting and helpful SFSW. I had no idea
Back to top
Profile PM
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 25th Jul 2020 17:02
Link

Thank you SFSW ! I had a suspicion that something had been implemented in Android 10 that was causing the trouble but had no workaround. Google

I had a quick play with the shared variable method and immediately realised it's great for sharing variables between apps, but not so wieldy for saving, say a text document for use in (in my case) a user editable config file or a very large .csv type file containing many values within a single file.

This is what I came up with for testing saving out and loading a simple .ini type file. Not winning any code of the day type awards I'm thinking.



I tried this on the Tab S4 and it worked, creating the 'Config.ini' in [/storage/emulated/0/SharedData]. Sadly, running it on the S9 returned all of the variables as empty and sure enough, no Config.ini was saved (or at least not saved in any accessible location where I could see it).

Looks like Google have stumped me with this.
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 25th Jul 2020 18:13
Link
Double check that the test code you are running has write permission on the device. And I would recommend just saving out each value to its own labelled index rather than one big string (just so its easier to manage and retrieve with far fewer lines of code and no required parsing).

IIRC, you can also edit the values as you would a text file by routing to the shared variable folder in a file explorer and opening the desired value in an editor.
SW3DG | Evochron
Back to top
Profile PM Website
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 25th Jul 2020 19:55
Link

So, I ensured the permission request was still in place and tested it by broadcast from the PC to the device on AppGameKit player and also compiled it with write permission ticked and ran it from the installed .apk.
Both brought up the runtime request for external write permission, which I granted but no file was generated. This is not quite the behaviour I experienced with trying to save using raw paths. This time
no message warning that I did not have permission appeared. In fact nothing happened, it just executed the code but didn't export the variables. Trying the same methods on the older devices worked
perfectly.

I agree the code I knocked up is a bit of a long winded mess but I was thinking of the end user being confronted by a series of files holding individual values when they were expecting a single config.ini
Similarly this app potentially generates a very large list of latitude, longitude, altitude, speed etc. data as a comma separated text output. Originally this was easy using WriteLine() to add a new entry to an
ever growing list. The idea was for the user to be able to copy the file, say to their PC and open it in a spreadsheet for processing however they felt fit. In testing I copied out a large block of lat/lon values
to paste into an online interactive map that drops pins at the co-ords, marking out the route. If I save out each value in it's own index I think you would have to load them all individually into the spreadsheet
and the folder would end up containing many thousands of files. At the minute it's fairly organised as it exports in the format: Route_2020_25_07_1.txt, Route_2020_25_07_2.txt etc. with each file holding the
entire dataset for that route.

I think I'm probably abusing AppGameKit here since the type of app I'm making is pretty far removed from being a game.

Thanks for taking time to look at this, much appreciated.
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 25th Jul 2020 20:23 Edited at: 25th Jul 2020 20:34
Link
Couple of things to try:

- Double check that the write permission was actually authorized before attempting to write out shared variables. That is, redundantly check for the write permission after you've prompted the user for permission, but before you attempt to save or load shared variables:



The message alert will appear if the permission check failed and then you'll know if there is something else simply preventing the player/app from being given write permission (settings or some other problem), even if the user selects to allow it.

- See if the 'com.YourGooglePlayID' folder is actually ever created in the \SharedData folder on the Android device. If so, check its contents to see if perhaps something was mislabeled or did not contain valid data.

I think you're on the right track with what you're trying to do. I'd just consider the 'Route' filenames your value prefixes (leaving off the '.txt' since you don't need it) and the values stored line by line in the same way you'd use WriteLine(). If sequential order is important for the dataset, then just add another number to the end of your 'Route' prefix value to keep them sequentially ordered and searchable (again though, only if that is important for your project).

Edit: Forgot to mention that since SharedVariables store everything under your GooglePlay ID, make sure to additionally prefix any stored values with some kind of character identifier for your specific app... especially if you plan to create other apps that need to store values also. That way, you avoid conflicts between your apps all trying to share the same \SharedData folder. So instead of just 'Route_A_B_C', something like 'MyAppName-Route_A_B_C'.

Unfortunately, this is an issue that all app developers must contend with. The changes probably broke more general purpose apps (pics, music, nav, files, etc) than it did games since games tend to use the sandboxed storage anyway. What you're trying to do sounds like a good fit for AppGameKit, just a change in how values are managed and stored in Android 10 and up in a way that is preserved even if the app is uninstalled.
SW3DG | Evochron
Back to top
Profile PM Website
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 25th Jul 2020 21:14
Link

The redundant write permission check confirmed that permission was indeed granted (i.e. no message appeared to say otherwise).

The Android 10 device didn't actually have a SharedData folder. The older devices have theirs located at [/storage/emulated/0/SharedData]

However, your mention of a Google ID has got me thinking though. I have not been using a GooglePlay ID at this stage of development (tbh I don't know if I would put this on the store unless it added value that couldn't already be found there in other apps. I'm certainly no professional programmer
or developer and get very apprehensive at the thought of putting out substandard software. It reflects badly on TGC when this happens e.g. some of the shockingly bad GameGuru titles that are essentially the bundled assets dropped in an environment, compiled and put on Steam.).
Anyway, back on track I am leaving the Google credentials blank but filling out the required Package field in the compiler window in the com.mycompany.my_app format. This package name then appears as the subfolder title within SharedData on the older Android versions, but not on Android 10
which doesn't even create the SharedData folder.

I'm sorry to come here and be awkward but it seems I've found a phone that was a politician in a former life - Yes, you have permission and you absolutely don't.

P.S. Could Google be blocking apps that aren't signed and aren't from the Play store from writing to external storage ?
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 25th Jul 2020 22:05
Link
It's certainly possible, might be beyond my level of understanding how that system works though. It could simply be that without a GP ID, Android 10 doesn't know where to route a shared data folder, so it just nulls out. No error would be thrown back, but nothing would happen when trying to store or retrieve values.

I would still expect it to work through the player at least though, since that does use a signed and ID'd app with an assigned '\com.thegamecreators.agk_player2' folder. So if that doesn't work, there may be more to explore to try and figure out a cause.
SW3DG | Evochron
Back to top
Profile PM Website
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 25th Jul 2020 22:58 Edited at: 25th Jul 2020 23:19
Link
Yeah I think I'm beat on this one. I keep going back and probing around but not getting anywhere. The info. online says that scoped storage was intended to improve security and reduce app clutter, providing the usual app private folder
which requires no call for read or write permission to use and is accessible only by the parent app itself. Additionally a shared data folder should be created that the app can store content to for public data access without blanket
permission needed for an app to access photos, videos, media folders etc. Sounds well and good but the shared folder isn't getting created, not even by AppGameKit Player when the code is broadcast to it, which as you say is a signed, Play Store
app.

I wonder if the AppGameKit devs themselves want to take a stab at it ? Any takers ?

Thanks once again for your time and help, very grateful.

EDIT: I removed the permission request from the code, ran AppGameKit player with Storage permission denied and broadcast the app. Result was that a permission request popped up anyway but even if you deny this the code I'd posted above
manages to pass data to storage with SaveSharedVariable() and load it back with LoadSavedVariable() and putting it back into the apps respective variables. However, no shared folder is created and the file is not indexed in search.
I can only conclude it gets written to the apps sandboxed private storage again, isn't externally accessible - I give up.
Back to top
Profile PM
Scribble
7
Years of Service
User Offline
Joined: 2nd Apr 2017
Location:
Posted: 26th Jul 2020 00:01 Edited at: 26th Jul 2020 00:03
Link
Just to add to what SAR1 has said. Even if the app is using all the valid AppID, uploaded to Google Play Console and installed via Google Play or even via apk, the Permission bug is still there.
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 26th Jul 2020 00:58 Edited at: 26th Jul 2020 02:14
Link
SAR1 wrote: "However, no shared folder is created and the file is not indexed in search.
I can only conclude it gets written to the apps sandboxed private storage again, isn't externally accessible - I give up."


An interesting test might be to uninstall the player, then reinstall it and immediately just try to retrieve the stored value. Both with and without write permission being granted. If it still exists after uninstalling the player in either permission condition, that would indicate that the value is indeed at least getting stored/recorded in a persistent location, but it's just being logged in a location we can't find.

BITBITBIT wrote: "Just to add to what SAR1 has said. Even if the app is using all the valid AppID, uploaded to Google Play Console and installed via Google Play or even via apk, the Permission bug is still there."


Which bug is that? As far as I can tell, it is still properly reporting whether write permission has been granted to the app or not. The problem rests with Android 10 in how it then treats the write permission. In API 29 and higher, write permission only grants access to shared variables and certain limited scoped locations, it does not allow for external file read/write access. So it's a change in OS behavior rather than an error or bug with write permission conditions.
SW3DG | Evochron
Back to top
Profile PM Website
Scribble
7
Years of Service
User Offline
Joined: 2nd Apr 2017
Location:
Posted: 26th Jul 2020 04:35 Edited at: 26th Jul 2020 04:51
Link
Sorry for being unclear. What I meant was the RequestPermission( "WriteExternal" ) will still not grant permission to write to external directory even with all the AppID filled in and doenloaded from Google Play, even if the CheckPermission confirms the permission is granted.

Now the RequestPermission seems to be pointless, because SaveSharedVariables and LoadSharedVariables still able to save and load even if the user denied the permission, while suddenly not allowing writes to external. Not a very transitional transition it seems.

However, from my test on Samsung Galaxy S10e running Android 10, it seems that a "SharedData" folder was successfully created along with the game folder and the saved variable inside it. It was located in the base directory /storage/emulated/0/.
This seems to differ from what SAR1 observed. I wonder if anybody else can try it too?

EDIT: It seems that the shared variables will only be saved in "Shared Data" if it's broadcasted from AppGameKit broadcast and ran by AppGameKit Player.
If the app is installed using .apk, it seems that no file or folder was created.
The 'saving' seems to be successful in-app, but if the app is closed, and another app tried to 'load' the variable, it will fail to load and revert to default value. This happens even if permission was given.
Back to top
Profile PM
Scribble
7
Years of Service
User Offline
Joined: 2nd Apr 2017
Location:
Posted: 26th Jul 2020 05:06 Edited at: 26th Jul 2020 10:21
Link
And, to add to the confusion, it seems that there are 2 'types' of 'requestPermission'. Note that this was observed with exported .apk (not uploaded to Google olay)

1st type: Prompt triggered by the AppGameKit RequestPermission command
2nd type: Permission promlt will still be triggered even if the RequestPermission command was not called in the app

For the 2nd type, no matter whether we give or deny the permission, the 'saving' seems to be successful, but no saved varible files or folders was created. (App installed by exported .apk)

For the 1st type, no matter whether we give or deny the permission, the 'saving' still seems to fail, and no variable file or folder is created. (App installed by exported .apk)

The only occasion the SaveSharedVariables and loading works, is when the app is broadcasted to AppGameKit Player.

At least, that is what I observed. I wonder if anybody else observed different behaviour?
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 26th Jul 2020 05:08
Link
BITBITBIT wrote: "Sorry for being unclear. What I meant was the RequestPermission( "WriteExternal" ) will still not grant permission to write to external directory even with all the AppID filled in and doenloaded from Google Play, even if the CheckPermission confirms the permission is granted."


That sounds correct. The change in behavior has to do with the OS, not AppGameKit or its permission request system. Just to restate, external file access is no longer available in Android 10 using traditional permission/write systems available in Android 9 and older. This is a forced requirement of Android 10 and higher. Permission is now granted only for the new restricted variable/write conditions. But the mechanisms to request that permission and checking if it has been granted or not are still the same as before. And both also still function as before within AGK. The only change is on the OS side of things and what it allows with that permission.

BITBITBIT wrote: "EDIT: It seems that the shared variables will only be saved in "Shared Data" if it's broadcasted from AppGameKit broadcast and ran by AppGameKit Player.
If the app is installed using .apk, it seems that no file or folder was created.
The 'saving' seems to be successful in-app, but if the app is closed, and another app tried to 'load' the variable, it will fail to load and revert to default value. This happens even if permission was given."


Interesting results. Are you side loading that APK? Or are you installing it through Google Play? Perhaps there are tighter restrictions on side loading in API 29 and higher. For both, managing variables works in both the player and in-app, but I've only used the Google Play signed APK method for these tests, not side loading.
SW3DG | Evochron
Back to top
Profile PM Website
Scribble
7
Years of Service
User Offline
Joined: 2nd Apr 2017
Location:
Posted: 26th Jul 2020 05:15
Link
For the SaveSharedVariable tests, I've only used the broadcasting to Player, and also exported .apk.
I've only used Google Play to test the WriteExternal permission along with writing files to external.

Just to double-confirm; you have tested the SaveSharedVariable that was uploaded to Google Play with all the Appid, and a SharedData folder was successfully created, along with the saved variable file? And it can be loaded?
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 26th Jul 2020 07:01
Link
AFAIK, users of my games with Android 10 devices can have data backed up using the SharedVariable commands with write permission enabled (TGC uses this for their own in house driving test app for mobile and has suggested it as an option for API => 29). What I can't yet confirm is that any particular folder is created in Android 10 nor where the values are kept. So further information in that regard could be very helpful in knowing -if- those values can be preserved (by saving them in an app, then uninstalling the app, then reinstalling it and attempting to retrieve the values). If they are saved, then we just don't know where that data is being held on Android 10 devices. It could be (as with other Android releases in the past) that the folder route/location to saving data is simply different in API 29 than it is in previous releases and we might just need to figure out where.

If values are just kept in a sandboxed app folder, then it's likely the values would be lost on uninstallation, which would defeat the primary backup purpose of shared variables entirely. But I haven't had reports of that being lost yet, although migration to the shared variables is fairly recent for my projects as I just ran into this problem myself about a month or so ago.
SW3DG | Evochron
Back to top
Profile PM Website
Scribble
7
Years of Service
User Offline
Joined: 2nd Apr 2017
Location:
Posted: 26th Jul 2020 13:52 Edited at: 26th Jul 2020 13:53
Link
I see. Looks like there's still inconsistencies with the shared save variable locations, and with permissions. I guess I will wait until next version of Android to let things stabilize first. Please do tell us if you guys had any more info. Thanks a lot for the enlightening infos.
Back to top
Profile PM
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 26th Jul 2020 16:38
Link

Sorry for the delay, the Android 10 S9 isn't mine - I have to keep borrowing it for testing.

So here are a few permutations:

All via broadcast to AppGameKit Player.

* All data wiped, clean AppGameKit Player install - Run to confirm all data is removed.

* Run with SavedVar write of data - App requests permission - Permission denied - Data is saved and reloaded

* Following the above, re-run with NO SavedVar write - App req perm - Perm denied - Data is reloaded

* Uninstalled + reinstalled - Run with NO SavedVar write - App req perm - Perm denied - No data can be loaded, i.e. did not persist after uninstallation.

* All data wiped, clean install - SavedVar write - NO perm requested by app, request comes from from Android - Request denied - Data is saved and reloaded

* Following the above Uninstalled + reins, no data wipe - NO SavedVar write - NO perm requested by app, request comes from from Android - Request denied - No data can be loaded, i.e. did not persist. after uninstallation

* Repeat as above but with Android requests allowed - Again, data did not persist uninstallation.


Finally, if the Storage permission is allowed through app settings (so no request is made at runtime) the data does NOT save and reload, even in the same session.
If the permission is revoked in app settings the save and load IS functional AND persists between app runs, even if the request for permission is denied at runtime.

It seems that if Storage permission is not set then the data is written to somewhere that is accessible to the app and persists between runs but not
following uninstallation (probably not the apps sandbox folder then, but where if not ?). If it is set to allow, then it attempts to write somewhere that it
can't but no error is shown.
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 26th Jul 2020 19:44 Edited at: 26th Jul 2020 19:47
Link
Interesting, thanks for the results. So it seems there is no persistent saving at all for Android 10 (post-uninstall), which would defeat the purpose of enabling write permission and then using SharedVariables. If this is a limitation of AppGameKit, then hopefully it's something they could remedy. Otherwise, something is not working as it's supposed to when it comes to ShareVariables. That's disappointing to hear that SharedVariables can't be preserved through uninstalls as that's specifically what they're supposed to be there for

Maybe it's just a matter of AppGameKit needing to point to a correct (new) external location in API 29 and higher when saving those values? Apparently, in Android 10, you can manually select to uninstall an app, but keeps its data. So there is that manual option if nothing else, but it has to be remembered and managed by the user.
SW3DG | Evochron
Back to top
Profile PM Website
SAR1
5
Years of Service
User Offline
Joined: 9th Feb 2019
Location:
Posted: 26th Jul 2020 21:22
Link

Agreed, this is a set back. I can confirm in my hands at least that Share Variables will only save and load if External Write permission is denied. Even then their save location is hidden and protected from access outside of the app, as would be expected if they are saved in the apps' own sandbox folder. They do not persist after the app is uninstalled. If External Write permission is allowed, the variables cannot be saved and loaded again even within the parent app. To me it seems that with permission granted the write path is set to a location that is illegal under Android 10+ scoped storage. I think to fix this requires a rework in how AppGameKit interacts with the file system for the more recent Android versions, preferably in a way that is legacy friendly, so not asking too much there then haha.

Does anyone know if the AppGameKit devs are aware of this issue ?

Thanks again everyone for helping to try figure what this issue is. Think it is out of our hands at this point unless anyone stumbles on a workaround.
Back to top
Profile PM
SFSW
21
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 26th Jul 2020 22:03
Link
I had brought it up in an inquiry to TGC when I encountered the file read/writing problem originally, but the SharedVariable route was the suggested solution. So I'm not sure if they are aware of it not working in API 29 or higher. Probably worth an official bug report with details on GitHub.
SW3DG | Evochron
Back to top
Profile PM Website
Back to top

Login to post a reply

Server time is: 2024-04-19 07:09:50
Your offset time is: 2024-04-19 07:09:50