Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Studio Chat / Sound bug when exporting to Android - Stuttering and frame drops

Author
Message
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 14th Oct 2024 18:43 Edited at: 16th Oct 2024 19:08
Hi,

recently I noticed stuttering while playing Space Haze. This was not always the case. I did some testing and found out it's sound-related.

If I export the following code to Android OR broadcast it to my phone, the app stutters as soon as the sound is played.
The faster I tap the screen, the more it stutters. At some point, the sound even breaks.
This is only the case for Android, not for Windows.
I tried different phones, different sounds, different .wav formats and .ogg files...makes no difference.
I also tried Studio and Classic.

Here's the code:


Here's the GITHUB link to the new AppGameKit repo including a ZIP file with source code and media:
https://github.com/TheGameCreators/AGKRepo/issues/26

Can anyone confirm?

PSY


PSY LABS Games
Coders don't die, they just gosub without return
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 14th Oct 2024 19:51 Edited at: 14th Oct 2024 19:52
Yes, this is a well-known trait with Classic and Studio which made it an impractical language/tool for mobile game development due to severity, inconsistency, and no viable work-arounds. The problem will vary depending on device and performance (oddly, some much older Android phones/tablets suffer less with the problem than newer ones). The problem was reported over the years, but wasn't addressed. It comes down to the fact that sound playback is single threaded and thus, hinders processing to play each audio clip and for every instance. So when more sounds/instances play, the problem compounds through increasing the resource load causing framerates to stutter/stop and other performance related issues.

Some additional reading on the subject:

2022: https://forum.thegamecreators.com/thread/228564#msg2671149

2017: https://forum.thegamecreators.com/thread/219938#postHolder

Github issue #680 in 2020: https://github.com/TheGameCreators/AGK-Studio/issues/680

Edited to add quick links...
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 14th Oct 2024 21:53 Edited at: 14th Oct 2024 21:53
I also have this issue and there is no fix as far as I know. My Samsung Note 9 doesn't have the issue but all my newer phones do. If I monitor FPS with sound off the frame rate is consistent, if I turn the sound on the framerate dips when a sound is played.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 14th Oct 2024 22:38
ARGH. That sucks. Big time.

So no more Android development with AGK...

Thank you guys!


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 14th Oct 2024 23:18
I'm wondering if this is something that Preben could fix if we reached out to him and perhaps he could do it via is Patreon page so that he would get paid for his time. This is a major bug that needs fixing.
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 14th Oct 2024 23:45
Indeed, it's certainly something fixable now with the codebase accessible. Although my time is limited, I'd be willing to help fund such a development objective.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 15th Oct 2024 00:41 Edited at: 15th Oct 2024 00:41
Same here.
Good idea!


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 15th Oct 2024 08:11
I have sent Preben a PM. I don't use Discord so if either of you want to reach out on there to him we may get a response a bit quicker.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 15th Oct 2024 12:28
Done


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 16th Oct 2024 03:18
@PSY, can you add this as an issue to Github and include your sample code and media so that the issue can be replicated? I think this is probably the most important bug to fix at the moment and I'm hoping that either Lee or Mike maybe able to help. I did have a look at it a couple if months ago but It's beyond my current knowledge unfortunately.
Preben
AGK Studio Developer
20
Years of Service
User Offline
Joined: 30th Jun 2004
Location:
Posted: 16th Oct 2024 10:02
Thanks for considering supporting my work, you can do this here: https://www.patreon.com/plemsoft
There is a few things for AppGameKit you can download if you do so. I do take suggestions from supporter for new features they like added and I also have a large list of suggestion but not any for AppGameKit, looks like mainly GGM users is supporting my work.
No guarantees, but the more that like a new feature added the higher it goes on the list, so you should only support my work if that's what you want to do.

Took a look at this here: https://developer.android.com/ndk/guides/audio/audio-latency#output-latency

So what you could try:

Loop a silence wav to avoid audio circuit warm up latency.

New devices can be slower if the sample rate do not match the Android native sample rate, use the native sample rate of the device: 44.1 kHz for older devices and 48 kHz for newer devices. This so no resample of the data is needed.

But mainly: PlaySound is set so it use signal processing that are slower, this is done to support changing Rate and Balance.

@Zaxxan this is what you could do to make playsound faster:

in: uint32_t cSoundMgr:latformCreateInstance( cSoundMgr::cSoundFile *pSound, int iVol, int iLoop, int iPriority ):
Change:
const SLInterfaceID ids[3] = {SL_IID_ANDROIDSIMPLEBUFFERQUEUE , SL_IID_VOLUME, SL_IID_PLAYBACKRATE};
To:
const SLInterfaceID ids[2] = {SL_IID_ANDROIDSIMPLEBUFFERQUEUE , SL_IID_VOLUME};
const SLboolean req[2] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE};
result = (*engineEngine)->CreateAudioPlayer(engineEngine, &(pSoundInst->bqPlayerObject), &audioSrc, &audioSnk, 2, ids, req);

Will make PlaySound faster but you will not be able to change Rate and Balance.

FYI: There is already a iPriority you could use to activate this, so you decide if you want PlaySound with Rate and Balance or the fast way iPriority=1.
agk:laySound( uint32_t iID, int iVol, int iLoop, int iPriority )

Hope it helps. And thanks for considering supporting my work
Subscribe: https://www.youtube.com/@MakingGames
best regards Preben Eriksen,
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 16th Oct 2024 11:43 Edited at: 16th Oct 2024 13:58
Hi Preben,

Thanks for replying so quickly and thanks for the advice, I also use GGMax but have not used it for quite a while as I've spent most of my time on AppGameKit, I know you spend a lot if time on GGMax and have made significant improvements to it it, if you could find time to spend some time on AppGameKit that would be great.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 16th Oct 2024 14:09
Hi,

I added the issue to GitHub. I'm a total noob regarding GitHub, so feel free to edit
I also replaced the download link with the GitHub link in my original post, as the link to the source was HTTP instead of HTTPS, triggering security issues when trying to download.


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 16th Oct 2024 15:12 Edited at: 16th Oct 2024 15:13
@PSY, Sorry my fault, I should have said to add it to the new repo

https://github.com/TheGameCreators/AGKRepo

The old studio repo is for information only now.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 16th Oct 2024 19:07
Oh, okay, no prob

Done, modified the link in my original post, it now links to the new repo.


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 18th Oct 2024 11:41
I've tried Preben's suggestions by resampling my sound effects to 48KHz and I made the suggested change to the code but unfortunately I am still getting the frame drops when a sample plays which creates micro stutter.
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 18th Oct 2024 20:02
Yeah, it appears that resampling isn't the full scope of the problem, stutters and delays still occur regardless of the sample rate of the sound file. Years ago, I tried all kinds of different sample rates and formats, none seemed to resolve the issue. It needs the same kind of audio handling the PC method has (or whatever technique/method other Android games employ).
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 18th Oct 2024 21:06
Yeah, I tried pretty much everything before I posted, no sample rate worked. I also tried Mono/Stereo and .ogg, same issue. Tried sounds longer than 1s, same issue. Background music is also playing all the time, so the audio circuit is always warmed up


PSY LABS Games
Coders don't die, they just gosub without return
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 18th Oct 2024 21:43
I've been reading up on this and it does look like the only solution is to put the sound on a separate thread as previously mentioned. I'm now reading about how to implement multiple threads on Android.
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 18th Oct 2024 22:49
That's the conclusion I originally reached also, the single threading of playback was the root cause. Hope you find a solution that's easy to implement.
Virtual Nomad
Moderator
18
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 19th Oct 2024 00:21 Edited at: 19th Oct 2024 03:19
i'm ignorant on this kinda stuff but seeing the recent FMOD and BASS dll's made me wonder if a DLL for android to handle sound could be a solution? is this along the lines of the previous plan to address the issue? add: i assume this means there's no notion of dll/plugins for android?

also worth mentioning, and this touches on a whole other can of worms, but when some were hitting close to api deadlines, i'd come across services (including a free one) that "converted" HTML to APK (and AAB) with the latest/required APIs (where it wasn't looking like TGC would meet the deadlines). add: THIS

i know there are sound limitations in HTML (including changing play Rate as mentioned by preben) but if devs didn't care about those limitations, i wonder if an HTML>AAB approach could be a solution? i'd guess there would be a performance hit in having another middle man but if the app wasn't too demanding in the first place, running an html app as an apk might help somehow? trying to replicate the bug in an HTML version of the game played in an android browser would be a good place to start?

another thought: do these issues exist with the Music commands which are somewhat interchangable? i expect others have tried but don't think i've seen specific mention/results.

finally, and no offense to preben, but i wouldn't contribute to a "maybe/if i have time" solution. if there was a sound fix (pun intended) that would eliminate the issue altogether where the single-threaded approach since AGK's inception was flawed, i'd kick in to make it go away (as i would for many issues). ie, if it was as simple as described above, i gotta believe it would have been done in the last 15 years.
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 19th Oct 2024 03:35 Edited at: 19th Oct 2024 19:24
Indeed, I'd guess it might not be very simple to fix (not like it is on PC anyway perhaps), but it seems most Android games employ some method for reliable sound playback with consistent performance (maybe many of them use some kind of audio library for Android?). I'm focused on desktop exclusively these days with AGK/S, but I would support continuing the effort for Android to maintain a well-rounded (broad) approach to AGK/S's usability as a development language. And this has been one of the biggest stumbling blocks keeping it from being viable for that mobile platform.

I'm not aware of plugin options for Android, but it is technically just Linux under the hood, so perhaps there is an option via that route. But a native solution would definitely be the preferred way to go for ease of use and proper functionality within the language itself, imo.

IIRC, I did test the music commands and encountered the same kind of stuttering when loading and playing back. With music though, you often load a song and then play it indefinitely, so there aren't as many stop/start actions as there are with sound effects. The impact is less in that regard, but it would be nice to have a sound system that doesn't impact performance overall.

I would also be hesitant for a 'maybe' solution, something committed and dedicated to make it work is what I'd be interested in supporting.
Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 19th Oct 2024 08:22 Edited at: 19th Oct 2024 10:40
I agree if it was a simple fix it probably would have been fixed by now. If I remember correctly didn't Paul allude to a library needing to be replaced? I think it's unlikely that Preben is going to fix the issue as like he said all of the requests he has are for GGMax so it's unlikely that there will be enough AppGameKit users who would contribute for Preben to even consider fixing the issue.

I have written several multi threaded apps in vb.net so I do have some experience of multi threading but none with Android. I will take a look, I don't hold out much hope though, if Paul couldn't fix it then it's doubtful I can.

Update: I think that the Android NDK that AppGameKit uses implements a version of OpenSL ES for audio which was deprecated a long time ago and is not recommended for new applications. The recommendation is to use Oboe or AAudio which I would think would require a major code rewrite so is probably beyond what we could do. Perhaps this was what Paul was referring to when he mentioned updating the library.
agkcoder
19
Years of Service
User Offline
Joined: 12th Dec 2004
Location: AGK Forums
Posted: 22nd Oct 2024 10:10 Edited at: 23rd Oct 2024 15:23
I came to the same conclusion as Zax that the API is outdated. Well, even if AppGameKit is in maintenance mode, they should also upgrade/maintain the sound API on Android...asking for the sound working properly is not really any kind of feature request. Anyway! I also posted my own findings that I gathered during battling AGK's sound limitations on Android. Posted them in the Android section, hope it helps some of your situations or give some ideas for your own better solutions while we're dealing with the current implementation.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 22nd Oct 2024 23:37
I found a solution that works for me.
I converted all ingame .wav files to .ogg and made sure they are longer than 1 second ( I just added silence with Audacity ).
I kept a couple of .wav files that are only used in the options menu, as the frame drop isnt visible in the options.

I now use PlayMusicOgg() with all the .ogg files. No matter how fast I play them, there are no frame drops at all.

If I want to change the system sound volume, I don't use SetSoundSystemVolume(), instead I'm using my own function that sets the volume manually, like:




Same goes for the system music volume:







PSY LABS Games
Coders don't die, they just gosub without return
Virtual Nomad
Moderator
18
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 22nd Oct 2024 23:56
PSY wrote: "I found a solution that works for me.
I converted all ingame .wav files to .ogg and made sure they are longer than 1 second ( I just added silence with Audacity )."

those are best practices that have been around (i've employed them for years) and while i'm glad your issues are resolved for you and your device(s), you might want to solicit testing on other devices before you put the issue to rest.

ideally, others test the old practices/version of your app and the current to compare "success" on their devices? coincidentally, a new channel was recently added to the AppGameKit discord server meant to assist playstore testing so you might solicit testing there (also a coincidence, i recently purchased an S24 and still have my old J3, so... sign me up ).

Zaxxan
Developer
4
Years of Service
User Offline
Joined: 17th Nov 2020
Location: England
Posted: 22nd Oct 2024 23:59
That's good news. I did try using PlayMusicOgg() last year but all my samples are quite short and I couldn't get it working properly but I can't remember why. I'll have to revisit it when I have time.

I'm busy trying to get Classic updated to API34 at the moment though....so close.
EdzUp
22
Years of Service
User Offline
Joined: 8th Sep 2002
Location: UK
Posted: 23rd Oct 2024 00:46
Fyi this issue comes from DarkBasic 1.x I had the same problem back in 1999
-EdzUp
Patreon: https://www.patreon.com/EdzUp
agkcoder
19
Years of Service
User Offline
Joined: 12th Dec 2004
Location: AGK Forums
Posted: 23rd Oct 2024 03:13 Edited at: 23rd Oct 2024 04:50
Quite an elegant solution PSY!

SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 23rd Oct 2024 18:46 Edited at: 23rd Oct 2024 18:47
The problem I ran into with the OGG approach was the instance-less playback. The clicking/popping was pronounced on stop/start events when sounds needed to be played multiple times as there was no layering or instancing. You could load multiple copies of every sound, say perhaps 5 or 10, for a limited trail of instances and then run your own buffer. But that would liklely have some negative performance and resource implications, and the sounds would need to be really short if played frequently. I woud also recommend trying it on multiple devices across a wide range of configurations.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 23rd Oct 2024 18:47 Edited at: 23rd Oct 2024 18:49
Quote: "those are best practices that have been around (i've employed them for years)"

I knew the problem with .ogg sounds < 1s, but I didn't know that PlayMusicOgg() would fix the stuttering. The stuttering wasn't always so severe....seems it got worse over the years, probably due to new Android / phone versions. I discovered it when I updated Space Haze to API 34.
The only way to fix the issue is using LoadMusicOgg() and PlayMusicOgg() ( with .ogg files >1s ), all other load/play sound/music commands cause stuttering or other issues.
I'll compile a test app with .ogg files tomorrow...

Quote: "I'm busy trying to get Classic updated to API34 at the moment though....so close."

Uuh, nice. Didn't know you were onto it

Quote: "Fyi this issue comes from DarkBasic 1.x I had the same problem back in 1999"

Loved DarkBasic. I joined pretty late, so I didn't have much experience with sounds back then.

Quote: "Quite an elegant solution PSY!"

Thank you agkcoder. I guess I'll update Molecools, also.

Quote: " I woud also recommend trying it on multiple devices across a wide range of configurations."

Yeah, I totally agree. I def need to do some testing first...


PSY LABS Games
Coders don't die, they just gosub without return
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 23rd Oct 2024 22:51 Edited at: 23rd Oct 2024 22:52
Quote: " i recently purchased an S24 and still have my old J3, so... sign me up )."

Very nice, thanks alot

Time to reactivate my S4 again


PSY LABS Games
Coders don't die, they just gosub without return
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 24th Oct 2024 15:25
I uploaded an OGG test code to the #google-play-store-tester-help! channel on AppGameKit Discord.
It utilizes music ogg commands instead of sound commands.

If possible, please test the code on your smartphone and share the results!

Thank you


PSY LABS Games
Coders don't die, they just gosub without return
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 24th Oct 2024 19:45
Tested on a Moto-E. As mentioned, almost no impact on framerate, but obvious stop/start clipping and no layering. With a loud screeching sound like that, it's more difficult to detect the clipping, but with a softer sound effect, it's more pronounced. The lack of instancing significantly limits sound playback capability, but this approach does resolve framerate reduction.

To manually apply instancing, here is the same code modified to provide up to 10 instances of the sound effect with playback through a simple queue system:



This works, but unfortunately requires multiple copies of a sound file to eliminate clipping and provide correct layering (the number of indexes loaded and limit would need to vary depending on the length of the sound effect). But if you're working with a very limited number of sound effects, it may be an option. Ideally (and because of the index limit differences), PlaySound should use the same instanced playback that it does on PC, leaving OGG properly aligned with single instance, single track music playback.
PSY
Developer
8
Years of Service
User Offline
Joined: 3rd Jul 2016
Location: Laniakea Supercluster
Posted: 24th Oct 2024 21:28
Hey SFSW,

thank you for your feedback
You're of course right about the clipping / layering, but the 8-bit sounds I'm using in Space Haze feel a bit "distorted" anyway, so the clipping is barely audible... you probably need to know it's there. I tried speakers and headset, and couldn't hear it.
Also, the only sounds in Space Haze that would theoretically be layered are the spawn and shot sounds, and I specifically don't want those layered, because that sounds crappy. ( in this case ).

In other Android games, where the sounds are more soft / smooth, instancing would makes sense.

But you're right, the best solution would be a PlaySound() fix.
Let's keep our fingers crossed

Thanks again for testing, very much appreciated!


PSY LABS Games
Coders don't die, they just gosub without return
Virtual Nomad
Moderator
18
Years of Service
User Offline
Joined: 14th Dec 2005
Location: SF Bay Area, USA
Posted: 24th Oct 2024 21:35 Edited at: 24th Oct 2024 22:06
Quote: "this approach does resolve framerate reduction"

the "instances" approach gives me the same FPS drop as below (i'm not listening for clipping so much as watching the frame drop).

feedback left @ the discord thread...
Quote: "if no OggMusic is playing, my FR drops to 57-58 every time i tap to play. if it's already playing and i tap again, no drop.

using the included .wav and PlaySound, identical results.

using SFSW "instances" approach https://forum.thegamecreators.com/thread/229746?page=1#msg2678344 and i get identical behavior.

GalaxyS24"

this was all via Broadcast. should it matter?

add: i thought to loop a Sound instance (of the ogg) at 0 volume; ie, This = PlaySound( SoundO, 0, 1 )
and play instances of that sound but received the same frame drop.

Music doesn't function like that and Playing then immediately Pausing 1 "Instance" doesn't help a Second instance from causing frame drop
SFSW
22
Years of Service
User Offline
Joined: 9th Oct 2002
Location:
Posted: 24th Oct 2024 22:50 Edited at: 24th Oct 2024 22:52
Behavior should be identical over broadcast versus native execution, it is the same player running both. So your results are consistent with what you would expect with a fully integrated game running on Android.

Playing OGG can have the same stutter as I recall, it just seems significantly diminished overall and more of a problem on newer phones and tablets (my Moto-E is pretty old at this point). It's definitely not a broad fix and the lack of instancing along with limited available indexes keep it from being a full solution.

The best way to measure any delay is to capture the time before and after a play event. If we modify the project code further to incorporate this option, this will provide a more accurate gauge of the stuttering than trying to measure with a framerate locked at 60:



Even on my old phone, there is around a 10-20 ms delay on a first few starts of playback, decreasing (likely due to caching) to around 2-3 ms after a while. So there still is a noticeable hesitation, but it's significantly lower than the attempt to use PlaySound (which tends to have a delay of around 100-300 ms on a first play, depending on length of sound and device state). On other (likely newer devices) I'd expect the delay to be much longer and more noticeable, even with OGG.

Login to post a reply

Server time is: 2024-11-17 22:33:07
Your offset time is: 2024-11-17 22:33:07