Unlike Funnell, I load everything in and then hide what's not currently being used. It's not really about right or wrong but about what works for your game, whatever causes the least problems. I suspect that it's more to do with what you think is easier to manage, hiding stuff or deleting it.
However, I don't literally load in everything at once (this really will cause problems). My general approach is to look for the natural breaks in the game - such as transitioning from the main menu to the start of the level - and this is where you can delete and load media, as the player will expect a bit of a pause anyway.
Normally I split the media up in the following way:
intro media - deleted when the player exits the intro and goes to the main menu
main menu media - deleted when the player starts playing the game
media that is used throughout the entirety of the game (bullets, explosion sound effects) - is only deleted when the player exits the game and goes back to the main menu
level specific media (map, npcs) - deleted when the level is over
You might have to experiment a bit to find what works. Personally, I'm quite flexible with how I approach each project and do whatever creates the least amount of work.
