GPUs were originally created to handle 3D. Nowadays, they are capable of doing a lot of things, but 3D was their main purpose.
So, roughly speaking, if your app does a lot of 3D calculations, it will run smoothly on a system with a powerful GPU.
If your app does a lot of complex calculations, like pathfinding for example, it will run fast on a system with a powerful CPU.
Sprites and particle systems are also based on 3D, so if you do a lot with "2D" sprites, a fast GPU will come in handy.
The main difference between a CPU and a GPU is the architecture. A CPU has some cores which can handle complex operations. A CPU also has interrupts and virtual memory, which a GPU lacks.
A GPU has hundreds or even thousands of cores, which can handle simple operations like matrix calculations ( which are essential for 3D ).
Here's a link that explains the difference pretty good:
https://superuser.com/questions/308771/why-are-we-still-using-cpus-instead-of-gpus
Cheers,
PSY