Btw, Ninjas were not stealthy Samurai. Ninjas were considered coweredly by Samurai as it totally defied their code of honour. While Samurai were facing each other and fighting to the death, Ninjitsu was all about shadows, stealth, entry, escapology and even impersonation.
As far as I understand it, it started from lowly towns folk that needed protection from Samurai war lords and developed hit and run tactics (in a similar way to the colonial americans used against the british), and it was very effective. But the eventual adoption and use of the ninja was more than just assassination. They were used mostly for espionage and also for kidnapping. Obviously they have some mythical mystique about them because of all the funky things they could do like block swords with their forearms (which was achieved with strips of metal sewn into their clothes) and climb up walls with spider spikes (and even across ceilings).
I mean, basically they were feared and despised, but their training and uses went far beyond assassin, and most of their legendary/mythical skills are based in some form of truth. Pretty amazing nuttjobs.