Ah, cognitive psychology
The people who have mentioned that we see patterns - not images - are correct (from what I learned at uni). An eye merely focuses photons on the retina, that then sends a series of electrical signals to the visual part of the brain. Those signals are more interested in contrasts and edges, than exact position and even colour.
Otherwise, we'd never be able to recognise a friend in a disco (different lighting) or from a funny angle (different shape).
Face recognition admittedly is a special case - human beings are 'hardwired' to excell at face recognition as it's so utterly essential for a social species like ourselves. From a survivial point of view, recognising who your enemy is, and who your ally is, is vitally important - it was when we lived in caves and hunted, and it is now when we sit around boardroom tables and cut deals over lunches.
Don't believe me? Then here's a face for you
If you've ever seen a human with a face like that, then I pity you - but still we all recognise it as a face. Two dots and a line are all we seem to need to recognise one. (Or shades of light and dark - you've seen faces in clouds before, I'm sure.)
Colour blindness / colour vision is usually a physiological matter - you physically have more or less 'cone receptors' to 'rod receptors' in the retina. One detects light/dark, the other detects colour (I can't remember off hand which is which).
If you have more of the light/dark ones, then you'll have less room for the colour ones. Thus - your eyesight will be better in the dark (tones of grey) but your colour vision will suffer.
Indeed, there is a community in Africa (I think, may be South America) of relatively primitive people who genetically have poor colour vision, but excellent night vision. They fish at night, under the stars because they can see so well.
Makes you wonder if amongst the 'non-colour-blind' masses, there aren't some people that have a more brilliant world of colour, but lack very much night vision at all, and others to whom colour is relatively bland, but they can see the dark a lot better. If colour blindness is the extreme, surely there are different levels?
It's also believed that the number of colours we see has not been a constant throughout the centuries/millenea. Red indeed would have been the first colour we (as a species) would have been able to discern - as it was the most important for our survival - blood is red, after all. What other colour would have been of more use?
And finally, on a much less scientifically sound line - who else here sees the extra colour in a strong rainbow, the one just past violet that looks a little bit greeny-purple?
</selfimportantwittering>