Our Lying, Visual, Brain

Here are some things that are true.

The first thing is the stopped clock illusion. This visual quirk reveals some interesting things about how we see. When turn quickly to look at a clock, the second-hand appears to freeze momentarily before starting to move. What appears to be going on here is that we do not actually see the clock for a moment. It takes time for our visual system to gather enough detail for our brain to construct the detailed and clear picture of the clock, and its second hand. And here's the creepy part:

Then our brain goes back and edits our memory, to create the illusion that we saw the clock clearly all along. Not knowing what to do with the second hand, it essentially paints it in place for those few moments.

Our conception of visual sensation as a sort of continuous high definition movie is in fact an illusion. It is not continuous, our short term memories are a lie. What we "see" is in fact a continuously rendered movie inside our head, built up out of memories of what we've been looking at a few moments before, ideas about what things ought to look like, best guesses and estimates. What's actually coming in through the eyes is actually pretty low resolution, it has detail only in a narrow field of view. We fill in the whole wide angle picture from a variety of sources, including our imagination.

The second thing that is true is that when we look at a picture with any kind of detail or content in it, we will discover new things about it for an astonishingly long time. Look at a picture of something, anything, for a couple of minutes. If you're attentive, you'll probably be seeing new things through the whole process. Perhaps minor details of how a shadow falls, or some minor object in the background, some detail of color.

The third thing that is true is that, presented with a picture of a decent model of some well known object, we can be fooled for a moment, or for several moments. If the model is ok, we might just do a quick double-take. If it's better, it might fool us for some seconds. If it's excellent, we might never be sure.

The details don't matter much, the point is that there are features of the model that are more and that are less important for that "fooling us" interval. I don't know or much care what the details are, but let's suppose that getting the color and overall shape are enough to fool us for a moment. Getting the shadows right might fool us for a few seconds more. There are, equally clearly, things that matter less for the purpose of fooling us.

This is surely related to the fact that we're not actually seeing what we think we're seeing. We see something that looks like the cathedral of Notre Dame, and we recognize it. It takes a little while for whatever the relevant details are to soak in, to alert us to the fact that it's a model.

So what does this all mean?

When we look at a picture, we don't actually see it. What we see when we look at a picture, or at anything else, is a collection of preconceptions, some imagination, some guesses based on other things we've seen before, with just a little bit of the actual picture's content thrown in there for seasoning.

As we look closer, over a longer period of time, we can soak up more of the content, but what we see, and more importantly what we remember, is not a true representation of the picture at all. It's a construct of our brain. A great deal of it is simply made up, a great deal more than than we'd like to think. We're remembering some visually important elements, probably related to whatever it is about a model of Notre Dame that would fool us. We're remembering an idea of the picture. We're remembering bits and pieces lots of similar pictures, and similar scenes from our own life, and they're all piling in to this construct. The picture itself is, I suppose, the single largest contributor to what we "see", and to what we later "remember", but it is by no means the only one and probably not the only important one.

Here's a quick test. In the well known photograph, "Migrant Mother", shot by Dorothea Lange, what is in the background? And how many of them are there? If you are like me, you feel that you can picture this thing in your mind with crystal clarity, but you're not quite sure about the background. If you nail that one, think of some other well known pictures, pictures you can "see" in your mind's eye in an instant. Ask yourself some relevant detail "what's in the background", "what's to the left of the tree", "does the river turn right or left as it leaves the frame" and see just how much detail that crystal clear picture in your mind's eye really has.