Computational Photography and Culture

One of my ongoing pet topics is photography's place in culture, popular and otherwise.

Let's start with a quick review of some of the basics. This isn't my stuff at all, this dates back at least to Susan Sontag in the 70s. There's this notion of a photograph's indexical relationship with the stuff being photographed, there is a physical relationship between the two. Light smashes into the world, bounces back through the lens, and crashes into a piece of film where it physically changes crystals of stuff. Some chemistry later, we have a physical chunk of film which bears the literal imprint of the world on it. A few more steps of the same sort and we have a print.

Note: the word indexical here is being used in a specific sense that has to do with photographs. It is a term of art, here, and the dictionary definitions you are likely to turn up in a quick search are actually unrelated. I apologize for the usage, but it's actually just the right word, and means exactly the thing I'm talking about. If you google it together with the word photography you'll get even more eggheaded wanking than I am writing here. Sorry.

In the 1970s and still today we treat this as important. It gives a photograph a very specific property of truth. Of course manipulation has always occurred and so on, but in the 1970s there was a presumption of a certain literalness to the print. What we see in the print is (usually) a literal rendering of what was in front of the camera at that moment, for whatever that's worth.

So, along comes digital. The physicality kind of drops out, because we enter an electronic/digital world almost immediately. Pictures are now mainly viewed on screens, and when they are printed they've taken a tour through a non-physical world.

Up through the previous century the idea of indexical was synonymous with no human actor. In practical terms, only way to make a picture that did not involve a human's translation of the scene was to take a physical photograph, with film, light, paper, and so on. When digital came along, we lost that physical connection, but the picture is still made without a human actor transforming it, as with a drawing or a painting. There is a very very strong analogy with the indexical, physically related, photograph. The analogy is strong enough that almost nobody noticed the change, or the loss. It may turn out to be irrelevant.

As it stands today, the analogy is pretty literal. Light still smashes about, bouncing off objects and through lenses, but now it lands on a sensor. Digital systems for handling this, in contemporary cameras, preserve the analogy with film quite closely. We record pixels, and retain the pixel-to-pixel relationships. The one at 1224,2788 is next to the one at 1225,2788 and they both originated with light bouncing off an object at two points next to one another on the object. The thread of literal connection remains, although it is no longer really physical.

Computational photography is going to separate this further. There will still be no human actor. There will still be no physical connection. But that digital zone in between will now bifurcate (at least) into "traditional" digital photography with literal pixels in specific X,Y relationships, and this other thing, where the resulting array of pixels is calculated from other arrays of pixels. The literal thread is lost, or at any rate extremely complicated.

The resulting pictures will look much the same, however. So we can consider, really, 4 different very very similar looking pictures:

• A traditional silver photograph (indexical)
• A traditional DSLR-made photograph (not indexical, but with a strong analog)
• A computed photograph (not indexical, with a weak analogy to indexical)
• A photorealistic drawing made by a person (not indexical, not analogous to indexical, introduces a human actor)

Of course if the viewer doesn't know what he's looking at, he's probably make whatever assumptions he makes by default about something that appears to be a photograph.

If the viewer does know, we know for certain that she'll view the first and the last as quite different objects. So far it looks like the first and second are viewed as more or less equivalent. What about #3, though? To my eye, it could go either way. Is it indexicality that matters here? Or the human actor? The human actor is surely much more important, but we're going to find out if indexicality matters. We believe, to some degree, in MRI and ultrasound imagery, which might give us a hint here. Computed photography is a lot like an MRI picture, but looks like a regular photo.

Do we treat it as more like a regular photo than an MRI, but otherwise essentially "true" (Probably)? Or is there some lurking uncanny valley territory here, where we trust the MRI specifically because it's unfamiliar material, and similar methods applied to "regular pictures" will generate some sort of backlash (Maybe?)?

Note that this is entirely separate from issues of manipulation. Digital photography has made photoshop and manipulation of pictures ubiquitous, which is definitely affecting the public perceptions of truth. Understanding changes in photography's place in society is going to be difficult, as the issue above and the issue of manipulation are inextricably tangled up together, despite being quite different.

At present, people assume that photographs are essentially all faked, or at any rate could be faked. We are distrustful of all pictures. I think to a large degree, though, we are still trusting in the indexical nature, even though that is lost. Photographs are no longer indexical with reality, but there's still that strong metaphor of indexicality remaining.

We have a vestigial sensation of that truth remaining, somewhere. Somehow, we feel that even these digital things on our screens ought to be as literally true, in the same ways, as the prints of the 1970s.

How do I know? Because we still get a hue and cry when a photo is revealed to be faked. Even though any properly cynical person assumes that they could all be faked, we're still upset and irritated to learn that a photo is faked, which reveals some sort of unconscious trust, or wish, for the photograph to have that underlying truth. We're much less upset to learn that a photo was cropped, or shot at just the right moment, to misrepresent the scene. But erase a mailbox from the background, and we are, as a culture, angry.

I have a vague theory that might address, to a degree, why.

Our vision is much like a computational photography system. It takes scrubby fuzzy terrible incomplete data from two badly made optical instruments, and computationally merges that data, interpolating and extrapolating out a sort of HD movie which is our conscious perception of vision. Educated people know that it's wildly unreliable, and we've all watched the videos where the bear runs through the frame and we don't even see it. Even unsophisticated people understand, I believe, at some level that their vision is not to be trusted.

The still picture, be it a painting, etching, photograph, whatever, can be handled well by our vision. We have time to scan it, to soak it up. Our visual system is given a gift, a scene that does not change, that has finite boundaries, that is graspable. Over time, we can literally see it all, we can extract every element, and we can build a complete and accurate picture of that still picture, something we cannot do with the real world.

If that picture has an indexical relationship to some part of the world, if it is a photograph, then we have at some remove entirely grasped a slice of the world. That's kind of cool, that's different, that's something we could not do 200 years ago, no way, no how. While we could grasp a picture entirely in 1800, that picture had been seen by someone's unreliable vision, and translated. We could not entirely trust it.

This is eroding. I don't think we've grasped it viscerally, although we generally get it consciously. Our hind brain still revels in the novelty and safety of these "real" pictures.

Computational photography will, I predict, erode this further. Not to suggest that people will grasp the details, but gradually the knowledge that there's been another step inserted, that we're another step further away from the real world, will soak into the culture.