Vision Machines

Over at Disphotic, Lewis Bush is writing about machine vision and worrying about the consequences.

I think it's a sort of interesting thread of thought. I imagine myself a Wodehouse character, pottering around and poking at it with some sort of absurdly ornate walking stick.

Lewis seems to be a little concerned that man will be replaced by machine in the very idea, the act, the mysterious cloud of whatever stuff surrounds the act of looking at pictures. I think.

I'm not convinced that this is "seeing" in any philosophically important way. The best the machine can do nowadays is basic feature identification, and we're not going to be doing any better for a long time, because we don't even know what seeing in the human sense is. It's more than feature identification, that's for sure, but our knowledge stops pretty goddamned abruptly and thoroughly at that point. I don't think anyone has an actionable plan for moving past this particular stop sign.

People seem to be pretty focused on fMRI scanning, which is very cool looking but is almost certainly not going to actually tell us anything, the code is not going to be cracked with brain scanning. The data we gather might prove useful later, but there's one or more deeply fundamental things we're missing which we don't even understand the vague shape of.

So, the ability to identify 7-UP cans, BMW automobiles, and long guns in photographs is certainly going to keep the spooks and the ad men excited and busy for a while, and it's probably going to, somehow, make our lives incrementally more miserable. But it's not some important change in the place and role of photographs in society or philosophy.

Lewis has a little bit of a tendency to concern himself with the more chic lines of thinking, and wanders off in to concern about the surveillance state. On the one hand it is a near certainty that the spooks are beavering away at exactly this problem. They are, almost without question, grinding through all the photos on flickr and facebook and so on, trying to find, I dunno, some damn thing.

Lewis' concern is, in my estimation, overwrought. The spooks are also, surely, arguing about what to look for, fiddling with algorithms, and finding either nothing or too much of something. Whittling a trillion photos down to a billion, or to zero, isn't particularly useful for anyone or anything. Well, not typically for government purposes. You can target missiles based on a billion hits if you have a lot of missiles, but you're gonna make even fewer friends than the first world nations have been making of late. You can target the hell out of ads, though...

The spooks, as they often do, will probably find the metadata much more interesting. The where and the when data, gently tied to an object or person in the frame, might prove interesting to them. Watch for cameras and phones adding encrypted metadata to your pictures, "manufacturer proprietary data" which turns out to be cell tower identifiers or something. The sharing/digital community as a whole is leaning on people to attach GPS data already.

Back to machine's looking at pictures, though.

Does the fact that photographs are, or will be, "looked at" in some loose sense more by machines than by people somehow important? I feel Lewis' thesis here, in a vague sort of way, and it does feel important. It's a shift, certainly, another sign and omen that we're not sitting on the floor going through a shoebox filled with snapshots any more.