Nnngh. The state of scholarship.

Here's a paper getting passed around by the usuals on social media. None of them will actually read it, but it will be cited as Necessary and Important, and over time will acquire meaning quite distinct from its content. What Does #Freedom Look Like?

If someone else would read it, and report back here — especially if I am wrong — I would count it a favor.

I read it, once. It's a bit thick on academic-ese, but in broad strokes what they did was this: They collected a set of Values (socially loaded words like "Health") from somewhere reputable. Then they gathered up the most popular hashtags on instagram, and as a team assigned one of more Values to each hashtag. Then they looked at a corpus of popular instagram pictures tagged with those hashtags.

They used a number of what appear to be rigorous methods to arrive at consensus at various points, although they seem to me to have been maybe a bit heavy-handed in scrubbing datasets here and there and throwing out things that were not going the right way. Let us stipulate, though, that their data handling, analysis methods, and statistics are all sufficiently sound. Evaluating those seems like actual work, ew, and there's lower-hanging fruit.

They then draw some conclusions with some charts and graphs and commentary.

Their goal is to connect instagram tropes (visual tropes) with Values. That is to ask, roughly, "what does instagram as an archive have to say about its user's ideas around, say, "Health" as a Value?" How do "we" (per instagram) visualize "Health?"

It probably isn't a bridge too far to propose that they want to show how instagram generates and perpetuates Gramscian hegemony. Which, on the one hand, of course it does if Gramscian hegemony is even a thing; but which on the other hand demonstrating it rigorously is, like, work.

They conclude that instagram's visual conception of "Health" is limited to fitness/workout related themes. There's a lot of p-values around, and citations, and so on, so it feels very rigorous.

But there's a problem. The study introduced the word, the Value, of "Health," it's an artifact of the method. They went and dug up a bunch of Values somewhere else, and then manually connected them with hashtags. Of the collection of most popular hashtags, the ones the team of authors connected with "Health" are: #Fitness, #Workout, #Bodybuilding, #Motivation, #Gym.

For some reason I am unable to determine, the hashtag #Health was not associated with the value "Health" according to their table. The #Health hashtag is, however, associated with the "Progress" Value. This might be an error in the table? There is probably some methodological detail I am missing here, but if so this suggests that the method might be, uh, flawed.

(Honestly, I cannot work out what the hell Table 1 in their paper IS? Maybe the associated hashtags that go with Beauty are hashtags that appear with #Beauty? So you wouldn't include #Beauty as a hashtag that appears with itself? The use of the word "term" in their method section doesn't seem to be fixed. It's also clear that hashtags like #Instagood are not in the Values terms they're starting with, because that would be insane, so there's some sort of dichotomy between Values and Hashtags in here.)

Ok, so, what they have actually determined, as far as I can see, is that instagram's visual conception of workout/fitness ideas leans heavily toward workout/fitness visuals, which is not, to my eye, a huge surprise. The connection to the Value "Health" appears to be entirely synthesized by the study itself.

(Or, possibly, what they're studying is hashtag clusters, rather than visuals, if #Health is in fact not a merely an externally introduced term, but also a hashtag they're studying)

When restricted to the most popular aspects of instagram, ideas like health care workers, proper diet, workplace safety, and whatever else you might reasonably associate with "Health" simply don't appear. These are not visually appealing subjects, in instagram's sense of the word.

This is, to be fair, something notable. Instagram absolutely focuses on specific slender tranches of culture and life, tranches which lend themselves to a specific style of visual representation. If this is the point the authors are striving to make, I don't think they've made it. Introducing the layer of Values terms and then laboriously (but rigorously!) wiring everything through them has simply muddied the waters, and produced spurious non-results.

A few other random notes:

I did not particularly dig in to any of the other 19 Values they worked with here, so I don't know if they same problems plague those, but to be honest I don't see how they can't. The introduction of the Values and the effort to study everything through that intermediate layer of analysis seems to me inherently flawed.

As is apparently some kind of requirement in this field, they include a Barthes citation which doesn't appear to mean anything. They cite Camera Lucida (natch) but I am pretty sure they mean to cite Mythologies (if they intend anything at all) based on the remark they're trying to support. I don't know most of the other cited material, but this is not a good look. I mean, it's possible that there's some side remark in CL that is actually relevant? But neither the actual thrust of the book itself, nor what are commonly understood as its theses, seem to be at all related to the thing they're trying to support.

They mention, a couple of times, the idea that instagram (and similar) encourages people to produce photographs in the same vein as already exists, that there is a feedback loop in play here, which produces an ongoing (albeit evolving) narrowness of vision around certain topics. The study in no way supports this thesis, and in fact does not pretend to, which begs the question of why they keep bringing it up. I mean, it's obviously true and a very appealing notion, but in the context of the paper it seems silly to mention it.

This is especially true given the above-mentioned flaws. If they want to assert that instagram's library of visual tropes both reflects and enforces a certain way of visualizing "Health" (and, to be honest, this is pretty clearly what they aim to do) they have quite a lot more work to do.

Overall one gets the sense that the ambition of the paper is to rigorously support something about cultural hegemony, to back up some of the commonly understood ideas about visual culture with alpha-tests, p-values, N=2000 statistics, and thoroughly cited theoretical underpinnings. In the end, though, even the very limited study they managed to bash their way through is methodologically a mess that produced, to my eye, almost no interesting result.

In the end, they have determined that on instagram, the photos align accurately with the hashtags attached to them but that trying to connect popular hashtags to general, broad, cultural values is a dicey proposition that doesn't seem to go anywhere no matter how much numerical crunching you do.

The state of scholarship in this field is really, truly, not good. I don't read any large percentage of the literature, but basically everything I touch after about 1990 seems to be really sloppy.