image sonification for historical inquiry, a video presentation for digital cultural history: a virtual roundtable @ the 2022 organization of american historians conference.
Below is my contribution to Digital Cultural History: A Virtual Roundtable at the 2022 OAH Conference. Presenting ideas from my ongoing research on defamiliarization tactics for historical inquiry, my talk uses one example of image sonification to reflect upon how it can bring together cultural and digital history by using adventurous computational tactics to alert historians to aspects of form and content, information and context, buried in the empirical record as it is preserved in archives. Rather than turning people from the past into things, this approach strives to activate things so that we can more effectively glimpse (or in this case hear) the people caught up in artefactual and archival representations. Embracing distortion and iteration as paradoxical paths to greater accuracy in historical analysis, image sonification has the promise of inspiring a kind of synesthetic hermeneutics, a way of bringing together close and computational reading by disorienting the perceptual sensorium through human-computer interaction to better orient ourselves toward the past: its lives, forces, stories, structures, negotiations.
A 14-minute video takes the viewer/listener through one very brief, very tentative image sonification and what even this one preliminary experiment suggests for methods that might bend cultural and digital history together in more fruitful ways. Below the video is a script for those who wish to access the talk in that format. For additional readings, essays, and information about this research, see my webpage GlitchWorks: Digital Defamiliarization Tactics for Historical Inquiry.
Script — Defamiliarizing Digital History
Michael J. Kramer, Assistant Professor, Department of History, SUNY Brockport
“Oh, my God! People back home will just never believe this!” — Mance Lipscomb to photographer Matt Hinton when given print of the “silhouette” photo, according to Sam Hinton
We see the silhouette of Mance Lipscomb against the audience in the Hearst Greek Amphitheater at the 1963 Berkeley Folk Music Festival. He’s playing an acoustic guitar in front of one microphone, with his hat placed carefully by his side. It’s a moment of cultural encounter as he performs for this predominantly white, middle-class, urban audience, encompassing issues of race, gender, class, and region. A male African-American musician and sharecropper from Navasota, Texas, Lipscomb was born to parents who lived through enslavement in the US South. Now in 1963, he takes his place center stage way out in California, on the campus of the state’s flagship university, in a structure funded by the newspaper magnate William Randolph Hearst and built in 1903 to signify the University of California at Berkeley’s claim at the turn of the twentieth century and thereafter that it was becoming a new bastion of democracy and wisdom, the so-called “Athens of the West.”
What did it mean for Mance Lipscomb to perform at the Greek on the Berkeley campus, at an institution that by 1963 was perhaps the preeminent public university in the nation, if not the world, a place of prize-winning scholars, top notch students, a place that ran the nuclear research program of the Cold War United States, among other things? This is very much a cultural historical question. Looking to material such as a photograph to try to address it is very much a cultural history move. Gaining currency in the historical profession during the 1980s and 90s, cultural history took social history’s efforts in the decades prior to study history from below, to broaden who counted as a historical actor and how, and applied them to a thing called culture. The term did double duty: the culture in cultural history stood for a turn to more aesthetic material, often non-textual, generated often by folkloric, popular, or mass entertainment people or institutions; culture also stood in for attention to value and belief systems, culture in the anthropological sense; and maybe even doing triple duty, culture pushed toward new methods of approaching the artefactual record, the archives, the empirical stuff or, for our purposes, the data or these data, through which we access the past as historians.
It is this last question of method that brings cultural history into contact with digital history, which has grown popular in many respects as a turn away from the cultural turn, a rejection of more semiotic, interpretive approaches of “reading” our historical data and a turn back to cliometrics, statistics, and a more positivist approach to the historical record. The so-called digital turn has often been placed by practitioners and critics alike as precisely a turn away from or even against the cultural turn.
To be sure, this is a simplification of the digital turn (and the cultural turn for that matter), but it’s a helpful and roughly accurate portrayal of the broad contours of these two fields as they have emerged. Yet I would contend that something has been lost in the false binary created between cultural and digital history. What has been lost is the question of how we might relate to historical inquiry what computer engineers call human-computer interaction, how we might think more critically about the ways in which human perception meets up with computational manipulation.
It’s not a choice between careful close reading by humans versus positivist, distant reading by computers. Rather, whatever scale one works at, whether one is looking at one artifact or many, whether one is pursuing network analysis, topic modeling, algorithmic approaches, or presenting digitized or born digital artifacts for shared inquiry through digital curation, cultural and digital history have the potential to come together in fruitful, even provocative, ways in service of more critically insightful (or as you’ll see, no, actually hear, in a moment, insoundful) understandings of the past and its peoples, its institutions, its issues, forces, and continued resonances in contemporary life.
Let us look again at the startling, expressive photograph of Mance Lipscomb at Berkeley in 1963. It was his second time appearing at the event, and the photograph is part of the over 10,000 photographs in the Berkeley Folk Music Festival Archive. The photograph is a visual glimpse at a sonic event, a silent rendering of one moment from the performative flow of a music concert. And it is of course already a technological creation, light captured on to film and then developed using chemical processes. Now that the photograph is digitized, it also carries in it not only the photographic technology that created it, but also the history of computer graphics software development, such as choices about how to translate it into digital data, compress it, and so on.
History here is highly mediated already. The artifact contains multitudes: a still image from a moving event; a silenced scene of sound being made; a glimpse out over the shoulder of a man whose audience is intently looking at and listening to him. Rather than only fight this mediation in the pursuit of some fantastical return to some pure present in the past, what if we embrace mediation and remediation as tactics of historical inquiry? What if we seek out ways of playing with the ductility of a digital image’s data in order to perceive what it represents in fresh, disorienting ways? By remixing, collaging, or transmediating the visual data into other outputs, for instance, we might even perceive as humans new information and aspects of what the image represents.
The image is silent, but what if by hacking existing software tools for historical interpretive ends, we turned its data into sound? To be clear, this does not mean we can magically recover the sounds being made when Lipscomb performed at Berkeley in 1963 from the image. We cannot, except perhaps in the most conjectural of senses if we took an archeological approach to the image’s venue and content, do so.
What we can do, though, is almost the opposite: not get back directly to the photograph’s content, but rather go forward, pay greater attention to its form through the ways that its digital data can be played, and played with, to produce new iterations of the artifact. Visual data can be outputted as sound so that we can ask strange, new questions, such as what does a photograph sound like?
Sounding out the visual data in the image of Lipscomb using Michel Rouzic’s Photosounder tool offers one very tentative experiment in this direction. I listened to the image’s pixel brightness from left to right using what sound engineers call pink noise. Now brace yourself: this really is a defamiliarization of the artifact as Russian Formalist Viktor Shklovsky imagined it in his 1917 essay “Art As Device,” sometimes translated as “Art As Technique.” Shklovsky wanted the defamiliarization tactic, in his words, “to impart the sensation of things as they are perceived and not as they are known…to make objects ‘unfamiliar,’ to make forms difficult to increase the difficulty and length of perception because the process of perception is an aesthetic end in itself and must be prolonged.” This sonification of image data that I did actually will go by really fast, and sound very disorienting, but notice how the pink noise quiets down right when we hit Lipscomb’s silhouette. This proved useful for me in using digital strategies to perceive something new about the cultural history of this particular artifact, this silent photograph of an intriguingly noisy cultural event out at the Berkeley Folk Music Festival. Here we go:
That is pretty far from Lipscomb’s beautiful blues playing and singing, but the disorientation and defamiliarization of the image, and of any audio recordings we have of Lipscomb from the time made in other settings, can still be quite productive. Even in this very strange rudimentary sonification of visual data, notice how it generates noise, then goes quiet as it passes over Lipscomb at the center of the photograph. The man making sound in the image is silenced here. This got me to thinking about how Lipscomb’s ability to move to the center of attention at Berkeley in 1963 was enabled, yet also somewhat undercut, by the ways in which he was a mysterious silhouette to many audience members. Defamiliarized himself from the rural community in which he typically played in Texas to this new audience in Berkeley, Lipscomb was in transit, at once linking together different strata of postwar American society yet also revealing the gap, the silences, between them.
The sonification defamiliarized the image, heightening my attention to the ways in which at the very same instance that Lipscomb appeared center stage, visible and appreciated, he was also rendered invisible. His knowledge was genuinely treated by the attendees at the Berkeley Folk Music Festival as equaling the worth of any of the institution’s most prestigious faculty members, yet Lipscomb’s own point of origin, his own experiences were also rendered somewhat silent, a rupture in the fabric of musical community in which he typically performed. Lipscomb becomes a kind of historical vortex here, bringing the silhouette of another cultural setting into a new light in the Greek Amphitheater on the Cal campus. In the photograph, he leaves a sort of hole punctured at the center of the proceedings.
All sorts of cultural negotiations of race, gender, class, and region are thus captured in this image. Even a strange sonification such as this one can help a historian suddenly perceive all of those negotiations more vividly in the photo by hearing their data play out on the sensorium. In a kind of synesthetic hermeneutics, sonification can alert the historian audially to things one only at first dimly sees in the visual artifact.
In this sense, defamiliarization tactics for historical inquiry such as sonification have the potential to address the danger of reproducing what Jessica Marie Johnson calls the “thingification” of people as data. Rather than treat lives as things, people reduced to dehumanized data, defamiliarization asks us to treat things as alive. Defamiliarizing an artifact can bring forward more boldly the traces of lived experiences in all their fraught negotiations of power, interaction, struggle, connection, alienation, pleasure, pain, joy, sadness. De-fetishizing the artifact as a thing by playing with the pliability of its existence as digital data, turning images into data correlated into sound compositions in this case, means being able to access what artifacts and archives represent: not a flattening of the archival record into things, but an activation of its things into heightened historical imaginings of the people, social forces, institutional dynamics, cultural encounters caught up in artefactual and archival representation.
Paradoxically, the futuristic computational distorting of the artifacts we study might allow us to get back more accurately to the pasts they preserve. Combining the cultural and digital history turns, bending them into each other, might enable us to read against the grain of the archive better by altering the very grain of the archive itself. Transforming image data into sonic correlations might quite literally, as well as figuratively, allow us to hear the silences of the archives, or at least tune our historical ears more effectively to the past in all its dimensions. Mance Lipscomb, and so too his audience at Berkeley, like all our ancestors, deserve nothing less.
[SLIDE Credits more info]
Photograph possibly taken by Matt Hinton. Preserved in the Berkeley Folk Music Festival Archive. For more information about the Berkeley Folk Music Festival, visit bfmf.net. For more information about defamiliarization research, visit michaeljkramer.net.