Reflections and Refractions: Experiments in Sound and Vision
A response to Jennifer Wicks’ Extended Compositions: Defining a Visual Language of Sound on CCA Annex.
Extended Compositions (in four frames)
1. Seven drawings. Lines and space, fractions of a circle spun out on the page. The repetition of shapes – arcs and chords and rays, separating, multiplying, returning – forms a pattern, a rhythm, a retinal pulse.
2. Two sculptural marks, their lines unravelling in cold steel. Listen as they gently stretch outwards, upwards, downwards and backwards. Circling form and space, their quiet hesitations tapping out a gentle beat.
3. A film etched in back and white. Circles pulse, lines tumble, spill out onto the soundtrack. Forming and reforming, re-re-repeating, the vertical lines slide apart, then together again. A rising tension plucks at the space between the frames.
4. A film in two parts, sometimes three, side-by-side, or one frame within another: a moving painting. A bright reel of colour, gliding rhythms of orange, of blue of pink and yellow, sound and image bleeding. Landscapes unfold under the microscope of translucent colour, in the particles of pigment and dust and hair. The skin of the film hisses and collides. The slap of impact, a quiver of the celluloid flesh, yellow on orange. Then waves of colour, synchronicity drifting in and then out. Out and then in.
In the final chapter of Michel Chion’s seminal book, Audio-Vision: Sound on Screen, the composer and critic proposes several methods of observation for those seeking to analyse audio visual work, what he refers to as ‘the audio visual canvas.’ [i] The first technique, ‘masking’, Chion explains, involves observing the sound and image tracks independently before viewing together, something he suggests ‘gives you the opportunity to hear the sound as it is, and not as the image transforms and disguises it; it also lets you see the image as it is, and not as sound recreates it.’ [ii] The advantage of this method is that each element of the canvas is given time and space for it to be considered in full, independently, before considering its relationship to the whole. It is an act of dissection, an expansion of our engagement with the audiovisual text, a breaking apart of the canvas under the magnifying glass before it is made whole again.
Jennifer Wicks’ new exhibition, Extended Compositions, adopts a similar level of scrutiny to the relationship between sound and image. Taking the form of an expanded animation, hosted on the CCA Annex website, the exhibition presents four dialectical works: Graphic Notations, a series of seven minimalist drawings; Marks #1 and #2, two steel sculptural responses to the graphic notations, presented as a film with violin and synth soundtrack; Re-re-repeat, a short experimental hand-drawn animation, which reinterprets the graphic notations through its inscription on the film’s soundtrack and digital remediation, featuring double bass from Jeremy Ward (BBC Scottish Symphony Orchestra); and finally, Suite for Extended Compositions, a hand-painted 16mm filmic response to the graphic notations, developed in collaboration with musicians Alice Allen and Katrina Lee (GAIA Duo). The presentation of the work itself, across four frames, could be read as a reference to the medium of film, and the individual cells of celluloid which, together, make the whole. Each work included as part of Extended Compositions, presents a different step in Wicks’ process of experimentation, a component part to the whole.
The project’s origins lie in an earlier work, Trace out the threads, follow their convulsions, a collaboration between Wicks and The One Ensemble, performed at the CCA in 2018. The event, featuring films set to a live score of composed and improvised music, and the live recording and looping of sound, afforded the opportunity to examine the sensorial relationship between sound and image, experiencing its evolution as it unfolded in the space and time of the performance. In this way the focus is not simply on the synchronisation of sound and image or determining a kind of aural and visual equivalence, but more on the experience and feeling of the dynamic exchange between the two. Extended Compositions, builds on this earlier work, and was initially intended as a live performance; however, for obvious reasons, was reenvisaged as an online exhibition (with a view to eventually stage a live performance of the work).
Although the pandemic introduced limitations, it also provided Wicks with the time to develop the work, perhaps in a more expanded way than originally intended. Like others during lockdown, who looked to hands-on activities requiring intense levels of absorption as a way of tethering their focus away from Zoom meetings and rolling newsfeeds, Wicks found the labour-intensive work required of her hand-drawn films was something that settled easily into the new rhythms of daily life. The material of film served as an antidote to what for many was a largely virtual experience of the pandemic, in a similar way that knitting allows you to do something with your hands, to touch at a time when texture and tactility are limited. But instead of counting rows and stitches, Wicks was counting frames. Sadie Plante’s essay, ‘Small Black Reptile’, featuring alongside Wicks’ exhibition, draws attention to the ways in which counting can provide a structure, something which may have been welcomed in the formless shape the days often took during lockdown.[iii]
Composed on her dining room table, the length of which Wicks determined equated to four seconds of screen time, the films were created in tandem. For Re-re-repeat, Wicks worked with four strips of 16mm lined along the table, so that she was able to synchronise the segments of film as she worked. This was important since although the segments are digitally pieced together in the final work, they were not digitally edited, so the relationship between the segments is directly related to the relationship as it was established on Wicks’ table. Working with the inherent rhythm in the medium of 16mm and the rate of twenty-four frames per second, Wicks worked out a numerical system to establish a tempo and how many frames there were per beat. In addition to drawing and painting the graphic notions onto the film, a leather hole punch with multiple tips in varying sizes was used, with the hole sizes adjusted in order to alter the pitch. The soundtrack for each film followed a slightly different approach. For Re-Re-Repeat, the bassist, Jeremy Ward, recorded a series of samples in response to the film’s graphic soundtrack which Wicks mixed, looping, layering and adding synth. For Suite for Extended Compositions, cellist and violinist, Alice Allen and Katrina Lee, were given six graphic scores, and a rough set of working notes for their performance, which were recorded remotely and mixed.
Experimentations with sound and image, of course, have their origins in the relatively early years of cinema. Even silent works such as Hans Richter’s Rhythmus 21 (1921), a black and white abstract animation which aimed to make rhythm visible, anticipate some of the earlier experimentations made with sound and moving image. The medium of film afforded unique opportunities for experimentation. As Richter writes, it promised a way to explore, ‘rhythmic processes not just in space and on a flat surface, but also in time.’[iv] Although some filmmakers and theorists warned that the arrival of synchronous sound and the talking pictures would spell the end of cinema as an art form[v], the early 1930s saw a variety Avant-Garde filmmakers explore the relationship between sound and image and synchronous sound through various experimentations. Oscar Fischinger and László Moholy-Nagy, for example, created hand-drawn sound films which feature nothing but the soundtrack on screen, offering a way of seeing sound which also represents the earliest experiments in electronic sound.[vi] Norman McLaren also began making animated sound films in the late 30s, first experimenting with minimalist patterns such as dots and loops etched onto the soundtrack and then later, while working at the National Film Board, Canada, and in collaboration with Evelyn Lambart, developing an approach to hand-drawn and hand-painted animated sound films, which included various systems and techniques for the creation of graphic sound. It was also in the 1930s that Mary Ellen Bute, a filmmaker whose work inspired Wicks, began making abstract animated films as a way of (so her title cards suggest) ‘seeing sound’ to ‘create moods through the eye as music creates moods through the ear’. Bute’s interest, as with many of the early experimental filmmakers, was in exploring a kind of audio-visual synaesthesia – in finding an ocular equivalent for the sound, and in effect, create a kind of visual music.
Wicks’ interest, however, is in exploring both – the original drawings and sculptural forms, offer their own visual notations of sound which are then re-interpreted and remediated through their application to film, live performance and digital composition. Furthermore, rather than maintaining an equivalence of image and sound, the work oscillates between moments of synchronisation, and the locking of sound and image, and the visual repetition or looping of sound, which provides an opening for striking new patterns to emerge. The approach perhaps has more in common with Guy Sherwin’s series of Optical Sound films (1971-2007), which are also abstract direct animation sound films that often include, looping, as well as phased looping and elements of live performance, and explore the relationship between sound and image. Aimee Mollahan writes of how Sherwin’s work plays with ‘not only points of synchronization but with the temporality and duration of the audio-visual content in order to draw attention to, both the construction and materiality of the film and the relationship between sound and image.’[vii]We see this in Wicks’ films too.
Across all of Wicks’ works which make-up the whole of the ‘Extended Composition’, there is an expansion through the process of repetition, an opening out of the elements into kaleidoscopic patterns. This is evidenced in the layering of images and framing, not just in the moving image, but also the drawings and sculptures as well as the soundtrack’s layering and looping. The textures and tensions produced between organic and synthetic forms and structures give further dimension to the work. The effect is also comparable to other earlier experiments with looping, such as Nancy Holt and Richard Serra’s video work, Boomerang (1974) which featured Holt’s speech delayed and played back, picked up by the microphone and then repeated again, forming audio feedback loop. Holt speaks to the camera, describing the sensation of the experience – the multiplication of sound, delay of time, and how the navigation of the ‘boomerang’ effect proves overwhelming. As Holt says, ‘For awhile I heard only my own voice at the time that I was speaking, but now I hear my own voice slightly delayed coming back in on top of me, so that I am surrounded by me, and my mind surrounds me, my mind goes out into the world and then comes back inside of me. There is no escape.’ It is ‘a world of double reflections and refractions’. This defamiliarizing effect is evident in Wicks’ work too. Repetition produces mesmerising kaleidoscopic patterns which, like a form of cubist art, further envelops us in its subject by expanding our points of engagement with it. The echoes produced through the repetition of sound and image and the interaction between them also invoke certain aspects of memory. With each repetition, as time passes and the work unfolds, our perception changes, we experience the work differently. In Extended Compositions this effect is also apparent in the process of remediation, in that the residue left behind by the old media forms used to initiate the project serves as a kind of technological haunting.
It is of significance that Holt and Serra’s Boomerang, although experienced as a video work, was broadcast live on television at the time. Similarly, live performance is also an important component of Sherwin’s work. Wicks’ Extended Compositions was also intended to be performed live and although it wasn’t possible for the exhibition in its current form, it is hoped that it will eventually be presented live in some form. But the live performance introduces elements of improvisation and the potential for the unexpected. It introduces an element of uncertainty and emphasises the work as an embodied act of experimentation in which the maker, performers and audience are all participants. Extended Compositions certainly does manage to offer an embodied experience and reference its performative elements through the textured layering of sound and image. There are many ways in which we see and hear the bodies involved in making the work – through the imprints left onto the moving image itself and the audible sounds of touch and reverberation produced by the musicians and their instruments. The musicians’ improvisation and Wicks’ remediation and sampling of sounds offer a glimpse of what a live performance might be like, but it would no doubt be a different experience to inhabit the physical space of a live performance, which would provide more of a sense of the work’s more sculptural effects – the comingling of sound and image with the sculptures, as well as the effects achieved by the presence of bodies in the space as well. This anticipation of the exhibition in a post-pandemic world, is perhaps an example of the future haunting the present, and my own reading of the exhibition. Nevertheless, Extended Compositions, and its expanded investigation of image and sound, offers the kind of expansion of observation which Chion calls for in relation to the analysis of the critic. The time spent watching and listening to Wicks’ work, enables a further deepening of our engagement with the audiovisual canvas, encouraging a more conscious consideration of sound and image at points of synchronisation, but also their component parts and function at other points too.
[i] Michel Chion, Audio-Vision: Sound on Screen, Columbia University Press, 2019, pp. 187 & 213.
[ii] Ibid., p. 187.
[iii] See also, Sadie Plant, Zeros and Ones: Digital Women and the New Technoculture, London: Fourth Estate, 1997, for the parallels drawn between knitting and computer programming.
[iv] Hans Richter, The Struggle for the Film: Towards a Socially Responsible Cinema, New York: St Martin’s Press, 1986, p. 56.
[v] See Sergei Eisenstein, Vsevolod Pudovkin and Grigori Alexandrov, ‘Statement on Sound’ reprinted and translated in Richard Taylor and Ian Christie (eds.), The Film Factory: Russian and Soviet Cinema in Documents, 1896-193, Cambridge: Harvard University Press, 1988, pp. 234-235.
[vi] László Moholy-Nagy’s hand-drawn sound film, ABC in Sound was a recent rediscovery made by the William Fowler and Bryony Dixon at the BFI. See Ian Christie, ‘ABC in Sound: László Moholy-Nagy’s rediscovered experiment in visual sound’, Sight&Sound, June 2019.
[vii] Aimee Mollaghan, (2014), ‘Audio-visual moiré patterns: Phasing in Guy Sherwin’s Optical Sound films’, The Soundtrack 7: 1, pp. 47–57, p. 55.
Many thanks to my colleague, David Archibald, for reading and commenting on a draft of this essay.