The Language of Gesture and the Moving Image

Representing speech by graphic means is something we humans have been doing for some 5000 years. This text is “what I’m saying”, captured in writing.

Of course, I have never really said this, never spoken these words. This text is born in writing. Like most written texts, it is a simulation of speech that awaits its realization in the reader’s imagination. In yours.

The letters of the alphabet more or less signify units of sound, phonemes. The relation between letter and sound however is purely conventional; letters are not images. There is nothing A-ish about the A, or F-ish about the F. But through these arbitrary signs this text may still represent what I’m saying through how it should sound. Very roughly.

Does language need speech, that is, vocal sound? No, it doesn’t. Sign language is a full-grown, albeit mute form of language. And yet, speech is worldwide the default mode. And no written text, I think, is completely silent.

It is unlikely that while you’re reading this text the letters will call to your mind the individual phonemes, or even the sound of the words. But maybe you’re imagining (however vaguely) a certain tone of voice. Punctuation and syntax play an important part in this.

As do white spaces.

Tone of voice, imaginary or real, is or should be every writer’s concern. It has traditionally been a focus of interest for teachers of rhetoric. And because little of the tone of voice, of the actual speech act, is actually captured in writing, some of these rhetoricians devised ways of representing the intonation separately alongside the text. Or attempted to transcribe it in musical notation.

The act of speaking is not limited to the voice. The whole body participates in expressing our intentions, which are only imperfectly realized in the words. Gesture, body language, “the eloquence of the body” too was part of rhetoric, a skill that could be taught, disciplined, rationalized.

According to the Irish clergyman and headmaster Gilbert Austin,

“On his commencement as a public speaker (which cannot be too early), it is necessary to teach [the pupil] every thing, and to regulate by rules every possible circumstance in his delivery; his articulation, accent, emphasis, pauses, tones, voice, countenance, and, along with all, his gesture”. (p. 282)

And like the words spoken, and the tone in which they are spoken, gesture was something that could be captured by graphic signs. This he did in his Chironomia (“hand management”), published in 1806, from which I’ve just quoted.

Besides being a teaching aid, the notation of gesture could also be a valuable tool in recording the actual performances by orators and actors. A text annotated by his method would allow the reader to recreate some of their performances in his or her imagination.

Austin recognized that, like speech and music, gesture is a continuous flow, not a series of discrete postures.

“The variety of gestures, of which the human figure is capable […] may almost be accounted infinite. In this great variety there appears however a similarity and relation among many gestures, which affords opportunity for classification and nomenclature: so that, however unattempted hitherto in this view, the art of gesture and its notation (that is the representation of any gesture by appropriate symbols) seems capable of being reduced to a regular system” (p. 293).

Classification and nomenclature: for this purpose, Austin broke down the flow of gesture into discrete elements, positions and movements. These he indicated in the text by a rather laborious system of graphic signs (ordinary letters).

Such a method is likely to produce an artificial result. Though it is not uncommon that a repertoire of discrete, identifiable gestures emerges spontaneously in practice (and much rhetorical teaching besides Austin’s has been based upon this premise). Just think – as a particularly depraved example – of the puppet-like hand and arm movements of Donald J. Trump. A product of involuntary self-caricature, and therefore a textbook case for contemporary studies of rhetoric.

Because gesture is visual, it is a small step to move from description to depiction. To clarify his system Austin made use of images. But images too can only capture a moment. Unless you draw various stages of the flow of action, that may be “read” as quasi-continuous motion. The curious fact is that some of the anonymous engravings in Austin’s book very much resemble film strips.

Austin 1806 Systematic Positions of the Arms

And it is in fact quite possible to animate them.
Austin 1806 animated © Lodewijk Muns

“The language of gesture bears more analogy to that of music than to the language of general ideas: and therefore it is named the notation of gesture. As the notation of musical sounds records the melodies and happy harmony of sounds which in their nature endure but for a moment, so the notation of gesture records the beautiful, the dignified, the graceful or expressive actions of the body, by which the emotions of the mind are manifested on great and interesting occasions, and which in themselves are no less transitory.”  (p. 276)

Gesture, for Austin, is a “language” more like music than like spoken language (“the language of ideas”), even though speech is no less “transitory” and may be equally charged with emotion.

Such a pictorial representation of gesture is, to my knowledge, unprecedented. John Bulwer’s much earlier book of the same title (Chirologia … Whereunto Is Added Chironomia: Or, the Art of Manuall Rhetoricke, 1644) is famous for its pictures of hand gestures. But those illustrations are a simple table of isolated gestural signs.

Does Austin’s Chironomia therefore deserve a place in the history of the moving image?

The technique of using a series of still images in rapid succession to suggest movement has come surprisingly late, considering the simple mechanics that is required. Even though the idea of combining moving images with magic lantern projection seems to have occurred to the Dutch scientist Christiaan Huygens as early as 1659, practical applications emerged only in the nineteenth century. The phenakistiscope, which contains images on a spinning disc watched through slits, was invented in 1832. The similar zoetrope was perfected in 1865. Most surprisingly, the simple flip book seems not to have been documented before 1868.

To suggest that Austin’s illustrations are part of this development is no doubt a far stretch. His text lacks any indication that the series of images must be “read” from left to right as a quasi-continuous sequence. Let alone, that he (or his engraver) had any idea that the slowness of our perception might trick us into seeing continuous movement where there is only a succession of distinct images.

On the other hand, going down to minute details of gesture, breaking a flow into elements and recreating it from those elements is the basic principle of animation.

With his method of analyzing and resynthesizing the “language of gesture” Austin attempted to animate the speaker’s body. In a more indirect way, a musical score animates the musician, directing her to produce a flow of sound-producing movements and gestures. And the letters of a text, this text, are animated as a flow of words, ideas, perceptions in the reader’s mind. In yours.


Filosofie van de kokermast (Help, de palen rukken op!)

(English version)Zoetermeer wintracks

En toen waren er ineens een heleboel: gladde grijze palen. Ontwiekte windmolenmasten die paarsgewijs de plaats innemen van de traditionele, uit honderden delen samengelaste eiffeltorentjes.

Continue reading

When will it come, this lovely Now!

A few personal notes on Bach’s Cantata BWV 82, Ich habe genug.

Recit. Mein Gott, wenn kommt das schöne: Nun! BWV 82

Mein Gott, wenn kommt das schöne: Nun! (Bach’s handwriting,

A daily dose of Das Wohltemperierte, during many years, has not quite cured me of a mild bachophobia. As much as I enjoy playing Bach, and less frequently, to listen to his music, it still leaves me with a feeling of discomfort.

Awe inspiring perfection and complexity may be part of the explanation. But more particularly it is the burden of christian religiosity, that seems to put a mark even on the secular works. Though no doubt Bach had a sense of humour (and a high sensitivity to other human sentiments), there is a kind of righteousness (Rechtschaffenheit) about the man and the music that puts me off.

A stern father figure, maybe.

The worst about this religiosity is its cult of death. And this is most explicitly present in one of the most popular, moving, and absurd cantatas, Nr 82, Ich habe genug/genung. Bach cannot be held fully accountable for the texts he set, not even for their general meaning, but inevitably the text is part of the work, and the music gives expression to the text.

Ich habe genug is a solo cantata for bass (in its first version) which consists of three arias and two connecting recitatives. Aria 1 is a free paraphrase of the Canticum Simeonis (Luke 2:25). Simeon was an old Jerusalemmer whose claim to biblical fame was being righteous and pious, and having received a personal promise from the Holy Ghost that he would live to see the Messiah. Having seen him (the infant Jesus), he “has enough”, and wishes “joyously to depart from here even today”.

Despite a coloratura on Freuden, the music of this aria has an elegiac quality — it is almost a lamento, in fact (the bass descends two steps instead of three), and one of the most heartbreaking, and at the same time comforting pieces in all of Bach.

In the context of the cantata this aria functions as a reading from scripture. What follows is a commentary upon this reading, applied to an anonymous “I”, the average Good Christian. Together with Simeon I behold the joy of that life-to-come … Ah! I wish I could take leave right now …, it says in the following recitative.

In the second aria the speaker is trying to prepare his body for death, as if it were sleep. Fall asleep, my weary eyes (Schlummert ein, ihr matten Augen). If dying were like this, it would not be so bad.

But simulating death by sleep seems not enough. The next recitative voices impatience: My God! When will it come, this lovely: Now! When I will dispatch myself in peace and rest in the cool ground and there in thy bosom? — I have made my farewell, goodnight to you, world!

The final aria is a lively bouncing dance, a celebration of death. I rejoice about my death, ah, would that it had come already. (Ich freue mich auf meinen Tod, ach, hätt’ er sich schon eingefunden). The wavering coloratura in the bass voice and the throwaway gestures in the instruments almost suggest a state of drunkenness. A grim minor-key comedy that seems to undercut everything that precedes it.

In the first two aria’s, a text which implies a certain affective state or action — contentment in the first, putting to sleep in the second — is coupled with music that suggests a slightly different attitude. A lament in the first aria, gentle persuasion in the second. It creates an ambiguity that leaves the performer and the listener interpretive space. Maybe the words aren’t quite right.

But what about the final aria? Bachian humour? Did the composer wish to expose the perversity of the text? I doubt it. 

I was reminded of this cantata while reviewing a recent book by the musicologist Lawrence Zbikowski, Foundations of Musical Grammar (OUP 2017). The cantata is repeatedly discussed in this book, that attempts to forge a closer relationship between music theory and musical experience.

Zbikowski’s interpretation is skewed somewhat by his mistake of putting all of the text into the mouth of Simeon, even though the old man has clearly left the stage after the first aria. Quite baffling I find the characterization of the second aria, Schlummert ein, as a “lullaby” (p. 57). The text evidently makes it a “sleeping song” (or “slumber aria”). In that sense it shares the basic function of a lullaby, or cradle song: putting to sleep. But lullabies are for babies, baby music. As Zbikowski notes, they typically have “melodies with a simple contour” and contain a lot of repetition (p. 87). I doubt that anyone who for a moment thinks of typical lullabies — a folksong such as Schlaf, Kindchen, schlaf, or Brahms’ Wiegenlied, will apply the term to this curiously ambigous, restful-restless aria. Or listen to Schlafe, mein Liebster, genieße der Ruh’ from the Christmas Oratorio (BWV 248/19). More aria than song, but very much a lullaby in its harmonic stasis and repetitiousness (ironically, it was originally sung by the allegorical figure of Lust in Cantata BWV 213).

Zbikowski does conclude that “the emotional range of the aria exceeds that typically accorded lullabies” (p. 58). But the characterization is wrong from the very beginning: a syncopated melody, a leap of a seventh, and a bass that immediately moves into instable regions. Even the short-long accentuation is “unlullabilike”.

Calling this aria a lullaby has a history — it goes back at least to Albert Schweitzer, who called it “the superb lullaby of death” (“das herrliche Todes-Wiegenlied”, J.S. Bach, 1908, 632; in the original French, “cette admirable berceuse”, J. S. Bach le musicien-poète, 1905, 167). It shows how a stereotype, even an absurd one, may keep echoing around and colour perception. Even the renowned Alfred Dürr speaks of “the cradle rhythm, for which the syncopation is responsible” (The Cantatas of J. S. Bach, OUP 2005, p. 665). “Pulsating rhythm” seems more appropriate.

Looking for other slumber aria’s, I hit upon the hauntingly beautiful (and nearly unsingable) Ruhe sanft from Mozart’s unfinished Singspiel Zaide. There are a few striking similarities between this and Schlummert ein, which become obvious when we put one below the other.

Top: Bach, BWV 82/3, transposed to G major, as in the Notenbüchlein für A. M. Bach. Bottom: Mozart, KV 344/3. Voice and bass only.

The mi-re-do beginning is, not coincidentally maybe, that of Schlaf, Kindchen, Schlaf — a quieting melodic “gesture”. So maybe a bit of a lullaby after all. The harmonic changes on these beats are also the same. The la-sol-fa continuation of the melody is similar too, though the correspondence is less obvious here.

Are the similarities significant? Yes and no. Music of the eighteenth century is full of patterns, formulas, schema’s. Sometimes, as seems to be the case here, one pattern lends itself particularly well for a certain expression or mood. But most fascinating is how one opening schema may be elaborated in very different ways.

Het Belletje Des Vaderlands

(No English version)

“De belangrijkste bel van Nederland is de stemmingsbel van de Tweede Kamer. Met deze bel worden de leden van de Tweede Kamer opgeroepen om in plenaire zitting te stemmen …”

Zo meldt Kamerstuk 2017D29437 van 19 oktober 2017. Bij de NOS kunnen we horen hoe parlementariërs zich beklagen over die “schoolbel”. “Bloedirritant,” zegt Pechtold, haast sidderend van verontwaardiging.

Het heeft wel iets aandoenlijks: parlementariërs die als schoolkinderen naar de klas worden geroepen. Nu is het wel een erg lelijke bel, zo’n telefoon- of deurbel met geëlectrificeerde klepel. Ik kan me voorstellen dat je die vervangt door een mooie handgeluide klok. Of, veel beter nog, een door het Kamergebouw verspreid carillon, zodat alle klokken en kamers één muziekinstrument worden.
Continue reading

Please Hold the Line (and Put Up with That Tune)

music box © Lodewijk MunsCircumstances have forced me to spend a lot of time making phone calls — and a lot of time staying on hold. It has given me a freakish and unsought for expertise in “on hold music”.

Complaints about this particular abuse are all over the internet. So I will swallow my frustration and just make note of a few things that struck me. First: how a very small number of tunes has a disproportionate share in the total repertoire.

One of the most frequent numbers is a famous-notorious piece called Opus Number One. Its synthesizer sound is very “eighties”, and it dates, indeed, from 1989. It is a strongly evocative piece — dingy office carpet, neon light and low suspended ceilings come to my mind.

Continue reading

De Raad voor Cultuur, of: de Grote Nivellering?

English version

“Raad voor Cultuur: Meer geld naar dance en Nederlandse lied”, kopte de Volkskrant in haar bericht over het deze week verschenen sectoradvies muziek. “[…] de tijd lijkt voorbij dat het subsidiegeld vooral vloeit naar klassieke muziek en opera, moderne muziek en jazz”.

Dat klinkt (voor sommigen) bedreigend, en dat zie je aan de lezersreacties. Vooral die glunderende kop van Marianne Weber doet het bloed koken. Moet daar het geld naartoe?

Continue reading