Code, Annotation, Composition

Much digital and analog ink has been spilled regarding the use of LLMs for coding tasks. This post does not seek to rehash the entirety of this conversation, but instead, offer another perspective similar to those voices combatting atrophy in seemingly mundane tasks that nonetheless require careful mental planning, experimentation, and adjustment through process. These writers have commented (quite accurately) that the process of writing is the important aspect of writing - it is through the examination of our ideas, translating them into a (more) persistent form that we are truly thinking. Naturally, I whole-heartedly agree; much of the day-to-day creative work I engage in is about subtle processes of becoming and slow-realization; these are marathons, measuring achievement in retrospection after weeks and months of repetitive engagement.

Musicians, artists, writers, etc. are all well acquainted with this process, but much of the conversation around LLMs centers upon their (seeming) ability to deliver product output at speed. The value of the model seems to essentially be speed of delivery rather than accuracy. In some cases, verbosity that would have otherwise been thought extraneous or cluttered is accepted as a minor consequence of labor offload. For some, outright depending on these models for consistency and accuracy has proven to be an exercise in faith.

And the speed is blinding, often impressive, frequently demoralizing; in a world that prizes product over process, the slow burn of carefully executed work seems to have lost its (adversarial) footing within the capital structure. In an effort to move faster, we run the risk of stumbling in ways not yet anticipated - a failing of imagination.

I wish to suggest, in this post, a different perspective on knowledge work and digital creativity’s mundane moments, the human effort in programming and scripting.

As someone who has not worked in a corporate coding structure, my perspective may be construed as glib or naïve, but my work with code is different in its aims, needs, and velocity.

I engage with coding as an exploratory adjunct to my musicking, a practice largely grown from Western classical traditions and structures. Although my deadlines are merely self-imposed, I have experienced the press of frustration with unfamiliar syntax, text that won’t parse as I wish, bugs, accidents happy and tragic. Nevertheless, the realization still yet arrives - adjustments to syntax, adjustments to structure, adjustments to logic, adjustments to concept - these all contain their own satisfaction and mastery even if the modification itself was slight. Through these modifications, it is ideal that the work becomes more elegant, more clear, more logically focused. Running opposite, added syntactical complexity that does not add a proportional amount of benefit clouds the picture of the program - it is noise (in the most undesirable sense of the word).

It is through the careful consideration of these technical aspects of the finished product that we arrive at an appreciation for the process of its construction. I have suggested aloud and in writing before that most appreciation of art lies in the subtle, sometimes transparent or non-aware, consideration of the artist’s process - from conception, to plan, to execution, to consumption.

These are never fixed in place, and may take on seemingly contradictory or nullifying qualities, but claims of divine inspiration or reliance upon aleatoric compositional policies are still valid artistic positions, even if they seemingly alienate the artist. That is to say, the artist’s own alienation through the processual realization of the work does not contradict our own admiration (or criticality) of the apparent process. (Indeed, our own struggle to decipher the process is itself another type of engagement with the work, one that is definitely postmodern in its hermeneutic centerings in a web of contexts both known and unknown.)

Within the structure of post-modern criticism (both institutional and within individuals), the primacy of the consumer (not to be read here as always consuming in a strictly capitalist sense) in the interpretive (hermeneutic) process is left undisturbed; what I suggest is that the examination of the original artist’s unfurling of process is still centered upon the viewer’s assessment of the process’s remaining evidence; postmodernity’s  solipsism is still yet unrivaled, even if the examination moves outwardly from the viewer’s primal, visceral reactions.

Put more simply, I wish to suggest that viewers are examining how they themselves feel about the examination of process (more or less) apparent in the work. This examination, I posit, is partly and partially (as opposed to impartially!) relived in subtle imaginings; “I could have done that!”; “I could not have done that…”; “How did the artist think of that?!”; are all statements enthralled in the replacement of the artist with themselves. I don’t suggest this to be altogether egotistical or narcissistic, but rather the necessary attempt at imagination’s empathy to understand the work, the artist, the time, and the place. These contextual webs of meaning and understanding are, of course, culturally and temporally contingent - it cannot be understood, experienced, or empathized with all at once in singular meaning. We are bound by our own interpretive teleologies, themselves beholden to strings of encounters, reflections, and forgettings.   

This is not a new idea - I don’t wish to suggest that it’s strictly original to my thinking, but has been subtly appearing in the work I’ve engaged with over years. More to this post’s point, and returning to our opening thoughts, I would like to think more carefully about music coding as human composition, to be appreciated by listeners and viewers for its processual realization. The LLMs’ ability to spew code should not bother us if we reformat our thinking toward process - the slow, stammering, and stumbling method of human written code contains a beauty if we are willing to slow down and examine it.

This appreciation already exists in live coding demonstrations, concerts, and presentations. While the sounds are reaching the audience, their eyes are (more likely than not) scanning the code (when it is frequently projected onto a screen or wall), examining the text, seeking to make SENSE of the process being revealed before them with multi sensory engagements. The code may be arcane, or perfectly intelligible  - this is reliant upon the viewer’s familiarity with the syntax. This is no different than engagement with cinema, painting, sculpture, or performance art; at every turn, symbols function as code and cipher within (or in contradiction to) a wider shared context. That is, a certain type of analytical appreciation grows proportionately to a viewer’s understanding of the works’ symbol-play.

For works that involve coding, consideration of the code’s features should enter into our engagement with the work. The underlying code has the potential to reveal ecstasies, frustrations, victories, frailties. Artistic coders, annotate for us! Let us see the humanity of the process, the unfolding of the work’s conceptual and computational logic, the revealing imperfections whose analogues lie in brushstroke, material obstinacy, performance eccentricities. Engagement with code, as composer and as viewer, will cause us to realize greater depth of the work, its human fingerprints, and the wonder of the work’s technical interiority.

Importantly, I’m suggesting that code, in these artistic spaces focused upon processual unfurling, be human generated. Yes, one could ask the LLM to annotate its output in an effort to explain the work, but in this process, we’ve offloaded an essential part of the work’s processual documentation - we’ve lost the critical moment of encounter to be discovered by those viewers wishing to live in the work’s logical concept and its own symbolic display. Put another way, by including the code and its symbolic play, we are ourselves, as artists, living in that same conceptual world. This is another door to working within the piece rather than from without, another vulnerability, another path to see our own humanity coming through the (culturally constructed) coldness of the logical syntax.

Perhaps this is a big ask. Not every viewer/listener/consumer may wish to understand process in such an intimate fashion, but it is a wish of mine, in the work that I strive to produce, that degrees of debasement and accessibility work in tandem to shape the impression of the piece with its multifaceted demands upon the viewer. I would like to suggest here that annotations (comments) in the displayed code text act not as an explanatory text, but rather as an invitation to the inner game and life of the work. The annotation is not a crutch, but a clue, another door to the richness of the work’s essential concept, rather than only its technical life.

There is more to be said on this topic. More to come! Thank you to those who got this far!