What's on My Desk: Tools for Real-Time Audiovisual Work
A walkthrough of the physical and digital tools that make up my working environment — from Arduino sensor arrays to VB-Cable audio routing to projecting onto walls.
This is the first entry in what I intend to keep as a running record of the work. Not polished writing — field notes. What I’m using, how it connects, what the setup actually looks like.
The projector
Everything starts here. A projector changes the relationship between an image and a room. It is not a screen you look at — it is a surface that becomes something else. Walls, ceilings, architecture, objects. The image does not sit inside a frame; it occupies the same space you do.
For my thesis setup, the projector connects via HDMI as a second display. TouchDesigner’s output window goes fullscreen on it. That is the entire output chain — one cable, one window.
Arduino and the sensor array
The thesis device integrates six sensors: CO₂ (MH-Z19), particulate matter PM2.5 (SDS011), temperature and humidity (DHT22), ambient light (BH1750), and sound. All read by an Arduino, sent over USB serial to the PC.
Arduino’s role here is not computation — it is translation. It converts continuous physical phenomena into a stream of numbers that software can act on. What makes this interesting as a creative tool is that these readings are never static. They drift. They pulse. A person walking into the room changes CO₂ and sound and light simultaneously. The data has temporal texture.
The serial stream lands in TouchDesigner as a string, parsed into individual CHOP channels — one per sensor. From there, everything is mapping.
TouchDesigner
TouchDesigner is the environment where everything converges. Sensor data from Arduino, audio analysis, video generation, and visual output all run in the same node graph simultaneously. The node-based workflow means the signal path is literally visible: data enters left, is filtered, scaled, and mapped as it travels right.
This is more useful than it sounds. When something is wrong — a sensor reading clamped at max, an audio value not updating — you can see exactly where in the graph the signal breaks. Debugging becomes visual.
VB-Cable
VB-Cable is a virtual audio cable. It intercepts audio output from any Windows application and makes it available as an input device elsewhere. When Spotify plays, VB-Cable captures it and routes it to TouchDesigner, which treats it as a microphone input.
No physical interface required. No latency worth measuring. For anyone starting with audio-reactive work on Windows, this is the first thing to install. It takes about five minutes and costs nothing.
Spotify
The audio content layer. For testing and development, Spotify is ideal: immediate access to wildly different genres, production styles, and loudness levels. You can go from compressed EDM to a sparse jazz quartet in ten seconds and see exactly how your reactive system responds to the change.
The one setting to adjust: turn off loudness normalisation in Spotify’s audio settings. The platform normalises playback levels by default, which compresses the dynamic range your reactive system has to work with. Without normalisation, you get the full signal.
Real-time audio reactivity
These tools together form a specific kind of system: one where the visual output is always a live function of input. Not pre-rendered, not triggered — continuously computed. Change the music and the visuals change. Walk into the room and the air quality data shifts the colour palette. The installation responds to its context rather than running a fixed programme.
This is what the thesis explores: not just the technology of making environments reactive, but what it means perceptually when invisible data — the air, the sound, the light levels — is made visible in real time.
Next week: deeper into the audio analysis pipeline — why the same TouchDesigner patch responds completely differently to electronic music versus acoustic recordings.