Video performance wall

Origins

Back in 2008, Firefly Makani was still an active musical group that would frequently play live shows. As the group was a duo, replicating all layers of the music on stage was impossible due to the limited amount of hands at the disposal of the two performers.

We thus adopted an approach where songs were subdivided into several (loopable) sections which could be controlled during playback (rearranged, extended, re-recorded, etc.).

The purpose of these loops was to keep time and key elements going while each member would play live instruments, making each performance unique where the songs reproduction differed from their recorded counterparts.

Playback control

Apart from being able to provide live melodic input and effects on top of the controllable recordings, an additional mechanism was developed in which all running audio streams - separated by rhythmic and melodic content - could be manipulated in realtime.

By using a short recording buffer (into which all live audio output is routed), the system could retrigger small snippets while being able to return to the raw, unprocessed signal at any time to maintain meter and tempo.

This part of the performance software also captured the activity of the multiple MIDI controllers used by the performers and was developed by Fedde ten Berge using Max/MSP.

Networking audio and video

The second laptop in our two computer setup was responsible for generating video output (in the form of either mathematically generated shapes or a layered collage of pre-recorded camera footage) which would be projected onto the stage.

The connected laptops synced the performance state, keeping the pulse of the images synchronized with the tempo of the music as the video was manipulated by the same controllers controlling the audio streams.

Whenever a controller change was registered, a UDP packet was sent to the video laptop, which would interpret the message and take action, keeping the visual aspect of the show tied to the music, making a badly executed move both aurally and visually painful.

Unified aesthetic

While Firefly Makani was the creative output of two individuals, our identity remained anonymous in favour of an insect themed lo-fi image, never displaying the names of the members, let alone their face.

This style was reflected in green and grey dithered images and through custom video recordings of insects crawling over computer hardware, jellyfish tentacles floating in water and similar goodness.

Above imagery reflected in the cover art, the website, and subsequently in this video performance wall, tying the visual identity together over all media and performance outlets of the group.

In case of the performance wall: in time to music performed live.

Program 2

Program "blitter" reacting to increasing glitches in audio playback

Particles galore

The available visualizations were split into several "programs", which could be changed at runtime via a controller directive.

Each of the programs would start in a neutral state, always displaying moving content. Programs were interchangeable and did not require a linear progression. This state would change by reacting to the controllers / changes in music, for instance: wild screeching sounds would be visualised as a violent explosion of particles or by the speeding up (or reversion of) a video feed.

Additionally, programs could be overlaid on top of another, which could be used to build intensity throughout a performance.

Uniform code base

The source of the video performance would reuse the components used in the composition of the music (e.g. the Max/MSP plugins) as well as the website which featured parts of the video material on its background.

Visitors of the website would thus get a sneak peek at new imagery while the performances would tie in with the overall theme.

Languages and technologies used:

Max/MSP, ActionScript 3, Adobe AIR, C.

Five meter wide epilepsy

Firefly Makani opening for Otto von Schirach @ Nieuwe Nor, 2009.