Cyber Aurality: Sound, Performance, and Space
on the World Wide Web

About Me

  • artist / programmer
  • the use of the web in music composition, hybridizing techniques
  • how the world of music(s) overlap with spatial computing, networked environments
  • My Website
  • started this research around 2020, Sonic Arts MFA, Brooklyn College
  • working with artists like Toni Dove, Kristin Lucas, Jacolby Satterwhite, Claudia Hart, LaJune McMillian

Table of Contents

  • Early Telematics
  • The Popularization of Cyberspace
  • Virtual Sound Performance
  • What Makes Sound in Virtual Environments Good?

Early Telematics


Yvonne Rainer : At My Body's House 1966

Leopold Stokowski

  • Conductor of the Philadelphia Orchestra starting in 1912
  • showman! adventurous supporter of new music, and pioneer of new orchestral techniques (technological and instrumental)
  • introduced stereo recording with Bell Labs Engineers to Wowed Audiences
  • Constitution Hall (1933) - Special Phone Lines operated by Bell labs would link the Philadelphia Orchestra back home with the D.C. Venue

Leopold Stokowski

“because through electronics, time and space have been conquered, so that we can see and hear at any distance and any interval of time … Unless all civilization and all inherited and newly created cultural values are destroyed by the madness of war, the contribution of electricity to music will always continue, but in new forms undreamed of at present, always leading to higher forms of musical creation, performance, and inspirational listening made available to Everyman.”

- Leopold Stokowski from Greg Milner's Perfecting Sound Forever

John Cage Future of Music Credo (1937)

  • When Theremin provided an instrument with genuinely new possibilities, Thereministes did their utmost to make the instrument sound like some old instrument, giving it a sickeningly sweet vibrato, and performing upon it, with difficulty, masterpieces from the past. Although the instrument is capable of a wide variety of sound qualities, obtained by the turning of a dial, Thereministes act as censors, giving the public those sounds they think the public will like. We are shielded from new sound experiences.
  • new technology beg us to consider the inclusion of all sounds into the musical lexicon

Experiments in Art and Technology

  • Headed by Billy Klüver beginning in 1960
  • Formalized Working Relationships between artists and engineers
  • Yvonne Rainer, Deborah Hay, Robert Rauschenberg, Andy Warhol, John Cage, Merce Cunningham
  • FM Transmitters, Photocells, Sonar Sensors for Motion Detection
  • 9 Evenings (1966)

Experiments in Art and Technology

"My project is simple to describe. It is a piece of music, Variation VII, indeterminate in form and detail, making use of the sound system which has been devised collectively for this festival, further making use of modulation means organized by David Tudor, using as sound sources only those sounds which are in the air at the moment of performance, picked up via the communication bands, telephone lines, microphones together with, instead of musical instruments, a variety of household appliances and frequency generators. The technical problems involved in any single project tend to reduce the impact of the original idea, but in being solved they produce a situation different than anyone could have pre-imagined.

Merce's Studio, Luchow's Restaurant, over public street, sanitation dept, aviary, the Time's press, Terry Riley's turtle tank 🐢

Experiments in Art and Technology

Maryanne Amacher - City Links (1967 - 1980)

  • "a collection where sounding resources of two or more remote locations (cities or locations within a city) are fed back to each other to allow for interaction between men and sound at distant locations."
  • The Kitchen, Walker Art Center, Museum of Contemporary Art Chicago, Hayden Gallery at MIT, and Cunningham Studio
  • installations and live performances
  • dealt with themes such as copresence, not always electronically mediated
  • sensing site, tone of place

No More Miles (City Links, 1973)

  • Walker Art Center Gallery
  • Budget Rent-A-Car
  • Site Specificity

Ephemera from NYPL (City Links Pieces)

  • where boundaries of ONE PLACE situations can begin to vanish
  • co presence non electronically linked

Construction of a Cyberspace

  • Tron (1982)
  • Neuromancer (1984)
    William Gibson

Construction of a Cyberspace

  • Hackers (1995)
  • Snowcrash (1992)

Immersed in Technology: Activities at the Banff Center (1992)

  • Art and Virtual Environments Project was initiated by Douglas MacLeod at the Banff Center
  • provide artists with an opportunity to work with, what was at that time scarcely available virtual reality computer systems to explore new possibilities around narrative time-based media art
  • NEXT/IRCAM machines running the music programming software Max, which was then only recently capable of processing sound in real time.
  • Uses familiar, physical gestures, to control a virtual environment, it's sound and visuals

Immersed in Technology: Convolvotron

  • https://spinoff.nasa.gov/node/8965
  • Convolvotron - Binaural 3D Sound Processing
  • chipset for realtime 3D panning
  • applications such as aviation, training (military), and of course art & gaming yay

Future of Music Credo Pt 2

  • The experiential designs of real-time "firstperson" sensory interactivity in converging new media are powerful because they individualize changing perceptual viewpoints, expanding visual, auditory, tactile, and motion-based experiences. Even more specific customization will be possible through materials science, neuroscience, and genetics, in health, in medicine – all these will have a relation to music as well. Perhaps we will soon tailor our sonic worlds for each individual's uniquely personalized bio-neuro profile, because it is clear that in the next fifteen years the foundations for a real knowledge of neurobiology will be achieved.
  • Modeling Cage's telescopic perspective, we can ask: How will such developments affect our thinking about the content, duration, and presentation of experiences in the works that we create? How will we take imaginative account of these and other developing trends in planning new work for the future? I believe that it is especially urgent to give serious attention and recognition to groundbreaking innovations in the field of the "emerging arts." Today media exist that begin to approach the range and subtlety of our perceptual modes. As immersive technologies expand and grow to mirror the sensitivity of our responsive energies, will the auditory arts delve consciously into these expansive sensory worlds? And in what ways? Or instead, will our sonic worlds be created with simpleminded "variation makers" – computer software that makes subtle or not so subtle variations and developments of preexistent musical materials?

Avatar Orchestra Metaverse (2006-Present)

  • Specialized Instruments that Use Linden Script
  • Development of Scores for Second Life

Aviaphone from Avatar Orchestra Metaverse

  • Created by Bingo Onomatopei
  • Development of Scores for Second Life

More AOM Please

  • Premiered at the 2010 Digital Resources for the Humanities and the Arts Conference at Brunel University, UK

Claudia Hart and Matthew Gantt with Kurt Henschlager

  • Presented by Hyphen Hub
  • lower technical barrier to entry
  • cartoon like - not uncanny valley - works! - maybe why Roblox is so popular?
  • Matthew Gantt "Perhaps a more interesting question than, how do we achieve lower latency so that we can perform a string quartet in VR, is what new types of musical experiences can we create in this environment."

Dimensions of Immersive Audio Experience - Hyunkook Lee (2023) AES

  • University of Huddersfield, Applied Psychoacoustics Lab
  • Acoustic Uncanny Valley

Remote Access: GlitchRealm

  • Yo-Yo Lin and Kevin Gotkin’s GlitchRealm (2021)
  • realized a space for “disability nightlife and access magic.”
  • Virtual Performances are not just substitute stand-ins for the "real thing" - they might be necessary counterpart to IRL performance, especially for those with assistive needs

Travis Scott & Fortnite: Astronomical (2020)

  • 27.7 Million Unique Players, with 200 Million Views on YouTube
  • Only could have happened in a Virtual Environment
  • brings the concert experience to surreal environments, plays with scale, magic, physics, to create a new concert going experience
  • virtual performance is now not fringe, it's popular 😏

Marshmello in Fortnite (2019)

  • How does this stand in contrast to the Travis Scott performance?
  • 1 to 1 Replication of a typical EDM Concert
  • Not taking full advantage of the possibilities of a virtual space, plays a little with scale
  • Doesn't build on top of the almost 30 years of Virtual Performances we have seen

Ensemble Uhhhhhmm

  • Animal Crossing
  • Uses assets earned or fabricated from the game environment as instruments
  • additionally assets may be purchased through 3rd party channels and sent to users
  • works composed for the environment are inherently spatial

What Works in Virtual Performance

Takeaways 😇

  • some sense of what is effective in networked performance or makes something meaningful
  • be wild in your approach to virtual space
  • what works in real life may not work in url life
  • instruments, performance tactics should be informed by the virtual space
  • interdisciplinarity forges new forms
  • thomasjohnmartinez@gmail.com

Credits

  • Mindy Seu
  • Toni Dove
  • Ludlow 38
  • Jace Clayton
  • Hyunkook Lee
  • Willa Koerner
  • Claudia Hart
  • Matthew Gantt
  • David Grubbs
  • Viv Corringham
  • Sarah Weaver
  • New York Public Library for the Performing Arts Special Collections

  • web tools
  • A-Frame
  • GSAP