re:MakingSense (2021)

The “re:MakingSense” series aimed to foster ongoing engagement and exchange among professionals in the sensory augmentation community. The sessions in this series were designed to stimulate collaborative dialogue across disciplines and encourage new inquiries into the future possibilities of sensory enhancement technologies.

The “re:MakingSense” series was created in 2021 as a platform to bring together researchers, artists, designers, and practitioners working within the field of sensory augmentation. This event fostered interdisciplinary dialogue to explore the philosophical, ethical, creative, scientific, and technological aspects of augmenting human sensory and perceptual abilities. The discussions aimed to broaden understanding and fuel innovation by engaging multiple perspectives on sensory enhancement.

Together with Lesley-Ann Daly, Imre Bard, and Stéphanie Gauttier, Michel Witter curated and organised the re:MakingSense series.

Session #1: Challenges and Opportunities for Sensory Augmentation

The inaugural session of “re:MakingSense” set out to assess the current state of sensory augmentation. It delved into key questions regarding obstacles to widespread adoption of sensory enhancement technologies, advances in understanding human perception, and the social and conceptual challenges shaping this emerging field.

Featured Speakers:

1. Marianna Obrist – Professor of Multisensory Interfaces at UCL
   Marianna’s research focuses on integrating touch, taste, and smell within human-computer interaction, applying these insights to areas including virtual reality, automotive design, and healthcare: https://www.youtube.com/watch?v=e2Q0M2XSgZY
2. Scott Novich, Ph.D. – CTO & Co-founder, Neosensory
   Scott emphasized non-invasive sensory augmentation through touch and highlighted the potential of low-power, scalable systems tailored to human sensory receptors: https://www.youtube.com/watch?v=Ia35tM7sNZs
3. Liviu Babitz – CEO & Co-founder, CyborgNest
   Liviu discussed “Plug n Be” devices, sharing his perspectives on the prolonged process the brain undergoes in adapting to new sensory inputs, likening it to the sensory learning journey of infants: https://www.youtube.com/watch?v=00uRzp6BtTg
4. Madeline Schwartzman – Artist, Author, Educator
   Madeline’s work was presented as an exploration of perception-altering art and technology, tracing developments in human sensorium from the 1960s onwards: https://www.youtube.com/watch?v=1Lw9lMaAqqg

#2 Translating, Substituting, Augmenting Senses

This re:MakingSense session focuses on the process of translation between sensory modalities. It questions how the attributes of one sense are mapped onto another, how this new sensory information is integrated into perception, and how novel information is translated into senses. What technology is needed to do this? What information is sent? What is lost? How does that affect the resulting perception? And how does this, in turn, affect social interaction and performance?

Our brains receive information from our sensory organs, interpreting the incoming electrical signals and conveying them to us as sensory perception. Today, technology makes it possible for individuals to receive information in one sensory mode and have it interpreted in another. For instance, ‘hearing’ light is possible. It is also possible to transmit non-sensory, or non-human sensory information to the brain to interpret as a sensory perception. For example, one can feel the Earth’s magnetic field.

For humans, this means that they either substitute some original (natural) senses with technology-based ones, try to add new senses to those they have, or perceive in an enhanced manner. Perhaps sensory augmentation technologies are even both substituting and augmenting senses at once.

Featured Speakers:

1. Dr. Giles Hamilton-Fletcher – Postdoctoral Research Fellow at NYU Langone Health
   Gilles his research focuses on finding optimal ways of turning visual images into sound, so that listeners can accurately reconstruct the original image. He holds a PhD in psychology from the University of Sussex. https://www.youtube.com/watch?v=LZjz7vxWbYU
2. Antal Ruhl – lecturer and researcher at Avans University of Applied Sciences
   Antal was trained as an industrial product designer at The Hague University of Applied Sciences. After completing a master’s degree in Media Technology at Leiden University, he focused on conducting research by creating custom interactive tools. https://www.youtube.com/watch?v=O1FDDLucgPY
3. Roseanne Wakely – Designer, Director & Co-founder of Rusty Squid
   Roseanne is a future-focused designer, artist, and maker, with playfulness and inclusive design at the core of her practice. She works within an ecosystem of robotic technology, digital fabrication, and social experimentation. https://www.youtube.com/watch?v=Km-xlohMQcU
4. Carl Hayden Smith – Director of the Learning Technology Research Centre at Ravensbourne University London
   Carl is also the co-founder of the Cyberdelics Society and the London Experimental Psychonautic Club. His work focuses on using both technological and biological means to alter, probe, and study the spectral nature of consciousness. https://www.youtube.com/watch?v=Km-xlohMQcU

#3 Work in Progress

The third and final part of the re:MakingSense series deals with conceptualizing, criticizing, and constructing meaning around sensory augmentation. Such work typically requires time, interdisciplinary knowledge, and various forms of input. Practitioners, students, and professors present their work-in-progress in diverse directions. The input and feedback from the participants and other speakers are used to make their ideas clearer and elicit useful feedback on early-stage work. This session also enables collaborations on ongoing projects with attendees.

Featured Speakers:

1. Alwin de Rooij – Assistant Professor in Creativity Research at Tilburg University and Associate Lector Situated Art, Design, and Technology at Avans University of Applied Sciences
   Alwin suggests that some physiological signals that cannot be (consciously) sensed by our bodies but can be detected with sensor technology, can be made available to a person through sensory augmentation principles. That is, via real-time mappings from sensor technology data to signals that the body can readily sense—a concept he calls “Augmented Interoception.” https://www.youtube.com/watch?v=q_NMe6xqJ_k
2. Louise Ward Morris – MA student in Fine Art at Central Saint Martins, University of the Arts London
   Louise’s current project, Total Parts, is a video and animation artwork created as part of her recently completed MA in Fine Art from Central Saint Martins, UAL. The work explores the reciprocal relationship between touch and sight in human sensory perception and how digital images and technological apparatus can disrupt this delicate balance. https://www.youtube.com/watch?v=Iiyc1M9xT1M
3. Hallidonto – Visual artist, London-based Cyborg Iconoclast
   After suffering a brain injury in 2006 that severely impacted his hearing, Hallidonto developed his own interpretation of sense. His work explores sensory input and alternative ways of experiencing the world, particularly in his multilayered opera production Sanctum Cyborgia. His current focus is on determining which immersive technologies best suit the opera’s vision while prioritizing accessibility, technological inclusivity, and economic sustainability. https://www.youtube.com/watch?v=FaOoFRyghZA
4. Michael Devine – UK Firefighter, Specialist Rescue Operator, Recruit Instructor, and Researcher at Ulster University
   Michael Devine started his career as a researcher, earning a Master’s in Medical Science before joining the Northern Ireland Fire and Rescue Service. Over his 17-year career, he has worked in diverse roles, including firefighter, specialist rescue operator, and recruit instructor. He is also a part-time researcher at Ulster University, examining the potential of wearable technology to improve firefighter health, safety, and well-being. His research is self-funded and done in his own free time. https://www.youtube.com/watch?v=Ai7HxcMZdXU

A word of thanks

We thank the Centre of Applied Research for Art, Design, and Technology (CARADT) of Avans University of Applied Sciences to host all contents of our re:MakingSense series of online symposia. We also thank the inspiring and thought-provoking speakers and all people that were present and participated in the discussions. You were awesome!

Video recordings of all individual presentations can be viewed here: http://www.youtube.com/@remakingsensesymposiumonse4316