Academic Paper

Read the academic paper published by UbiComp/ISWC 2025: The State of Performance Art and Interactive Technology

Project Description

I made a simulation of myself that you can speak to and it will respond in my voice, in realtime. On the tech side, it utilizes multi-modal AI, using both the OpenAI and ElevenLabs APIs. A custom fine-tuned version of ChatGPT acts as the ‘brain’ and mirrors my personality and communication style, a voice clone created in ElevenLabs makes it sound like me, and it utilizes RAG for memory/biography.

This project culminated in a durational performance installation in which I engaged in a two hour, unscripted conversation with my simulated self from behind a two-way mirror, so that the audience could see me, but I could not see them. Viewers were invited to submit text prompts that I was required to speak aloud to the simulation, provoking responses that looped between my human and digital self.

Initially undertaken as a technical exploration of whether I could create a convincing simulation of myself that could be interacted with in a realtime conversational manner, the scope rather expanded as things developed, into a larger exploration of philosophical questions surrounding the fluidity of identity, who we are to ourselves and others, the impossibility of being objective about oneself, technical gaps and the truths they expose, and how little might be needed in the space of an interaction to effectively evoke at least the gist of one’s identity.