State of the art

To mark the launch of the digital stage HAU4 at the Hebbel am Ufer Centre in Berlin in 2021, Hito Steyerl wrote ‘The Fifth Wall,’ a poem-statement in which she proposes an additional imaginative level of interpretation for the performing arts and digital theatre dynamics:

The 5th wall opens onto a sea of pixels.

The 5th wall is elsewhere […]

The 5th wall encloses no theatre of cruelty but a theatre of extraction

The 5th wall does not reveal a spectator but hides a user

The 5th wall does not spark a V-effect but a VR feeling

The 5th wall swaps the souffleuse booth with a monopolist platform

In front of the 5th wall everyone is a stage.

(Steyerl, 2021)

HAU4 is one of the institutions that emerged to monitor the rapid development of digital performance online (Dixon, 2007). Before and after the pandemic, online 3D virtual environments emerged as crucial sites of social experience. Online platforms such as sandbox and open-world games, as well as online 3D collaboration platforms (Unity et al.) and MMOs (Massively Multiplayer Online Games), structure and facilitate conversation, creativity, collective endeavour and commercial exchange as they facilitate and implement the possibilities of designing virtual experiences through native or external software. Seen as virtual stages, these environments increasingly host performative, immersive collective experiences.

In the 21st century, online digital performances are real-time, augmented, decentralised and collaborative. To quote Boccia Artieri (2023), they are OTONI, ‘oggetti teatrali online non identificati’: online performance actions and live performances in the (proto-)Metaverse, framed in a time from the second half of the 1990s to the present. These objects coagulate around communities, creators and users. They localise and interpenetrate different levels of reality, physical and virtual, online and offline, under transdisciplinarity and DIWO (Do It With Others, or DIT Do It Together).

After Eva and Franco Mattes’ ‘Reenactments’ on the 3D virtual platform Second Life in 2007, several digital performances on 3D online platforms have stimulated a re-evaluation of the nature of being online, leading to a re-contextualisation of the concepts of reality, simulation and community.¹

Three of these digital performances on 3D online platforms are the focus of this article: ‘Connessione Remota’, a telematic performance by Giacomo Verde, broadcast from the Pecci Museum in Prato in May 2001. It is considered one of the first experiments in live-streaming performance. Another notable case study is ‘Second Soup’ by the artist Gazira Babeli on Second Life (SL). The final case study is ‘Toxic Garden – Dance Dance Dance’ (TGDDD) by Kamilia Kard on Roblox. I followed its creation, development and documentation closely (Patti, 2023).

Focusing on the concept of hedonic experience (Hassenzahl et al., 2010) as encountered by users, this article proposes to read these three cases according to the way users interact with the technology, understand, perceive and describe the experience, emphasising psychological well-being through non-instrumental and user-oriented product attributes.

Figure 1

The User Experience Wheel Infographic designed by Magnus Revang in 2007. A model to explain user experience categories. Source: Revang M. (2007): The User Experience Wheel, https://userexperienceproject.blogspot.de/2007/04/user-experience-wheel.html.

Analysis

Reading a digital performance involves identifying its tools, the human-machine dynamics it activates and the experiential effects it produces (Pizzo, 2001). Proposing a poetic reading framework that is more intuitive than empirical, this paper proposes an interpretive analysis of selected artistic examples. Focusing on extended authorship and DIWO, particular attention is paid to the quality of the generated user experience, the relationship established by the technological system and the virtual place.

The case readings focus on whether or not each technological system influences the performative, participatory, and community-building poetics of these kinds of virtual events. These selected case studies were conceived and developed on different platforms, implementing the design possibilities of virtual environments and objects through native software or external graphic design software. An attempt is made to define how these systems allow the viewer to interact with these specific virtual environments, what setting was created, and what UX qualities were generated by the system or by the artist’s intervention.

Figure 2

Verde, Giacomo 2001 Connesione Remota. The performance stage. © Archivio Giacomo Verde.

As mentioned, ‘Connessione Remota’ is considered one of the first experiments in live-streaming performance. Although it did not take place on a 3D platform, I chose it as an emblematic case of the beginning of experimentation in digital performance and web theatre. In ‘Connessione remota’, viewers in ‘remote connection’ could gather in a virtual space and watch the performance over the web, engage with each other online, chat, and interact with the performer in real-time. The virtual space was utterly flat, open source, and entirely modelled by Verde, as was the user experience designed to encourage voyeurism and remote dialogic interaction via chat. Once connected, users could text and watch Verde create short performance actions while wearing one of the first Sony headsets. The integration of real-time chat facilitated audience participation and interaction.

Before 2000, the network had a speed of 56 KB, and it took a long time to load the whole page; the cost was that of a long-distance call – to an Internet service provider. After 1 January 2000, with the ADSL system – with broadband over the 144 kbps threshold – it made sense to land on the Internet with one of the first telematic services. Despite these difficulties, the idea of the Net as an art form, as a generator of connective social relations, put into practice by many underground artists and collectives through hacking and popularised by theorists, prevailed (Bazzichelli, 2006).

Once connected to the webcamtheatre.org site, everyone could see a typical ASCII screen divided into three columns: one contained an introductory text entitled ‘From interpretation to performative interaction’. The second column was connected to the webcam to see Verde’s real-time performance. The third column housed the chat between users and with Verde himself, through which Verde gave users some written instructions, such as ‘starting time 15’, but also ‘start one thing at a time’. Given the complete novelty of the experiment, it was imperative to facilitate the UX, interaction and user experience through explicit instructions. For this, at the bottom of all three columns were both the ‘close’ buttons for the content of each column and an indication of the start time for reopening the content itself.

I guess that the video connections will be quite slow and difficult.

They can only decently support up to ten visitors at a time.

Anyway, the perf. It should last about 10 minutes and is repeated

probably with an interval of 10 minutes. – We also hope to make

10/10 success -³

(Verde, 2001)

Describing the ‘Connessione Remota’ in terms of the main dimensions of UX, both accessibility and usability were truly innovative for the time – even if primitive compared to today – centred on a robust space-time experiment. If the languages used were chromatically and aesthetically synthetic, the emotionality aroused was directed towards the novelty of the dialogue at a distance; thus, usability induced positive emotions in the user – almost bridging the technological gap that limited desirability. Although minimal compared to today’s possibilities, the level of interactivity was advanced for its time, even if it only centred on the possibility of opening/closing content in columns and commenting live on the entire experience via chat.⁴

The reference mental model for Verde and users in 2000 was the original Internet UX of the time: the interaction was conveyed by writing the code, and only some users had the hardware and software capabilities to interact. In particular, the sites/platforms were designed to collect information and documentation that would be stored and freely shared by all who could connect, in line with the philosophy, the democratic ‘dream’ that characterised the vision of the Internet at the time. Nevertheless, they were only partially modifiable.

To open source, Verde adds the dynamism and philosophy of interactivity typical of his theatrical artistic style: he creates a website where users can watch a performance and exchange feedback in real-time. To be able to contribute only partially to modelling the dynamics of the experience.

‘These experiments confirmed my intuition that it is possible to make theatre with/for the Net, taking into account the sense of community often activated on the Internet more convincingly than in many other material venues’.⁵ Verde accepted the hacker’s idea but first thought of the technology as a social tool, an opportunity to create a community. The ambitious project was weakened by the fragility of the users’ connection, the network’s different speeds, and the constant and inevitable need to refresh the page to update the data at regular intervals. Although this gimmick ultimately acted as an interface/stage for Verde (Monteverdi, 2024).

As a second case study, artist Gazira Babeli’s ‘Second Soup’ in Second Life (SL) is worth mentioning. Founded in 2002 by Linden Lab CEO Philip Rosedale (a.k.a. Philip Linden), initially as a commercial platform, SL was one of the first 3D online virtual spaces for the development of specific social interaction projects, fostering, in particular, the use of chat and, from the second half of the 2000s, the first forms of user feedback systems, such as skins and emoji. In March 2005, version 1.6 of Second Life was launched, with several innovative user features, including QuickTime multimedia streaming and a standardised interface for building environments. This version allowed users to create original and customised animations for their avatars, creating more opportunities for social interaction. In January 2007, SL released its client as open source. Residents (over 1 million at the time) participated in the Architecture Working Group (AWG) to influence its development by developing the protocols that would make SL’s grid manageable by Linden Lab and other users.

Figure 3

Babeli, Gazira, Second Soup. You love pop art, but pop art hates you. Scripted cans, still from video. May 2006 © the artist. Source: http://gazirababeli.com/secondsoup.php.

Towards the end of the 2010s, access to the platform required a computer with relatively high hardware and software system requirements. Once logged in, the user experience for SL residents was (and still is) characterised by customising their avatars for surreal but routine activities. For the first time, native functionalities focused on the possibility to customise the user’s avatar easily; then, using WASD keys, it was possible to explore the environments with no purpose and interact according to verbal, aesthetic-symbolic, musical and movement languages borrowed from real life (RL), but also invented, jargon. Being open source, the possibilities of interaction and participation in the platform’s development quickly spread through coding and compatibility with external software, virtual objects, gadgets, entire buildings and cities.

Gazira Babeli has been living and working in SL as an artist, performer, and filmmaker since 2006. That year, she published recordings of some ‘unauthorised performances’ online and joined the Second Front group.⁶ In her words:

The real performance starts with the login; the rest is a recording of the performance.

The avatar tries to forget that it is a code.[…] I prefer thinking of the whole SL environment as (a kind of) frame space.

It means that scripted and built objects, avatar-people and their behaviours become parts of the artwork…a ‘world in a valise’.

In SL, you forget the ‘computer’. It disappears, and you are completely inside the frame-space. […] Everyone is an actor, director and audience together. But is that so different from what we call RL (real life)?

(Babeli, 2007)

She became famous online for her performances in unconventional virtual environments, such as squares, streets and open spaces, in front of unwitting audiences who usually react badly. As Steve Jobs unveiled the first iPhone, Gazira presented her first retrospective at the ExhibitA gallery on SL in the spring of 2007. It was an opportunity to collect and re-present many of the performative actions the artist had created on SL the previous year in the form of interactive virtual installations, including the most famous ‘Second Soup’, an experience in which giant cans of Campbell’s soup were activated and made aggressive by the avatars of passing users. Even more obviously, this UX was deliberately focused on accessible, amusing and irreverent dimensions: as with everything SL, there was no purpose or functionality – let alone findability and credibility. The experience is entirely user-centric: the space in which it takes place is anonymous, with the only personalised element being Campbell’s interactive cans, activated by the simple passing of the avatar. The performance is only created when the avatar passes by, and the UX is designed to maximise the emotional return of the interaction.

The notion of non-pragmatic (or non-utilitarian) hedonic qualities – which are pleasure, enjoyment, excitement, fun, and happiness, but also novelty and interactivity-social element in the context of technology (Stelmaszewska et al., 2004) – originated in consumer research and was applied to interactive products by Hassenzahl (2003). According to this view, instrumental, task-oriented, and pragmatic attributes (e.g., ‘useful’ and ‘controllable’) are primarily related to behavioural goals. In contrast, hedonic attributes emphasise psychological well-being through non-instrumental, user-oriented product attributes. When hedonics was first introduced, Hassenzahl proposed an ‘extended concept of usability’ that focused on user satisfaction, in line with gamification and challenging the then-common notion of the computer as a tool to be taken ‘seriously’. The emotional impact is one of the most studied effects of contrasting and manipulating hedonic qualities. Studies have essentially shown that hedonic experiences generate more positive emotions than less hedonic/more pragmatic products (Diefenbach et al., 2014).

Drawing inspiration from sources as diverse as Dada, Fluxus, the Situationist International and contemporary performance artists such as Laurie Anderson and Marina Abramovic, Babeli has created performances and choreographic interventions that amplify the native UX and fundamental assumptions of SL, exploring what it means to be a virtual being in this space. Babeli not only reinterprets works from art history and contemporary performance by recoding them and placing them in a virtual environment but also reinvents them according to the desecrating and interactive mood typical of SL, exploiting the situationist and hedonic or non-activity-oriented UX typical of SL and focusing the performative action on the possibilities and variety of responses and interactions with the large community that freely inhabited these public spaces.

According to the analysis of critics and curators who have inhabited the platform and analysed the community (Flimflam et al., 2007), SL’s UX can be said to have brought each inhabitant closer to being a performer – Dadaist, Situationist – by the simple fact of inhabiting a collective virtual space and interacting with it through instructions/codes that were displayed and interpreted into objects, images, movements—a collective happening, simultaneous and in real-time. In the intervening years, the UX of SL has become the mental model for most of the social platforms of mass interaction.

Figure 4

Kard, Kamilia, Toxic Garden – Dance Dance Dance. Digital performance on Roblox, still from video. © the artist. Source: https://not.neroeditions.com/ballare-nel-metaverso/.

The third and final case study is Kamilia Kard’s ‘Toxic Garden – Dance Dance Dance’ (TGDDD) performance created in Roblox in 2020.⁷ This popular massively multiplayer online game allows users to come together to create and share virtual map content. Launched in 2006, Roblox has quickly become one of the most popular games in the world, with millions of active users every day whose sole purpose is to freely create environments and related activities of all genres, from adventure to role-playing, from simulator to massively multiplayer, using the tools provided by Roblox Studio, a free ‘in house’ development software. In multiplayer games such as Roblox or Fortnite, among others, the user experience focuses first on the customisation of the avatar (as in SL, but for a fee), then on promoting and facilitating the communication component through emotes – small animations of the avatar – and the customisation of movements, environments and objects, both free and for a fee. Emotes allow the user’s avatar to express itself through more complex gestures and facial expressions, such as a short dance to express satisfaction, a hump to express disappointment, and more.

At first glance, Roblox’s UX appears highly hedonic, focusing on desirability, interactivity-social elements and the agile transmission of users’ moods. De facto, the hedonic impression is given by the emotional impact that all dimensions, even the task-oriented ones, are charged with. In the apparent nonsense of the proposed experience, we find the keys to discoverability, credibility and usefulness. This UX philosophy is not exclusive to Roblox. On the heels of SL, most of today’s densely populated social platforms focus their UX proposition on maximising desirability and extreme simplification of usability, which disappears from the other dimensions. This is also a marketing strategy: the platforms – as brands, as technologies and as values – are associated with the pleasantness of their experiences.

TGDDD focuses on the affordance of Roblox (Van Dijck et al., 2019), which focuses on engaging children and adolescents in massive social and virtual interaction through this hedonic UX. Dance, social interaction and music are central elements in Roblox and TGDDD. In many of Roblox’s maps, this formula is widely used, and the maps become a place of encounter and exchange, characterised by simple graphics, where avatars meet to get to know each other and form small dance groups. However, unlike Roblox, TGDDD does not allow to customise avatars. As soon as entering the environment designed by Kard, users ‘lose their uniqueness’ and are given a common, randomly assigned avatar. According to the artist:

Making an avatar dance comes naturally.

This spontaneity is partly due to the proliferation of movement animation libraries that are primarily composed of dance or combat steps (such as Adobe’s Mixamo, or amateur open-source libraries) and partly due to a kind of liberation of movement, a letting go that is expressed through digital doubles and dance.⁸

(Kard, 2023)

In a ‘toxic’ map created ad hoc by the artist, the first part of the experience allows the viewer to explore the space, as in many open-world games. In the second part, improvised avatar crews are involved in group dances (with choreographies taken from contemporary dance mocap libraries), automatically synchronised with the movements of Kard’s avatar, KKlovesU4E. The association between avatar, communication and dance combines the textuality of chat with the body’s expressiveness – albeit digital and often stylised – in real-time.

The spatial environment modelled by Kard is reminiscent of a poisonous garden, a metaphor for toxic human relationships. The theme of TGDDD emerges through the encounters between the avatars and the comparison with what is loaded on the map: the garden is striking for its bright colours and exaggerated architectural plants. To compose the experience, in addition to creating the environments, Kard modelled the movements and interactions between the avatars using various systems, from motion capture to AI. The result is condensed around the virtualisation of dance steps as individual symbolic units of feelings and attitudes related to toxic relationships.

The TGDDD can be seen as a device that promotes sharing one’s own experiences within a regulated and a priori environment, where a confrontation with the experiences of others on the topic of toxic relationships – expressed through the graphical interface of dance steps – takes place. One of the central aspects of the experience that Kard offers Roblox users is the inability to customise their avatars once they enter the map. Kard prevents the game from functioning naturally by assigning participants predefined, random skins she designed. In this way, she disrupts Roblox’s native UX and makes this change the focus of the performance. In addition to inviting and allowing the user to explore the garden of poisonous plants, she focuses the interaction between the user’s avatars on using and interpreting the movements of a library composed by her rather than the Internet. By limiting the customisation features typical of Roblox’s UX, she limits its pleasurable and emotional effects, making them fall into dimensions closer to findability more clearly – those aspects designed to make the UX more brand and marketing-oriented. TGDDD’s UX completely undermines the mental model behind Roblox’s UX; This inversion helps to metaphorise toxic relationships.

Conclusions

The selected case studies were designed on different platforms, implementing native user experience design functionality through native or external software – depending on the time and stage of development of the digital environment and the Internet – to create specific UXs that differed from the established ones.

In the early 2000s, Verde designed a unique, completely open-source UX from scratch. Babeli in the late 2010s and Kard in the early 2020s started from the basic functionality of an already massively populated platform to interact with the community of residents: Babeli optimised the pre-established UX both technically and ideologically; conversely, Kard disrupted the mental model of UX, its fundamental dynamics both technically and philosophically. While Verde proposed a minimal environment with minimal but innovative interaction possibilities for the time, explicitly creating a community to interact with, Babeli and Kard operated in already heavily modelled private platforms, riding usability partly at the expense of privacy but in favour of accessibility and more massive interaction. All three are artist-coders, using code to model a well-defined UX that, through the active participation of users, becomes the centre of the proposed artistic operation.

None of the three modelled experiences is functional for achieving pre-established goals or objectives, and no third-party information is conveyed. The artist indicates to the users a space-time of encounter, draws a few objects and a few rules of composition, choreography and interaction. Then, the composition is open: by interacting with the space and other users, the spectator shapes his or her own experience, individually and collectively. Starting from Verde and arriving at Kard via Babeli, it is possible to follow a trajectory in which even the most functional dimensions of a UX are progressively and systematically given emotional and aesthetic connotations. This aligns with studies on the evolution of user experience design, which note a progressive hedonisation of the technological experience.

All three case studies can then be described according to a UX philosophy favouring the ‘hot’ dimensions of Revang’s wheel – accessibility, desirability and usability – which focus on the aesthetic, psychological and emotional qualities of the experience. This, above all, takes a user-centred perspective on experience design at the expense of the more pragmatic dimensions of findability, credibility and usefulness. Like the Babeli performance, the hedonic philosophy of UX could be described as Situationist, open: a ‘forget the computer’ experience.

Therefore, the three artists’ choice falls on a hedonic UX design associated with highly positive emotions. Thus, the subversion of the mental model – or creating an open source and interactive UX – is deliberately associated with positive emotions. In these cases, the hedonic philosophy of UX corresponds to a ‘Dada’, alternative and purposeful artistic vision of interaction with the system, which is crucial for imagining non-commercial forms of storytelling, interaction and participation and for building diverse communities outside of the marketing/hyping dynamic of private platforms.

In conclusion, to quote Hito Steyerl again, the Metaverse is a highly performative and interactive dimension. As artists do, it can be experienced and designed as a theatrical backdrop that raises both aesthetic and ethical issues of inclusion, collectivity, economic and environmental sustainability and justice.

Competing Interests

The author has no competing interests to declare.

Author Information

Federica Patti is an independent curator and lecturer. Her research focuses on transmedia practices, digital performance, posthuman issues and the metaverse. She is a PhD student at the University of Turin, researching ‘The Fifth Wall. Posthuman performativity and extended experiences’, online digital performance and user experience design. She actively collaborates with various institutions (among others, the Romaeuropa Festival), curating art, science and digital humanities exhibitions. She is a member of the curatorial collective LaRete Art Projects and IKT – International Association of Curators of Contemporary Art. Since 2020, she has been a tutor of ‘Residenze Digitali’.

In 2010, she graduated at the University of Bologna Alma Mater Studiorum with a thesis on interactivity in contemporary art.

She composes the newsletter LUNARIO – segnali dal Metaverso every full moon.