In ‘How We Think’, the American pragmatist John Dewey describes knowledge as artificial and thinking as a skill (1910). As he portrays it, our thoughts are designed modifications upon the natural order of things: when we think, we put things in order and create relationship and continuity between them. By designating thinking as an artificial apparatus, Dewey does not mean that our thoughts are factious or unreal. Rather, he invites us to view the word ‘artificial’ as associated ‘with the idea of art, or expert skill gained through voluntary apprenticeship’ (Dewey 1910: 56–7). His description emphasizes the cultural value of knowledge gained through thinking since our artificial knowledge-systems advance the possibility of foresight. Yet, he stresses the contingency of our thoughts – in relation to the processes, the orientation and the contexts of their production. Processes of realization might be truthful or not. A failing might happen if while being immersed in our explanations we have ‘false notions of concrete and abstract.’ In this case, a thinker misconstrues judgements as sense perceptions (Dewey 1910: 135). As he stresses: ‘[a]ctually, all dealing with things, even the child’s, is immersed in inferences; things are clothed by the suggestions they arouse, and are significant as challenges to interpretation or as evidences to substantiate a belief’ (Dewey 1910: 135). Thinking is ‘an operation that may go wrong as well as right’ (Dewey 1910: 19). For that reason, in ‘How We Think’ Dewey emphasizes the need for training reflective thinking. Through reflection we start to notice how we think, and the contextuality, relevance and significance of our knowledge becomes clearer. Dealing with the intermedial performative practice of dancing with VR technology, I ask you to view the creative media of VR technology and bodily performance as artificial apparatus, in which we can both be immersed and develop reflective awareness to the alchemy of leading our visions and movements.

In 2017–2018 I directed and choreographed the practice-based research experiment ‘Playing with Virtual Realities’ as part of my postdoctoral fellowship at the excellence cluster ‘Image, Knowledge, Gestaltug – an Interdisciplinary Laboratory at Humboldt University of Berlin’ (Katan-Schmid 2019). The Interdisciplinary Laboratory clustered representatives from more than 40 different disciplines, who were collaborating in fundamental design processes and co-explored imaging and knowledge production as creative practices. One of the research foci of the Interdisciplinary Laboratory was to investigate how imaging influences perception, thinking and action. In ‘Playing with Virtual Realities’ we explored the relationship between imaging and activity, via challenging the movement habits of dancers and asking them to dance in a virtual environment. The initial outlook, which was leading the project, is that dancers enact movement by following embodied imageries (Fisher 2017; Katan-Schmid 2017). In my philosophical research I deal with dancing as a creative process of realization – one in which images, metaphors, scores, or any other conceptual, cultural and technical understanding of spatiality are explored, embodied and brought into sensual existence. By putting dancers in a virtual environment, we wanted to explore what happens if the imageries dancers relate to are really present in front of their eyes. The motivation for the research was to explore our media as platforms for knowledge production. We related dancing as a form for embodied thinking and VR technology as a medium for immersive exploration. By using VR technology as a metaphor for virtual vision that is provoked by technology, we associated the notion of ‘virtual reality’ with the symbolic order (or epistemology) provoked by our techniques of knowing, among them the techniques of the body (Mauss 1994; 1934). Eventually, the project brought together two dancers, a philosopher/choreographer, two theatre scholars/dramaturges, two computer scientists/gamers, and two media experience designers to co-explore how the embodied practice of dancing can interact with HTC Vive, a virtual reality headset developed by HTC and Valve Corporation. As part of the research we developed a movement practice in mixed realities. We composed physical performance, in which the dancers learned to assimilate awareness to the actual space, to a dancing partner, and to the audience, while negotiating their immersive experience within the virtual environment. The VR applications we used for designing our composition were Tilt Brush by Google, Masterpiece VR by I-Illusions, and the arcade game Space Pirate Trainer by I-Illusions.

At the outset, moving in VR technology interrupts the dancers’ somatic awareness. While dancing dancers tend to integrate attentiveness to their bodily feelings in order to complete their movements. Staging dancers in VR is a challenge for their artistic perception since the VR setting converts the experience of how dancers normally lead their dance. While moving within a virtual environment there is a disruption and discontinuity between what the dancers see and how they feel their bodies. Thus, at first, it might seem that VR technology throws embodied thinkers into a dualist – Cartesian – state of mind, in which visions, actions, and thoughts are disconnected from the sensual body. For example, in the drawing applications Tilt Brush and Masterpiece VR, the users draw with handheld controllers in 3D space: the movements of the controllers become brush strokes, so the imagination of the artist gains a virtual form and the users become absorbed in their own creative process. The idea of using the drawing applications followed William Forsythe’s CD-ROM publication ‘Improvisation Technologies: A Tool for the Analytical Dance Eye’ (Forsythe & Haffner 2012) and the research project ‘Motion Bank’ (deLahunta & Hennermann 2013). In ‘Improvisation Technologies’, Forsythe explains his method of improvisation. The imageries he describes in words and demonstrates in movements are annotated, so the viewer can witness the precision of his bodily movements in relation to the score of the dance. ‘Motion Bank’ is a bank of online digital scores which were created in collaboration with leading chorographers with diverse movement approaches, like Forsythe and Deborah Hay. In both projects the score of the movement gains a digital graphic shape. In ‘Improvisation Technology’ and in ‘Motion Bank’, however, the score of the dance is created by digital artists, who interpret the choreographic instructions and the movements of the dancers, so the dancers can be oblivious to the score as an external entity to their bodies and imagination. While dancing, they lead their dance systematically according to their techniques and their comprehensive methods of negotiating imageries as bodily feelings. Withal, by using ‘Tilt Brush’ the dancers are confronted, while moving, with an external image, which is created by their very movements, and represents them. In ‘Playing with Virtual Realties’ we found this disruptive for two opposing reasons. During the first rehearsals, the dancer Lisanne Goodhue could not relate the virtual representations and her dance at the same time. Goodhue described her experience in VR as fragmented – she could either draw or dance. She could not bridge the two experiences immediately and intuitively into one comprehensive practice. To the contrary, the dancer Nitsan Margaliot tended to integrate the score immediately into his dance, to be carried away by it and to identify it, as he reported, with his own imagination. At times, Margaliot became aware of the limits of the score and felt as if he was stuck in his own movement patterns, as if his own imagination was limiting him, rather than the digital representation. This feeling of identification, between the virtual representation and his own imagination, caused frustration which interrupted his experience. Thus, the problems we faced were either a feeling of discontinuity between the virtual environment and the experience of dancing, or a feeling of lack of methodical and emotional distance between imageries and bodily feelings, which seemed to us necessary for liberating curiosity and releasing the flow of embodied imagination.

Furthermore, within the experience of dancing in actual environments dancers normally follow both their imagination and their visual perception. They enact sensory information with imageries, embody them and bring them into actual life, according to – inter alia – environmental and physical understanding. If I want to perform a grand jeté, I direct my landing in the room; if I perform a pirouette, I keep a focal point while spinning. The perceptual affordance of the actual environment enables a feeling of physical trust and competency, since sensory information inputs from the different senses are continuous (Gibson 1975). As a dancer I can direct my grip, my jump, and my expression as a gestalt between my actual perception, imagination and technique. However, while using the VR headset, the users cannot see their bodies. Thus, the usual gestalt dancers are used to lead and follow is converted and interrupted. In addition, if there are spatial obstacles like a wall, furniture, or another person, the immersed user might run into them. Dancers, who are used to relay upon their vision to perceive spatial relationships, try to keep in mind where they are moving in relation to the physical space, and often lose orientation. In our experiment the dancers became aware to the distance between the environments – the virtual and the actual. The division between the environments affected their trust. The awareness to the gap between environments and to the multi-layered information accelerated, as Goodhue described it, as a ‘fragmented experience’. Moreover, as I witnessed, the feeling of fragmentation disoriented the dancers emotionally, as well. In the first rehearsals, Goodhue reported that, while normally she would feel indifferent to being watched while dancing, the lack of trust she felt within the virtual environment triggered an inconvenient feeling of self-consciousness. Margaliot testified that moments of rivalry between the environments provoked in him frustration and a feeling of incompetence.

Another layer of disruption for dancers who are moving within a VR setting and lose full awareness to their physicality, is their limitation in designing a creative choice-making. While moving in a virtual world, experiencers follow the media design of the VR. In ‘Space Pirate Trainer’, for instance, gamers fight off relentless waves of droids, shielding themselves with handheld controllers while shooting back. The gamers’ movements follow solely the VR setting, so they become oblivious to the physical space. In addition, the VR setting of ‘Space Pirate Trainer’ is highly immersive. While experiencing and rehearsing, it was often difficult to communicate with Margaliot, who wanted to win the next points in the game. As the media experience designer Sabiha Ghellal, who contributed the research and creative process, distinguished: in ‘Tilt Brush’ the experience was too ambiguous for the dancers, while the dancers’ choices in ‘Space Pirate Trainer’ were too prescribed by the game (Ghellal 2017). Beyond the psychological challenge for balancing an experience for users, which is the emphasis of media experience designers, if the dancers could not perform clear decision making, it dislocated their performance from being a creative endeavour. The dancers who are immersed in the virtual realm become too occupied with the virtual vision and do not lead their movements in relation to a composed information. Consequently, although Margaliot’s physical movements were more athletic than the movements of typical gamers, when he was merely following the experience design in ‘Space Pirate Trainer’, we found his performance unexciting to watch (Photo 1). It seemed that, being immersed in his experience, his movements could not be recognized by us as artistic expression. In ‘Art as Experience’, Dewey defines expression as an excitement without discharge:

Photo 1

Nitsan Margaliot interacts with Space Pirate Trainer. Screen-shoot from documentation by Harumi Terayama.

to discharge is to get rid of, to dismiss. To express is to stay by, to carry forward in development, to work out to completion […] where there is no administration of objective conditions, no shaping of materials in the interest of embodying the excitement, there is no expression (Dewey 1980 & 1934: 64).

As we defined it in our research, playing a physical game was not enough for the dance to be an artistic utterance, we needed the dancers to be leading agents, who altogether sense, direct and comprehend the experience they undergo.

The choreographic endeavour in ‘Playing with Virtual Realities’ needed to assimilate considerations from both dance-knowledge, dramaturgy and media experience design. We looked to provide the dancers with reasons to interact with diverse information in both environments, while both reciprocating with a dancer in another realm and articulating experience for the audience. We used a beamer projection on the wall upstage, which exposed the point of view of the dancer inside the VR. The projection of the viewpoint in VR was staged to unfold the perspectives and the exchange between the two dancers. For example, in the first scene a computer voice – representing (a fake) artificial intelligence – introduced the technology to Margaliot. When the beamer projection turned on, the AI voice said: ‘let’s see what you see’. By that, we staged the connection between the beamer projection to the dancer’s perspective. In another scene, Margaliot shared his experience with Goodhue, and before they exchanged the headset, he asked her whether she wanted to see what he sees. Additionally, we staged the differences between the virtual and the physical spaces, deconstructed the experience design in the VR applications, and then created a continuity and correspondences between both realms. For example, in the first scene, using Masterpiece VR, Margaliot pointed out the limits of his virtual environment. Accordingly, Norbert Schröck, a computer scientist, gamer, technician and researcher in the project, marked the limits of the virtual environment in physical space. Additionally, in the opening scene Margaliot and Goodhue moved in parallel spaces: while Goodhue was outside of the VR and followed a method of room drawing from Forsythe’s ‘Improvisation Technologies’, Margaliot wore the headset and drew with the controllers in the virtual room. From a dramaturgical point of view, this scene enabled us to articulate the analogy between following a score while dancing and drawing in 3D space. Later on, we deconstructed the experience design in ‘Tilt Brush’ (Photo 2). One dancer wore the headset, while the other dancer drew scores with the controllers. The dancer who wore the headset and was absorbed in the virtual realm, replied to the drawings as if they were metaphoric choreographic instructions. We associated the haptic-like texture of brush strokes and their positions in space with sensual feelings: thus, the dancer inside the VR embodied the score. The role of metaphors in contributing immersive experiences and bodily presence was explored as a cross-over method in both experience design and choreography. While experience designers use metaphoric concepts, which direct users how to behave in digital environments (Murray 1997: 106), in choreography metaphoric instructions are often used to embody a new physical understanding. When I instructed the dancers how to move by using dance metaphors – a common occurrence in Gaga, Ohad Naharin’s movement research (Katan-Schmid 2016: 65–76) – Ghellal looked for comprehensive metaphors which may reflect the bodily experience of the dancers, among the drawing possibilities in ‘Tilt Brush’. The crossover metaphors in the virtual realm and in the dancers’ bodies provided the dancers with a reason to follow the virtual space, while generating the impression that the virtual realm is an expression of their inner world. In the last scene we layered the information between the environments and staged the virtual environment on the actual theatre stage. Goodhue designed the virtual room with bubbles from the palette in ‘Tilt Brush’, while embodying the bubbles as if they were running inside her flesh. At the same time, Margaliot filled the actual stage with soap bubbles, and mirrored Goodhue’s expression, as if he was learning and connecting with her inner world. This scene continued with a unison, to express the communication and empathy between agencies moving within different environments. Later on, a child performer (Aurica Mosse) came from the audience, entered the virtual realm and embodied the digital bubbles with her performance, while the spectators entered the stage and filled the actual environment with soap bubbles. The divisions between actual and virtual worlds, as well as between performers and spectators were finally merged into a multi-layered, yet continuous, experience. The spectators, who supported the child, seemed as if they were staying by and carrying forward a development of the girl’s vision. By that, the audience took part in articulating her expression (Photos 3 and 4).

Photo 2

Lisanne Goodhue and Nitsan Margaliot communicate while experiencing Tilt Brush by Google. Screen-shoot from documentation by Harumi Terayama.

Photos 3 and 4

The virtual vision of Aurica Mosse is supported by the audience. Screen-shoot from documentation by Harumi Terayama.

Immersive experience is a mirage, which is not developed without comprehensive and systematic administration. In ‘The Theatre and its Double’ Antonine Artaud defines the mirage of artistic symbols as ‘virtual reality’:

All true alchemists know that the alchemical symbol is a mirage as the theatre is a mirage. And this perpetual allusion to the materials and the principle of the theatre found in almost all alchemical books should be understood as the expression of an identity (of which alchemists are extremely aware) existing between the world in which the characters, objects, images, and in a general way all that constitutes the virtual reality of the theatre develops, and the purely fictitious and illusory world in which the symbols of alchemy are evolved. (Artaud 1958: 48)

The creative endeavour of the performing arts includes a process of embodying a reality, which is not there in advance. The ‘virtual reality’ of dancing comes into life when imageries and ideas become movements, feelings and sensations. The feeling of presence in dance signifies a continuous experience within correlation of body and mind. Nevertheless, despite being a mirage, in participatory media, like performance, immersion is also a reflective labor. In ‘Hamlet on the Holodeck’, Janet Murray writes: ‘in a participatory medium, immersion implies learning to swim, to do the things that the new environment makes possible’ (1997: 99). As Murray indicates immersion, it is a metaphorical term for a psychological response to an exceptional, intense, and ultimately transformative experience of ‘diving into’ a new environment (Murray 1997: 98–99). At the same time, ‘diving into’ implies creative attunement; ‘we do not suspend disbelief so much as we actively create belief’ (Murray 1997: 110). In performance and in VR technology, the creative labor of designing immersion seek to integrate sensory perception, intentionality and skill. Both in performance and in VR technology we ask to deliever an artificial world of continuous identity between all experiential elements – we create worlds of ‘make believe’. Nevertheless, returning to Dewey, in ‘How We Think’ he stresses that despite the artificiality of our knowledge production, our knowledge is not necessarily factious or unreal. Virtual reality is also a reality and it has the positive potential to ease our living. Furthermore, both performance and VR technology are chances for developing reflective thinking. By exploring the alchemy of perception, through their media of design, we also generate a chance for observing immersion, while being immersed. As Bertolt Brecht stresses the dialectic of distancing in performance: ‘[i]f one has learned to think dialectically one can find it possible that a technique which is taken from the realm of magic can be used to combat magic with’ (Brecht, 1961: 134). By exploring our media of thinking we learn how to maintain a mental and reflective distance to our knowledge production, while undergoing it. The potential of this dialectics is within the possibility of comprehending our knowledge in context.

The intermedial and interdisciplinary exploration in ‘Playing with Virtual Realities’ forces us to reflect our knowledge-production, in order to find out why our knowledge ‘make sense’ and how it can be adjusted, when we face other knowledge-systems and environments. In our practice, we needed to create a new method for exchanging between sensory-information and virtual visions. We generated a new immersive exchange by defining the relationship between visions and sensations as analogous metaphors. As a result, the aesthetic labor of the dancers could emphasize the embodiment of this relationship, while being aware to the alchemy of their dance. On the day of the premiere, Goodhue told me she felt she was dancing the same as always, that she did not convert her dance into a new skill. This statement, as I understand it, demonstrates that eventually Goodhue could swim effortlessly within the mixed environments. Nevertheless, the choreography, the experience design, the dramaturgy and the dancing had to press out the relationships between the realms, between the players, and between performers and spectators, in order to ‘create belief’ and to facilitate for all participants an easy navigation between diverse realities. As an interdisciplinary and intermedial project, ‘Playing with Virtual Realities’ dealt with a variety of know-hows –the VR technology, media experience design, dramaturgy, choreography, philosophy and the embodied techniques of the dancers among them. We needed to play with our knowledge-systems and with our habits to produce immersive explanatory relationships between diverse environments and to emphasize how they could coexist and intertwine. As Dewey describes thinking, we needed to put things in order and to create relationship and continuity between them (Dewey 1910: 56). Immersion is the proof that our methodical practice ‘made-sense’, which generates a comprehensive meaning and a comfort within a multifaceted world. Our practices, knowledge-systems, methods, and skills are techniques for generating a feeling of competence. Nevertheless, they are artificial and cultural. When we put our knowledge in intermedial encounter, or in a setting, in which ‘make believe’ is within awareness, we learn to reflect, observe and value the alchemy within embodied processes of realization.

Photos 5 and 6

Alexander Katan-Schmid.

Acknowledgements

References