Introduction

This project focuses upon body-movement perception and body sense perception within body-movement interaction installations exploring new media aesthetics and technologies.[2] Within this arts-practice as research, I created a series of interactive installations that explore different methods of stimulating a more creative experience in the engagement within the interactive 3D audio-visual environment. This active realization of body-movement perception is an alternating or parallel manifestation of reflective and immersive moments, which I have shown in the non-Cartesian Interactive installations.

I position my critical and artistic practice within the field of contemporary new media practice and explore Bergson's theory in my own practice, that the body is the 'Centre of in-determination' that 'enframes' the image.[3] This 'Centre of in-determination' is a framing function of the body in cognitive perception that perceives what itself needs, out of the universe of images surrounding it. By altering body-movement in the digital audio-visual interactive environment, would a more creative cognitive experience be achieved in the creative imagination? Body-movement perception in the digital interactive environment is not thought here to aid creative professional performance or dance. It is the subtle changes in body-movement that affect perception and consciousness in the creative imagination in interaction with the audio-visuals, that is central to this research.

The dialogue between my experimental practice and the theoretical concerns in the interactive designs focus on how new technologies can have an impact on the body's consciousness and perception. The affect of digital media and virtual reality on the body raises issues on embodiment, presence, perception and consciousness. The first section of this paper discusses my 'Non-Cartesian' interactive installation in the 'Emerging technology Exhibition' at Asia Siggraph 2012. This was an art installation for the purpose of exploring how interactive design digital media can enhance imagination by effecting consciousness of body movement, that are embodied in audio-visual media.

I explored interrupting body-movement perception as the 'Centre of in-determination', in the interactive digital environment. The starting point for these interactive designs were experiencers' accounts of their own experience of the non-Cartesian interactive installations. These are accessed and recorded, by video documentation and video-cued interviews. Later I found my own subjective experience more valuable, and the later section of this paper discusses my findings in relation to body-perception and embodiment theory drawing upon phenomenology and cognitive science with the aim to discover if a new aesthetic or language to experience an interactive digital artwork, via body-movement, can be achieved to enhance creative imagination. This practice-as-research developed its inquiry through parallel conceptual and experimental frameworks. The conceptual frameworks of body perception theories informed the interactive designs. The experimental frameworks were the series of experimental interactive designs for the body-movement interactive audio-visual installations.

Non-Cartesian interactive installation

In the Asia Siggraph 2012, 'Emerging Technologies Exhibition' I created an interactive installation focussing upon interactive audio-visual perception and its affect on the experiencers' creative engagement with the audio-visual environment. This was one of a series of investigations in interactive design exploring perception, consciousness and embodiment in audio-visual digital environments. This particular installation focussed more on interactive sound, in combination with interactive 3D visuals.

The visuals in this interactive installation were projected as a virtual non-Cartesian digital environment within the physical, spatial environment of the 'exhibition'.[4] It formed a dialogue between the experiencers' known stasis of the body and the non-Cartesian virtual environment. The non-Cartesian environment was chosen and designed to transform known feelings and experiences of Cartesian dualism space within the digital interactive virtual space. One of the reasons for this is that we are embodied into an architectural world of verticals and horizontals and its relation to our anthropological state of being. The interactive 3D audio surround adopted a dissociative meta-language of sound for interaction in combination with the 3D visuals, forming a quasi-non-Cartesian sound environment.

The single channel projection with surround sound had three adjacent walls surrounding a dark open space, large enough for body-movement-interaction. In the virtual simulated non-Cartesian projection were quasi-tectonic 3D forms that had transparent, laser-ray like structures that formed a non-Cartesian aesthetic. The body-movement xyz Cartesian 'live-data' of body motion and gestures of 14 body-limb nodes' could manipulate the 3D virtual non-Cartesian structures, as well as the surround sound.

Within the large space I used 'Kinect' sensor technology with a Max MSP programming environment to track position, velocity and gestures of body joints. Max MSP translated live xyz spatial and orientation data from body-joints, so that experiencers could engage, by body-movement interaction, including head and limb-mobilisation with the virtual and quasi-tectonic forms within the quasi-non-Cartesian space.

Figure 1: Non-Cartesian (2012) © Ken Byers

Experiencers' walked into a darkly lit space (Figure 1) within the 'Immerging Technologies Exhibition'. They began to interact by body-movement with displaced images and surround sound within the embodied environment, which affected a parallel or divided attention, of consciousness in the experiencer. These non-Cartesian spatial images seemed incongruent with their own embodied movement and the organic and analogue synth sounds (Figure 2). This formed a new environment for the experiencers' to relearn body movement, breaking from their past-embodied experiences of the world.

The experiencers' body-movement interaction of 3D visuals could zoom in and out, rotate, change velocity, change direction and distort. The experiencers realized that their embodied movements did not correspond fully with what they had expected from the visual and surround sound (Figure 3), yet they still felt a sense of presence and meaning. The reason for this is that unconscious processors are convinced of this quasi-non-Cartesian reality, a virtual reality, whilst being embodied within the physical space.

The dynamics of the interactive design algorithms cause interruptions of body behavior and gestures for the experiencer, randomly changing every three minutes. This caused the experiencer to re-engage their proprioceptive habitual movement within the construction of the audio-visual, forming a new kind creative engagement. Creative transformation of the audio-visuals and aesthetic engagement are made possible by the alternating or a parallel manifestation of reflective and immersive moments. This draws the attention of the experiencer to the temporal continuity of their embodiment, whilst in parallel the newly obtained 'virtualized subjectivity' is projected. By 'virtualized subjectivity' I mean the subjective state of the experiencer is mirrored onto the screen.

The 'live body data' of body limb gestures were translated to control parameters of a 'Max MSP' poly-synth. Gestures control the synths filters, reverb, velocity, frequency, and pitch. The interactive body-gestures of the experiencers' cause the sound to have a displacing affect on the body, which in turn affects body-movement perception. It's here that striking disparities of the sound cause an effect of movement and posture, as well as change in visual perception. Body movements could consist of full body motion, and movements from all body limbs. The live-data body-movements from all body limbs engage with the non-Cartesian.

Figure 2: Non-Cartesian 'Warped Space' Topographical (2012) © Ken Byers

The installation was sufficient to cause body-movement awareness and aesthetic distance during the interactive experience. The experiencers naturally relearned or preformed interactive movements in relation to the movement of the 3D structures and sound. The embodied affect of the combination of both 3D interactive manipulation and sound in the 'disruptive' non-Cartesian forms a new aesthetic, a dialogue between the body's movement and the cognitive perception of the experiencer's imagination.

Interactive body: 3D audio surround

In the body-interaction, non-Cartesian installation experiencers' could interact with the predominance of sound or visual fluctuations, to develop a more integrated meaningful aesthetic in the interactive environment. This particular installation was predominantly set up and designed for audio interaction. However the complexity of audio-visual perception in these digital environments caused a more meaningful result for the experiencers.

Experiencers interacted and perceived the digital surround sound from where the speakers were positioned, the digital relocation of the meta-language sound and the form of their own body movements. Six speakers were positioned around the space at different heights. In the non-Cartesian installation, spatial information, and spatial content, in combination with the 3D non-Cartesian caused dissociated feelings, within the experiencers. The dissociated feelings described by the experiencers caused a more active engagement with the interactive art installation. Although in visual perception we are aware of space, with audio perception we are aware of things in space. For example, if a high note is generally associated with verticality but is heard lower down, it causes dissociation in the body. Temporal aspects of sound, such as time being suspended or moving forward, slow or fast, at a distance or in close proximity or with upbeats and downbeats, feedback to the interactive body in motion. In this way the dissociated interactive visuals were explored in conjunction with the interactive meta-language of surround sound.

I have experimented with the dissociation of embodied meta-languages within the interactive design as a method for challenging the experiencers' hearing and visual perception. Verticality for example is commonly understood in the realms of pitch and harmony i.e. the perception of high notes and low notes. Within these parameters there are rhythms, upbeats and downbeats, rhythms that can be grounded or floating, as well as the sense that time can be suspended or moving forward. Embodied meta-language was included in the interactive design to enhance the virtual non-Cartesian environment. As sound affects the sensation of bodily position, presence, and reciprocity, interactive body limb movement may also affect synthesized sound. The bridge between the body's re-embodiment and the fraction of time it takes to preform a movement in relation in the audio-visual-visual non-Cartesian triggers the creative imagination of the experiencer.

To make clearer what I mean by this, in Dancing with a Virtual Dervish Diane Gromala and Yacov Sharir demonstrate a method of incorporating body perception theory into contemporary artists' work. Gromalas' Virtual Reality HMD project incorporates an enormous simulation of inside the human body, including rib cage, kidneys, heart and so on 'that allows the participant to dance through the inner spaces of the 3D body parts, via a non-Cartesian interface that transforms the body as a geometric volume into a dimensionless topological intuition' (Hansen, M. 2004: 180). 'Gromala goes on to explain, that her project aims to exploit proprioception, (the inner sense of what we are in our bodies) as the basis for "re-embodiment", a 'reconfigured and enhanced experience of [the] body' (Hansen 2004: 180). Gromalas' VR project was thus the first to consider Virtual Reality as an embodied experience by traveling into the deep space of the body was the desire of escape of the body. Previously Virtual Reality where previously Virtual Reality had encapsulated the desire to escape, to enable it to travel or fly through outer worlds through an immersive disembodied environment.

My own project and contribution to knowledge exploits proprioception in body-movement interaction with audio-visuals, as a means of designing better interactive installations. Through a series of live-data new media interactive audio-visual installations that cause the experiencer that cause the experiencer to aesthetically reflect, on their movement and interaction. It places perception and consciousness outside the body into the aesthetics of the audio-visual non-Cartesian so this thus is not one of immersion that explores the inner feelings of the body, like Gromalas'. In Figure 2, many complex structures are attached to the various limbs of the body of the experiencer and are drawn, moved, rotated, through interaction. Whilst the surround sound is played by their control of gesture. The skin of the body, the extremes of the limbs, reaches out, beyond the body into the techno-aesthetic environment of the non-Cartesian structures, forming a new configuration between body-movement and the aesthetic of the non-Cartesian.

Most of the technological advancements in audio technologies have been designed with particular regard to creating the effect of immersion, or presence, as in cinema and virtual reality (Dyson, 2009: 140). It is only when the recipient experiences immersion that they can experience presence. The experiencer alternates between experiences of presence and reflection. In these interactive audio-visual installations I have explored what happens when sound is dissociated from the usual cognitive perceptions of sound. The dissociated digital sound forms a new aesthetic on perceiving the environment, in which the physical body is momentary 'stalled' and is in a state of cognitive perception and consciousness, because it is not used to such disparities It is during these preformed states of interactive movement that an aesthetic reflection takes place, when the experiencer is not immersed.

Sound affects the sensation of bodily position and presence just as the body position or movement of an experiencer may also effect sound associations. Sound combines a sense of spatial dimension and extent. For example, surround headphones can blur vision just as sound and vision can zoom in and out, alternating between figure and ground.

As Francis Dyson states:

'Sound or rather audio, surrenders its intimate relationship to the body, it's unquestionable access to the interiority and truth, it's camaraderie with the un-representable, the emotional, the mystical' (2009; 137).

In this sense, sound has a more immediate impact on the emotions and feelings of the recipient than a visual aesthetic, as it is more direct and immersive. In this way the experiencers can have a more intuitive body response to the interactive sound installations (Rokeby, 1998).

In this way body-movement interaction intuition is more natural in the interactive sound environment and naturally responds to sound, rhythms, beats, and cadences. Even so, it is still questionable whether the body is more able to en-frame sound than visual information as it is more direct. The directness of sound is another form of aesthetic realization, or 'immersion'.

In this research I incorporated an approach that causes the body to be conscious and therefore in a status of aesthetic reflection. This is caused by the body-movement interruptions of proprioceptive interaction and by the algorithms in the interactive design. The main difference and complexities that arise that cause this interruption to the body is the inclusion of both visual and audio interaction. This method had measures of success and failure, which has now led to further research of interactive design and new methods of 'disrupting' body-movement-interaction.

The 'active realization' of body-movement perception is an alternating or parallel manifestation of reflective and immersive moments. By 'disruption' of the experiencers'' body-movement perception though algorithms in the interactive and combined interactive audio-visual perception techniques. This method was employed to create a flexible body-movement aware state where proprioception has to be relearned with a real-time response to the interactive aesthetics of the non-Cartesian audio and visual media. It causes 'presence' and a virtual enhancement of the senses, and also the alternating awareness of their body-movement interaction (Lombard & Ditton, 2006) This in turn allows experiencers to interact in a more meaningful way, broadening their understanding of the parameters involved.

My art practice-as-research explored the question: By affecting body-movement and by turning the body inwards in an embodied interactive audio-visual installation, can a more creative aesthetic experience be gained? This has been shown to be true in the video-cued interviews and my own experience. It is known that the body's kinesthetic sense is capable of continually updating in micro-changes, via proprioception's ability to relearn and store this in memory. The interventions of body-movement produced by the interactive design flushes and de-stabilizes kinesthetic memory. A new body condition results, causing the experiencers awareness of their proprioceptive full-body-movement and in comparison to the virtual audio-visual environment. These micro-kinesthetic changes stored in proprioception allow a reflective perception that causes us to reflect on our body sense and state of being in the designed non-Cartesian world, which activates an aesthetic imagination.

Body Perception Theory

In New Philosophy for New Media (2004) and later extended into Bodies in Code (2006), Hansen conjectures that the body becomes a more active framework for the image in the digital environment. Hansen goes on to maintain 'that the body continues to be the "active framer of the image", in the digital realm'. (Hansen, 2004: 3) Hansen states: 'On Bergson's account, the body functions as a kind of filter that selects, from among the universe of images circulating around it and according to its own embodied capacities, precisely those that are relevant to it' (Hansen, 2004: 3) Hansen defends Bergson's philosophical theory stating that the body is the 'Centre of In-determination', which emphasizes the role of the affective, proprioceptive and tactile dimensions of experience within the constitution of space and by extension, visual media.

My intention within my artistic research was to explore this theory in relation to the complexities of digital media. Hanson turned to contemporary artists to develop his 'Bergonist vocation' of redeeming the body as the 'Centre of in-determination'. In my own research the complexities of the audio-visual environment, as well as the complexity of 'full-body-interaction' with 'audio-visual', are unaccounted for in Hansen's research. I have thus demonstrated that affectivity in the forcefulness of movement within the embodied interactive digital environment becomes even more emphasized and is open to further research, within this new media arts practice.

In his view of 'affectivity' as an active role of the body in the perception of the image and more so the digital image in its own in-determinacy, Hansen writes:

Motion functions as the concrete trigger of affection as an active modality of bodily action. Active affection or affectivity is precisely what differentiates today's sensorimotor body from the one Deleuze hastily dismisses: as a capacity to experience its own intensity, its own margin of in-determinacy, affectivity comprises a power of the body that cannot be assimilated to the habit driven, associational logic governing perception (2004: 6).

Hansen's claim that the body becomes a 'more active framework of the image' in the digital environment does not take into account the complexity of body-movement perception of audio-visual materials, especially within the interactive live-data environment. Within this context, important issues surrounding perception and body-movement interaction with audio materials have largely been placed aside.

My own research into body-movement interaction within audio-visual perception has shown that complex relationships within the body take place and can reciprocally confound the body, especially in full-body-movement interaction. For example, vision can lose its priority when it is disrupted by sound, where shifting occurs, creating a zooming in and out between figure and ground. Immersed in sound, experiencers can therefore lose themselves 'in creating interiority', since 'sound destroys the subject/environment and interior /exterior distinctions' Therefore the body as 'en-framer' of the image that Hansen conjectures reduces its authority in the audio-visual environment. The bodies own volition in the making of perception of the image and the affect of sound on the body-movement in the interactive audio-visual environment therefore can be understood to have several complex relationships.

Cultural theorist Brian Massumi has shown far reaching implications of a shift in perceptual modality to a more 'haptic', mode of perception, grounded in bodily feeling. He discusses proprioception, stating that: 'the hinging of the proprioceptive on the visual in the movement of orientation is a synesthetic interfusion (2002: 188). Within my work I have shown how sound affects body-movement first before body-movement interacts with the visual components of the installation. He has suggested that there is a missing period of time between the bodily beginning of an event and its completion as an outwardly directed expression of emotion. The missing period of time is the affective duration during which bodies sense a sensation. He continues: 'Another way of putting it positionality is in an emergent quality of movement' (2002: 8).

Following on from this premise, I propose that it is the body's 'proprioception' in motion that is important to the interactive projections of the body translated by computer algorithms. The interactive designs within such research are thus co-created interactive digital environments that cause the mind/body to perceive within a state of changed perception and perspective. This is most likely because there are both unconscious and conscious states of proprioception. Proprioception can be made conscious through body awareness techniques. Theorists have argued for the importance of proprioception as a kind of 'sixth sense' than enables the body to orient itself through its habitual movement within space. [5] In this sense proprioception can be thought of as the recurrent patterns that form as the body's sensory motor system generates microscopic 'kinesthetic' transitions while simultaneously negotiating time and space within the world.

In this research I have therefore focused specifically on the relation between disrupting proprioception and interaction within audio-visuals installation as an aesthetic departure. I do so not as a means to understand the aesthetics of body performance or dance but as a means to uncover a new relationship between body-movement interaction and audio-visuals media.

Seeing proprioception as an internal memory and not a spatial memory, choreographers and performance theorists attest that dancers may sometimes make mistakes based on body misrepresentations: 'a dancer might proprioceptive perceive his or her knee as perfectly straight, when it is in fact bent,' e.g. (Montero, 1999: 239). Such phenomena tend to occur when the dancer cannot see a reflection of their body in a mirror. This shows that proprioception is an internal awareness that does not necessarily relate to spatial awareness.

Proprioceptive perceptual mistakes are also evident in perception of interactive installations within my own research, as the experiencer's body relearns complex movements within its awareness of the virtual and physical space. What is more important is that previous learned body-movement reactions are transformed by interaction with the audio-visuals. The stalling of body-movement is a 'preforming' action that takes place between the aesthetic of the audio-visuals and the cognitive creative imagination.

The aim within my interactive installations was to extend the 'centre of indeterminacy' via the bodies framing function of body perception by going beyond what's 'important to itself', to explore a more 'creative cognitive imagination' via the body within the interactive digital environment. By 'creative cognitive imagination' I mean the experiencers' mind state of creative cognition, as co-author of the art piece and access to meaningful insight or as Lesley Stevenson states: 'the ability to create works of art that express something deep about the meaning of life' (2003: 238). This leads the experiencers' to explore a changing and evolving cognitive perception in real-time. This is a form of 'virtualized subjectivity' - the creative imagination of the experiencer, which is stimulated by the interruptions of body-movement that in turn formulates the virtual environment. I have shown that these interactive installations cause the participants' aesthetic subjectivity to be stimulated by the provocations of embodied interaction, generating a constantly renewing, changing body perspective. This arises from the newly obtained kinaesthetic memory and proprioception, which is continuously evolving. This causes the experiencers' awareness and perception to form a virtualized subjectivity. It also challenges cultural body inscriptions that inform our thought systems, making us aware of 'embodied perception' for the purposes of creative imagination. Most of our bodily actions are governed by proprioception, so by designing interactive environments with the intention of disrupting habitual body movement, a new type of creative engagement can be experienced.

Conclusion

The project has examined the theory of body memory perception, proprioception, for the development of contemporary new media body-movement-interaction installations. These full-body-movement-interactive audio-visual installations explore body perception, techno-aesthetics and digital technologies. The human condition is embodied, and is affected by our surroundings; socio-geographical, socio-political and socio-cultural models and the digital technological environment. The installations explore disruption of proprioception, from an embodied psychological perspective. Interruptions of body-movement in the interactive digital environment cause a momentary aesthetical re-consideration or reflection of preformed movement in relation to the interactive virtual imagery and surround sound. The body's inscriptions are brought to awareness during this brief moment. The interactive installations concentrate on aspects of the body in the way it moves to achieve perception, imagination, and consciousness.

Notes