Panel: Towards the Interbody: Embodiment and Interface

Patricia Pisters, Perfecting the Self in Digital Media Culture: Neurofeedback and Embodied Self-Knowledge

Neurofeedback is used in closed-loop settings, where subjects learn to control the activation of specific brain regions when presented with a measure of that regions activation (Cavazza et. al. 2015) Neurofeedback is used increasingly in the development of Brain-Computer-Interfaces and sometimes make use of virtual agents to provide realistic visual feedback in order to enhance mental behavioral and emotional control. This paper will investigate how the experimental settings of neurofeedback systems, where a mind reflects back on itself in order to learn, correct, perfect or control, can be extrapolated to a larger and complex media context. Several fields governed by similar looping patterns. A first cultural feedback loop system that can be recognized as such is related to CCTV systems and other surveillance mechanisms (including drones, facial recognition software and other ‘nonconscious cognitive agents’); these systems and devices actually have turned the world into a closed feedback loop circuit, which includes the embodied brain of the human subject. Besides this collective political level, there is another level of looping that operates on our individual consciousness: the camera switch mode of our cell phones have encouraged the cult of the selfie, making us all reflect back on ourselves in a narcissistic tradition of self-reflection and perfect self-presentation. Thirdly, filmmakers have turned their camera’s increasingly inward, filming by and large from ‘inside out’, showing us the world of their protagonists in all their fantasmatic and regularly even psychopathological dimensions. It is this third level of brain reflexivity in contemporary digital cinema that I will focus on in this presentation. In this context I will refer to two particular cinematographic cases of ‘neurofeedback looping’ and ‘avatars’ (Darren Aronofsky’s Black Swan, 2010; and Simon Pummel’s Brand New U, 2015) that are telling of the kind of problems of embodied (self)knowledge we are facing today.

Vicente Raja and Paco Calvo, Augmented Reality: An Ecological Blend

Augmented Reality (AR) makes reference to the real-time perception of an environmental setting that has been enhanced by means of computer-generated virtual components. The design of current AR devices is based on the real-time superimposition and alignment of a symbolic virtual layer with the real environment. This way of augmenting reality rests upon the representational-cum-computational principles of cognitivism: cognition and perception are to be accounted for in information-processing terms.
Unfortunately, designing AR devices along representational/computational lines results, we contend, in cognitive overload. AR users are faced with the problem of having to connect two pools of data as they deal with disparate sources of information, the symbolic and the environmental—in some devices, problems of attention and headaches have already been reported. It seems that current AR devices serve to augment, not reality, but rather cognitive demands. This is more pressing once we consider that some users’ cognitive skills may not be fully developed or are somewhat diminished (e.g, infants and Alzheimer’s patients).

Our thesis is that we may be able to bypass these problems by endorsing, when designing AR devices, the principles of ecological psychology, and not those of cognitivism. The main tenet of ecological psychology is that information does not need to be processed cognitively, since it is rich enough and already specifies opportunities for behavioral interaction in the form of perceived affordances. Information only needs to be detected. In this way, ecological augmented reality (E-AR) devices could contribute to enriching environmental information by providing novel specifications and by generating new affordances or by highlighting those already present. Crucially, the type of information to be exploited by E-AR is in the blend of the virtual layer with the environment itself. This is how augmented reality becomes E-AR. We review some sensory substitution devices for the sake of illustration.

Adnan Marquez Borbon, Perceptual learning and the emergence of performer-instrument interactions with digital music systems

The relationship between human and computers within the area of HCI has shifted from being represented by cognitive, information-processing approaches to more recent situated and embodied perspectives. This matter has analogously developed within interactive digital musical system research.

However, the nature of this interaction, while commonly drawing upon J.J. Gibson’s theory of affordance for analysis, often fails to characterize the emergent relationship between performer and musical system. This has led to regard affordances as mere design features, as well as taking for granted the skillful contributions of the performer. Interactions in this manner are represented as static.

In this paper, I present the findings of a long-term phenomenological research study describing the evolving nature of performer-instrument interactions. I resort to E.J. Gibson’s perceptual learning approach to frame the experiences of performers, both individually and socially, as they learned to play a new digital musical instrument, the Pulley Synth. I argue that the nature of such interactions is grounded on learning and that the changing perception of the system contributes to the process of enskilment. In this view, more than being an acquisition of conceptual knowledge of the instrument’s operation, learning suggests a holistic behavioral change towards the system in which musical and sonic possibilities of the entire performer-instrument-environment ecology are explored and enacted.

I conclude that perceptual learning is significant within the context of HCI and musical technology, given that current design and performance practices with interactive digital music systems remain influenced by established musical practices (for example, Classical music). If one of the aims of this domain is the development of new musical interactions and practices, the ability to overcome the influence of musical traditions and to perceive new sonic possibilities must be considered in both the design and instrumental learning processes.

Shannon Cuykendall and Thecla Schiphorst, Untying the Knot of Dance Movement Expertise: An Enactive Approach

Many cognitive scientists are turning to dance experts to explore the relationship between bodily knowledge and perception. Dancers have a unique skill set that integrates physical and expressive abilities, making them intriguing participants in studies that examine how the action observation network, action prediction, learning, memory, and aesthetic preferences are related to one’s physical experiences. While findings from these studies suggest that dance experts perceive movement differently than novices; the definitions of what constitutes a “dance expert” are far ranging, making it challenging to compare results across studies.

We discuss the variety of dance experts that are recruited for scientific studies in movement perception and draw upon research in dance education to form a richer definition of dance expertise. One possible way to parameterize expertise in dance is by role (e.g. performer, teacher) and style (e.g. ballet, hip-hop). However, the challenge in differentiating expertise by style or role is that many many dancers have trained in multiple styles of dance and commonly have overlapping roles–thus creating a knot of expertise.

To untie this knot, we propose an enactive approach. Rather than label dancers as experts or novices based on an arbitrary requirement of the number of years a dancer has trained in a particular style or role, we suggest that broader categories of expertise will emerge from the data that go beyond both style and role. We predict four main types of expertise: Virtuosic, Expressive, Kinematic and Expressive expertise. Through combining methodologies such as psychometric measurement, eye tracking, brain imaging, phenomenological and analytical accounts of movement we can develop a more complete understanding of how expertise relates to the ways in which dance experts observe, learn, and articulate movement. This research can both inform the study of movement perception and help define gaps in dance education.