Panel: Embodiment and Materiality in Art Experience

Emanuele Quinz and Samuel Bianchini, BEHAVIORAL OBJECTS : A New Paradigm for Art and Design?

The notion of behavior and even the power to act (agency) is becoming increasingly central to contemporary art. But instead of situating behavior on the side of living, as is usually the case, how can we invert the perspective and consider this aspect in connection with the works, objects, dispositifs, and environments themselves? How to analyze, understand, theorize, test, and design artworks that include a behavioral dimension, that is, possess the capacity (especially the “physical” capacity) to act and react in relation to their environment and their audience? While more work is being done on this subject in engineering, it is mainly in the areas of robotics and artificial intelligence, primarily using a functionalist approach and with a focus on representational robots. Relatively little research, however, is occurring in the fields of contemporary art and design.
In the frame of the research project “Behaviors: Strategies and Aesthetics of Behaviors Between Art, Science, and Design” (2012-ongoing, in partnership with Université Paris 8, the École Nationale Supérieure des Arts Décoratifs, and the Centre Georges Pompidou, Paris), we proposed the notion of “behavioral objects” : non-anthropomorphic, non-zoomorphic, and more broadly speaking, non-biomorphic roboticized objects that do not intrinsically possess an expressive capacity through their form, but express behaviors by their movement.
The Lecture will address this notion, and retrace a genealogy in the field of art and design – notably going back to the historical avant-gardes and to their intersection with scientific and technological research, from cybernetics to electronic engineering, robotics, from cognitive sciences to computer programming.

Frances Joseph, Sapient Textiles: Materiality, mediation and embodiment

The traditional role of textiles as a second skin, mediating between body and environment, is being reconsidered in light of technological developments and new theoretical perspectives. The notion of skin as a sensory interface or ‘fringe of the virtual,’ has been recognized in digital embodiment theory, New textile processes involve physical and digital dimensions, both in terms of the processes of making that constitute textiles as material artefacts, and through the development of fabric based electronics in the form of smart textiles that can sense, communicate, respond, and even harvest energy. These ‘second skins’ and the new frameworks and sensibilities produced through such technological infrastructures are the focus of this paper. Smart textiles are considered in terms of materiality, mediation and embodiment in relation to a series of projects investigating the development and performance of knitted e-textile structures.

The emerging field of e-textiles is an interdisciplinary domain that is underpinned by a radical repositioning of bodies, textiles and computing. These ‘wearable, washable, drape-able computers’ pose questions about dualities of material and immaterial; exterior and interior; body and mind; object and subject. The notion of ‘sapient materiality’ where consciousness and cognition are part of the specifics of materiality rather than defined in opposition to a material world, is discussed in relation to ongoing research into knitted textile transducers that translate variations in physical quantities into electronic signals. Relationships between the structure and behavior of smart knitted textiles are considered in terms of configuration and performance as embodied interaction.

Elisabeth Nesheim, Physical sketching, abstract movements and creative processes – the promise of updating the body schema with haptics – telepresentation

What is at stake when the sense of touch, postures and gestures of the hand, aided by haptics become key players in a creative process of designing tools, services or even artistic works? Although haptic technologies are becoming more common we still do most of our everyday computing through screens. Such ocularcentric interface design offers an unbalanced mediation of the multisensory way we experience the world. The sense of touch has long been noted as a reality checking device in what I touch is real. By downplaying the role of touch and motor-sensory perception data in interface design, we ignore a vital source of information of the world—which in turn has its impact on our thinking about the world. This paper develops Mark Hansen’s claim that new media technologies can broaden the what he labels the pre-personal domain—“the organism–environment coupling operated by our nonconscious, deep embodiment” (2006, 20), as it offers us the option of translating the unobservable (or rather the pre-consciously perceivable) into the visual realm, granting us access to new multi-sensory and synesthetic experiences. I do so by unpacking the two points present in Hansen’s claim: 1) the notion that the pre-personal domain is extendible, and 2) that new media technologies (digital, haptic and sensor technologies) are in a distinct position to do so. Hansen’s idea has its roots in Merleau-Ponty’s notion of the body schema. By identifying the processes governing the extending, revising and preserving of the body schema—unpacking MP’s concept of abstract movements and investigating embodied cognition researcher David Kirsch’s hypothesis on physical sketching—we are given a clue as of how the pre-personal domain is expanded. Finally, I discuss a particular haptic technology (the data glove) and connected artistic practices, to see how such technologies are in unique position to extend our body schema.

Aurélie Besson, Senses of movement and embodiment in interactive artworks

Interactivity in art has opened a whole new research field in which the visitor’s gestures have become central since they modify the artwork and the visitor at diverse degrees. These artworks require from the visitor to enter in relation, in motion, with them.

What happens during the interactive artwork experience, between the perceived artwork and the visitor’s body? And what does it imply at the level of the five senses and the body’s movements?

This presentation is based on a study that analyzes the experience of interactive artworks, conceived with or without computer technologies, that have the particularity to provoke the visitor’s gesture or movement. This research is based on the fact that we actually perceive with the whole body and numerous embodied sensory channels. As described by Alain Berthoz (1997), the gesture is both an initiator of an action and a mode of perception.

This presentation is based on an interdisciplinary study I am leading during my Phd research, at the crossroad of aesthetics and cognitive sciences questioning how these artworks destabilize or stimulate the senses of movement (proprioceptive, kinesthetic, vestibular senses, etc) and offer an embodied experience that enables a certain degree of conceptualization. I am leading an auto-ethnographic analysis based on researches made in the fields of interactive art and embodiment, art and perceptions, and embodied cognition.

Yelena Gluzman, Interdisciplinary Approaches to Interaction

In the past twenty years, neuroscientists found evidence for embodied and inter-subjective theories of human cognition, suggesting that higher-order cognitive events (e.g., learning and creativity) in the central nervous system are non-trivially contingent upon interactions with the body and social environment. This has led to a widespread effort to identify specific neural correlates of embodied social behavior in humans, and this work is often done using non-invasive brain imaging techniques, like electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Such work, as in the case of the Pineda lab, often focuses on populations with social dysfunction, like people diagnosed with Autism Spectrum Disorder (ASD), and investigates whether behavioral deficits are linked to atypical neural function. One of the challenges in doing such work is the limit imposed by technologies like EEG and fMRI, which often require subjects to be immobilized and/or isolated. Tests of social behavior are therefore done based on interactions with video screens and highly constrained movement. Efforts to address this problem have focused on the development of imaging technologies to incorporate data from traditional brain imaging, body motion capture, and eye gaze tracking, allowing for participants to move and interact, while also affording researchers information about the bodily and visual actions of participants.
Though new technologies hold great promise for more ecologically valid ways of analyzing the coupling between neural, bodily, and social events, the experimental paradigms used in concert with such technologies remain largely unchanged from those used with more limited imaging technologies. In traditional experimental paradigms, researchers will choose a fixed sequence of “social” stimuli that participants observe, and measure brain activity during the perception of and response to these stimuli. This approach can yield a great deal of legitimate data about what brain systems are involved in particular tasks. However, since stimuli do not change in response to participants’ behavior, such studies cannot analyze a dynamic interaction in real time, in which participants collaborate to interpret and make sense of each other’s action.
Both theater and social science offer resources to explore this problem. In theater, it is widely acknowledged that the design of fictional situations can facilitate actual experiences and interactions; moreover, these theatrical experiences are available to performers as well as a variety of co-present participants. Therefore, using theatrical paradigms as experimental situations allows for both a repeated, experimental structure, while also allowing for an emergent and actual social interaction. Social science is also interested in how the social emerges from interaction. Sociologists and communication scholars have been able to ethnographically observe dynamic interactions and, using approaches from semiotics and ethnomethodology, could show the minute steps through which interacting pairs coordinate to achieve local instances of shared meaning, learning, and collaboration. Such research has been able to identify semiotic, multimodal resources that participants use to coordinate with one another. Of course, the studies done in this manner can only address interaction at the scale of behavior, and do not seek to relate these to biosemiotic events at the scale of neurons, muscles, etc.
This paper reports on an ongoing research project in which we attempt an interdisciplinary approach, combining methods from theater (performative enactments), with social science (multimodal semiotic analysis), with methods from cognitive neuroscience (brain imaging) to develop a novel paradigm for the analysis of social interaction. Such an approach can both examine neural correlates of social behavior, while also considering the social to be not fixed, but rather emerging through embodied interaction. Concretely, the project will look at two-person interactions in which either member spontaneously takes on the role of a fictional character in order to coordinate with the other. These interactions, situated within the context of an interactive performance installation, are instances of dynamic social interaction that are then analyzed in the lab. In the preliminary findings of the study, we report on embodied strategies used both by interacting pairs to coordinate with each other, and also by the researchers themselves as they attempted to code and analyze these interactions.