Projective embodiment in Videogames and Digital Spaces

Brothers_header_601x232This article argues that gaming is an embodied phenomenon which is distributed across multiple conceptual domains. Videogames are, as Gee notes, “’action-and-goal-directed preparations for, and simulations of, embodied experience’” (23)[1]. However, gaming is more than just what happens on screen. It is a highly mediated experience (the screen sits between that player and the game) in which the player straddles two worlds. They simultaneously exist in the ‘virtual’ world as their character on the screen as well as in the ‘real’ world as they press buttons and manipulate the interface of the game. Indeed, Juul argues that playing a game is a “dual structure” in which “the actions we perform have the duality of being real events and being assigned another meaning within the fictional world” (141)[2]. Thus, when I click the mouse, I perform a real world action (moving my finger to press the button) as well as a symbolic action in-game (moving a character or selecting an item). Whereas Gee was primarily interested in what happens between the player’s head (mind) and the screen, I intend to examine embodiment across this dual structure of physical/virtual experience—that is, not just in the game but in the game play.

Embodiment is a metaphor for a way of thinking, of processing information through our experience with the world and our capacity to act upon and be acted upon by the larger world we inhabit. The body (and mind) stands as a symbol for the self, a persistent perceptive system that exists and is enmeshed in a world around it. It is through that persistent perceptive system – the mind-body – that thinking occurs. Glenberg claims that “the meaning of an object, event or sentence is what that person can do with the object, event, or sentence” (“Memory” 3)[3]. The implication is that we think in terms of how we can act upon the world (or how the world acts upon us), and it is through the body that this action occurs. Since videogames are “action spaces” there is a natural fit with this way of understanding game play.

This conceptualization of embodiment relies on a particular understanding of the body’s (and mind’s) role in the way we make meaning by extending the mind into other tools, the world, and even other minds. The body (and mind) exists within a constellation of other entities. It is part of a network of things. Some of these can be incorporated ad-hoc into the processes of thinking (using a certain number of rocks to count, say); some are designed to be incorporated into thinking (paper and pencil); others are simply sensory information (the color of a car passing in the street); still others are things we have no access to and are essentially meaningless. Nevertheless, these objects, as part of the extended network that constitutes the world, contribute to creating meaning making through our experience with them. Together, the mind-body unit and these networked objects make up the cognitive (embodied) system through which we create meaning.

This paper extends Gee’s notion of the “projective stance” to describe how a player—networked with other nodes in the system—makes their experience of gameplay meaningful. I will argue that projective embodiment is one way of understanding the relationship between the player’s mind-body, the tools of the game, and their interaction. I will begin with a brief outline of three key concepts: embodiment, networks of knowledge, and mediation. Then, I will explore the concept of projective embodiment in a particular game, Brothers: A Tale of Two Sons in order to describe how the player, through the  extended mind-body, plays the game.

Embodiment

Why the body? What is important about the body that shapes our thinking? In the most literal sense, the body is ever present in our experience. As Glenberg notes, “our bodies are our constant companions. Bodies are there during all of our development, they are there during all social interaction, they are there when we think, and they are there when psychological processes run awry” (“Embodiment” 3)[4]. We might also say that a person ‘embodies’ certain values or beliefs, and in doing so we privilege the body as a container, a boundary which separates it from the world around it. Here, the body is a metaphor for being which is tied to its physicality.

Embodiment is experiential, in that it is a sensory-rich experience of objects and actions in relation to the body. As Lakoff and Johnson explain, “Our experience of physical objects and substances provides a further basis for understanding – one that goes beyond mere orientation…[to where] we can identify our experiences as entities or substances, we can refer to them, categorize them, group them, and quantify them – and, by this means, reason about them” (25)[5]. Sensory information about objects – visual cues, tactile feedback, even sensations about the temporal duration of an event – allow us to make sense of them, to, in Lakoff and Johnson’s terminology, “reason about them.”

However, embodiment is more than just the presence of a physical body. It is a way of thinking and knowing the world and is a part of the mind, too. For Dourish, embodiment is “sense of ‘phenomenological presence,’ the way that a variety of interactive phenomena arise from a direct and engaged participation in the world” (115)[6]. Embodiment involves a vehicle to engage with the world (the body) but also the ability to act within it to create meaning.  That is, we are a being capable of doing. Dourish argues,

There is no homunculus sitting inside our heads, staring out at the world through our eyes, enacting some plan of action by manipulating our hands […] We inhabit our bodies and they in turn inhabit the world, with seamless connections back and forth. (102)[7]

The mind and body are inseparable; the body is an extension of the mind, the mind a part of the body, both part of the larger system that is the world. Dourish suggests is that thinking is inseparably derived from being – that existing is a prerequisite for thinking, and therefore thinking is just a way of being. We think through the body.

Networked knowledge

Making meaning is a critical component of Dourish’s argument. He argues that “[e]mbodiment is the property of our engagement with the world that allows us to make it meaningful” (126)[8]. To make meaning of a state of being or potential for acting, it is necessary to consider the context for that action or state. The body as entity metaphor exists simply to draw a distinction about one part of the system (the body) in order to describe it effectively. Lakoff and Johnson suggest that making these distinctions are practical ones, for “[h]uman purposes typically require us to impose artificial boundaries that make physical phenomena discrete just as we are: entities bounded by a surface” (25)[9]. We remain, bodies and minds, enmeshed in the world.

Furthermore, Clark contends that the mind-body does not simply end at the physical boundary of the flesh; rather, thinking extends to the environment in which that thinking occurs, and is distributed across multiple domains:

What the brain is best at is learning to be a team player in a problem-solving field populated by an incredible variety of non-biological props, scaffoldings, instruments, and resources [….] a complex matrix of brain, body, and technology can actually constitute the problem-solving machine that we should properly identify as ourselves. (26-27, emphasis in original)[10].

In this view, the tools I use become a part of the mind. When I write, the piece of paper and pencil are incorporated into my cognitive processing which enables me to think. Clark explains that “our continual experience of closely correlated action and feedback routines running via these nonbiological peripheries allows the brain to temporarily generate a new kind of ‘body-image,’ one that includes the nonbiological components” ( 62)[11]. I have off-loaded an internal process (memory) onto the outside world; I have effectively extended the mind onto a technological tool.

These tools and objects (and other minds, too) themselves have certain capacities and limitations. Incorporating them into the cognitive process requires using them in the right kind of way by capturing their affordances. Gee (following Gibson) describes an affordance as a “feature of the world (real or virtual) that will allow for a certain action to be taken, but only if it is matched by an ability in an actor who has the wherewithal to carry out such an action […a]ffordances are relationships between the world and actors” ( 25)[12]. There needs to be a fit between the action at hand, the tool, and the actor in order to function properly (i.e. to think).

These tools/objects can extend the mind (both thinking and acting) in ways not possible to the bounded mind-body unit alone. Writing, of course, is a prime example: it can record and store vast amounts of information. It works differently than the “native” processing of the mind but can supplement (and even alter) the way we make meaning and act in the world.

Indeed, in this sense it is even possible to consider that these tools and objects “know” things themselves, and our engagement with them creates a distributed network of knowledge which we activate for some purpose. Affordances are one kind of knowledge. For example, I leverage the potential for action possible in a hammer (that is can be used to hit a nail), but it does some work in the system (it actually hits the nail); the hammer “knows” how to do this work in a kind of way possible because of its shape and the laws of gravity and so on. But it also requires me—and my mind-body unit, which now includes the hammer (as well as the nail, the wood, gravity and so on)—to activate this affordance.

In this case, most of the knowledge is stored in my bounded mind-body unit and the tool is a low-knowledge node in the network. Other tools may store greater knowledge; clicking a mouse to open a program on the computer activates a great deal of knowledge on the part of the mouse (the physical object) as well as the computer’s operating system, the software of the program, the screen on which it displays, the electrical system of the computer and a host of other nodes. Here, a simple action by my bounded mind-body (clicking a button) triggers a host of other actions (knowledge) to accomplish the action at hand (opening the program). My mind-body unit is low-knowledge in this case (I know very little about the code or electricity), while the tools/objects know a great deal which I leverage accordingly.

Since embodiment is a metaphor for thinking (and acting), even these networked and distributed systems are a function of the body across space. I “project” my mind into other nodes in the network which become a part of how I think and act. These nodes also “project” back onto me and influence, extend, and change how I think and act as well. I adopt a projective embodiment which incorporates these extra-body affordances, limitations, and knowledge.

Mediation

These tools/objects are both conceptual and concrete links between the mind-body and the world; they mediate my experience. For example, when gaming the screen sits between me and the virtual spaces it displays; the keyboard sits between my input (pressing the keys) and the input into the program (electronic signals). Seen in this way, mediators are things that stand between; they separate, symbolically and literally, object and subject, thought and thinker.

There is another view, however, which suggests that mediators act as extensions of those things that they mediate rather than obstacles or objects in their own right. As McLuhan argues, “[a]ll media are extensions of some human faculty – psychic or physical” (26)[13]. In other words, I am not looking at the screen itself, but ‘through’ the screen at whatever it is I am doing within the virtual space. The screen is not ‘that which lets me see’ but is seeing. Certainly, the computer monitor does not disappear – it remains firmly on my desk. It functionally disappears, however, as I configure my projectively embodied mind-body to play the game. I remain aware of it as a physical object, but incorporate this concession into my actions and do not focus on the tool as an object but rather as a means of acting.

Dourish summarizes Heidegger’s explanation of this phenomenon by describing the state that tools exist to us conceptually; they can be zuhanden (“ready-to-hand”) or vorhanden (“present-at-hand”):

They are ways, Heidegger explains, that we encounter the world and act through it. As an example, consider the mouse connected to my computer. Much of the time, I act through the mouse; the mouse is an extension of my hand as I select objects, operate menus, and so forth. The mouse is, in Heidegger’s terms, ready-to-hand. Sometimes, however, such as when I reach the edge of the mousepad and cannot move the mouse further, my orientation toward the mouse changes. Now, I become conscious of the mouse mediating my action, precisely because of the fact that it has been interrupted. The mouse becomes the object of my attention as I pick it up and move it back to the center of the mousepad. When I act on the mouse in this way, being mindful of it as an object of my activity, the mouse is present-at-hand. (109, emphasis in original)[14].

It is important to stress here that it is not the tool that changes, but our perception of the tool.

When it is apparent in its function, it becomes an obstacle, and the mediating effects of the technology become evident. This is a key distinction: the tool, the mediator, can be both transparent and opaque, depending on how I perceive it. Projectively embodied cognition further suggests that the tools effectively become an extension of myself; what they mean to me is what I can do with them and how I relate to them in terms of my enmeshed existence and experience in the world.

 

Brothers: A Tale of Two Sons and Projective Embodiment

To see more clearly how this theory of projective embodiment works in a videogame, I will examine the game Brothers: A Tale of Two Sons. There are many kinds of videogames, from simple puzzle games (Tetris, for example) to fully realized worlds with complex character interactions and a free-form, non-linear plot (Grand Theft Auto, Flight Simulator, World of Warcraft). Games can also be played on many different kinds of devices, such as computers, consoles, handhelds, and mobile devices. Brothers is designed to be played with a modern controller (with two joysticks and several buttons) on a computer or console. The player navigates a three-dimensional world in search of a cure for the main characters’ sick father. The player must solve different puzzles (such as crossing a river with no bridge) using the characters they control in order to beat the game.

Here is an example of the gameplay in Brothers:


Despite these various forms, all videogames share some inherent features; they are, as Juul describes, “rule-based system[s] with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels emotionally attached to the outcome, and the consequences of the activity are negotiable” (36)[15]. There are two key points in this definition for the purposes of this discussion: the player provides input to the system (acts on it) and the system provides an outcome (result of that action).

This relationship between user input and game output is typically referred to as interactivity, where the player and the game work collaboratively to produce the outcome. This term is highly debatable, certainly, but for the purpose of this analysis I define interactivity as a process where human input and output and tool input and output meet to produce some change in the system. There are times when there is a (nearly) direct correlation between my actions and the outcome in the game – I press a button, the game responds; there are other times when the game provides some input for me (narrative, say, or instructions on how to move) which I must process in order to act. In this way videogames are “action-spaces;” I act, the game responds; the game acts, I respond. Further, as these processes are spread across both the real world (I see the screen, I press buttons) and the virtual (my input is translated, the simulation runs), my actions are mediated by the videogame between two conceptual domains, the real and the virtual.

Part of what makes Brothers so interesting—and so demonstrative of projective embodiment in games—is the unique way the player interacts with the game. In most games, players inhabit an avatar (a representation of the player such as a character or other tool like a race car) to navigate the world. In Brothers, the player actually controls two avatars simultaneously. Each character is controlled by a joystick so that the player is moving, manipulating, and interacting with the game world in double.

The avatars are the most direct representation of the player’s mind-body in the game. They are the primary tool through which the player acts, and they serve as the object that most directly affects the player’s behavior on screen. There are other objects in the game (other characters as well as the world itself), but the player is bound to the avatars. In this way, they are the player’s agent by proxy.

Each of these characters has different features and capacities; the older brother is bigger and stronger, while the younger brother is smaller and faster. The player uses them in different ways, and the puzzles are designed to capitalize on these differences. For example, early in the game the brothers must cross a bridge by pulling a large lever. The lever is too heavy for the younger brother, so the player must use the older brother to complete the puzzle. Similarly, there is a locked gate that the younger brother can fit through but the older brother cannot, so the player must use the younger brother to sneak through to the other side of the gate and open it. The player needs to utilize the affordances of each avatar in order to complete the puzzles.

The player must also combine the affordances of both avatars, not just use each individually. At several points, the player must get past a cliff which is too tall for either character alone to climb. The older brother must kneel down so that the younger brother can climb over him to reach the top, and then let down a rope so the older brother can join him. The game is designed in such a way as to force the player to incorporate both avatars (surrogates for the player’s actions) into their problem solving strategies.

Since each avatar has particular affordances and limitations, they “know” different ways of acting within the system. The player activates these affordances at the right time to complete the puzzle. The avatars also know how to act contextually; pushing the “action” button on the controller does different things at different times. For example, pushing the “action” button near a wall makes the avatar jump and grab the ledge, while pushing the same “action” button near a ball causes the avatar to pick up the object.

This contextual knowledge highlights a key feature of projective identity. The same action for the player (pressing the “action” button) creates different changes in the system through the “knowledge” of the avatar. In one way, the player is a low-knowledge node (they don’t need to know how to climb a wall or pick up a ball) while the avatar has high-knowledge (they know how to climb and grab). But activating this contextual knowledge requires the player to choose it at the right time, since without the player the avatar can do nothing. It is only together (the player + the avatar) that the system changes, and it is through the networked, projective mind-body that the player activates these affordances.

Knowledge is not limited to just the avatars, however. There are many different objects in the game which also “know” things within the system. For example, early in the game the player encounters a troll who helps the avatars cross several large chasms. The troll throws the avatars across, or stretches his arms across them so the player can climb over him. The player has no “direct” control over the tool (the troll) but the game “knows” how to act in order to provide the player with the opportunity to complete the puzzle. Here again, the player’s problem-solving process is networked across the extended mind-body unit which incorporates the nodes under the players control (the avatars) and the game world (the troll).

There is one other core feature of Brothers (and other videogames): the controller itself. The controller is a kind of boundary or “threshold object” (Murray 146 )[16] that serves as the most visible mediative tool in gameplay. It is through the controller that the player instructs the avatars on how to act, and it is the primary site where the player “connects” with the avatars. Certainly the screen (which “shows” the avatar) is another important site of mediation, but it is the controller which serves as the primary instrument through which the player projects their embodied cognitive processes.

Brothers brings this boundary object to the forefront in an important and illuminating way. In doing so, it further demonstrates the networked nature of projective embodiment. The unique control scheme itself is an interesting feature, one which challenges some conventions of gameplay; this is most obvious for experienced gamers, who normally use the two joysticks to control features of one avatar at a time (such as movement and camera angles). By adding a separate avatar to each joystick, Brothers emphasizes the controller and the way it mediates the interaction with the game world.

At a crucial moment in the game, the player loses control of one of the avatars. They are then forced to solve a puzzle that seems possible only with both characters; they must swim across the river, but one character is afraid of the water. In previous encounters with water, the brother who is afraid clung to the other brother, who swam across. Now, the brother who is afraid must cross the water alone without the aid of the brother who “knows” how to swim. Initially it is a moment of frustration. It also highlights the nature of the controller itself and the controller’s relation to the avatars. Without one node in the problem solving network (the brother who knows how to swim), the player is forced to reconsider their controller. It forces the player’s relationship with the controller to shift (in Heideggerian terms) from ready-at-hand to present-at-hand. Ultimately, the solution is that the brother who is afraid of the water can swim by using the joystick “vacated” by the other brother. This moment—changing the control scheme—is not only a powerful narrative moment, it emphasizes the relationship between the player, the controller (mediative tool), and the game as a network for problem-solving (thinking).

What the game mediates, then, is the player who inhabits a physical space and the conceptual space of the game world simultaneously. For Nicholls and Ryan, “[g]ameplay exists not in the hardware or game software, not ‘in’ the player or in an idealist concept of game, but in the interface and interplay of ‘as’ and ‘as if’ structures, the connections and passages between playerspace and gamespace” (np)[17]. The division between the gamer and the game is a productive space with multiple meaning-making possibilities; it is a multimodal state, where I act in overlapping spheres of reality and virtuality. That is, the division of subject and object—the projective embodiment of the player’s mind-body networked with the tools and knowledge of the game—is the game. Playing the game is about shifting across these networked nodes and conceptual domains.

 

Implications

This conceptualization of projective embodiment has several important implications, both in gaming and beyond. For game designers, creating experiences which capitalize on the relationship between the player’s mind and the designed tools is a crucial challenge but one which is at the very heart of gameplay. Understanding how to capture (and challenge) these relationships (such as the way Brothers highlights the mediative tool of the controller) is a core design principle. For players, much of the same holds: understanding how they relate to and utilize tools and objects is central to making their gameplay meaningful while also providing an important point for critique.

Beyond games, projective embodiment plays an important role in our interaction with all kinds of tools and other media. As we increasingly incorporate the virtual into everyday life—from digital representations of our social selves through things like Facebook and Twitter to fictional role-playing in massively multi-user virtual worlds like World of Warcraft and Second Life—it becomes especially important to understand just where the mind is in relation to the world, and how we interact with it.

Finally, more broadly, the notion of projective embodiment highlights an important critical and methodological stance: analysis of any phenomena, gaming or otherwise, increasingly requires understanding the system in which that phenomena occurs. Our world is becoming more complex, and these complex systems create new and greater challenges. Exploring these systems is more necessary than ever, and projective embodiment is one key way of understanding just where in the world we fit in.

 

References

Clark, A. (2003). Natural Born Cyborgs. New York: Oxford UP.

Dourish, P. (2004). Where the Action Is: The Foundations of Embodied Interaction. Boston: MIT Press.

Glenberg, A. (1997). “What Memory Is For.” Behavioral & Brain Sciences. 20.1: 1-19.

Glenberg, A. (n.d.) “Embodiment as a Unifying Perspective for Psychology.” Unpublished.

Gee, J. P. (2007) Good Video Games and Good Learning: Collected Essays on Video Games, Learning, and Literacy. New York: Lang.

Juul, J. (2005). Half-real. Cambridge: MIT Press.

Lakoff, G. and Johnson, M. (2003). Metaphors We Live By. Chicago: U of Chicago P, 1980. 2003.

McLuhan, M. (1967) The Medium Is the Massage. New York: Bantam Books.

Murray, J. (1997) Hamlet on the Holodeck. Cambridge, Massachusetts: MIT Press.

Nichols, B. and Ryan, S. (2003). “Game, Space and the Politics of Cyberplay.” fineart forum. 17.8.

Advertisements
Tagged , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: