There is an earlier draft of this analysis as well that goes a little deeper into the theory, though it has changed since I originally wrote it. I’m also working on a longer, revised version that fleshes out my thinking based on all the feedback I’ve received. This version is what will appear in the GLS 9.0 proceedings. In short, there are many versions of this theory and the joy of digital media is that you can watch me draft and revise publicly. I love the future.
This paper provides a methodology for analyzing the visual elements of videogames, and in particular how those elements can help players understand the contexts of the game and prepare them to act within it. Visual elements help orient players to the mechanics of the game (what they do) and to the stories they enact (why they are doing it and how). There is a risk in isolating visual elements from other modes of meaning-making in games, as gameplay is about how these modes work together to make the meaning of the game possible. However, this analysis considers how the visual elements point towards these other modes, how the game cues players in how to interpret and act, and how to use the visual features to “do” the game. Building on a framework developed by Serafini (2010), this methodology for visual analysislooks across several interrelated features: the representational and orientational elements within the game screen; structures and conventions called upon; and ideological choices and frames used by the designers in creating the game and by players when interpreting and enacting that design. If, as Serafini suggests, we think of these as nested layers within a sphere, then the outermost level is the ideological frame, the middle is the structural and conventional frame, and the innermost is the representational elements or “noticings”; we look inward through ideologies, through conventions, at the “noticings,” which reflect back to us those other features. It is important to understand that the boundaries between layers is porous; conventions are certainly ideologically motivated; color is both a semiotic structure (Kress and van Leeuwen, 2002) as well as a noticeable element. These features blend into each other and isolating them is useful only in the most abstract deconstruction. In the everyday world players experience these things simultaneously and as compound meaning-potentials.
The work of meaning: Orientation and Preparation
Gameplay is about the mechanics of play, about the specific tasks and actions, the inputs and outputs, the systems that govern those interactions, win states and status states and more. Games are also about new worlds and new identities, ideas and stories, narrative tales and player enacted happenings (Gee, 2003). They are both tools and stories. Some games focus on one feature more than the other; they are all videogames though they may present their tools and stories differently. Visual elements must help orient players to the gamespace and to the tools of the game; they must help players come to some understanding about what they are supposed to do and how to do it. In this way, these visual elements also “prime” players to act effectively and continue the game. These cues can be geared towards the mechanics of the game (icons indicating actions, highlighting key elements, textual cues), or cues into what the player should believe (the narrative they are participants in, what mood or feeling the game is portraying, what ethical or moral beliefs make up the world itself). It is important to recognize that these visual elements are moving and changing throughout gameplay; games may intersperse cutscenes, menus, action sequences, text and more in a dynamic way. Playing the game involves navigating a shifting nexus of representational elements over time. Specific examples and use cases of the tools described below are included separately in these proceedings (Holmes et al., 2013).
Analysis Tool #1: Representational/Orientational noticings
The representational/orientational elements of games help players make sense of what the world is and what they should do. These elements are things like the buttons and labels on the screen, the avatar and characters, the world or space players inhabit, textual elements, colors and shapes. Noticings also include tone, what characters and other objects looks like, what fills out the world, and what colors are present. These elements point to the conventions the designers chose to make the meaning-potentials of the elements and help indicate the ideological beliefs they used to design the game.
NOTICINGS TOOL: What do you notice? What stands out to you? What seems important? What indicates how you are supposed to act in the game? What tells you about the story or world you inhabit? What features are part of the interface? What features are part of the “story”? How are these related? How do they influence how you act in the game?
Analysis Tool #2: Structures and conventions
Structures refer to the composition of the representational/orientational elements, including color, size and placement, salience, framing and emphasis, point-of-view, and others (Kress and van Leewuen, 1996). Conventions refer to the ways these structures are used to promote particular meanings and how various visual elements are used across different games, different genres, and different media. Structures and conventions work in similar ways. The color palette of the game, for example, provides cues to the content and the actions of the game (e.g. a bright cheerful color usually indicates safety or peacefulness,a dark color scheme often indicates danger, and thus how a player might expect to act). Perspective and point-of-view often tell players how they will play (first-person games are often shooters; detached, distantperspectives are often utilized by RTS or puzzle games). They also help orient players to the story elements of the game (players are the character in a first-person game, while characters in RTS gamesare oriented more as tools to be used). Features like salience and framing help players understand what is important and what is worth noticing, and what these things might mean within the larger gameworld.
STRUCTURES AND CONVENTIONS TOOL: How are the elements arranged? How are they related to each other? What is emphasized, and how? What is the color palette of the game? What does this indicate to you? What is the point-of-view of the game, and how do you relate to the game through this perspective? How are these elements represented in other games? How does that influence how you understand them, and act in the game? How are these elements presented outside of the game? Does the game conform to these extra-game conventions or does it challenge them?
Analysis Tool #3: Ideologies
Ideologies refer both to the worldviews and beliefs of the designers when creating the game (and how they choose to represent them) as well as the worldviews and beliefs of the player as they play (which informs what they notice and how they interpret it). This frame provides players and designers a chance to reflect on what they believe about the gameworld and how they want to act within it. Ideologies are manifest both in the representational elements (what things look like, why those appearances and not different ones, what is included or excluded) as well as the mechanics of play (do players shoot and kill things, do they organize and arrange things, is conflict a driving motivation, or is resolution and so on).
IDEOLOGICAL TOOL: What are the values and ethics promoted by the game? How are these manifest in the representational elements? What elements are present (or absent) which inform your interpretation? What do you believe about the world? Does this match or conflict with what the game is “about”? How do these beliefs drive you to act within the game?
Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York, NY: Peter Lang Publishing.
Holmes, J., Cope, A., Hoffman, R., Pincus, B., and Shedd, J. (2013) Visual Analysis Toolkit: Four Use Cases. Poster presented at Games, Learning, and Society Conference 9.0, Madison, WI.
Kress, G. and van Leeuwen, T. (1996). Reading images: The visual grammar of design. New York, NY: Routledge.
Kress, G. and van Leeuwen, T. (2002). Colour as a semiotic mode: Notes for a grammar of colour. Visual Communication (1.3):343-368.
Serafini, F. (2010): Reading multimodal texts: Perceptual, structural, and ideological perspectives.Children’s Literature in Education (41): 85-104.