Howtonotsuckatgamedesign.com

An introduction to semiotics for game designers and artists, and a detailed analysis of Batman Arkham City’s complex and effective visual language.

Semiotics What? Why?

Why care about semiotics and what is semiotics anyway?

In short, semiotics is the study of how humans interpret visual language and how they use signs to communicate with each other. Here is a more elaborate quote from wikipedia, which is actually quite an accurate description:

“Semiotics, also called semiotic studies or (in the Saussurean tradition) semiology, is the study of signs and sign processes (semiosis), indication, designation, likeness, analogy, metaphor, symbolism, signification, and communication. Semiotics is closely related to the field of linguistics, which, for its part, studies the structure and meaning of language more specifically.”

For anybody creating visual media, semiotics, in my opinion, is a must know and important tool. It is basically a way of understanding, analyzing, cataloguing and utilizing visual language. Yeah, knowing semiotics is like having a thesaurus for images.

Video games are largely communicating with the player through visual means. In-game graphics and GUI (graphical user interface) together with sound, music and occasional controller rumbling are used create the actual playing experience. While just screenshots, trailers and promo art are used to drum up interest for a game in other media, like websites, television or magazines.

Getting your visual communication right is key for the success of any type of video game. And especially when working in teams, consolidated terminology and goals are key to have everybody push for the same effective imagery.

Categorizing Signs

A sign is any piece of visual information in the widest possible sense of that term. Signs are not only images we recognize are made by other humans, but also accidental visual occurrences and visible natural phenomena. If you can see it and process it as information,… it’s a sign.

In semiotics we differentiate between 3 categories of signs:

Icon – a graphical or photographical depiction of a visible thing
Symbol – a graphical substitute for an invisible concept
Index – a piece of visual evidence for an occurrence

A sign fits a certain category, depending on the intention of the sender (if there is a sender at all) and the interpretation of the viewer. Considering the fact, that the intended meaning of a sign and what it means to the viewer isn’t always congruent and considering that many indexes actually have no sender whatsoever,… categorization of signs is kinda arbitrary.

Here is an example of multiple ways to categorize the image of a lightning bolt:

fig 1: This is an icon of a lightning bolt. A simple graphical abstraction of what a lightning bolt looks like.
fig 2: This is a symbol for danger and high voltage. The lightning bolt is used as a graphical metaphor to visualize electricity and the yellow triangle base (which is a sign itself) is used to help the viewer recognize the image as an official warning sign.
fig 3: This is a photograph of a lightning bolt. The lightning bolt is an index for a thunderstorm.

To be needlessly specific, fig 3 is of course technically an icon of a lightning bolt, since it is just a photograph of a thing, not the actual thing itself. Like Ren‚àö¬© Magritte’s not-pipe. But since images are the only thing I can put up here, we have to ignore that fact for now.

While the warning sign above can be viewed as an icon or symbol, or both. The actual lightning can not be viewed as either of those two things. Icons and Symbols are always man-made, indexes not necessarily. So as long as you don’t believe Zeus is using lightnings to tell us how angry he is, the lightning in the photograph above is an index. What the viewer thinks who made the sign and why factors into the interpretation as much as the actual visual information itself. Content is key and so is context.

Here are a few contexts and sign combinations, that give even more possible semiotic properties to the image of a lightning bolt:

lightning bolt + Frankenstein monster = symbol for life force
lightning bolt icon on costume of The Flash / DC Comics = symbol for speed
lightning bolt icon striking an icon of a specific body region = symbol for pain
icon of a hand holding a lightning bolt = iconic depiction of the greek god Zeus
icon of a lightning between two poles = symbol for connection
icon of a lightning bolt above a button on a camera = symbol for flash photography
scar resembling lightning bolt on Harry Potter’s forehead = index for traumatic occurrence

So you see, while the meaning of a single individual sign is arbitrary (some would say even non-existent), combining signs and providing context allows us to guide the viewer’s interpretation. You can never be a 100% certain, that your viewer understands your signs in the way you intended. But you can get pretty close, if you know who your viewer is, what his viewing habits are and what his surrounding cultural conventions for the meaning of images are.

Why differentiate between icons, symbols and indexes?

It allows us to understand and optimize how well a sign is read by the average viewer. Icons are the most widely understood signs, since they just replicate how things actually look. Visual reception in general is not dependent on cultural conventions like for example spoken language totally is. Symbols on the other hand need to be explained and rely on a similar cultural understanding between the author and the viewer. Written words for example are arrangements of symbols and understanding them heavily depends on being able to read in the used language.

Looking at the two emergency exit signs above, which one do you think is understood in more languages? One is iconicly showing somebody running to a door, while the other one is showing an arrangement of symbols.

Now using an index always has a component of time, cause and consequence in it since it displays the current state of things. The viewer sees the current state of a thing and then can either comprehend the current situation, deduce what happened before or anticipate what may happen next.

Cognitive Principles

Before the viewer can make any conscious or subconscious attempt at deciphering your sign language, the sign has to grab the necessary attention from the viewer first and secondly appear to be of significance. I wont dive into cognitive psychology here. This would be too far off the subject at hand. But you can go here to learn more. Big recommendation.

The thing is, especially in interactive media, you have to make any sign a point of interest to the viewer before(!) the viewer actually starts understanding it. To achieve this, a sign must visually stand out from the rest of the screen to catch the eye and also instantly indicate significance to warrant further investigation.
I already wrote a couple of articles about using colors, contrasts and viewing habits to grab the players attention here and here.

Analyzing Batman – Arkham City

So, what are we going to do now?

Following the semiotics goals I defined earlier, we will explore the complex network of sign language of AAA games, comic books, the Batman universe and related pop-culture, we will explore the narrative themes behind it all and we will examine how Rocksteady implemented said sign language using semiotic principles.
For this we will dig through the characters and locations of the game (I took tons of photographs) and analyze what is there using three simple methods:

Step One: Scout for seemingly important images.
This is a cognitive task. A filter. Recognizing patterns and recognizing emphasized images is the first step.

Usually, if images appear in patterns, meaning there are dots to connect, it is a pretty good hint at those images being of significance. In Arkham City, some patters are easy to spot others require a lot more attention to detail.
When it comes to emphasized images, we see something that appears to be created to catch our attention. When we feel that somebody created the image to convey information to us, it immediately becomes significant. Special colors, arrangement of elements, camera focus, written descriptions, all those things can make an image seem to be a direct message, even if we not yet know what it means.

When I was taking photographs, I already was applying this filter by only shooting stuff I found to be significant. Though impractical, it would be per definition correct to examine every single little piece of visual data, since every element was placed there by an artist, lending it significance. But we keep it practical here.

Step Two: Assign images to themes.
Now, once we’ve collected all the images we deem to be important, it’s time to search for themes. This is basically refining the process of pattern recognition by not only using your cognitive capabilities but also some additional back knowledge.

The way we establish themes and assign the available images to them is completely a matter of personal interpretation. But this personal interpretation can be more or less substantiated by knowledge about already popular themes in the source material and our culture.

Assigning images to themes is a guess basically and, if done properly an educated guess.

Themes are the context in which we assign meaning to images. The theme we pick for the images to be part of heavily influences what we think it means.

An example:
Bare cognitive skills make us recognize that there are a lot of bat images in Arkham City. A guy in a stylized bat costume is on the box cover and always the centre of attention, the word ‚Äö√Ñ√∫bat‚Äö√Ñ√π is in the title of the game, bats are part of the GUI, the protagonists weapons are icons of bats and an icon of a bat is on his chest. So it’s natural to attribute some significance to bats here.

Without resorting to any back knowledge, the only theme we could extrapolate from this pattern would be “a guy likes bats”. Knowing how Bruce Wayne decided on picking the bat as his totem animal of choice (see part two of this series) allows us to establish the theme “the bat is a symbol that strikes fear in the heart of criminals”. Suddenly throwing boomerangs shaped like bat icons is no longer just a gimmick. It becomes psychological warfare.

Now, let’s say, somebody doesn’t know who Batman is (just roll with it, fanboy!) but sees him and is reminded of Dracula. The bat fetish, prowling at night, cloaking himself in a black cape… it’s not too far off to draw a connection. The theme would easily shift to be ‚Äö√Ñ√∫vampirism‚Äö√Ñ√π and justifiably so.

You see, knowledge of popular themes decides what connections we can draw. Having a vast mental library of themes and images helps a lot to understand signs in the most objective and reliable way. So, do your research and read some classics from time to time. As designers, once we have confidently decided on a theme for our own work, we can use this theme as a pool of signs to incorporate in creating further designs. And keeping close to widely known and popular themes increases the chance of our sign language resonating well with our audience.

Step Three: Re-evaluating signs using themes.
Now that we deduced which themes to look out for, we can revisit the material and analyze how and how well the designers articulated their themes.

Which icons are used to convey the theme? Which indexes and symbols are used?
How often are they used? Where are they used?
Are there enough relevant signs to allow the player to easily recognize the theme?
What methods of sequential storytelling are used to convey the theme?
How much viewer attention is wasted by images hard to assign to any theme?
Are there images that contradict the theme?

All those questions help to judge the quality of the presented visual communication, art direction and visual narration.

Reverse Engineering

Finally, why is all this useful?

Well first, understanding the signs of the Batman universe is always nice trivia on nerdy parties and enhances your viewing pleasure when consuming batman media… your welcome. But, wait! There is more!

Reverse engineering themes and the ways to convey them from impressive case studies like Arkham City gives you the tools to convey your own themes. And conveying themes is half the battle in all your visual storytelling efforts (the other parts are staging and sequential methods).

Check out the following parts of this rather massive article series and find out how that works. We of course begin with dissecting the dark knight himself and his nemesis the clown price of Gotham.

Hit the Batman tag for all articles of this series.