How Your Brain Understands Visual Language

Or, “why email is still represented by an envelope.” Design professor Josiah Kahane explains.

How Your Brain Understands Visual Language

Semioticians classify signs or sign systems in relation to the way they are transmitted. To illuminate one way semioticians look at things, let us peek at stage and film semiotics, which study the various codes and signs on stage and in the movies, and how they are interpreted by the viewer in spite of their artificiality. In film, the actors often stand one behind the other, facing the audience and not each other; recalling the past is indicated by switching to black and white or via a wiggly picture; squeaky doors and shadows moving on stairways are associated with horror, and so on. Historian Pamela Smith‘s book, Symbols, Images, Codes: The Secret Language of Meaning in Film, TV, Games and Visual Media provides many examples.

Alfred Hitchcock–a master of film symbolism.

Designers are quite familiar with product semantics, which is a branch of semiology (Semioticians prefer the terms “Design Semiotics” or “Product Semiotics”) that studies of the use of signs in the design of physical products. Product designers frequently resort to metaphors to exhibit original and aesthetic solutions to design problems. They may use metaphors as a tool or method in the design process, which help to identify, frame, and solve design problems.

Familiar products may also be used to express ideas or words. If you closely inspect the latest iPhone screen icons, you will be surprised by signs still represented by objects that we barely see any longer, especially not anywhere else in the iPhone itself: a 35mm reflex camera represents digital photography, time is represented by an analog clock, an envelope stands for e-mail, zebra striped film-board represents video, old telephone handset stands for telephone. Probably most anachronistic are mechanical cogwheels representing computer setting. Symbolic icons tend to last.

We use outdated products as symbols because they are ingrained in our mind’s dictionary.

Insights From Linguistics
In 1975 the linguist Eleanor Rosch asked 200 American college students to rate, on a scale of one to seven, whether they regarded the following items as a good example of the category “furniture.” These items ranged from chair and sofa (which scored the highest score) to a love seat (which ranked tenth), to a lamp (thirty-one), all the way to a telephone (ranked sixtieth). This was the beginning of the prototype theory that explains how the brain employs cognitive category classification. Classification may be quite complex, depending on what classification category we employ. A somewhat parallel linguistic classification approach, called “Semantic Feature Comparison Model” obeys the familiar saying “If it looks like a duck, walks like a duck, quack like a duck, and flies like a duck, it is a duck.” The features define the object we observe.

The philosopher Ludwig Wittgenstein, in his second book, Philosophical Investigations (1953), compared the various uses of the term “games” (board games, card games, ball games, Olympic games, and so on) and came to the conclusion that they do not have a common denominator, but rather possessed what he called family resemblance. The notion of family resemblance is better expressed today by another notion—conceptual distance.

We identify features of airplanes in spite of different configurations, orientations, or colors (Mokhtarian
& Abbasi, 2000).



Recent evidence suggests that culture and language can influence perception; for example, there is evidence that people tend to perceive things in ways that are influenced by the manner in which they have learned to think in order to function efficiently in their ecological setting. In English we call the device that records incoming telephone messages “answering machine” (a genderless mechanical device). In Hebrew the term used is “electronic secretary” (a humanized feminine electronic gadget). From these different terms we may infer that the visual expressions of that telephone device in these two languages may differ. The English-speaking design student may envision an electronic gadget, maybe with a reference to the telephone. The Hebrew speaking design student may envision a product with feminine features with an earphone-microphone device or a shorthand writing pad.

A question designers may ask linguists has to do with how language changes over time. If language constantly changes because people and their culture evolve, why should the more visual aspects of language not follow suit, including the language of man-made objects?

Languages and dictionaries change continually.

Visual Language
Language is not just verbal or written. Speech as a means of communication cannot strictly be separated from the whole of human communicative activity, which also includes the visual. The word “imagination” definitely suggests that we can also think in images. Visual language is defined as a system of communication using visual elements.The term visual language in relation to vision describes the perception, comprehension, and production of visible signs. Just as people can verbalize their thinking, they can visualize it. A diagram, a map, and a painting are all examples of uses of visual language. Its structural units include line, shape, color, form, motion, texture, pattern, direction, orientation, scale, angle, space, and proportion. The elements in an image represent concepts in a spatial context, rather than the time-based linear progression used in talking and reading. Speech and visual communication are parallel and often interdependent means by which humans exchange information.

What we have in our minds in a waking state and what we imagine in dreams is very much of the same nature. Dream images might be with or without spoken words, sounds, or colors. Classical Greek philosophers believed that a replica of an object enters the eye and remains in the soul as a memory as a complete image. It is amazing that such an insight originated more than two thousand years before the workings of the brain were unraveled.


Visual Thinking
Designers are trained to solve problems, frequently in a visual way. Designers can form images in their mind’s eye, manipulating and evaluating ideas before, during, and after putting them on paper or on the computer screen. Visual thinking used by designers institutes a cognitive system equivalent with, but different from, the verbal language system. And not just trained designers. It seems that human beings have an innate capacity for cognitive modeling and its expression through sketching, drawing, construction, acting out, and so on, that is fundamental to human thought.

Visual thinking by designers.

Literacy (communicating in words by means of writing and reading) and numeracy (communicating information by means of numbers) are already well-developed achievements of the human species. The development of the visual aspect of human communications as a parallel discipline to literacy and numeracy has been referred to as graphicacy, still far from being a familiar term. The ability to think and communicate in visual terms—both the understanding and conception of the visual—becomes of equal importance with that of literacy and numeracy in today’s learning process.

Visual instruction for coffee making.

This excerpt from The Form of Design: Deciphering the Language of Mass-Produced Objects was published with the author’s permission.

About the author

Josiah Kahane is a professor emeritus of design at the Holon Institute of Technology (HIT) in Israel. He has been a tenured design faculty at HIT for 37 years