19th
Mark Changizi on how we read
“Writing was invented only around five thousand years ago, far too recently to have affected our brains. In fact, most of us don’t have to look back more than several generations to find ancestors who couldn’t read.
How, then, do we have reading areas for a brain that didn’t evolve to read?
Stanislas Dehaene, neuroscientist and author of Reading in the Brain, argues that our brains have undergone “neuronal recycling,” where writing has shaped itself over time to be easy on our visual systems.
And what’s the trick to getting writing to fit into our illiterate visual system?
In my own research I have suggested how it happened: culture shaped letters to look “like nature.” Oliver Sacks describes the research this way:
“Such a redeployment of neurons is facilitated by the fact that all (natural) writing systems seem to share certain topological features with the environment, features that our brains have evolved to decode. Mark Changizi and his colleagues at Caltech examined more than a hundred ancient and modern writing systems, including alphabetic systems and Chinese ideograms, from a computational point of view. They have shown that all of them, while geometrically very different, share certain topological similarities. (This visual signature is not evident in artificial writing systems, such as shorthand, which are designed to emphasize speed more than visual recognition.) Changizi et al. have found similar topological invariants in a range of natural settings, and this is has led them to hypothesize that the shapes of letters “have been selected to resemble the conglomerations of contours found in natural scenes, thereby tapping into our already-existing object recognition mechanisms.”
(…) Reading and writing is a recent human invention, going back only several thousand years, and much more recently for many parts of the world. We are reading using the eyes and brains of our illiterate ancestors. (…)
Good Listening
That’s what good listeners do. They rewind the story if needed, or forward it to parts they haven’t heard, or ask for greater detail about parts. And good communicators tend to be those who are able to be interacted with while talking. (…)
Even though we (arguably) evolved to speak and listen, but didn’t evolve to read, there is a sense in which writing has allowed us to be much better listeners than speech ever did. That’s because readers can easily interact with the writer, no matter how non-present the writer may be. Readers can pause the communication, skim ahead, rewind back to something not understood, and delve deeper into certain parts. We listeners can, when reading, manipulate the speaker’s stream of communication far beyond what the speaker would let us get away with in conversation. (…)
When one’s eyes are free, people prefer to read stories rather than hear them on tape, and the market for books on tape is miniscule compared to that for hard copy books. We humans have brains that may have evolved to comprehend speech, and yet we prefer to listen with our eyes, despite our eyes not having been designed for this! (…)
When we speak there are typically only a small number of people listening, and most often there’s just one person listening (and often less than that when I speak in my household). For this reason spoken language has evolved to be a compromise between the mouth and ear: somewhat easy for the speaker to utter, and somewhat easy for the listener to hear. In contrast, a single writer can have arbitrarily many readers, or “visual listeners.” If cultural evolution has shaped writing to minimize the overall efforts of the community, then it is the readers’ efforts that will drive the evolution of writing because there are so many of them. That’s why as amazing, as writing may be, it is a gift to the eye more than a gift to the hand. For example, a book may take six months to write, but it may take only six hours to read. That’s a good solution because there are usually many readers of any given book. (…)
Harness the Wild Eye
Just as horses didn’t evolve to be ridden, eyes didn’t evolve for the written. Your eyes reading these words are wild eyes, the same eyes and visual systems of our ancient preliterate ancestors. And yet, despite being born without a “bridle,” your visual system is now saddled with reading. We have, then, the same mystery as we find in horses: how do our ancient visual systems fit so well in modern reading-intensive society? (…)
Eyes may seem like a natural choice for pulling information stored on material, and indeed vision probably has inherent superiorities over touch or taste, just as horses are inherently better rides than rhinos. But just as horses don’t fit efficiently into culture without culture evolving to fit horses, the visual system couldn’t be harnessed for reading until culture evolved writing to fit the requirements of the visual system. We didn’t evolve to read, but culture has gone out of its way to create the illusion that we did. We turn next to the question of what exactly cultural evolution has done to help our visual systems read so well. (…)
Word and Object
Is there something beneficial about drawing objects for the words in writing? I suspect so, and I suspect that it is the same reason that animal-call symbols tend to be animal-call-like: we probably possess innate circuitry that responds specifically to animal-call-like sounds, and so our brain is better able to efficiently process a spoken word that means an animal call if the word itself sounds animal-call-like. Similarly, we possess a visual system designed to recognize objects and efficiently react to the information. If a word’s meaning is that of an object (even an abstract object), then our visual system will be better able to process and react to the written symbol for that object if the written symbol is itself object-like. (…) 
Our brains evolved to perceive objects, not object-parts, because objects are the clumps of matter that stay connected over time and are crucial to parsing and making sense of the world. Our brains naturally look for objects and want to interpret stimuli out there as objects, so using a single stroke for a word (or using a junction for a word) is not something our brains are happy about. Instead, when seeing the stroke-word sentence in (a) in the “rain in spain” figure, the brain will desperately try to see objects in the jumble of strokes, and if it can find one, it will interpret that jumble of strokes in an object-like fashion. But if it did this, it would be interpreting a phrase or whole sentence as an object, something that is not helpful for understanding a sentence: the meaning of a sentence is “true” or “false,” not any single word meaning. Using single strokes as words is, then, a bad idea because the brain is not designed to treat single contours as meaningful. Nor is it designed to treat object junctions as meaningful. That’s why spoken words tend to be written with symbols having a complexity no smaller than visual objects. (…)
If written words must be built out of multiple symbols, then to make words look object-like, make the symbols look like object parts. That’s what culture did. Culture dealt with the speech-writer dilemma by designing letters that look like the object parts found in nature, object junctions, in particular. That way written words will typically be object-like, so that again our visual system can be best harnessed for reading.”
— Mark Changizi, The Man Who Mistook His Y for a Hat. Oliver Sacks and how we read, Psychology Today, July 15, 2010
The Topography Of Language
The Variety of Visual Signs
“The evolution of ornamentation, art, painting, and other non-linguistic visual signs (i.e., signs not part of language) has gone on unabated, diversifying into millions of non-linguistic symbols used over the ages, and occupying nearly all aspects of our lives, including pottery, body art, religion, politics, folklore, medicine, music, architecture, trademarks and traffic.
(Click image for larger size)
Writing (i.e., visual signs distinguished by use as a means of visually recording the content of spoken language) has also undergone an evolutionary explosion in variety. The earliest writing appeared several thousand years ago, and occurred independently in Sumer, Egypt and China (and much more recently in the Americas). These earliest linguistic visual signs were pictograms, evolving later to logograms (where a character denotes an object, idea or action), and a single logographic writing system (such as Chinese or Linear B) can have many thousands of distinct visual signs. It wasn’t until about 2000 years ago in Egypt that phonemic writing was invented and used, where each character stands for a constituent of speech rather than having a meaning as in logographic writing. Many hundreds of writing systems have evolved and diversified from this ancestor (e.g., Latin, Arabic, Avestan, Mongolian, Phags-pa), varying widely in geometrical shape and style, and in the aspects of speech the characters represent (e.g., alphabets represent consonants and vowels, abugidas represent just consonants, and syllabaries represent syllables).
Amongst both non-linguistic and linguistic signs, some visual signs are representations of the worlde.g., cave paintings and pictograms, respectivelyand it is, of course, not surprising that these visual signs look like nature. It would be surprising, however, to find that non-pictorial visual signs look, despite first appearances, like nature. Although writing began with pictograms, there have been so many mutations to writing over the millenia that if writing still looks like nature, it must be because this property has been selectively maintained. For non-linguistic visual signs, there is not necessarily any pictorial origin as there is for writing, because amongst the earliest non-linguistic visual signs were non-pictorial decorative signs. The question we then ask is, Why are non-pictorial visual signs shaped the way they are?
Previous efforts at answering this question have primarily concentrated on the differences. In particular, some of the shape differences among different (non-pictorial) visual signs are due to the kind of writing implement used, whether impressions in clay tablets with a blunt reed, rounded writing on leaves, or the physical details of a modified feather-tip point. Little attention has been devoted to uncovering the similarities, however, and as we will see here, there are deeper visual regularities that hold across human visual signs, independent of the writing mechanism (regularities that are also found in nature).
It is as if someone had noticed that throat size causes male and female voices to sound differently, without noticing that male and female speech possesses a critical deeper regularity, namely that they utter the same set of phonemes, morphemes, words and sentences as one another (within a single language speaking community). We will find that, despite superficial differences in their shapes, visual signs appear to possess similar underlying “visual phonemes.” (…)
We have seen that human non-pictorial visual signs appear to possess a characteristic signature, and we have seen that this signature is not a result of chance. Before attempting to explain this signature, a natural first question is, Does this signature appear to be good for the eye, or good for the hand (or any other writing mechanism)?
There are at least two reasons for expecting that visual sign shapes are designed (by cultural selection) for ease of reading, not ease of writing. First, visual signs are written once, but can be read many times. Second, writing speed is typically limited not by the motor system, but by the time taken for the writer to compose the sentence; that is, writing is not like talking, where we can talk effortlessly without feeling as if we are composing our thoughts. (…)
Natural to the Eye
The topological shapes of non-pictorial visual signs are, then, for the eye, not the hand. But we are still left with the question, Why does the eye like these shapes? Here is where the evolutionary, or ecological, hypothesis enters into the story. Because over millions of years of evolution our visual systems have been selected to be good at processing the conglomerations of contours occurring in nature, I reasoned that if visual signs have culturally evolved to be easy to see, then we should expect visual signs to have natural topological shapes.
Where are these topological shapes in nature? What were conglomerations of strokes for visual signs are now conglomerations of contours for natural scenes. Contours are the edges of objects (as seen by the eye), not, of course, strokes in the world. For example, an L occurs in the world when exactly two edges of an object meet at their endpoints, like an elbow. A T occurs in the world when the edge of an object goes behind another object in the foreground. A Y occurs, for example, at the inside corner of a rectangular room. (…)
(i) we wish to read words, not letters; and (ii) we have evolved to see objects, not object-junctions. In this light, we expect culture to select words to look like objects, so that words may be processed by the same area in visual cortex responsible for recognizing objects.
Logographic characters (e.g., Chinese) and non-linguistic symbols do tend to be more object-like, possessing many more than three strokes. For phonemic writing, however, there are severe limits to how closely words can match natural objects, for the manner in which letters combine is determined by speech. However, by having letters shaped like natural object-junctions—rather than natural contours or natural whole objects—written words become combinations of natural junctions, and thus more similar to objects and more easily processed by our visual system.
Evolution by natural selection is too slow to design our brains for reading, and so cultural selection has come to the rescue, designing (without any designer) visual signs for our brains. Because our visual systems have evolved to be good at perceiving natural objects, cultural evolution has created non-linguistic symbols, logographic symbols, and written words in phonemic writing that tend to be built out of object-junction-like constituents, and are thus object-like.
In particular, this explains why letters tend to have around three strokes and have the topological shapes they do. We expect that these insights will be useful in designing optimal alphabets or visual displays.
Because culture is capable of designing for the eye, the visual signs of our culture are a fingerprint of what our visual systems like. Akin to the linguistic study of the auditory productions humans make, the “visual linguistic” study of the visual productions people make is a currently under-utilized tool for vision research.
There is every reason to believe that the study of visual linguistics will aid traditional lab experiments on vision and brain design as much as linguistics has supplemented lab experiments on cognition.”
Mark Changizi, cognitive scientist, author, The Topography Of Language, Science 2.0, Sep 17, 2009.
See also:
☞ Mark Changizi on brain’s perception of the world
☞ A brief history of writing, Lapidarium
☞ Mark Changizi, Music Sounds Like Moving People, Science 2.0, Jan 10, 2010.
☞ Mark Changizi on How To Put Art And Brain Together
☞ Maria Popova, A Visual History of the Alphabet, The Atlantic, Jun 21, 2011
