The Territory Is The Map

A sensorimotor intervention

Matis men hunting in the Amazon

What I aim to do in this article is to introduce non-representationalist, or “model-free” theories of perception, for consideration - the sensorimotor theory of O’Regan et al. in particular. Making this intuitive to an interested lay audience is my goal, albeit since I spent my life thinking of perception as representation myself, it’s still just beginning to seem intuitive to me.

I have decided to do go about doing this by first subverting Korzybski’s old nugget of wisdom: “The map is not the territory”. The original intent of the quote notwithstanding, I’ve seen it being used as a kind of thought-terminating cliché, to shut down discussion that direct perception of the world is possible. In any case the quote only applies to representations.

To possibly make the idea that our perception of the world is simply not a representation intuitive, to someone like myself whose mind is entrenched in this map-territory talk: I realized that beyond talking about neuroscience of perception or CogSci, I will first need to dismantle the logic of the metaphor itself.

1. We don’t need maps to get around

You are a man of the Matis tribe, hunting deep within the Amazon jungle. Surrounded on all sides by uniform greenery, not seeing beyond your arm’s length. You must find your way home. Fortunately, you have left a trail: strategically breaking off plant stems on your way here. These are easy to miss though and at one point you indeed lose the trail. Not to worry though: There is a big tree here. You could climb it, the same way you’ve been doing since childhood, so you can see the forest from a bird’s eye view; or you can bang on the tree with a stick. The sound reverberates through the jungle. Someone else in the distance, responds with the same rhythmic banging. You are close.

You are a Bedouin man in the Arabian Desert and someone has stolen from you. You must track down the thief. In the dry desert ground you see camel trails all over. This camel was walking bush to bush; it is a wild one, grazing. This other camel has deeper footprints; it may have been carrying something. You find its dung and in it, grains: someone was feeding it. This is the trail to follow.

You are an aboriginal Australian in the outback, walking from one watering hole to the next. Without thinking about it, you head North. You don’t need to check where North is; you learned as a child to unconsciously track the cardinal directions. It would be like mistaking left and right. The stars above are guiding you too, the same stars you stared at ever night before falling asleep, all your life. Getting lost is something only children, the very old and senile, and the white fellows do. You know exactly where you’re going.

Thousands of years before there were maps, before there was writing or even much painting, humanity explored and populated this planet. How? They followed stars, winds, rivers, the tracks of migrating animals and the landmarks in the songs and stories of people who walked before them. They used the territory as a map.

2. Your head inside your head

Your cognitive psychology professor shows you the classic inattentional blindness demonstration video. The gorilla suit man walking behind people passing a ball in a circle. Everyone watches with disinterested smugness because everyone has seen the video countless times, you are psychology undergrads. If there was ever a time when you did actually miss the gorilla man passing through, you don’t remember. Indeed you all see the gorilla man. Then the professor reveals the video was actually a variant; the real trick is that the curtains in the background have gradually changed from green to blue via a digital edit which indeed nobody noticed. Nobody really likes this professor.

Inattentional blindness is scary because it seems to contradict our very strong preconception that we have a unified pictorial representation of the world in our mind: the brain, like a computer, takes in the data stream from our senses, and “renders” it as a kind of continuous video feed, which “we” are watching. A lot of problems in philosophy of mind all revolve around who or what is this “me” sitting in my head watching this video feed, and is there another video feed in his head and so on ad infinitum.

Of course cognitive science already tells us this isn’t so. The gorilla suit thing is just one of many cracks in the edifice. Here’s a question: where, in your visual field do you see colors better: in the center or in the periphery? What is it? You see colors the same way, no matter where you are looking? Wrong. You have more photoreceptors, but especially cones, in the center of your retina. In the focal point of your vision, the fovea, you see color perfectly, and at the very edge of vision, not at all. The appearance that it is otherwise and you can see color everywhere just fine is a trick by your brain, fooling you into thinking you see the world as it really is. You can test this by staring at a fixation cross, taking a colored pencil out of a box at random and slowly moving it into your field of vision while staring ahead. Chances are you won’t be able to tell what color the pencil is. To me, if I strain myself to describe it, it always looks like a bland fluctuating brownish-yellowish-bluish-grey before I focus on it for real and see it is bright green or something. Spooky stuff. Your vision is lying to you.

If you knew this already you also definitely know that the retinal image is also upside down. The brain has to rotate it, the absolute scoundrel. Blind spot, binocular rivalry, the evidence for this ruse is everywhere.

So the video feed in your brain, the story goes, isn’t actually rendered from your sense data stream like an online video game: your brain is an entire film studio, using its neurons as VFX sweatshop slave labor to edit the crappy image recorded by your eyeballs into something watchable, so that the audience doesn’t walk out of the Cartesian theater in protest. You don’t see the gorilla man because this fake video fails to update in time, still showing you the people passing the ball back and forth like normal.

Step 2. ???, step 3. profit. You really have to wonder why this is, why evolution has given us lying brains, has hidden the truth of the world from our eyes deliberately, but based on the evidence, it’s difficult to see how another interpretation could be possible.

3. Sight is a form of touch

What’s interesting is that none of this weirdness related to sight comes into consideration when (on the rare occasion, in philosophy of mind and cognitive science) we discuss our sense of touch. But there are exact analogies.

You go to a dark room and, by chance, you grab an object with both hands. Simply by grabbing it you don’t learn much about it other than that it exists. So unknowingly, reflexively, immediately, you start moving your fingers around to determine its shape. But throughout this process, you are not freaked out by the fact that the object seems solid and consistent, even though you don’t perceive anything at all wherever your fingers aren’t currently touching. Your eyes only have 1 blind spot each; your hands have 4, each gap between your fingers. So why is it that you don’t perceive gaping chasms in the object itself, perhaps 10 separate objects, hanging together magically, when you grab onto it?

Once we move away from the metaphor of pictures and video feeds, the answer here seems blindingly clear. You don’t perceive gaps because there are no gaps that you can know about. In the first instant of grabbing the object you don’t perceive gaps, nor do you not perceive them; you just perceive that you have grabbed something, and that’s it. But the first instant is just that, an instant; immediately, your fingers start feeling around in the dark, until they identify the object.

Could it be that your vision works the same way?

4. Cybernetics 101

You are a brain, the brain of something just recently born. Life isn’t easy. Sensory signals with no meaning to them come in, and you can make motor signals go out… to somewhere. And that’s all you have to work with.

You cannot be sure how your sensory signals relate to the things in the external world that caused them, or how you could use them to learn about those things. If there are things in an external world at all. Furthermore when your motor signals go out, you cannot know whether they have actually successfully moved something out there, and if so, what. How are you supposed to “represent the external world” without actually having access to it? How could you possibly know that your model of the world is accurate to the real thing, then, when you have no way to compare the two? How do you even begin?

You begin like this. You know that your own motor signals change your next incoming sensory signals, in reliable ways. When you send this signal, that signal changes, in this way. All those other signals don’t change. Some signals change on their own, without you doing anything. You can’t paint a picture of the world, and put it next to the real thing. But you can, over time, feel out the myriad ways your own perceptions are contingent on your actions, and the ways in which they aren’t. Your job will be then, in order to survive, to predict the sensory consequences of your actions; observe the actual consequences; and try to minimize the difference between the two.

(Actually it might be that you can’t directly minimize this difference and you have to minimize a proxy variable, its theoretical upper bound called “variational free energy” but that’s beside the point right now).

5. The brain is a pathfinder

The point is this: forget this idea of lying, cheating senses. Your senses are extremely honest to you. You are just confused regarding what the senses are honest about.

It is not the place of your vision to tell you anything about the “image” on your retina. Your vision is about the world as it relates to you, and what it tells you are the visual consequences of your own movements. The reason your peripheral vision seems just as colorful as your focal vision is because the world itself does not change color depending on where you look. It’s not that your brain falsely represents its own capacity, telling you colors are available to you when they aren’t. It accurately tells you colors are available- if you were to look there.

If you’re like me a student of cognitive science with a head full of false imaginings computer metaphors, you can think of it this way: the visually accessible world acts as a sort of external memory; vision is a process similar to memory indexing. No picture is ever getting filled out with false information; but any information is only ever available upon query.

In terms of our map metaphor: no, the map is not the territory. The map isn’t anything: we have no map. In its basic capacity to perceive and act upon the world, the brain is not a cartographer, drawing images; it is a pathfinder in the Amazon, breaking off plant stems to keep track of its own steps.

Further reading: O’Regan, J. K. (2011). Why Red Doesn’t Sound Like a Bell: Understanding the Feel of Consciousness. Oxford University Press.