notes

Hi. Thanks for having me here. [smile]

This talk was originally intended for an audience not well-versed in the arts of rationality or the heuristics and biases literature. So this will be pretty basic for most of you, especially if you've read the sequence "A Human's Guide To Words". I hope, though, that one or two of the ideas that I talk about tonight will be genuinely new to you. Or at least give you a new way of looking at something you already understand.

I'll start with a quote. [lamb quote] Maybe some of you already know where this is going.

Briefly, so you know what to expect, I'll talk a little bit about
    - my background and why you should be listening to me
    - what communication is
    - how to do it efficiently and what happens when you optimize for information transfer
    - confirmation bias as a roadblock to communication
    - and finally why telepathy is so dang hard, plus the structure of human knowledge

So, who is this guy?

I'm a software engineer based in San Francisco. (That there is a definition symbol. Whether it's a good definition we'll leave for another time.)

My academic background is pure math and computer science. For most of my career I've worked in casual games and now I work for a company that builds a tool that transforms large datasets.

In the last few years, not coincidentally since I started reading lesswrong, I've been getting more interested in Bayesian statistics, machine learning, and data visualization. I'm also pretty knowledgeable about how technology is being used in education, so if you're interested in starting an education company come talk to me afterwards—I may know some people you can talk to.

A quick disclaimer: I'm not an expert, I'm just a guy who likes to think about things in a structured way.

So what is communication? Communication is a broad topic and I'm really only talking about a small, but I think important part of it. I'm talking about the kind of communication that happens all the time in my industry, say, between a designer and an engineer. The designer has an idea in their head and the engineer is the one who has to build it, so somehow the idea has to jump from the designer's head to the engineer's.

I'm talking about model transfer.

Here's a quote from Douglas Hofstadter: [read Hofstadter quote]

So what he's saying is, the thing that happens in my brain when I think about a tree or dogs or mathematics is not in any way the same thing as what happens in your brain when you think about a tree or dogs or mathematics.

But the patterns are nearly isomorphic.

For the programmers out there, they present the same interface.

Communication works because (and to the extent that) we share handles (such as words, phrases, or icons) to these nearly isomorphic thought patterns.

Now that we know what we're talking about, how do we do it well? We're actually shockingly good at this, and we can do it despite severe handicaps. Even deaf-blind people are able to learn language and communicate, provided they do it in a certain window of time. (By the way, I heard Noam Chomsky say that in an interview, but I couldn't find a good cite. If you're familiar with the research please let me know.)

Sometimes it helps to think about how a system works when you apply severe constraints—often the solutions you find will generalize to less constrained systems.

Let's look at an example. You're playing 20 Questions: you can ask anything you want, but you only get binary responses.

Bad questions are ones that are either too specific or too general.

Is it a zebra? a penguin? These are too specific: you'll go through half the dictionary on average before you find the answer, and that's assuming the answer's even in the dictionary!

Is it made of atoms? This question actually might be ok, since many things (love, or the Portugese language) are arguably not made of atoms. But once you narrow down the possibilities a little bit it's easy to imagine a question being too general.

Good questions: is it a mammal? is it fictional? What makes these good? They're good questions because they divide the search space roughly in half a each point. You're *guaranteed* to eliminate large swaths of the search space. So they have a high expected VOI (assuming you really care about getting the answer).

Pinning down a concept in concept-space is best done by asking questions whose answers divide the remaining space into regions of equal probability.

Fortunately we're allowed to use English, so we're not quite so bandwidth constrained, but communicating some concepts can still be quite difficult.

[What can possibly go wrong?]

What can go wrong?

Confirmation bias can get in the way. Often what we're trying to communicate is not just a single point in concept-space, but a region, or a category delimited by a certain rule. If we have some notion of what that rule is, we'll tend to seek information that confirms our notion of the rule rather than trying to eliminate it. Psychologist Peter Wason studied this effect in the 70's. Here's an example which you may have seen before.

[demo]

This might remind you of a certain game played with pyramid-shaped pieces. If I were still making games, I'd be all over a Zendo clone.

This is rationality 101. If you can't explain something or can't understand something, notice your confusion, and check your assumptions. Chances are your assumptions and the other person's don't match up. And this shouldn't be too surprising: you haven't had the same experiences, so you're going to understand the world differently even when presented the same evidence.

Pointing these things out can be a delicate business, and informal protocols tend to appear in environments where egos are involved but communicating ideas matters. It's usually a good idea to learn these protocols, but that's a topic for another talk.

How are we doing on time?

Let's talk about telepathy.

Stephen King says that writing is a kind of telepathy: you literally transport your thoughts across space and time. So why, besides confirmation bias and status games, is it hard?

We've already touched on one reason: categories are fuzzy and we're often not even aware of their shape. Also we're open to equivocation, intentional or not.

Second, most communication is highly compressed and, because most human knowledge is heavily hierarchical, vital background knowledge is typically left unarticulated.

More technically, knowledge is structured like a directed acyclic graph, a dependency graph. If I want to learn quantum mechanics I probably have to learn calculus first and basic physics, and before that algebra. You simply can't understand the set of concepts that make up QM without first understanding all those other things. This is clear in academia, but it's true in any specialized field, and this is the reason jargon exists. For instance in casual game design we talk about "appointment mechanics" and "energy" which are actually rather complicated in their details. Jargon is useful, but can also hinder communication with newcomers to a field or across different fields, especially when terms have overloaded meanings.

But I would argue that it's not as bleak as it sounds. For one, the human knowledge graph is not as dense as it seems, and many dependencies are softer than they seem. Also we're actually pretty good at working with gaps in our knowledge—through things like analogies—and often a general understanding of things is enough to get the job done.

And when we fill those gaps with the right background knowledge things seem to fall into place and we experience that "aha", that click of epiphany.

Thank you.