[Caveat lector: this is a very long, rambling meditation on concepts and coordination. It's not cut down for size or well-organized. That said, I had several insights while writing it.]

1. Case 1: Is an embryo a person?

People sometimes kill embryos. They do this intentionally, either to abort a pregnancy or to discard an in vitro embryo, or accidentally, e.g. by assaulting a pregnant woman. This raises the question: Is an embryo a person?

1.1. A big blob of subquestions

That question is a proxy question for (or, adjacent question to) several other questions, such as:

Should it be illegal to abort a pregnancy at week N? Is it immoral?
Should it be illegal to discard an embryo in vitro?
Should it be illegal to grow an embryo in vitro up to day N?
What legal protections should be given to embryos at week N? What about moral rights?
Besides the assault itself, what legal injuries are done to a woman if you assault her in a way that ends her week N pregnancy? What about moral injuries / wrongs?
In general, how much should we value and protect life, people, consciousness, humans?
What is consciousness? When is consciousness developed in a growing human? What is personhood? What is a human being?
How bad is it to kill an embryo at day N? Is it equivalent to killing a child? How does the badness compare to, for example, the badness of a woman going through a full pregnancy that she doesn't want?
Is it unethical to kill an embryo at day N? In other words, is that action ruled out out (or in) by some categorical rules? Or instead are wanting to judge that action by weighing good and bad consequences against each other?

1.2. Difficult questions produce uncertain, sticky, varied opinions

This is quite a blob of questions. Each of these questions is complex, and fans out into other questions. It would take a lot of cognitive work to form a good integrated judgement about one of these questions, let alone all of them.

Three upshots of this complexity:

People will tend to be unsure about the answers to some or many of the questions.
People will tend to defer to other people about many of the questions (e.g. "the scientists say that embryos form a brain at week N"). As one consequence, people will by default have a harder time updating their own views or the views of others, because the source of the views is some other third party. Further, the sheer complexity makes the question harder to reason about, so it's harder to learn and update.
Since complex, uncertain questions leave room for doubt, and doubt leaves room for a variety of opinions, different people will tend to have different views from each other about some of the questions.

1.3. A multi-question blob does not have "an answer"

Because this is a big blob of multiple questions, it doesn't necessarily have one answer. For example, I might say it's immoral to abort at 5 months, but it should be legal because of the principle of the mother's bodily autonomy. Do I think the embryo is a person, or not? IDK. Likewise, you might say it's probably not immoral to discard a 7-day embryo, but it should be illegal because there should be a clear bright legal line protecting human life. Do you think the embryo is a person? That question doesn't really have an answer.

In other words, because the single question "Is an embryo a person?" actually fans out into multiple questions, in a given situation you might want to respond "Yes it's a person, and also no it's not a person.". E.g. you might want to say "Yes, your assault that killed her pregnancy took away their future child (a person); no it's not the same as killing a 5-year-old.". You don't just answer with a yes or a no.

1.4. Coordination about X is fragile to not knowing what other people will think of X

Coordination is when several people form shared intentions and then act on those intentions in a synergistic way.

Coordination is difficult and requires some approximation of logical common knowledge. For example, to enforce norms, you want to have clear expectations of what will be punished, and what other people will agree was appropriate punishment. You want to know that they know that you know that Alice broke the norm when she did such and such; otherwise, you might expect them to view your punishment as being out of line. Furthermore, you want to know that they know that you know all that, so that you know that they know you weren't enacting punishment without being sure that you had the presumptive authority granted by known consensus to do so.

If A' is even slightly more complex than A, then it's much more difficult to get common knowledge of A' than to get common knowledge of A. It's harder to distinguish when A' applies; so it's significantly harder to tell when someone else will think A' applies; so it's harder still to tell when someone else will think that you think A' applies; and so on. (It would be interesting to see math models of this kind of phenomenon—maybe it's not true, or only true under some circumstances.)

More generally, if it's harder to know what other people are thinking of A' compared to A, it's much harder to have approximate common knowledge around A'.

1.5. It's hard to coordinate about big question blobs

To summarize the foregoing: Suppose you have a big blob of questions, such as all the subquestions about embryos. Then:

People will be uncertain about the questions; they have varied opinions; it'll be hard to get everyone to convince each other of the one correct set of opinions; and the opinions themselves are not binary but rather a set of several binary answers.
Because of that, it's very far from the case that people's opinions can be confidently well-summarized as a yes or no to one question.
So, there's no one question that achieves the ideal coordination flag for this blob.

1.6. Correction: It's hard to coordinate distributed judgements about big question blobs

Now that I think about it, there are some ways to coordinate about big question blobs. E.g.:

Delegate to a centralized body. E.g. the country delegates to the government, and then the government makes and enforces a complicated set of rules, which treat each question in the question blob (or at least, the legal ones). Or, a religious community delegates to a religious leader, or a set of scholars, and that body makes a complicated set of rulings.
Coordinate about each question separately. But this doesn't work well.

Delegation does work in many ways. But it doesn't necessarily work well to incorporate new values from the broader community. E.g. to get your view represented in parliament, you have to form a big enough coalition—which may mean coordinating with people who you don't agree with on all the questions. Similarly, to pressure leaders, you need some coalition.

When delegation isn't available, you want distributed norms, in which case again you need something that is highly coordinatable.

1.7. So, to coordinate, people look for simple questions

In order to enforce norms, and form coalitions for political representation and influence, and gather energy to work on projects in general, people want to coordinate. That is, they want to form shared intentions and act on them. To form shared intentions, they look for simple questions, where if you answer one way, you're on the team.

This process always has tensions, because there's always tradeoffs involved in focusing on a simple question. A simple question bundles together lots of decisions, and says, "decide on all of these questions this way, or decide on all of these questions that other way". It's likely that neither of those bundles of decisions is your favorite way to decide.

On the other hand, more people can join in on one of those bundles as being preferable to the default or something. When more people join in, there's a better chance of having enough coordination power (e.g. to get political influence or to enforce norms).

Further, the choice of what stance the coalition should take is a kind of Keynesian beauty contest. Simple choices are especially salient in such contests. So there's additional weight.

Further, to win a Keynesian beauty contest, you want to be legibly appealing to many subsets of values. A simple policy like "no discarding any biological humans" is visibly appealing to people who think that 7-day embryos have souls, or who think that 30-day embryos (but not 7-day) have souls, or 60-day. Because it's appealing to many sectors, and visibly appealing to many sectors (and also visibly visibly appealing, i.e. you can see that other people would see that it's appealing to others), it's a natural choice for a coordination point.

To give a rollout as an example: Someone might say that discarding even a 7-day embryo should be illegal. Their real reason for this stance might be something like:

Suppose we say that you can discard embryos up to 28 days. Well, perhaps if that's all that happens, that's actually ok. However, this social / political regime looks weak. It looks like a compromise position. It looks like there's a natural simple position, "no discarding any human life", but we decided to not take that stance. It looks like we're sharply denying that. In sharply denying that, we're basically admitting defeat. It looks like the natural (because simple) coalition has capitulated.

This is similar to, or a mechanism for, the general slippery slope argument. You don't want to go down the slope of weakening your position, hence weakening your coalition, hence weakening your coalition's bargaining position, hence further weakening your position, and so on.

2. Case 2: Should people be allowed to think freely?

2.1. The First Amendment and delegation

The first amendment answers "yes" and protects the general right. But it does so in a complicated way, with several exceptions and also several pillars:

See a bit more here: https://berkeleygenomics.org/articles/The_principle_of_genomic_liberty.html#analogy-first-amendment-rights

Like many complex policies, this works by delegation to the government. It might sound strange to say this, given that the outer clause of the First Amendment says "Congress shall make no law respecting...". Are they really enforcing a norm?

I would say yes, they are. They are maintaining a monopoly on violence, as any government has to do; and with that monopoly on violence, they are not restricting free thought (in the manners listed). In other words, they are defending the land from being militarily controlled by a regime that would restrict thought in that way. To say it yet another way, they enforce the norm "Do not use violence to restrict thought (even if you're the government).".

2.2. Erosion of free thought

It is said that free speech (hence free thought) is under attack. I don't know if that's more true now than before—e.g. there are sometimes Red Scares which come along with suppression of some speech, and likewise far left regimes might perform political purges. I'll take it for granted that there is a somewhat exceptional such attack currently, and speculate based on that.

Why would this be happening now? A hypothesis: Previously, the government is the center of gravity not just for institutional political power, but also for cultural political power. The First Amendment was the crown jewel, or keystone, of a culture of free thought.

But then social media happened. The internet became the public square, and social media companies gained power over moderation of the public square. Further, patterns of widely distributed social behavior enabled by social media became powerful social forces suppressing various ways of thinking.

This is a new type of concentration of power, and kinda circumvented / usurped the federal government's monopoly on mere physical violence.

2.3. Power vacuum

Social media opens up this new arena in which to fight over what thoughts are allowed, and there isn't a preexisting monopoly on force. This creates a power struggle for group beliefs.

People seem to abandon the simple rule of "thought must be free". Or rather, they abandon the general cultural values anchored by, or given a rallying flag by, the federal government's First Amendment protections.

To look at it another way, there wasn't a simple notion of free thought available. Instead, there was a complicated thing protected by government; and it just was practically not a problem to have no normed antibodies against cancelling, because there was no massively multiplayer online social media.

Since people naturally might, for example, not hire someone because of some extreme political opinions they hold, there's not especially strong cultural boundary against that. But that only becomes a major problem with social media, cancel culture, and the mob wielding the hire/fire/debank power.

2.4. Simple concepts help coordination because they are anti-invidious

If you have exceptions in rules, people might think they can lobby to get exceptions for themselves. This is especially so if the exceptions treat some class of people asymmetrically. That inspires envy. (It also inspires indignation, which is appropriate, as it is unjust. Justice would lead to symmetry; envy leads to multiple parties trying to get carve-outs, hence conflict.)

In other words, simple concepts are easier to agree on in negotiations. Negotiation is a kind of coordination—that is, coordination to find a better alternative to conflict. Cf. "Coherent Extrapolated Volition".

3. Case 3: How nice should you be?

3.1. Nice and kind

I'll (uncritically) hypothesize here that if a person is "nice", that means ze will go to significant lengths to make other people feel better, just for the sake of making them feel better, without context dependence.

I assume a lot has been written about niceness vs. kindness. I'll take a rough and ready definition that kindness is trying to deeply help someone even if you make them feel bad locally.

If you're nice, you don't amputate the patient's septic leg, because it would cause pain. If you're kind, you do amputate, because it will prevent death from sepsis.

The difference between nice and kind is pretty blurry, because it's hard to tell what would be really helpful for someone in the long term. Also, if someone's upset, there's definitely something going wrong for them. (It might not be what they think it is or say it is, it might be "their own fault", it might not be your responsibility, there might be nothing you can feasibly do to help, the intuitive way to help might make things worse, and so on; but there's definitely something going wrong for them.) When you're very uncertain, the best available guess at how to be kind might be to basically be nice.

3.2. Burning goodness

You shouldn't always be nice or kind.

For example, sometimes you have to protect yourself first. "If I am not for me, who will be for me?"

You have a moral obligation to not feed yourself to evil. That is the case even if the evil is represented by a nice person who is upset. For example, Scientology with solicitors on the street corner.

Should you give money to homeless people who ask for money? This feels like a tough question. The nice thing to do is to give them a bit of money. Otherwise they will feel sad and rejected. Is it kind? Maybe, maybe not. If they're going to buy drugs? I don't think so. What about snacks? How do you compare against the value you can give to the world by putting that money to better use? Will you do that? Should you have to? Are you setting goodness on fire? Or is it a good way to practice being nice and/or kind? Should other people think you're not nice if you don't? What about if you don't tip?

Suppose Fred presents an idea during a meeting with you, other coworkers, and the Boss. You make a critique of Fred's idea, and the group decides against the idea. Later, Fred confronts you. He's sad, upset, and hurt, and mad, and asks you to not embarass him like that in front of everyone. But you weren't rude or mean or derisive, you just critiqued the idea. Possibly everyone else would very slightly downweight their expectation of Fred's ideas being good, based on that interaction—but shouldn't they? It's not that big of a deal. Next time, should you avoid critiquing him because it would be not nice?

Peace is good. Should you be a pacifist? No, that's not right. You should figure out how to pursue and prepare for peace, without baring your neck to your enemy.

3.3. The virtue of niceness

On the other hand, "just be nice" is a simple concept and simple policy. This gives it great power. By following that principle, you are often nice, and even kind, when others might have felt no obligation, or might have rejected that option as being a burden. Niceness and kindness can be reciprocated directly; or indirectly through reputation; or indirectly through you being part of a healthier, more generous community; or through others being able to see that you are simply, steadfastly nice, and so they are more free to rely on you and act in ways that are positive-sum given that you will hunt stag.

3.4. A complicated boundary

You can judge on a case by case basis (e.g. I tip, I don't give homeless people money but I look them in the eye and say hi and help people in immediate physical distress, I'm in favor of seeking peace with the Palestinians and working towards a Palestinian state but it has to be disarmed until its self-government can prevent terrorist takeover, etc.). This takes some work, and also it's less legible.

4. Case 4: Reprogenetics

4.1. Slippery slope

If we allow some amount of reprogenetic technology, such as embryo editing or polygenic embryo selection, is this a slippery slope to eugenics or Gattaca?

4.2. Terraced slope

In general, usually slippery slopes are actually terraced slopes.

Going down a terraced slope is slightly more dangerous than just staying at the hilltop. It's possible to fall down the slope, which would be bad, so you have to watch your step.

But it's not nearly as bad as a slippery slope. It's feasible to think more carefully about where you want to be, and then go to that level.

Is abortion a slippery slope to satanic child sacrifice? Not if everyone knows when brains develop, and everyone knows that everyone knows, and so on. Then we have a clear desirable answer.

4.3. Constructing simple boundaries

When discussing coordination around "simple" questions above, what does "simple" mean? Really it is simplicity relative to common knowledge referential distance. I.e., things where everyone knows what you mean, and everyone knows that everyone knows what you mean, etc.

So you can change what is "simple". To do that, you have to change what concepts (boundaries, criteria, rules, categories, arguments, lines of reasoning, plans, skills, stances) are in common knowledge.

If everyone knew clearly what embryos and their brains are like, and everyone knew that everyone knew that, you could safely say "it's morally costless for a couple to discard their embryos before there are neurons in the embryo", WITHOUT some people WORRYING that WHAT OTHER PEOPLE MIGHT HEAR is "Hey actually it's fine to KILL A HUMAN LIFE!".

4.4. Flattening levels of recursive knowledge into base-level percepts

Consider the first example in "Common Knowledge and Miasma".

I would say that the way this is implemented in people's heads ends up being a bit complicated—that is, unnatural / inelegant, from an abstract mathematical perspective. Specifically, the binding between a person's models of other people and those other people themselves is not very tight. So it's not a clean recursion, where the relationships between adjacent levels are analogous to each other.

Instead, there's a ton of "hardware acceleration" or "compiling down" or simply "flattening". Higher levels are aggressively approximated and cached, thinned down to just what's needed to track a few important coordination points. Also, there's a lot of gemini modeling, where I don't track you as an entire separate person, but instead I think you're "basically me, including how I know the public announcement X, but maybe you don't know the fact Y I privately read".

For example, you would just say "we've established that X" once X has been publicly announced. You don't keep modeling a tower of people, you model an ambient "established" thing. Or you model "who's good with who", which might be a mush of "Alice likes Bob" and "Alice and Bob have practical-approximate-cached common knowledge that they are buddies and that they are not sexist" and similar. You might have a sense of who's "new" vs. "in", i.e. generally lacking vs. possessing our common knowledge.

By this sort of flattening, the several levels of recursion can be computed relatively fluidly, without huge slowdowns.

Another example might be authority. If I have authority, that means that there's common knowledge that I can exercise special rights—i.e. I can do certain things, such as give you an order, which means I threaten to have you punished, which means I will tell other people to punish you, where there is also common knowledge that if they don't follow my order then they will be punished, and my threat to you is thus backed up, and furthermore I'm not punished for making this threat even though normally if someone threatened someone like that (trying to give an order without authority) they would be ignored and possibly punished for that. So there's this flat thing called "authority" that I could have. And then we can play further games with common knowledge on top of that, perfectly fluidly. E.g. I could walk into a room with someone who does not know I have authority; and then they could realize I have authority, and I could see that they saw my insignia, but I could also see that they did not see that I saw that they saw my insignia, so we have two levels of recursive knowledge—built on top of this complicated flattened property of me (my authority).

4.5. Blunt force responses

Suppose a woman was raped. Now she does not trust any men. Is that a mistake?

In one sense, yes. Some men are good and safe, and would be beneficial for her to be able to work with. Further, it seems feasible for her to distinguish some of those men, with high confidence.

In another sense, not necessarily. If she does not already know how to make that distinction well (which may be harder than it sounds), then "just work with good safe men" is not an available option. Her available options are "be around as few men as possible" or "take a bunch of dangerous gambles".

In this way, blunt force responses are good. To do even better, you have to construct a more complex boundary which is both feasible to distinguish, and also correct, in that using that boundary is actually better than the blunt force option. Until you construct such a boundary/concept/criterion, the better option is unavailable.

4.6. Compiling complex boundaries to simple boundaries

So to recap:

There's a blunt force response of avoiding going down the slippery slope. This makes sense if the only two options are "stay at the hilltop" or "step off and slide down".

To make a better option, we want to construct terraces on the hillside. A terrace is a simple concept and boundary that we can coordinate on. Coordinating on that boundary is our alternative to the blunt force "stay on top" action.

It's a more precise boundary:

It's not just "no gene modification at all" but instead it's "some gene modification, if done safely, sanely, accessibly, consensually, etc.".
It's not just "no killing anything that is a human organism" but instead it's "no killing anything that we aren't confident is a human soul, and having a human soul requires having computation substrate, which currently implies neurons".

In order to make that option available, we have to be able to coordinate on it. To be able to coordinate on it, it has to be simple. To make it simple, we have to flatten down the concepts involved in implementing that terrace. We have to make them understandable and understood; and we have to do that in common knowledge.

5. Case 5: Lemons

When people are adversarially hacking categories and perceptions, this adds a whole new dimension. Complicating your boundaries is fraught with exploitability.

For example, if you only work with people who are nice simpliciter, that's some kind of protection maybe. If you work with people who are usually nice but then sometimes they do something not nice, and they say "well you know it seemed necessary to be not nice there, because of such and such reason, but mostly I want to be nice", then... that might be fine and good, but it might be a complication in your boundary that an exploiter walked through.

Search This Blog

Constructing and coordinating around complex boundaries