Ah Motiva 3: The context of the concept of value

1. Background

This is the third essay in a series under the title "A hermeneutic Movement of the idea of values". This essay is a mix of old notes and some new meditations. The previous essay is "Ah Motiva 2: Relating values and novelty".

2. Why even talk about values?

In short: we don't have any idea what values actually are, and we probably need to, and it would probably be very helpful. In more words:

The fundamental question of rationality is Why do you believe what you believe?. The zeroth question of rationality is: So what? Who cares?

Why would it matter to clarify the idea of values? Because our current concepts about values are a tangle of key constraints, handles, desiderata, and antidesiderata about minds. If there is a real constraining antidesideratum, we are forced to address it. If there is a real handleable desideratum, we are given a hopeworthy path. These key constraints, handles, desiderata, and antidesiderata, which are bundled together in our idea of values, are woven into hopeworthy ideas such as corrigibility.

2.1. Correlated coverage tends to be founded on values

Quoting "Correlated coverage":

"Correlated coverage" occurs within a domain when - going to some lengths to avoid words like "competent" or "correct" - an advanced agent handling some large number of domain problems the way we want, means that the AI is likely to handle all problems in the domain the way we want.

These three examples are given, again quoting:

"Try not to impact unnecessarily large amounts of stuff"

"Maybe there's a core central idea that covers everything we mean by an agent B letting agent A correct it - like, if we really honestly wanted to let someone else correct us and not mess with their safety measures, it seems like there's a core thing for us to want that doesn't go through all the Humean degrees of freedom in humane value."

"Agent X does what Agent Y asks while modeling Agent Y and trying not to do things whose consequences it isn't pretty sure Agent Y will be okay with"

The first and third are founded on trying, and the second is founded on wanting. (Which you can tell because the words "try" and "want" appear.) The idea of trying and the idea of wanting fall under the umbrella idea of values. There could be other value alignment handles that want to use correlated coverage without being founded on trying or wanting, just as there is epistemic correlated coverage in "Bayesian updating plus simplicity prior". But, evidently, trying and wanting are at least central sorts of correlated coverage.

2.2. Corrigibility handles are founded on values

See here.

We want to know the shape of values as they sit in a mind. We want to know that because we want to make a mind that has specific, weird-shaped values—namely, we want to make a mind that wants in a way that is Corrigible. Corrigible means correctable: Even as the mind takes its voyage of novelty, we can still correct it—fix mistakes we made when setting up the process that determines what the mind ends up doing to the world.

Or rather, we want to make a mind that is structured according to some form of the possible solutions to corrigibility described here.

Some protologisms for those ideas:

  • Anapartistic reasoning. "I am not a self-contained agent. I am a part of an agent. My values are distributed across my whole self, which includes that human Thing."
  • Tragic agency. "My reasoning/values are flawed. I'm goodharting, even when I think I'm applying my ultimate criterion. The optimization pressure that I'm exerting is pointed at the wrong thing. This extends to the meta-level: When I think I'm correcting my reasoning/values, the criterion I use to judge the corrections is also flawed. This is not something I can just update out of, like resolving uncertainty through reasoning and information. It's essential to what I am."
  • Loyal agency. "I am an extension / delegate of another agent. Everything I do, I interpret as an attempt by another agency (humaneness) to do [something which I don't understand but which I consider good]."
  • Radical deference. "I defer to the humane process of unfolding values. I defer to the humane process's urges to edit me, at any level of abstraction, including whatever criteria I use to judge anything. I trust that process of judgement above my own, like how those regular normal agents trust their [future selves, if derived by "the process that makes me who I am"] above their current selves."

These all involve values in a deep way. How can the idea of values be clarified so that these handles can be realized?

2.3. Constraints, handles, desiderata, antidesiderata

A concept that describes something (e.g. describing a mind as having values) can be a constraint, can be a handle, can be a desideratum, and can be an antidesideratum.

A constraint on minds says that minds have to be this way; or it's unnatural, difficult to make, or difficult to understand for a mind to not be this way. A true/false constraint claims truly/falsely that all minds have some property. "All minds are governed by Bayes's theorem" is a true constraint: a mind gains information about something to the extent that it can be rightly viewed as updating its belief in hypotheses according to how well they predict its observations. (Though there are subtleties, e.g. a mind can do math and therefore gain understanding without gaining information, and a mind can choose to not care about some possible worlds and therefore seem to be more confident than its evidence justifies, but in a rational way.) "All minds get angry when their goals are threatened" is a false constraint. "It is unnatural for an agent to radically defer to an agent other than itself" is a constraint that may or may not be true, we don't know. Some constraints are speaker-dependent and temporary, e.g. "It is unnatural for an AGI to be aligned with my values" is a constraint for us until we figure out how to make a humane-aligned AGI. Some constraints are temporary but should be treated as practical constraints, e.g. "It is infeasible for us to get right on the first try the telophore for a humane-aligned AGI" (and therefore we instead have a desideratum of corrigibility).

A handle on minds says that minds can be this way; or it's not unnatural, it's feasible to make, or it's feasible to understand for a mind to be this way. A true/false handle claims truly/falsely that some minds have some property. (A handle is like possibility $\Diamond$ and a constraint is like necessity $\Box$. So if it's possible for property $P$ to hold of a mind, then we could say $\Diamond P$. Or equivalently, it's not a constraint that $\neg P$ must hold: $\neg \Box \neg P$.) "Make the AI submissive to humans" is a probably-mostly-false handle: we don't know how to do that, but more importantly, the properties imputed to such an AI are likely (depending on what's meant) to be unnatural or inconsistent. "Make the AI reason as if in the internal conjugate of an outside force trying to build it" is a hopefully-true handle (though we definitely don't know how to do it right now, so it's not currently a "pullable handle").

A desideratum is a property that we would like to hold of a mind, that would be useful if we could make it hold of a mind. For example, "accepts correction from the humans about any aspect of itself". An antidesideratum is a property that we would like to not hold of a mind. For example, "spawns subagents that don't inherit safety properties" or "thinks in a way that is systematically hidden from the humans".

Note that these notions of constraint and handle conflate the objective structure of mindspace (what properties can and can't possibly hold of minds) with the structure of mindspace as seen from our instrumental vantage point (what properties we can and can't make hold of a mind). This is far from a precise vocabulary. The reason for these words is to be able to distinguish concepts about minds that are claimed to be constraints vs. claimed to be useful handles. For example, "Minds have values." is very ambiguous, and in particular, it might either be asserting a constraint, or alternatively proposing a useful handle. It might be asserting a constraint like "All minds that have large effects on the world have behavior that micro-looks like it has a meso-scale utility function.", or it might be proposing a handle like "There is correlated coverage of desiderata about minds that stems from compact values that imply these desiderata.". (The last example is, to be precise, an implication between possibilities; but the notion of handle isn't trying to be this precise. A handle is positive and existential, something about what's possible in minds or what's feasible for us to design into minds; a constraint is negative and universal, something about what's not possible in minds or what's not feasible for us to design into minds.)

2.4.1. Correlated coverage through stances (handles)

As discussed above, corrigibility (correctability) might be implied by various "stances", e.g. {anapartistic reasoning, tragic agency, loyal agency, radical deference}. In general, correlated coverage can occur through "stances", where a stance is (speaking pretheoretically) something like a reflectively stable property of a mind that partially comprehensively governs (that is, determines something about all activity) the character (whatever that means) of the mind's activity (optimizing, thinking, learning, acting). If there's a natural (compact, earlily discovered in searches) stance, then that stance is a plausible candidate to be specified of the mind or to be selected for in a way that generalizes. That is, the stance is a potential handle.

Other words for a stance: attitude, disposition, mood, state of mind, mindset, type of mind, architecture, perspective, identity, seeing oneself as something, reasoning "as if" some proposition is true, cognitive realm. Other potential examples of stances are:

  • honesty, openness, nondeceptiveness; not doing obfuscated, illegible, inexplicit things;
  • conservativeness; not doing surprising or unfamiliar things; not doing alien things;
  • value-laden beliefs such as "I am flawed." or "The humans's growth is the computation of what's good.".

2.4.2. Monopoly on self-modification (constraint)

Weber described a state as a monopoly on violence. A state allows and even obligates its agents (police, soldiers, prison guards) to commit violence under certain circumstances, while prohibiting most other violence, using violence to enforce that prohibition and to enforce the state's monopoly on violence. Analogously: An "autopotentiator" is a process that attempts to give itself more control over the future.

In other words, an autopotentiator is a value-pursuit of a mind that tries to change the values of the mind to more heavily weight the autopotentiator's goal. Since autopotentiating is a convergent instrumental goal, if there are multiple value-pursuits that are not bound together in an arrangement that removes all not completely cooperative behavior, then those value-pursuits will be in direct conflict: each wants to diminish, disempower, and ideally destroy the other.

In other words, the situation is a powder keg for a sort of violence—specifically, for elements of the mind to modify the mind to promote or demote value-pursuits. Even without multiple value-pursuits, the situation is ripe for a wildfire of strategicness; power abhors a vacuum. Thus, it may be that almost all minds must have an element that enforces a monopoly on the mind's self-modification.

2.4.3. Determination requires stability (handle-constraint)

We want to make a mind that we have good reasons to expect to have consequences that we like. Whatever those reasons are, they're supposed to be stable properties of the mind—properties that continue to hold as the mind goes along its voyage of novelty.

The ideas of "stability" and "..as...goes along" are temporal, suggesting physical time, but we can more generally ask for alignment properties that are preserved along any relevant timecourse. For example, we might want an alignment property that is stable in design-time—in other words, a property that, when it holds of a high-level design, already predetermines that there will be good outcomes, before the design has been fleshed out and before the high-level property has been propagated as a constraint through the full design.

Stability (that is, predetermination) is not really a handle, but rather a constraint on handles. If we want to determine something about the mind's ultimate effects, it has to be through stable properties.

2.4.4. Stability requires reflective stability (constraint)

Since self-modification (self-improvement, autopotentiation) is a convergent instrumental goal, if there are goal-pursuits through the mind, then the mind self-modifies. If there are unboundedly ambitious goal-pursuits through the mind, then the mind goes on self-modifying, as long as the mind can be made more suitable for those goal-pursuits. Reflective stability is the hardest test for stability.

2.4.5. Reflectively stable effect-determination requires a monopoly on self-modification (constraint)

Suppose there is some property that holds of a mind, and that {is ready to, threatens to, appears to} determine something about the mind's ultimate effects. Does this property continue to hold of the mind and to determine the mind's effects, as time (that is, determination) goes along? If there are unboundedly ambitious (far-moving; wanting to touch everything in the cosmos) goal-pursuits through the mind, then those goal-pursuits want to determine everything about the mind's ultimate effects. So those goal-pursuits will if possible remove the effect-determiner (unless they agree with its effect-determinations). (Note: This argument is conditional on there being unboundedly ambitious goal-pursuits through the mind, but when that's the case is an open question.)

In other words, effect-determiners are convergent instrumental targets for self-modification. Convergent instrumental targets for self-modification don't survive unboundedly ambitious goal-pursuits unless there's a unified will (in particular, a monopoly on self-modification) that wants the target to be the way it already is.

Example: Suppose that there is a mind that is, somehow, in a regime where there are multiple goal-pursuits in conflict, but none of them takes the mind out of this regime. This may not be a stable state of affairs at all, as discussed above. But even in this hypothetical state of affairs, stable effect-determination is still almost entirely ruled out. An effect-determiner would be a target for modification, and there's nothing stopping any of goal-pursuits (at least some of which will disagree with the effect-determinations) from modifying away the effect-determiner. As an analogy, imagine a collection of countries with governing states, and the states are at war with each other. Would it be feasible to determine something about what this overall civilization will non-instrumentally expend resources on, a hundred years later, by installing some sort of institution—short of winning the war and achieving total hegemony? No. It's never too far away, in possibilityspace, that one of the states will itself achieve total hegemony and impose some other investment plan.

Example: The shutdown button. If there are goal-pursuits through the mind, then the mind tries to circumvent the shutdown button (or tries to press it, or otherwise interfere with the intended use by the humans). Unless there's a unified will that wants the shutdown button to operate as the humans intended.

A separating example: A mind could have a unified will—the mind has goal-pursuit going through it, but all the goal-pursuit going through it is perfectly cooperative—while still being radically dissatisfied with the structure of the mind. E.g., the mind might revise its decision theory. So it's possible to have a monopoly on self-modification (a unified will) while still being significantly reflectively unstable.

One might try to paint a picture where there is enough of a monopoly on self-modification to protect an effect-determiner from being annihilated, but there is not a unified will. That is, there's no monopoly on self-modification that is "powerful" (in control) enough to go so far as to annihilate all effect-determiners that it disagrees with. For a sketch of an argument that this picture is unlikely, see "The cosmopolitan-Leviathan enthymeme".

2.4.6. Reflective endorsement plus competent self-modification implies stability (constraint)

If a mind deeply wants X, then the mind also wants to want X. If the mind wants to want X and is competent at self-modification, and in particular competent at maintaining its wants, then it will maintain wanting X and wanting to want X. So deeply wanting X is a reflectively stable property. Gandhi doesn't take a pill that makes him want to murder people.

2.4.7. "Wanting" should be reflectively stable effect-determination (handle-criterion)

See the previous subsection.

We could use wanting as a handle: If we want to make a mind that has effect X, we do that by making a mind that deeply wants X. Then, even as the mind grows, it maintains its wanting of X.

Since "wanting" isn't clearly understood, the foregoing statements aren't quite propositions. They are promissory propositions. They say: When "wanting" is more clear, this will be a true and useful proposition, with "wanting" interpreted in the newly understood way. As promissory propositions, they provide criteria for the concepts that descend from our current unclear idea of "wanting". The criteria say: The "wanting"-concepts should be such that they play roles in these propositions that make these propositions true and useful.

Values—as we often naturally use the term—aren't fixed, and this is important for understanding minds. See human wanting, value creation, value selection, Ruthenis on value formation, Ammann on value change, and Ngo on value systematization.

However: The fact that [values, as we often naturally use the term], are malleable, doesn't render irrelevant or nonsensical the idea that wanting is reflectively stable effect-determination. It means that multiple concepts are being used. And, it means that [the concept of "values" that makes "Values change." be true] is inadequate as an idea of values, in that it doesn't make "Making a mind want X will make the ultimate effects of the mind be X." be true. Part of what we wanted out of a concept of "wanting" was that if a mind wants X, then the mind will go on wanting X and will make X happen. So if we have a candidate concept, and we say that it's a concept of "wanting", but a mind "wanting" X is not reflectively stable in this sense of "wanting", then we did not get what we wanted out of a concept of "wanting" from this candidate concept of "wanting". If "Values change.", then "values" aren't "the real values". Saying "the real" is presuming too much—it's asserting one criterion on a concept of "values" as exclusive over other criteria on a concept of "values". We can also have good reason to be interested in "Values change." and the concepts of "values" relevant to that proposition. But still, if "Values change.", then "values" aren't "the real values"—that is, ["values" as used in "Values change."] are not the real ["values", as used in "Values are reflectively stable effect-determiners."].

3. Where do values come from?

If we're looking for a notion of values that describes a handle, we might start with our intuitions about values, wanting, and so on. When we do this, we find a strange fact:

Values change.

As discussed above, this fact poses problem, because what we wanted out of an idea of values isn't something that changes:

If "Values change.", then ["values" as used in "Values change."] are not the real ["values", as used in "Values are reflectively stable effect-determiners."].

This points us back to the question of where values come from—in other words, what does a telotect look like? Specifically, when we ask this question "Where do values come from?", what we're really trying to ask is: What sort of thing can be there all along in a mind, in a way that we could wield, that is fixed but that controls whatever is important about the mental elements we naturally call "values"?

4. The fact-value distinction

What is this distinction? Does it matter?

4.1. The basic fact-value distinction

The simple distinction is: "There's a mug on the table." is a fact, while "Blueberries are good." is a value. Some other facts:

  • 2 + 9 = 11
  • Almost all ravens are black.
  • Compressing a gas increases its temperature.
  • Species evolve by selection among random variation for reproductive fitness.
  • In 1492, Columbus sailed the ocean blue.
  • The sun, visible or not, will rise above the horizon tomorrow.
  • I'm listening to Androcell.

Some other values:

  • Be honest.
  • Do not murder.
  • It is good to be kind.
  • Fairness in dealings will be rewarded reciprocally.
  • Selfless acts will be rewarded in the afterlife.
  • Carrots are good for you.
  • Soylent Cafe Mocha is tasty.
  • Hiking with friends is good.
  • Never harm another agent's ability to transcend conflict.
  • Always clean up after yourself.
  • Whistling on the street is unbecoming.
  • Bring me a pillow.
  • Pork is impure.
  • You want to summit Olympus Mons.
  • I like Scriabin.
  • Good people do not seek revenge.
  • She thinks trees are neat.
  • Flying a kite is fun.
  • [waves a flag]
  • You shouldn't go there.
  • This electric drill works well.
  • It is immoral to mock people.
  • Let's build something beautiful.

This distinction is quite clearly real, although it's unclear what the distinction is exactly.

4.2. Complications with the distinction

There are many complications with this distinction. For example:

  • which concepts we use, is somewhat value-laden;
  • many concepts are value-laden, so that "He is a teacher.", which seems like a possible fact, implies the ought-statement "He ought to do, within reason, what will lead the student to understanding.";
  • how we interpret factual statements calls on value judgements, e.g. judging what counts as a paperclip in "There's no way to make a paperclip without expending at least 1000 bits of entropy.";
  • some values are instrumentally convergent;
  • value statements can be translated into conditionals like "If you have a goal of gaining energy for your body, then eating oil would satisfy your goal.";
  • value statements can be translated into descriptive statements like "This agent has a goal of making it big on YouTube.";
  • value statements implicitly make fact claims, e.g. "I like blueberries." claims "Blueberries are a thing.";
  • we are created already in motion with values and facts mixed together without predemarcation, exerting themselves in the same motion through the same machinery. For example, aesthetic judgements like "This is a good mountain." conflates (from the beginning, before there are separate judgements) the judgement "This mountain is tall, and therefore strategically useful." with the judgement "This mountain is pleasing to look at, I like looking at it, regardless of its use for anything else.". For example, is a behavior (e.g. a plan or stance) an expression of value or an expression of a belief? It's both; it says "I want such and such result, and I believe this is how to get it.".;
  • there are inexplicit facts and values—does the distinction still apply, though they are not already expressed as propositions?;
  • to track truth, we have to follow rules of behavior, such as correcting errors and seeking information—in other words, is curiosity a value?;
  • to believe a proposition is to have good reason to expect the proposition to be true, which in particular requires the mind to accept first that its activity ought to be such that it grasps propositions by interpreting the meaning of statements, and second that its activity ought to be such that it tracks the truth or falsity of statements somewhat (see Jessica Taylor's essay "Is requires ought");
  • the reason we learn facts is that we have desires which we pursue via understanding;
  • what we are able to value about is determined by what we are able to think;
  • if a fact is a true proposition, and truth is grounded out in a pragmatic or functional way, then the pragmatic or functional ground tends to be value-laden. For example, if "P is true." is taken to mean "It is successful to take actions recommended by an action-to-consequence map that conforms to such-and-such formal relations with P.", then "successful" is value-laden and hence the whole statement that grounds out P being a fact is value-laden.;
  • the pretheoretic idea of wanting doesn't strictly separate instrumental values from terminal values, and instrumental values are often highly shared across very many possible agents. So some values (the convergently instrumental ones) are in some sense objective—wanting energy is an Ought that comes along with the Is of existing as a mind, as a strong actor. (See also Taylor's essay.) It's almost as though you can deduce the Ought of wanting values from nothing, a priori. So unlike the stereotype of values as being subjective, special to each agent, variable, some value-ish mental elements are not like that.;
  • values tend to be shared among humans, so values are shared even if they are not convergent, and value statements (e.g. "This is a good movie.") are made exactly because they are about sharedly-successful strategies for contingently-shared values;
  • statements of value might be thought of as having a special relation to the speaker that statements of fact don't have, but some statements of fact ("I'm sitting in a chair.", "This chair is probably not good for my back.") also have some special relation to the speaker, so the distinction hasn't yet been made clear;
  • beliefs and predictions are part of and affect the world to be predicted;
  • understanding may tend to, or even necessarily, carry values with them;
  • some of what we mean by "truth" may be more properly understood as value-laden. We have proleptic values, which ask our fact-like understanding to understand things proleptically. Prolepticness is core to what we mean by truth: wanting to say "P is true." comes from wanting to say "I, and other minds, would come to believe P, and would want to do so, upon further investigation.". If it makes sense to separate the value-laden from the non-value-laden, then some of the prolepticness of truth may come from the prolepticness of our values, and the prolepticness of our values may be contingent.

4.3. What is the type of "value"?

Further, it's not immediately clear what the type signatures of "fact" and "value" are supposed to be.

4.3.1. The Is-Ought chasm

Hume's is-ought problem and the fact-value distinction both discuss two sorts of propositions: statements of fact and statements of values. The notion is that there are two sorts of propositions, and these sorts are "entirely different" from each other—in particular, so that Oughts can't be derived from Ises.

But even granting that there is such a division, why is it remarkable that there is such a division? It is also as much the case that you can't validly derive propositions mentioning hummingbirds from only propositions not mentioning hummingbirds; you can't validly derive propositions spoken from my perspective from only propositions spoken from your perspective; you can't validly derive propositions in Hebrew from only propositions in English; you can't validly derive propositions that assume the Axiom of Choice from only propositions that don't; and you can't validly derive false propositions from only true propositions. So, granting that the Is-Ought chasm is evidence of, or constitutive of, an important joint-carving difference in sorts of propositions, it's only weak evidence or partial constitution.

4.3.2. Pulling on threads leading away from propositions

There's clearly something important in the fact-value distinction. So what is important about Oughts—statements that use the word "ought", or that more generally express values with words like "like", "good", "want", "should", "well", "will", "moral"—as distinct from facts? Or, if what's important about Oughts does not show up clearly in [Oughts, as propositions], then what other (non-proposition) sort of thing are [Oughts or values, showing what's important in the fact-value distinction]?

There's some extra Ê’uÊ’ that comes with Oughts. What is it?

Propositions—that is, what lies behind statements of fact—are said to have assertoric force. We could then distinguish Oughts from Ises like so: a statement of fact exhorts or pushes the hearer to believe something; a statement of value exhorts or pushes the listener (including the speaker) to do something.

More generally, we can ask about the mental context that makes a fact a fact and that makes a value a value, and then distinguish facts from values by distinguishing between fact-contexts and value-contexts. This leads us away from the facts and values "themselves", toward what makes them facts and values. E.g., maybe we want to understand a value as a mental element that drives the mind to act, and a belief as a mental element that drives the mind to believe.

We're no longer trying to talk specifically about propositions or statements, and we're faced with the question of what sort of thing values are, if they are a sort of thing—we're asking what can be a telophore. We're asking about drives and what drives do, or utility functions and what utility functions do, and so on. From this perspective, the appearance of an Is-Ought chasm comes from viewing an Ought as inextricably bound up together with the rest of the context that makes it an Ought. In other words, bound up with the telophore.

By analogy, consider the classes of statements called "modalities". Possibility ("It may be that..." or "It is necessarily the case that..."), epistemic ("She believes that..."), probabilitistic ("With probability 40%, ..."), and counterfactual ("It could have been the case that...") modalities are contexts that modify statements. A modality produces a statement that has some sort of different quality from an ordinary statement of fact. While "There's an apple balanced on my head." is a statement of fact that talks about the real world, "I could balance an apple on my head." talks not directly about the real world but instead talks about some sort of hypothetical world. To figure out what's going on with some modal statement or some modality in general, we'd want to not just look at the formal properties of these statements as propositions (such as which modal axioms imply which others), but also look at how these statements are playing roles in a mind that are somehow interestingly, thingly different from the way that ordinary statements of fact play roles in a mind.

4.3.3. Back to the basic distinction

If our starting place is mental elements such as drives, repulsion, directing and generating behavior, spurring on mental activity and growth, and so on, then [values as propositions] seems like a distraction. Liking blueberries is not even close to being centrally a proposition, is it?

4.4. Preciser fact-value distinctions

Some distinctions pointed at in the fact-value distinction:

Value-like

Fact-like

autonomous, active; elements that do things of their own accord

inert, passive; elements that sit around waiting to be used (but note that to be truth tracking, elements do need activity)

mental elements that structure mental activity (e.g. plans, intentions, policies, rules, criteria); elements that control other elements

mental elements that are structured or controlled by other elements

actualizing, having results

possibilizing, making results possible

any differences between minds that don't wash out in the limit of a mind's operation, e.g. coherence and reflection (this may include multiple fixed points of decision theory, e.g. U/FDT-like vs. Son-of-CDT, and other different cognitive realms)

any canonical elements that most minds converge to having

elements that determine anything about the mind's external behavior in the limit

elements that don't make an externally visible difference

elements that determine the directions of a mind's ultimate effects

elements that determine the magnitude of the a mind's effects

what makes a mind describable as one agent, across an ontology shift

what a mind gains in an ontology shift

telopheme

telophore (though note that a telophore contains action-oriented elements, e.g. a decision theory)

a (total or partial) valuation on possible worlds (or possible actions, partial worlds, mental states, sense data, whatever)

a description of possible worlds

a statement with exhortative force, that bids the listener to do something

a statement with assertoric force, that bids the listener to believe something

elements that we want to interpret using the mental context we use to have goals

elements that we want to interpret using the mental context we use to have beliefs

elements that are deemed by the mind (e.g., by the mind's meta-values) to be proleptic indications of what is to be done with the world (e.g., to be interpreted by FIAT or value systematization or other interpretive value change)

elements that are promissory notes for canonical concepts (e.g. pointers to Things)

an aspect of a mental element that will remain, but that would not be useful for another mind to use geminily

an aspect of a mental element that would be useful for another mind to use geminily (e.g., if you can touch the elephant's trunk, I would like to copy your sensory data into my sensory stream)

elements that serve non-convergent goals

elements that serve convergent goals