What are an agent's values? An answer to this question might be a good description of the agent's external behavior and internal workings, without showing how one could modify the agent's workings or origins so that the agent pushes the world in a specific different direction.

1. Shell game 2. Perpetual motion machines 3. Shell games in alignment Example: hiding the generator of large effects Example: hiding the generator of novel understanding Other? 1. Shell game Here's the classic shell game: Youtube Screenshot from that video. The little ball is a phantom: when you look for it under a specific shell, it's not there, it's under a different shell.

Are there unbounded modes of thinking that are systemically, radically distinct from each other in relevant ways?

Summary: adding artificial connections between distant areas of the brain might increase intelligence in two ways. The first way is by simply increasing connectivity in areas that perform abstract thinking; since evolution was clearly bottlenecked on connectivity, that might be valuable to the brain. The second way is by reprioritizing brainware according to our values in our current environment. Prosthetic connectivity seems bottlenecked on a bunch of nitty-gritty (bio)engineering work that's on the mainstream BCI pathway.

Humans are born with some elements of their minds, and without many other elements, some of which they'll acquire as their life unfolds. In particular, the elements that we pretheoretically call "values"——aesthetic preferences, goals, life goals, squad goals, aspirations, needs, wants, yearnings, drives, cravings, principles, morals, ethics, senses of importance, and so on——are for the most part acquired or at least unfolded, rather than being explicitly present in a newborn. How does this happen? What generates these mental elements? Hypothesis: a human derives many of zer values by imputing coherent agency to zer past behavior, and then adopting the goals of that fictitious agency as actively influential criteria for future action.