I gave a talk on 20 Feb 2024 in the PIBBSS speaker series . It's a talk version of " A hermeneutic net for agency ". Here's the talk: https://www.youtube.com/watch?v=3ZaxlegV90w Here's the discussion: https://www.youtube.com/watch?v=vznEzOCAmho Abstract: We are fundamentally confused about minds, and about what about a mind determines what about the world . Our concepts don't automatically support the inferences and design choices we would like to make using those concepts , and there are strong forces that will break weak supports . Drive-by attempts to rework one or a few concepts in isolation don't work . Minds are too big and structurally entangled within themselves to centrally unravel with a reductionist piecemeal method . The relevance of the most relevant mental elements is essentially provisional and requires the full context of a mind to be understood . The only source of useable data about minds and their intentions is our own minds . W

Exploring the ruins of an alien civilization, you find what appears to be a working computer——it's made of plastic and metal, wires connect it to various devices, and you see arrays of capacitors that maintain charged or uncharged states and that sometimes rapidly toggle in response to voltages from connected wires. You can tell that the presumptive RAM is activating in complex but structured patterns, but you don't know their meanings. What strategies can you use to come to understand what the underlying order is, what algorithm the computer is running, that explains the pattern of RAM activations?

[This post is labeled בבל, meaning it's especially experimental. See: בבל disclaimer ] If one were to complain, one would hear, whether from the outside or from the inside, something like this:

[Caveat lector: I know roughly nothing about policy!] Suppose that there were political support to really halt research that might lead to an unstoppable, unsteerable transfer of control over the lightcone from humans to AGIs. What government policy could exert that political value?