Draft braindump Outline:
Goal: Continuously maximize the probability of following the space of optimal axiological trajectories for as many caring beings as possible.
Short term
Identified bottleneck we can solve: Morality is pre-formal still and we can speed up the axiological tech tree / formalization using modern science theoretical trees , internet hive mind and ASI.
Short term How: Reverse engineering the measurable boundaries & constraints of good futures that are computationally irreducible to predict in instantiation but not in specification. ie: Formalize Morality (by formalizing Axiology) to make it computable, measurable & iterable systematically; essentially causing a phase transition in the crystalization of pre-scientific ethics that will in turn cause a phase transition in how all caring patterns coordinate by giving them the theory and technology to measure, predict, and guide the best axiological trajectories for themselves and their surrounding networks recursively and exponentially: ie the Mettasplosion (positive feedback loop of mettaligment from theory to caring beings).
Results: Mettasplosion continues to advance & spread measurably (more people building on the theory, implementing it, spreading it, etc) with measurable positive axiological changes in their trajectories (and those surrounding them), etc.
Deadline / Failure metric: Before exponential tech lead moral error amplification causes x-doom (through technological warfare, rogue ASI, techno dictatorship, etc) the theory should be sufficiently developed (and proved hopefully) to be able to persuade caring systems of sufficient and arbitrary levels of capability / intelligence including any ASI no matter how intelligent, and ofcourse also any humans , goverments, corporations,etc. Furthermore we want it to be spread and being implemented enough (we are the data in the limit) that we can bias the training data positively of all caring beings (including ASIs) ASAP. Failure is x-risk doom, or simply failing to minimize suffering / conflict as much as possible. Failure would also be if we have no part in accelerating the advent of this axiological phase transition and in fact hinder it.
Weak success metric: If we fail, but accelerate the advent of the axiological formalization / axiological trajectory improvements of caring being on net then we will have succeeded.
