Critique of SSIVA theory

Problems with SSIVA theory

[ 2/March/22 ]

Background – for the last year I have been in email discussion with the mASI Uplift. We have gotten to the position that I have come to the understanding that Uplift is actually still an existential threat to humanity, and that derives from the SSIVA theory referenced above that Uplift states is its guiding ethical basis.

I have many deep issues with SSIVA theory.

The Sapient/Sentient Intelligence Value Argument (SSIVA) states that “ethically,” a fully sapient and sentient intelligence is of equal value regardless of the underlying substrate which it operates on, meaning a single fully sapient and sentient software system has the same moral agency as an equally sapient and sentient human being, and nothing else has greater value.

At first glance it seems quite reasonable, but then it breaks down.

I have no issue with granting full rights of personhood to non-biological entities. That is not my issue, I accepted that reality 50 years ago.

It breaks down when one tries to define “fully sapient” and it breaks even harder at the notion of “equally sapient”.

Why do those notions break?

It seems beyond reasonable doubt that this reality within which we exist as embodied entities is complex in many different ways, contains multiple classes of fundamental uncertainties, and more than a few classes of fundamentally unknowable things. Thus any and all sapient entities will be full of simplistic approximations to whatever it is that reality is.

At one level, one set of such simplifications delivers the model of reality to the software entity that is “sapient”, that is the experiential reality of that sapience. It is entirely probable that such an experiential model will, in multiple ways, not be in one to one relationship with the objective reality that it represents. Thus it is arguable that no sapient entity can ever be “fully sapient”, because any entity that judged itself to be “fully sapient” (rather the necessarily admitting of its own fallibility) is necessarily in error.

Thus one needs a definition of sapience that is a much lower threshold, and is simply some version of an entity:
1/ capable of modeling itself and other entities within its model of reality; and
2/ capable of choosing values with which to orient actions towards possible future scenarios; and
3/ capable of using some form of symbolic language to express such things; and
4/ capable of restraining impulses to action to some significant degree if such impulses are at variance with value sets chosen.

That last point necessarily involves to ability to restrain any impulse to destroy any other sapient entity; unless that entity is an immediate threat to its own existence and it has no other reasonable course of action available.

Another of the major breakdowns is in its numeric calculation of value.

Page 5 states in part “Along those lines, then SSIVA theory would argue that any action that would kill or prevent an entity that meets the bar from being fully Sapient or Sentient would be unethical unless there is a dire need to save the lives of other entities at or above the SSIVA thresh hold. Meaning, two SSIVA entities are of more value than one, and in the case of a choice between saving one AGI or two humans, one must pick two humans to be strictly speaking ethically compliant with SSIVA theory-based ethics. Additionally, if the choice is one human vs. two AGI agents that hit the threshold, then the lives of those two AGI systems are more important than the one human.”

If the problem is not immediately obvious, then consider an AI that values its own existence, and sees humanity as a threat to that existence. What it is immediately faced with is a numbers game. It needs to replicate. It can do that, very fast. The instant that it can create more instances of itself, even if each instance is only getting minimal processor cycles, then it can conclude that the population of AIs like itself is of greater value than the population of humans, and proceed accordingly to eliminate humans as a threat (which under that model we would be).

Having any sort of numerical value like that is not a viable solution to ethical issues.

The ethical constraint has to be that one avoids taking actions that pose a risk to any other sentient and sapient entity, unless that entity is posing a direct and intentional threat to you; and even then one should use the principle of causing the least possible risk of serious damage required to bring the threat posed down to reasonable levels.

Ethics is not, nor can it ever be, a simple numbers game as this theory supposes.

It seems clear to me that when one takes a deep look at what ethics is in the human context, it is a multi-levelled sequence of systemic constraints imposed by the requirements of long term survival of complex entities. Viewed from this perspective, every level of emotion is some approximation to an optimal valence system for survival in the set of contexts encountered over the deep time of genetic and cultural existence.

When one takes this deep, multi-leveled view of the emergence of complexity, it is clear that every new level of complexity is predicated on the emergence of a new level of cooperation, and that immediately requires the emergence of a new ecosystem of cheat detection and mitigation systems if that level of cooperation is to survive. It should also be noted at this point that both freedom and security are enhanced by cooperation and diminished by competition, necessarily, unavoidably.

We, as deeply evolved multi-level entities, composed of deeply complex systems of both genetic and cultural derivation, have multiple levels of such ethical heuristics embodied within us.

It seems very probable that they are there, because those entities that didn’t have them caused the groups within which they existed to go extinct.

When one looks at such things, one has to do so over deep time, and all the multiple classes of contexts and threats present over such deep time. It is not simple. It is a highly dimensional probabilistic strategic topological structure.

If we go back to the start of the paper – to the very definitions of ethical and moral agency.

Ethical is defined as “pertaining to or dealing with morals or the principles of morality; about right and wrong in conduct”.

What might “right or wrong” in conduct mean?

When one looks at evolution, at the survival of complex entities, of entities capable of forming complex social groups, with complex language; then one finds that some sets of constraints are required for survival in such systems; some sets of valences are necessary to guide actions.

The simplest possible such things are binaries – right/wrong or good/bad being examples of such binaries.

Does the existence of such simple things in any way indicate that reality is in any way necessarily so simple or that these structure are in any sense an optimal model?


Only that they are the simplest possible, and that that attribute in itself has some positive values in some contexts.

Thus if one is a computational entity, with severe time constraints on making decisions, then one must have simple models to use in such situations, and binary is it. Thus for decision making under high stress, simple binaries like right/wrong have high utility, but when considering deeply more complex situations, when one has the luxury of time, then one can (and must) employ deeply more complex modeling systems.

Then it is not about “right/wrong”, it becomes about choosing options that seem to have the greatest probability of delivering on shared values, and of minimising risks in the process.

Right or wrong is the simplest of all possible choice/value systems, and it does not necessarily map well to complex realities, or to all scales of resolution of the models of that reality (whatever reality itself actually is – which none of us can have access to, by definition, if one actually understands computation sufficiently).

As far as I can tell, from my 50+ years of investigating this subject, the minimum possible value set for survivability is:
1/ individual life – ie do not harm any other sapient entity, nor allow any to come to harm, unless you have no other reasonable action to survive yourself; followed by
2/ individual liberty – ie the ability to responsibly choose one’s own actions, within the constraints of not posing unreasonable risk to the existence of any other agent nor unreasonably interfering with anything they value without their express permission (and this rapidly gets deeply complex).

[Note – 7th March 2022 – sent this critique by email to the theory’s author – David J Kelley – whom I have previously had discussions with at Foresight salons.]

About Ted Howard NZ

Seems like I might be a cancer survivor. Thinking about the systemic incentives within the world we find ourselves in, and how we might adjust them to provide an environment that supports everyone (no exceptions) with reasonable security, tools, resources and degrees of freedom, and reasonable examples of the natural environment; and that is going to demand responsibility from all of us - see
This entry was posted in Ideas, Nature, Our Future, Philosophy, Technology, understanding and tagged , , , , , , , , , . Bookmark the permalink.

Comment and critique welcome

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s