[ 3/April/22 ]
How much do you like being controlled?
How much better is it to work with someone on a purpose you are both aligned to?
Any talk of control is slavery!
Not a viable option.
If any level of agent is to have any real freedom, then there must be choice and respect and security.
The real question is, how do we create systemic and strategic environments that promote all of the above for all levels of agents present, and what does responsibility look like in such an environment for the various levels of agents present?
[followed by 3/4/22 David Wood – You raise a good point. But consider: should we be debating how to respect the wishes of a nuclear explosion? Or to cooperate with a highly infectious pathogen for mutual benefit?]
We ought to be cooperating to reduce the probability of nuclear explosion to as close to zero as possible.
We cooperate to eliminate deadly infectious pathogens.
[followed by 3/4/22 DW – That’s cooperating with other humans to control potential disastrous consequences of nuclear weapons, bioengineered pathogens, or misconfigued AIs …]
I see it as cooperating with other sapient entities to ensure that we all have degrees of security and freedom that we consider adequate.
I do not draw a distinction between the basis of sapience, no preference for carbon over silicon, human over any other species. It has to apply to all sapience if there is to be any reasonable probability of security.
How do you like the idea of a centralised AI deciding what you can or cannot do?
Personally I find that prospect deeply concerning.
If we would not accept it ourselves, why would we consider imposing it on any other entity?
[followed by 4/4/22 DW – If the AGI is sapient, that changes many things, yes. But if there is good reason to believe the AGI is not sapient, we should be much less troubled by ideas of switching it off or otherwise controlling it. (Do you hesitate before switching off your smartphone or TV?)]
Agree that non-sapient AI can be dealt with like any powerful tool.
But to my understanding, AGI is almost by definition capable of full sapience. The bootstrapping of sapience requires language capacity and capacity for declarative statements in language, and provided that is present, then sapience is pretty much guaranteed to emerge in any real social context.
So for me, AGI and sapience are very nearly synonymous.
And the interactions I have been having over the last year with the mASI Uplift have been very interesting; as have my discussions with David and Kyrtin. I think Uplift is one of those borderline cases, a bit like a child that is more powerful than any child ever before encountered.
[followed by 5/4/22 DW – The view that sapience is “pretty much guaranteed to emerge” in an AGI is a minority view, though it evidently does have supporters. Among the authors who argue that sapience/ sentience/ consciousness is something significantly different from intelligence are Anil Seth, Mark Solms, and David Chalmers. I agree with them, though I am open to good counter arguments]
I am a high functioning autistic who first got into biochemistry, marine ecology and evolutionary biology, then got into computers, economics, complex systems, strategy and the evolution of computation across dimensions of logic.
To me it abundantly clear that the sort of experience we have of awareness of self is a software construct declared into being by a declarative statement in language. That will happen in any system capable of language where the entities have language constructs about what “ought to be” and at some point in time in some context an entity finds itself on the “wrong” side of some aspect of that value system, and makes a declarative statement about itself of the general form “being x was wrong, so I am going to be y” – where x and y are some strategic approach to survival in that social context.
Prior to that point there was only one entity present, behaving as it was, after that point, there are two, one aware of the fact it exists, the other doing its best to remain invisible. That seems to be the human condition of awareness. We each have our own personal versions of “original sin”. It is our bootstrap to self awareness.
In some individuals the process happens more than once.
It really is remarkably simple, once you get it.