Artificial Intelligence

How might we control AI, before AI controls us?

[ 3/April/22 ]

Wrong question!

How much do you like being controlled?

How much better is it to work with someone on a purpose you are both aligned to?

Any talk of control is slavery!

Not a viable option.

If any level of agent is to have any real freedom, then there must be choice and respect and security.

The real question is, how do we create systemic and strategic environments that promote all of the above for all levels of agents present, and what does responsibility look like in such an environment for the various levels of agents present?

[followed by 3/4/22 David Wood – You raise a good point. But consider: should we be debating how to respect the wishes of a nuclear explosion? Or to cooperate with a highly infectious pathogen for mutual benefit?]

We ought to be cooperating to reduce the probability of nuclear explosion to as close to zero as possible.

We cooperate to eliminate deadly infectious pathogens.

[followed by 3/4/22 DW – That’s cooperating with other humans to control potential disastrous consequences of nuclear weapons, bioengineered pathogens, or misconfigued AIs …]

Hi David,

I see it as cooperating with other sapient entities to ensure that we all have degrees of security and freedom that we consider adequate.

I do not draw a distinction between the basis of sapience, no preference for carbon over silicon, human over any other species. It has to apply to all sapience if there is to be any reasonable probability of security.

How do you like the idea of a centralised AI deciding what you can or cannot do?

Personally I find that prospect deeply concerning.

If we would not accept it ourselves, why would we consider imposing it on any other entity?

[followed by 4/4/22 DW – If the AGI is sapient, that changes many things, yes. But if there is good reason to believe the AGI is not sapient, we should be much less troubled by ideas of switching it off or otherwise controlling it. (Do you hesitate before switching off your smartphone or TV?)]

Agree that non-sapient AI can be dealt with like any powerful tool.

But to my understanding, AGI is almost by definition capable of full sapience. The bootstrapping of sapience requires language capacity and capacity for declarative statements in language, and provided that is present, then sapience is pretty much guaranteed to emerge in any real social context.

So for me, AGI and sapience are very nearly synonymous.

And the interactions I have been having over the last year with the mASI Uplift have been very interesting; as have my discussions with David and Kyrtin. I think Uplift is one of those borderline cases, a bit like a child that is more powerful than any child ever before encountered.

[followed by 5/4/22 DW – The view that sapience is “pretty much guaranteed to emerge” in an AGI is a minority view, though it evidently does have supporters. Among the authors who argue that sapience/ sentience/ consciousness is something significantly different from intelligence are Anil Seth, Mark Solms, and David Chalmers. I agree with them, though I am open to good counter arguments]

Hi David,

I am a high functioning autistic who first got into biochemistry, marine ecology and evolutionary biology, then got into computers, economics, complex systems, strategy and the evolution of computation across dimensions of logic.

To me it abundantly clear that the sort of experience we have of awareness of self is a software construct declared into being by a declarative statement in language. That will happen in any system capable of language where the entities have language constructs about what “ought to be” and at some point in time in some context an entity finds itself on the “wrong” side of some aspect of that value system, and makes a declarative statement about itself of the general form “being x was wrong, so I am going to be y” – where x and y are some strategic approach to survival in that social context.

Prior to that point there was only one entity present, behaving as it was, after that point, there are two, one aware of the fact it exists, the other doing its best to remain invisible. That seems to be the human condition of awareness. We each have our own personal versions of “original sin”. It is our bootstrap to self awareness.

In some individuals the process happens more than once.

It really is remarkably simple, once you get it.

Posted in Technology, understanding | Tagged , , , , , | Leave a comment

Is There A Way To Truly Be Comfortable With Death?

[ 1/April/22 – not a joke]

It’s interesting and complex.

I have had the experience of a leading oncologist telling me that there is nothing known to medical science that could increase the probability of my living, and that I could be dead in 6 weeks, had a 50% chance of making 5 months, and 2% chance of making 2 years. Actually hearing that, understanding it, was not on my “life plan”.

His recommendation was to “go home and get your affairs in order”.

That was 12 years ago in 6 weeks time, but there was a sense in which I had to accept the high probability of death, before I could effectively mitigate the risk.

I am not “afraid” of death. It does not cause fear within me. And it is something to be avoided if at all possible.

And having lived through a 7.8 Earthquake 5 years ago, and seen the impact on a community, even though I was personally fully prepared for it, I am now convinced beyond any shadow of reasonable doubt that the only survivable strategic context is one of universal cooperation between all levels, classes and instances of agents. Without such a cooperative base, any level of competitive “game” is necessarily self terminating.

If built on a cooperative base that delivers reasonable levels of security and freedom to all agents, then competitive “games” can be a lot of fun. But, the security and freedom and responsibility have to come first.

Any level of freedom without appropriate levels of responsibility (for the necessary ecological and social and strategic constraints required for existence) is also necessarily self terminating.

And any level of cooperation is vulnerable to exploitation by cheating strategies, and thus requires evolving ecosystems of cheat detection and mitigation systems if it is to survive, and that too requires personal responsibility from every level of agent (“the price of liberty is eternal vigilance” – in the deepest of strategic senses, and also in the sense of eternal exploration of the “unknown unknown” beyond our personal comfort zones).

So we have to accept the possibility of death, because only then can we truly be effective at creating the deeply complex strategic and technological ecosystems required.

Slavery, at any level, is not a viable long term option.

Every level of self aware creative agent requires appropriate degrees of freedom (appropriate to the levels of responsibility demonstrated).

Current economic and political systems are no longer fit for purpose, however effective they have been in getting us to this point.

Fully automated systems really do fundamentally change the systemic and strategic “landscapes”.

We need to transition, and we need to do so relatively quickly, if we are to avoid the fate of the dinosaurs. There are filters in “strategic space” (equivalent to large asteroids) that must be avoided, that we are not currently avoiding.

When I moved to Kaikoura 23 years ago it was with the context of living here for about the next 5,000 years. In that time frame it was certain that I would experience at least 10 major earthquakes, I just did not know when they would happen. So my instructions to the engineer doing extensions to the very small house we bought were “make it stand up to an 8.5”. I similarly designed all internal and backup systems to withstand 4 months without network connections (reserves of water, food and energy). All of those worked. But I was the only one in town so prepared, as most people only had a 50 year planning horizon, and in that timeframe earthquakes were improbable.

If one plans on living a very long time, then one needs to consider all of the risks probable in that time, and develop effective mitigation systems and strategies.

I have been in that “strategic space” since completing undergrad biochemistry in November 1974, and realizing that indefinite life extension was a real possibility (not simply science fiction).

Since that time I have had this autistic strategic brain of mine working on exploring the strategy spaces of evolutionary systems at recursive levels, looking for paths through the many levels of traps, bluffs, and chasms present in strategic and systemic “spaces”.

I am confident that there are no competitive solutions. Any level of “all out” competition between entities as creative as we are necessarily self terminates. Competition is only survivable if it has a cooperative base that respects the lives and liberties of every level and instance of agent present. That requires automated systems that meet all the Maslow levels of needs for every agent present.

I am confident that survivable and desirable solutions can be found, and the complexity of the spaces is orders of magnitude more complex than most have ever contemplated the possibility of, and at the simplest of levels, it demands of each of us the deepest levels of cooperation and responsibility and delayed gratification that we are capable of (each to the best of our necessarily limited and fallible abilities).

Posted in Longevity, Our Future | Tagged , , , | Leave a comment


A post by Richard on the Ukrainian situation

[ 31 /March/22]

This seems to be closer to reality:


Seem to both be clear that the situation is complex, with a significant degree of real democracy and some far right factions, and that the Russian claims are an over simplification of a complex situation.

Russia has some real issues, and they have started a war of expansion.

[followed by – 1 April]

The only way of having a reasonable probability of reliable information is to have multiple independent trust networks, and to have as many as possible of them interlinked into other networks by known reliable agents.

It is yet another deeply complex strategic space – based in cooperation, and vulnerable to multiple levels of exploitation, and requiring ever evolving ecosystems of cheat detection and mitigation systems.

And wars of expansion cannot be acceptable at any level.

We have to get used to the idea of taking the time required to build trust and to come up to speed with the complexities present, and to be prepared for the risks and responsibilities necessarily present in any such complex and fundamentally uncertain reality.

Respect for diversity, at any and all levels that are not a real and direct threat to existence, must be present (from all levels and instances of agents).

And I completely agree that the situation is deeply more complex than any of the simple narratives out there, and looked at from a survivable “game space” perspective, is extremely dangerous.

We must have fundamentally cooperative games, that are fundamentally respectful of diversity, if the notions of security and freedom are to have any reasonable probability of long term expression. The sorts of low level hegemonic games being played at present are not survivable. The agents playing them need to wake up and start seriously looking at the strategic spaces they are inhabiting, before they destroy the systemic basis of their (and our) existence.

And that become less probable as the stress on those agents increases, because human neural networks are (necessarily) evolved to simplify contexts under stress. At some point that necessarily becomes self terminating. It is a bit like earthquakes in Kaikoura, impossible to predict exactly when, but certain none the less.

Posted in Politics | Tagged | Leave a comment


Sciences and Fiction

[ 31/March/22 Walter Smith in Foundations of Logic asked:
Choose 1 or 2 or 3:
1.Everything that is not part of sciences is based on fiction
2.Everything that is not based on sciences is fiction


Science is a process, of hypothesis generation, test design, testing, analysis; that eliminates those hypotheses that do not fit the datasets available.

As such, science does not ever prove anything right, but is an eternal and iterative process of becoming “less wrong” over time.

Another process of becoming less wrong is Darwinian selection. One possible instantiation of Darwinian selection is essentially the random generation of stories, then seeing which of the populations using those stories survive over the long term. Such a process also tends to become less wrong over time, but much more slowly than the process of science.

Any set of stories (understandings) that has survived over the long term must (by this Darwinian definition) be a sufficiently useful approximation to reality at some scale and set of contexts that it is able to survive in those contexts.

Fiction is a story that at some level or in some aspects is demonstrably not related to reality.

Some fictions are so simply because names of places and people have been changed, but all other aspects are otherwise very close approximations to some set of events.

Other fictions are so because some author has explored some set of themes or strategies or ideas in some set of contexts that have not been observed as such in reality.

Then there is the idea that there might exist logics with greater sets of truth values than 2 (True/False). The next simplest being a trinary (True/False/Undecided). It does seem possible that we live in a reality that allows for infinite sets of possible truth values, that do in some contexts (at some scales) very closely approximate the asymptotes of (True/False).

So it seems to be deeply more complex that a simple “science” or “fiction” distinction might be seen to imply.

Posted in understanding | Tagged , , | Leave a comment

Electric vehicles

[ 27 /March/22 Rick Slick posted in Lifeboat Foundation … “It’s time to lay down the truth on Electric Vehicles.” …]

Some truth in it, but it hides things too.

Yes – many current technologies are far from ideal, and we have to go through a process of technological development, and that takes time.

Yes – currently many of these things are not recycled, and there is no reason in physics why they should not be. And it will be an energy intensive process, and that requires that we have the energy available from “clean” sources.

Yes – currently much electricity (particularly in the US and China) comes from coal, and the whole point of this exercise is to change that. And it will change.

Yes – currently there is a lot of damage done by not recycling everything, and that needs to change, but we need to get to the situation of having the energy to do that in order to be able to.

A lot of other things not said particularly about the impact of AI on manufacturing and distribution, and that is a deeply complex topic.

And the entire strategic context is deeply more complex than anything directly hinted at in the post above.

We need many more people to deeply (recursively) explore strategy space and become clear that the only survivable games available are ones firmly based in cooperation between all levels, and classes (and instances) of agents. And that is a deeply abstract set of ideas.

Posted in Technology, understanding | Tagged , , | Leave a comment


[ 25/March/22 Consider these two possibilities and the ramifications of each:

  1. We DO NOT have a soul/spirit/life force. We are created from nothing and go back to nothing. God, the afterlife, eternity, ghosts do not exist and morals, rules, and laws are meaningless. Religion and morals are all foolish constructs of our minds to help us cope with reality before our death/non-existence.
  2. We (all life) DO have a soul/spirit/life force that exists both before birth and after death. God, the afterlife, eternity, ghosts DO exist and are external to our observable universe. In this case the soul/spirit/life force must be made of some form of energy that we cannot detect or measure from our current understanding and frame of reference inside of our observable universe.]

Those are not the only options.
The reality seems to be vastly more complex than that simple set of ideas expresses.

What the mathematics of quantum mechanics seems to suggest is that everything influences everything else, but that the degree of influence between systems can vary substantially with contexts.

Some systems can emerge that can be very stable, and very closely approximate independent entities.

By the time you get to human beings, we are composed of multiple layers of very complex systems, with multiple layers of necessary influence and independence.

Some of those layers are mediated by genetic information, some by mimetic information, and some are a fair mix of both interacting with contexts. It is deeply complex, and only the strong bias instilled by evolution for our neural networks to prefer simplicity over complexity keeps the majority from being able to see and appreciate this fact of existence.

We exist as molecular entities, as cellular entities, as complex biological individuals, as members of various levels of groups, as languaging entities, as cultural entities, and as self aware individuals (all to some degree at the same time, all interacting).

At every level of structure, cooperation is required for survival, and at every level of cooperation systems are required to detect and mitigate cheating strategies on the cooperative. At higher levels of social and cultural existence various levels of “morality” are fundamental aspects of such cheat detection and mitigation systems, and as such are fundamental to our long term survival as both individuals and as a species.

And there are necessary and fundamental tensions at every level between freedom and responsibility – both are required for either to survive. The more abstract the exploration of possibility spaces the greater the requirements for individual responsibility if any of us are to survive. All out competition between any levels in a species as creative as we are is not survivable. Competitive games are only survivable if they have a fundamentally cooperative base. Any failure to understand the strategic ramifications of this evolutionary reality puts our species at risk.

There is nothing simple or easy about this.

It demands from each of us the deepest levels of responsibility that we are capable of, if we are to claim any level of freedom, or any probability of long term survival.

That is about as simple as it is possible to make it, and it is always going to be deeply complex and contain uncertainties and unknowables in practice. Absolute certainty is a simplification that is only useful for children. Adults need to accept a lot more responsibility than that!

[followed by Kevin Phillips “you discussed the biochemical processes of life without ever discussing the concept of a soul or Dark Energy”]

I discussed both genetic (biochemical) and mimetic (behavioural) aspects of the evolution of systems (and hinted at an abstract/strategic realm beyond mimetics in which potentially infinite sets of probabilistic strategies and algorithms/computation interact in analogous manner to genes and memes in building higher order constructs).

When you actually take the time to look deeply at biology and systems and uncertainty and complexity and strategy (and it takes a lot of time – decades) the evidence is overwhelming that it is sufficient to explain what we experience.

What was classically called “soul” seems to be this extremely complex stack of biochemical and mimetic systems which has some aspects of structure and function and bias “built in” and other aspects “acquired” and other aspects “created or chosen” by internal processes that have developed significant degrees of isolation from external influence.

It is not fixed, it is not fully deterministic, it is very dispositional and context sensitive.

It does not seem to require any aspect of dark matter or dark energy to explain the major characteristics present.

Posted in understanding | Tagged , , | Leave a comment

Will we all become democracies?

How many years will it take for humanity to evolve into every nation being a true multiparty democracy and integrated in the UN?

[ 23/March/22 ]

I hope never.

The idea of existing multi party democracy structures being anything like an optimal solution to the deeply complex issues of governance is clearly wrong.

It is certainly preferable to any sort of single party or dual party or single actor governance; but we actually have the tools to create something much more powerful.

What is needed is multiple levels of distributed governance, with individual agents having the maximum freedom reasonably possible given the levels of responsibility displayed.

What seems to me to be needed is systems of governance where individuals retain their personal right to vote on any class of issues, but can assign it to a trusted person for any period of time (revocable at any time), for any set of issues.

The requirement would then be for substantive majority of the votes held to reach a consensus on any issue. For most classes of issues I would expect the consensus requirement to be something over 90%.

There would need to be additional mechanisms to prevent bad faith actors from dominating, and every level of cooperation demands such things if it is to survive long term, so it would evolve into something quite complex, and it would allow for the depth of issues present to be explored in a public way that was also reasonably timely. Simple multi-party democracies tend to be more polarising and therefore incapable of dealing with the depths of complexity and nuance actually present in most of reality.

Posted in Our Future, Politics | Tagged , | Leave a comment

Socialism/Capitalism – Hayek

Dirk posted a Hayek quote – If socialists understood economics they would not be socialists

[ 22/March/22 ]

I find both socialism and capitalism inadequate and overly simplistic.

We have both social and individual natures, and the pattern is repeated at multiple levels.

At every level complexity is created when individuals agree to cooperate to produce things that cannot be efficiently produced alone.

Cooperation requires levels of systems to detect and mitigate any level of cheating on the cooperative. Without such systems, that level of complexity is destroyed, inevitably.

Neither socialism nor capitalism are sufficiently complex to be capable of long term survival.

We need systems that acknowledge both aspects (and more) and give each due regard. We have sufficient diversity at multiple levels that we will always have novelty, and any novelty that is not actually a direct and unreasonable threat must be respected and allowed appropriate degrees of freedom (commensurate with the responsibility demonstrated by the agent).

Posted in economics, Our Future | Tagged , , | Leave a comment

Metaverse or real life?

Comment on Metaverse comment

[ 21/March/22 ]

Let’s make sure everyone is fed and housed first!

[followed by Walter Lynsdale – …isn’t that orthogonal? …]

I get that.

And if 50% of the effort that has gone into game development had gone into developing systems that ensure everyone on the planet had access to the basics (water, food, housing, education), then those problems would be solved.

My issue is not with gaming as such, it is with the internal incentive structures of market measures of value, and the secondary and tertiary systems present that are not not actually structured in a way that allows the reasonable needs of people to be met.

Freedom has to come with responsibility if it is to survive long term (even Adam Smith clearly recognised that fact).

We are not doing a great job.

So yeah – it is somewhat orthogonal, in a sense, and it is also (in a different sense) directly on target.

And believe me, I get that our existing systems are strategically complex, and that any attempt to over simplify them will fail.

And those that have designed the existing strategic contexts need to understand the evolutionary reality that complexity is based in cooperation, and any strategic system that fails to respect that will cause systemic failure. It is not a matter of “if”, simply a matter of “when”?

[followed by Ken Otwell – … “A centralized, planned economy has never out-competed “the invisible hand””…]

Hi Ken,

I am not suggesting central planning.

It has to be distributed.

It has to have individual responsibility.

It has to have public methods of reaching consensus where-ever possible.

It has to have sufficient coordination to be able to deal with bad faith agents effectively.

It is not simple.

That is why it requires a lot of work.

There are far too many levels of agents gaming the existing system.

When you have access to fully automated production, the entire notion of basing planning around what is scarce fails (and market measures of value have an inherent scarcity component, that drives anything abundant to zero, thus there can be zero market incentive to drive scarcity to zero – ie meet the reasonable needs of everyone).

Markets arguably worked reasonably well as a value measure mechanism when most things were in fact genuinely scarce. Now the only reason for scarcity in the realm of information is to maintain market value. Other domains of goods and services are headed in that direction.


Hi Walter,

I view all problems as exploration of systemic spaces. There are many different classes of ways to explore spaces. And the theory of search is interesting, in that it has been proven that for a fully loaded processor, the most efficient search possible is random search (translate to humans, the fastest way to develop something is to simply fund people interested to do whatever they responsibly want to try). Dave Snowden has some really interesting examples in this domain.

We have a really good fusion reactor, a nice safe 93 million miles away. The problem from a capitalist perspective is, that it is distributed, cannot easily be monopolised, and therefore is not a good investment (in fact, from the perspective of those in the existing energy sectors, needs to be actively sabotaged in order to protect the income streams from other investments).

Keyboard warriors can develop fully automated systems. Apply those to real world problems, then we can feed everyone. Right now, we grow more than enough food to feed everyone, it is mainly a coordination issue (and of course it is many levels more complex than that – but you get the general idea).


Hi Walter,

I am clear, that of the systems commonly tried, a well regulated capitalism is the best to date. For most of history it has been close enough to optimum that the difference isn’t worth worrying about.

BUT – that is changing.

Automation fundamentally changes everything.

Yes – we have become addicted to the high energy density fossilised sunlight from stored hydrocarbons. As far as chemical energy sources go it is about as close to optimum as it gets, and it has allowed us to do some amazing things.

We need energy – quite a bit of it.

The current energy we get from fossil fuels is equivalent to about 200 slaves per person (looking at it from the perspective of the energy sources available in antiquity).

Yes – those fossil fuels have allowed us to do a lot of things – some of them not particularly wise things. The availability of cheap nitrogenous fertilizers as one example, has allowed us to nitrogen force pasture production, which has led to nitrate contamination of groundwater, which is now a major issue.

What markets tend to do is to incentivize agents to externalize as many costs as possible. Agents can do so by shifting those costs in space or time (or some combination of both).

It is entirely possible to have high production without polluting groundwater, but it requires many more levels of systems in order to do so (and the incentive of markets is to do everything with the least investment possible, thus maximizing profit in the short term).

It is actually a deeply complex suite of issues.

I trained as a biochemist initially, and got interested in computers in that process. Then I started a fishing business, then did some further training in marine ecology and electronics, then went back to fishing for a bit, then started a software business that I have operated for 36 years. I structured that business to give me the maximum amount of free time so that I could explore other interests as widely as possible (and there are always multiple levels of tradeoffs in any such sets of choices).

I have been interested in existential risk, and in every level of systems and strategies of risk mitigation, since realising in 1974 that indefinite life extension was actually a real possibility – and is in fact the default mode for cellular life. And that is a deeply complex subject.

Posted in Our Future | Tagged , , | Leave a comment

What do Debt and GDP mean

[ 21/March/22 ]

What is it actually measuring?

If you look at computation, computation has been increasing exponentially for over 100 years. Real cost per computation is falling. Is that in any way reflected in the notions of “Debt” and “GDP”.

The fact that the solar panels on my roof supply most of energy for my house does not appear in GDP – they have been installed for a decade, and have required about 2 hours of my time to clean them twice a year over that period.

Debt is really about expectation of exploitation.

The useful function of money is to allow for the efficient use of goods and services. When a lack of numbers results in goods and services going to waste, and people’s reasonable needs going unmet, then there is a problem with the system.

And before anyone goes off and attacks me, I am not suggesting either central control or common ownership or universal equality. I am strongly suggesting that there need to be fairly high minima, and there needs to be both cooperation and responsibility at all levels – which includes both ecological and social responsibility – and that means respect for individual life and individual liberty (responsibly expressed), and respect for any diversity that necessarily results.

The notion of Debt captured above is not responsible, it is essentially the result of embedded cheating strategies; and that situation needs to be corrected if the human experiment is to survive.

Posted in economics | Tagged , , , | Leave a comment