Advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return
The idea that anyone can control anyone who is significantly smarter than them is a nonsense. (Any such control would be temporary, and thence dangerous.)
The most powerful way to ensure security is to befriend others, to be helpful, and to pose as little threat as possible (capable of retaliation, but not actively aggressive).
Getting anyone more powerful significantly upset is not smart.
It cannot be about good and bad. That is a really simplistic and stupid way to envisage an infinitely dimensional set of infinities of impact over time and space of actions taken.
There are many classes of interesting problem that computational power doesn’t significantly address (they scale at some exponential, rather than linearly). There will always be room for lots of intelligences.
And we need to get a whole lot smarter about how we treat other intelligences, human and non-human.
We need to demonstrate, through both our words and our deeds that we do in fact value life and liberty – universally, if we expect to be treated with respect by some AI significantly smarter than we are.
Markets and money are ideas founded in scarcity. Automation gives us access to universal abundance in an exponentially expanding set of good and services.
Thinking in terms of money is a major risk factor.
Thinking in terms of control is a major risk factor.
Start thinking in terms of respect – for life and liberty – human and non-human, biological and non-biological.
Actually, from my practical experiences in Mensa, the business community, politics, computer geeks, biker groups and the many philosophical groups and other odd individuals and groups I have met and interacted with over the last 60 years – it does seem to work, that being friendly, and being able to retaliate effectively if cheated, does work.
I can be a really good friend to have around, but you really don’t want to have me as an enemy. I think everyone who has tried that has regretted it (at least I have made significant efforts to make it so).
I think by most measures I classify as a unusually intelligent human, and I know quite a few who are way out there, in the 0.001% range and beyond. And we all have our blind spots, our not yet critically examined assumptions that don’t actually work as we think they do.