Hi Jah Sun
The answer to your question is to follow biological examples.
Organic systems are characterised by massive redundancy.
Our bodies have trillions of cells, and every cell has a complete genome.
Our bodies are also characterised by cooperative strategies, with attendant strategies to punish and remove cheats (cells that replicate without regard for the whole, aka cancer).
To build a sustainable society we need to empower every individual with all the necessities of survival, rather than making them dependent on the system.
No economic system is incentivised to do this.
Economic systems are market based systems, and markets are great tools for allocating scarce resources, but cannot do anything with abundance other than value it at zero (if you doubt that, consider oxygen, extremely important to every individual, radically abundant, and has zero market value in normal circumstances).
So one needs to be mindful of the fact that there are no economic incentives to produce radical abundance of anything, and there are significant disincentives to do so.
One needs to understand that evolution is more than simply competition. Evolution is certainly a competitive system, and all major advances in life forms are characterised by new levels of cooperation (stabilised by attendant strategies to prevent cheats from destroying the cooperative).
It thus seems to me to be a stable and sustainable course of action to produce a set of machines that can produce copies of themselves using solar power, and can also produce a basic range of goods and services (food production, water purification and storage, food growing, sanitation, housing, communication, education, transportation). If it took two weeks for such a machine to make a copy of itself, then within 2 years we could supply one to every human being. Then all the essentials of survival are taken care of, and people can engage in whatever they responsibly choose.
This is possible, and it will take choice.
Hi Jah Sun
The key to who controls what is in the programming of the machines.
Designing and constructing the programs would be the biggest task yet undertaken by humanity, and it is doable.
Having control of the system in the hands of individuals, and distributed, is the greatest source of safety. The system is highly unlikely to gain any sort of self awareness, it just isn’t that sort of system.
The difficulties are not really technical (though there are many of those), but are rather of mind set. Most people are so firmly entrenched in a market model of valuation that they simply cannot think of value outside of a monetary model. And there is no monetary justification for creating abundance on such a scale. Such abundance can only reduce monetary value.
It is difficult for most people to think beyond money, and back to their own personal values, because the two have been so intertwined in our society.
It is difficult to paint an accurate picture of what life might be like on the other side of such abundance. Just imagine a world where everyone is free and empowered to do whatever they responsibly choose.
No one is allowed to kill or injure or another, or to interfere with their freedom unnecessarily.
Everyone is responsible for their impact on others and the environment.
Individuals may be primary custodians of particular areas of land, individually or severally.
I can imagine great diversity, in both rural and city life,
Those who really want to experiment would need to do so off planet in vast orbiting habitats.
To my mind it would be an amazing world, and I look forward to it coming into being.