by Robin Hanson DRAFT Dec. 29, 1993
[If you got sloppy in your argument, you know I`d come down on you, right? So, please, let me have it with criticism. This draft is private, not for distribution. When the paper is better, I'll publish it. Robin]
----------
The future is hard to predict. We may feel confident that eventually space will be colonized, or that eventually we'll make stuff by putting each atom just where we want it. But so many other changes may happen before and during those changes that it is hard to say with much confidence how space travel or nanotechnology may affect the ordinary person. Our vision seems to fade into a fog of possibilities.
The scenario I am about to describe excites me because it seems an exception to this general rule -- more like a crack of dawn than a fog, like a sharp transition with sharp implications regardless of the night that went before. Or like a sight on the horizon much clearer than the terrain inbetween. And, as scenarios go, this one seems rather likely. Here it is.
If so, it may well be that technological progress and economic growth give us computers with roughly the computational power of the human brain well before we know how to directly program such computers with human-equivalent intelligence. This would not mean computers and robots couldn't do better on many specific tasks, or that computer-aided humans wouldn't be many times more productive than unaided-humans.
But it could mean that human intelligence continues to be very productive - that on average the amount of valued stuff that can be produced decreases by a substantial fraction when the amount of human labor used to produce that stuff decreases by a substantial fraction. Thus, as today, and as standard economic models predict, most folks would still spend most of their time working. And most wealth would remain in the form of people's abilities to work, even if the median worker is incredibly wealth by today's standards. We are, after all, incredibly wealthy by the standards of the ancients, yet we still work.
In contrast, having human-level A.I. could be more like each person owning a hundred human slaves, each as skilled as herself -- in this case there is hardly any point in working, unless for the pleasure of it.
A limited understanding of the brain and biology in general would also suggest that humans would not be highly modified - that inside we are basically the same sort of people with the same sort of motivations and cognitive abilities, and are likely still mortal as well.
Imagine further that we know how to take apart a real brain and build a total model of that brain, by identifying each units, its internal states, and the unit's inter-connections. A "good enough" model for each unit should induce in the total brain model the same general high-level external behavior as in the real brain, even if it doesn't reproduce every detail. That is, if we implement this model in some computer, that computer will "act" just like the original brain, responding to given brain inputs with the same sort of outputs.
That model would be what we call an "upload", software with human-level intelligence, yet created using little understanding of how the brain works on anything but the lowest levels of organization. In software terminology, this is like "porting" some software to a new language or platform, rather than rewriting a new version from scratch (the A.I. approach). One can port software without understanding it, if one understands the language it was written in.
Now some will doubt that such a brain model would "feel" the same on the inside, or even feel anything at all. But it must act just like it feels, since it must act like the original brain, and so many folks will believe that it does so feel.
Now without some sort of connection to the world, such an upload would likely go crazy or attempt suicide, as would most proto-uploads, not-quite good-enough brain-models that fail on important details like hormonal regulation of emotions. But with even very crude fingers and eyes or ears, uploads might not only find life worth living but become productive workers in trades where crude interaction can be good enough, such as writing novels, doing math, etc. And with more advanced android bodies or virtual reality, uploads might eventually become productive in most trades, and miss their original bodies much less.
Thus some folks should be willing to become uploads, even if their old brains were destroyed in the process. And since uploads should be productive workers, there should be money to be made in funding the creation of such uploads. The time at which uploads begin to be created in significant quantity is the "dawn" I referred to above, a sharp transition with clear and dramatic consequences.
To a fast upload, the world would subjectively seem more sluggish. Computers would also seem slower, and so fast uploads should find less value in them. Space colonization would seem a slower and more forbidding prospect. Interest rates would seem smaller, making investing in the future less attractive for a given set of values.
Fast uploads who want physical bodies that can keep up with their faster brains might use proportionally smaller bodies. For example, assume it takes 10^15 instructions per second and 10^15 memory bits to run a brain model at regular speed, and that upload brains could be built using nanomechanical computers and memory registers, as described in [Nanosystems]. If so, a ~7mm. tall human-shaped body could have a brain that that fits in its brain cavity, keeps up with its ~260 times faster body motions, and consumes only ~16W of power. (footnote: By consuming 3.5 times more power, the brain could run ~2.5 times faster than the body, and by consuming 35 times more, it could run ~10 times faster.)
Many billions of such uploads could live and work in a single high-rise building, if enough power and cooling were available. To avoid alienation, such uploads might create tiny familiar-looking trees, houses, etc., and live under an artificial sun that rises and sets ~260 times a day.
Or they might withdraw more into virtual realities. For relaxing and having fun, virtual realities could be anything folks want them to be. But for getting real work done, "virtual" realities could not be arbitrary; they would have to reflect the underlying realities of the software, knowledge, or social worlds they represent. Since the human brain seems especially good, compared with software we write, at dealing with the physical world, and since dealing with physical objects and processes should remain a big part of useful work to be done for a long time to come, many, probably most, uploads should remain familiar with the physical world.
If, as seems reasonable, upload brains are given extra wiring to allow the current brain state to be cheaply "read out" and "written in", then uploads could change bodies or brains relatively often, and could be transported long distances by ordinary communication lines. "Backups" could be saved, allowing near immortality for those who can afford it; if your current brain and body is unexpectedly destroyed, your latest backup can be installed in a new brain and body.
But the most dramatic consequences for both uploads and everyone one else come, I think, from the fact that uploads can be copied as well as backed-up. The state of one upload brain might be read out and written into a new upload brain, while that state still remained in the original brain. Thus at the moment of creation there would be two identical upload minds, minds which would then diverge with their differing experiences.
Uploads who copies themselves at many different times would produce a zoo of entities of varying degrees of similarity to each other. Richer concepts of identity would be needed to deal with this zoo, and social customs and law would face many new questions, ranging from "Which copies do I send Christmas cards to?" to "Which copies should be punished for the crimes of any one of them?" New forms of social organization might be needed within families of copies of the same original mind; some families of copies might be very loyal, while others might fight constantly. And of course political institutions like "one man, one vote" would require substantial modification.
Would an upload population explode? For a little perspective, let's review ordinary human population growth. In the short term one might take people's values as given. In that case reproduction rates depend on values and per-capita wealth, and per-capita wealth depends on values and reproduction rates.
People choose to have more or fewer babies depending on their values and culture, how much such babies will cost them, the wealth they have to give, how much payback they expect get from their children later, and on how their children's lifestyle will depend on family size. Technology and wealth also influence contraception and the number of babies who make it to adulthood.
Per capita wealth changes, on the other hand, depend not only on reproduction rates, but also on how much folks value current consumption over future consumption, and on the rates of growth possible in physical, human, and knowledge capital. And knowledge capital growth rates seem to grow with the size of the human population.
The net result of all these factors is not clear from theory, but since we have observed rising per-capita wealth for the last few centuries, we might suppose the net tradeoff, given current values, favors rising per-capita wealth.
A few centuries is only about a dozen generations, however. And Darwinian arguments suggest that after enough generations, the values in a species should evolve to favor the maximum sustainable population for any given technology, and the maximum sustainable growth rate as technology improves [Hansson&Stuart]. Basic human values, for resources, health, leisure, adventure, friendship, etc., don't seem to have changed much in the last few centuries, and these values do seem well suited for promoting growth in the sort of environments faced by our ancestors long ago.
Thus given enough generations, human values should evolve to promote maximal growth in our new sorts of environments -- one may still worry that small minorities who value exceptionally large families (footnote: There are U.S. religion groups which have averaged 10 kids per family for a century), for example, will eventually come to dominate the population.
Fortunately, however, this Darwian process is slow, and if economic growth rates continue their historical acceleration, they should soon exceed the maximum rates at which ordinary humans can have babies. From then on, per-capita wealth would have to increase, at least until artificial wombs were created, or until raw materials or knowledge progress started to "run out", and could no longer expand exponentially with the population as they have so far. For now, the world seems to be changing too fast for Darwinian evolution to catch up.
How do uploads change all this? An upload considering making a copy is much like a parent considering making a child. An upload would consider the cost to create a copy, the lifestyle that copy could afford, and how much they would value having another entity like themselves. Uploads may on average value having copies of themselves more or less than ordinary folks now value having children somewhat like them - this is hard to predict.
But what is clearer is that upload reproduction rates can be very fast - the upload population could grow as fast as factories could crank out new upload brains and bodies, if funds could be found to pay these factories. Upload copies, after all, do not need to be raised to adulthood and then trained in some profession; they are ready immediately to become productive members of society. Thus the main limitations on reproduction, and hence on Darwinian evolution of values, are economic and political. Who would want to pay how much to make upload copies? And who would try how hard to stop them?
Note, however, that if an upload expects wages to be high enough above an their minimum required income, they might agree to a copy even with a *negative* initial endowment. That is, if a copy were to be loaned enough money to buy their new brain and body, that copy might still find life worth living even under the burden of paying back this loan. (footnote: Such a loan might come from the original upload or any other source, and might involve more risk-sharing than a simple loan -- more like a joint investment.)
If we now add in the original upload's values for having copies around, presumably positive for having more company but negative for the added wage competition, we should find that such an upload has some minimum expected income at which they would be willing to spin off copies. And given that this upload has decided to make a copy, they may or may not prefer to transfer some of the original's wealth to the copy.
Now some uploads, perhaps even most, might not accept this line of reasoning. But those that do should, if not forceably prevented, keep making copies until their minimum income threshold is reached. Thus wages for tax lawyers should quickly fall to the lowest wage any one such upload tax lawyer is willing to work for. At this point many previous tax lawyers would find themselves displaced, even though the total number of tax lawyers has risen. And a large fraction of all tax lawyers should be copies of that one upload!
Of course abilities vary, and the lack of an ordinary body could be a disadvantage for early uploads competing with ordinary workers, limiting the number of ordinary workers uploads could initially displace. And reduced diversity of thought among a large family of copies may put them at a disadvantage in artistic trades. But in many trades, like tax law, a large number of standardized workers might have special advantages, especially in reputation-building.
It also takes time for a labor market to absorb new workers; each job is somewhat different, and it takes time for folks to learn each new job. Uploads running faster than ordinary folks might quickly master the relevant book-learning, but for most jobs most learning comes from watching and working with co-workers. At first, most co-workers will not be uploads, and most physical processes being managed would be tuned for ordinary human speeds, so being very much faster than usual may not be worth the cost of the faster hardware.
But as uploads became a larger part of the economy, upload communities which standardize on faster speeds would become more economical. If the rate at which faster uploads can grow wealth can grow to match their faster speeds, then interest rates should grow with the speed of such uploads. Slower folks would then be much more tempted to save instead of consuming their wealth.
What about trades where no upload has prior training? Even if the cost to upload folks were very high, or the number of volunteers very low, upload workers should still displace other workers, though at a slower rate. If the wage in some trade was above an upload's minimum, even considering the costs to become trained in that trade, and if loans could be arranged, copies would be created intending to specialize in that trade.
This population explosion should continue until it reaches limits, such as those of values or of subsistence. Values limits would be reached if almost no capable, versitle upload found copies worth having at the prevailing low wages. Subsistence limits would be reached if uploads simply couldn't make ends meet on a lower income; lowering their standard of living any more would lower their productivity, and hence wages, by so much that they could not afford even that lower standard.
Would values limit this explosion? Yes, of course, if typical values were held constant; few folks now who would make productive uploads would be willing to work at subsistence levels. It seems, however, that values will not be held constant. With upload copying, the potential rate and selectivity of reproduction could once again be comparable to the rate at which the world changes; Darwinian evolution (this time asexual) would have caught up with a changing world, and be once again a powerful force in human history.
As wages dropped, upload population growth would be highly selective, selecting capable people who value life even when life is hard. Soon the dominant upload values would be those of the few initial uploads with the most extreme values, willing to work for the lowest wages (footnote: these wages are per product produced, not per time spent). From this point on, value evolution would be limited by the rate at which people's values can drift with age, or can adjust to extreme circumstances.
Investors with foresight should be able to make this evolution of upload values even faster than ordinary "blind" biological evolution. Investors seeking upload candidates to loan money to, or upload copies to loan money to, would likely seek out the few capable people with the most extreme values, and those folks with the most pliable values. After all, these candidates would, all else equal, have the best chances of repaying their loans. Thus the evolution of upload values might be very fast indeed.
What would values evolve to? Would wages hit subsistence level limits? We expect that over many generations (i.e., times copied) Darwinian selection should favor maximum long-term generation of "wealth" that can be used to buy new copies. That is, since upload reproduction can be so directly bought, we expect evolution to favor uploads whose values induce them to take actions which give their copy lineage the maximum financial return on their investments, including their investments in new copies or in "leisure." (footnote: Regarding investment risk, I suspect evolution would favor maximizing the expected log return.)
Uploads who are overly shy about copying would lose out, holding less of the total wealth, measured by market value of assets, and constituting less of the population. Similarly, uploads who go whole hog in copying, just because they like the idea of having lots of copies, would become more numerous in the short term but lose out in the long term, both in wealth and population. Thus we don't expect uploads to become as poor as possible, though we do expect them to eliminate consumption of "frills" which don't proportionally contribute to maximum long term productivity.
We should also expect an evolution of values regarding death and risk. Imagine situations in which making a copy might pay off big, but most likely the copy would fail, run out of money and have to be "evicted" from its brain and body. Many folks might decline any such opportunities, because they so dislike the prospect of such "death". Others might consider this not much bigger a deal than forgetting what happened at a party because they were too drunk; "they" would only lose their experiences since the last copy event. We expect evolution to prefer the later set of values over the former.
Perhaps the hardest values to change in uploads will be our deeply-ingrained values for having children. Early upload technology would likely not be able to create a baby's brain from scratch, or even to upload a child's brain and then correctly model brain development processes. And even when such technology is available, children would likely be a poor investment, from a long-term growth point of view. New children may offer new perspectives, but with enough adult uploads, these benefits should only rarely exceed their high costs. Adults can offer new perspectives as well, and for much cheaper.
Eventually, human-level artificial intelligence may be achieved at competitive hardware costs, or we may learn enough about the high-level organization of our brains to modify them substantially, perhaps merging distinct copies or splitting off "partials" of minds. The upload era would have ended, and many of the consequences of uploads described above may no longer apply; it seems particularly hard to project beyond this point. But before then the upload era may last a long time, at least subjectively to uploads running at the dominant upload speed.
If level heads can be found, however, they should be told that most everyone can be better off if uploading and copying are allowed. While an upload transition might reduce the market value of ordinary folk's human capital, their training and abilities to earn wages, it should also increase total wealth, the total market value of all capital, including human capital of uploads and others, real estate, company stock, etc.
So, for example, if most everyone had the same fraction of their wealth in each form of capital, including owning shares in firms that make loans to fund uploading and copying, and if a large enough fraction of upload funding came from such loans, then most everyone would get richer from the transition. Even if you weren't one of the highly-copied uploads, your reduced wage-earning ability would be more than compensated for by your increased income from other sources. You could stop working, yet get richer and richer; and by uploading and resisting copying, you could become effectively immortal.
Or course even by then we may not have developed the institutions, financial and otherwise, required to allow most everyone to so diversify their assets. Folks might not be able to trade fractions of their future wages for shares in mutual funds, and tax laws might still encourage undiversified real estate holdings. And even if folks are able to so diversify, they may not choose to do so, yet later demand that politicians fix their mistake.
If forced to act, politicians would do better to tax uploads and copies, rather than forbidding them, and give the proceeds to those who would otherwise lose out. (footnote: Note that such a tax would be a tax on the poor, paid to the relatively rich, if one counted per upload copy!) Total wealth would grow more slowly than it otherwise would, but grow faster than without uploads. Of course there is the problem of indentifying the losers, and politicals systems have often failed to find such win-win deals in the past, and could well fail again.
What about folks who have values and abilities compatible with becoming part of the few highly-copied uploads? Would there be great inequality here, with some lucky few beating out the just-as-qualified rest?
If the cost to create an upload brain model from an ordinary brain were very high relative to the cost of creating a copy of an upload, or if computer hardware were so cheap that even the earliest uploads were run very fast, the first few uploads might have a strong advantage over late-comers; early uploads may have lots more experience, lower costs, and may be a proven commodity relative to new uploads. (footnote: Many initial uploads might well be cryonics patients, if legal permission to dissect and experiment with their brains were easier to obtain.)
Computer technology should keep improving even if work on uploading is delayed by politics, lowering the cost of copying and the cost to run fast. Thus the first-mover advantage should increase the longer uploading is delayed; delaying uploading should induce more, not less, inequality. Similar arguments suggest that a delayed transition might be more sudden, since supporting technologies should be more mature. And a sudden transition may risk inducing more military instabilities. All of these points argue against trying to delay the upload transition.
K. Eric Drexler, "Nanosystems", John Wiley & Sons, Inc., New York, 1992.