Friday, 23 April 2021

Elon Musk; Simulated Universes; Moore's Curse; The Fermi Paradox; Mars colonization; and Technological Hubris

Overall, I'm a fan of Elon Musk. He is smart; articulate; a great innovator, engineer, and entrepreneur; an incredibly hard worker; and in many ways a very inspirational figure - someone willing to try bold new things and tackle difficult problems. Oftentimes controversial, to be sure, but a net force for good in the world. The world needs more people like Elon Musk. As I've commented in the past, Musk is the virtual embodiment of George Bernard Shaw's quote: "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man". Musk is the paradigmatic unreasonable man.

However, as I've also commented in the past, while progress may depend on the occasional, unlikely success of the unreasonable man, more often the unreasonable man is forced to adapt themselves to reality. One of the reasons why opinions on Musk are so divided - apart from his sometimes controversial approach to business and regulatory compliance, and his "fake it till you make it" tactics in building Tesla - is that he comes across as an absolute genius to those of limited technical understanding, but to domain experts, some of what he says/believes seems to be highly questionable, if not at times borderline fraudulent/wilfully misleading.  

In my view, the truth is somewhere in the middle of the two extremes of opinion - adulation and skepticism/condemnation. Without doubt, Elon is a fantastic entrepreneur; engineer; and business builder. A lot of what he has achieved has come from successfully raising capital (as well as investing in his own), often at what are de facto negative costs of capital; hiring a lot of smart people; and then inspiring them with BHAGs (Big Hairy Audacious Goals). He has also managed to inspire customers and charmed his way to receiving special treatment from regulators, as well as generous EV subsidies/policies, which have worked to his advantage. He is clearly a very competent engineer and creative problem solver, but his breadth of technical and scientific knowledge is not without its limits, and is also constrained by the fact he works 100 hours weeks. He has learnt a lot about electric cars and rockets, but he isn't necessarily an expert on other areas of science, and like everyone, he has blindspots.

One of Elon's greatest strengths is also his greatest weakness - his tendency to view things through the prism of sci-fi, and one sometimes gets the sense he may have read a little too many sci-fi books for his own good. On the positive side, a sci-fi frame of reference allows him to dream of possibilities others don't, and pursue innovations others wouldn't. He dares to be unreasonable, and sometimes through a combination of good fortune and sheer force of will, may succeed in innovating his way to actualizing those ambitions. But many other times he won't, and will end up pursuing ventures doomed to fail right from the start, because he analyzed things through the prism of sci-fi rather than economics and energetics.

As cool and powerful as technology is, economics have always trumped technology, and always will. The reason we don't have flying cars is not because we lack the technological capabilities as a species to make it happen. It's because of economics, including energy economics (energetics). It is simply far easier and more energy efficient to roll something along the ground than it is to elevate it and sustain that elevation. The costs of doing so far exceed the benefits. 

A couple of people can push a car along a flat even surface up to a speed of perhaps 5-10km/hr. How many people would it take and how much energy would it require to lift a 2MT car just one meter above the ground and carry it at a speed of 5-10km/hr? Orders of magnitude more. That's why we don't have flying cars (other practical issues exist too, including noise pollution and safety, but economics is the main barrier). We don't have an unlimited amount of energy, and it's far from free.

This is the same reason why commercial air travel speeds have plateaued at around 900 km/hr, and the supersonic carrier Concord went out of business. It's not due to a lack of technological capability (fighter jets fly much faster than that). It's because 900 km/hr is aerodynamically the most energy efficient cruising speed. Concord's flights were 50% faster, but they cost 3x more. Most people were not prepared to incur the extra cost. An hour of human life spared of inconvenience/discomfort has a finite value, and the market for people willing to spend thousands of dollars to avoid it was too small. 

Aside from financial economics, energetics - which includes the all-important concept of energy return on energy invested (EROI) - also appear to be unduly de-emphasized or omitted from Musk's sci-fi-addled world view. It creates blindspots and causes him to incur errors of judgment, in my view - both in business, and with respect to metaphysical speculation.

Elon, for instance, has argued at several public appearances that we are most likely living in a computer simulation, and that the probability we are living in a "base" universe to be one in billions. What's his reasoning? Elon's argument basically boils down to the following: we have seen very rapid increases in computing power over the past 50 years, and our ability to simulate worlds via computer games has exponentially improved. If you extrapolate any pace of continuing development forward, it is only a matter of time before we will be able to simulate whole universes, as well as entities operating within those universes. If you assume that - once in a position to create such environments - we would in fact do so (seems likely), and would reach a point of being able to do so before becoming extinct (likely in Musk's view), then it is highly probable we would birth simulated universes - perhaps many of them. Furthermore, simulated entities within those simulated universes would themselves simulate universes. As a result, you would end up with billions upon billions of simulated universes, and the probability you were living in the base universe would thus become extremely small.

This might sound superficially persuasive, and particularly if you are sci-fi minded, or a philosophically minded academic. But as soon as you understand some real world practical constraints, the plausibility of this argument rapidly falls to pieces. Of course, anything is possible. It is possible, for instance, that the energy and physics of the base universe are so completely different to ours that such constraints do not exist. But if you're going to assume that, you can't justify the argument by reference to progress in our microprocessor, AI, and computer game technology, and you might as well just argue "God made it that way". I will proceed on the basis that the putative base universe is at least somewhat akin to ours.

So what are some of the problems with this reasoning? They center on "Moore's Curse". This is the idea that Moore's Law and the remarkable, exponential improvement in computing power we have seen over the past 50 years, has caused people to believe that exponential progress in computing and technology is some kind of inevitability that we can extrapolate forward ad infinitum. But this is a fundamentally flawed assumption, even for compute - let alone many other domains that are often falsely analogized to Moore's Law, such as progress with the reduction in costs of batteries and solar panels. Indeed, it is yet another example of the oft-seen error of "naive extrapolation".

Moore's Law has changed the world, but the total improvement in computing power we can hope to ultimately extract from Moore's Law is subject to some fundamental physical limitations. The vast majority of improvement in computing power and speeds has come from simply making transistors smaller (Moore's Law, in its original/literal interpretation, is doubling transistor density every 2 years or so). This allows us to pack more of them onto a chip, and being closer together, for them to communicate and transfer electrons amongst their circuitry more rapidly. However, the problem is that there is only so small you can make transistors before you start running out of atoms. We are already getting down to the scale of some 5-7nm, with 3nm nodes already under discussion. To put this into perspective, 1nm is the width of about two silicon atoms. We are already rapidly approaching hard limits on the degree to which we can extract further gains in computing power from miniaturization.

Ok, so for supercomputing applications, then why don't we just make chips bigger (while keeping the smaller transistors)? Why not make them the size of say football fields, once we hit our limits at the scale of the small? Little known is that the maximum size with which chips can be made is also subject to some fundamental limits, and at commercial scale, chip sizes have so far not exceeded about 800mm2, and that limit is unlikely to change (for scaled & cost effective solutions at least). 

One of the fundamental constraints is the speed of light and how fast electrons can move. Modern processors operate at 4-5GHz, or 4-5 billion operations per second, and even on a small chip, there are several km of complex wiring pathways, which sometimes result in electrons having to weave there way through complex pathways as they cascade through multiple transistors. If you make the chips physically larger, electrons have more distance to travel, and that takes time (as well as energy). Processor clock speeds have started to top out in recent years for this reason, and speed gains have started to focus on things like architectural improvements, or the efficiency with which chips are wired so that the electrons have less distance to travel, on average, to perform necessary operations. Increasing the physical size of chips would therefore require clock speeds to be reduced, offsetting the whole point of increasing sizes, while total energy consumption would also rise significantly. 

There are also manufacturing challenges. A very small wafer defect at the atomic level can render a chip faulty, and such defects occur with a certain degree of frequency. The smaller you make chips, the less likely there is to be a defect on any given chip and thus need to be discarded, but the bigger you make the chip, the more likely a defect becomes, and this likelihood (and hence the cost) goes up exponentially. This limitation might be able to be overcome with better manufacturing techniques, but it also might not be.

What about 3D cubic chips? Setting aside the issue of whether it would even be technically feasible to manufacture a 3D cubic chip at the nanometer scale - very far from a given - the limiting factor here is heat dissipation. Microprocessors generate heat, and a cubic chip would find heat dissipation a major challenge due to its low surface area relative to its volume. Already, high-spec 2D GPUs for home gaming rigs require cooling fans to be installed to avoid overheating. However, even if 3D was not a limiting factor - which it very likely is - it too would eventually run into fundamental 3D size constraints.

What about distributed architectures, or wiring multiple chips together, rather than making individual chips better, and processing computations in parallel. This can and is already being done, and promises further significant gains. But even here, there are fundamental constraints in the speed with which distributed chips can interconnect with each other.   

The upshot of these realities is that in the not too distant future,* humanity is going to be confronted with some fundamental constraints on how much additional computing power we can derive - from traditional computing methods at least - and face the once unthinkable reality that we may have reached the limits of this technology. Just because we have seen exponential growth in computing power and cost reductions over the past 50 years, does not mean that that pace of improvement can be forever extrapolated into the future, or indeed any improvement at all beyond a point

We are likely to be able to prolong the gains for a while yet by moving to different architectures, increased GPU offloading, etc. Quantum computing is postulated by some as the next generational leap, but that technology appears highly speculative at present and mostly appears to have emerged precisely because people understand that the limits to conventional computing are rapidly approaching. While nothing about the future is assured, there is not a low likelihood that at some point the amount and speed of computing power humanity can harness will top out, rather than keep increasing exponentially.

If that is the case, Musk's assumption that we can extrapolate forward gains in computing/simulation power ad infinitum could well be unsafe. And how close are our current computing technologies - where transistors are already being packed together 20 atoms apart - to being able to simulate a universe? Our most advanced supercomputers can barely simulate perhaps a square centimeter of atomic-level molecules, fluids etc and all their interactions. It's too computationally intense. Can you imaging the truly astronomical (literally) amount of computing power and energy that would be required to simulate an entire universe, let alone billions of them?

Furthermore, it is important to remember that any subsidiary simulated universes created by simulated beings themselves would still need to be processed on the base universe's computing hardware. If we created a computer simulated universe today, and the entities within that program created computer programs that simulate yet more universes, the processing of all of those universes would have to take place on our silicon. This makes the argument for billions upon billions of stacked universes all the more implausible.

I'm aware some explanations of quantum phenomena argue that one of the reasons for quantum superposition is that the wave function only collapses into a specific value once observed in order to conserve hardware resources (render only what needs rendering). But this applies at the quantum scale, not the atomic/molecular level and above, and even simulating the latter is extraordinarily computationally intense.   

When these limits are taken into consideration, it leads one more towards the conclusion that the probability we are in a simulation with billion to one odds (there need to be billions of simulated universes for those odds to be true) to actually be very low, because the computing and energy resources needed for that to be true are astronomically large. For Musk's assertion that the probability we are in the base universe to be one in billions, you would probably be talking about energy requirements above all the energy currently available in the known universe, maybe billions of times over, and that's assuming it is even physically possible to build computer chips that are able to handle that many equations in the first place, which in all likelihood it's not (for the reasons already described).

The universe is really big, but once you start thinking mathematically and in orders of magnitude, it is not so big compared to many other mathematical phenomena. The size of the universe is something like 8.8 x 10^23 km. You "only" have to add 23 zeros to 8.8km. When you a talking about stacking orders of magnitude of additional computing power on top of one another, the size of the universe's available energy resources suddenly doesn't appear that large by comparison. Such is the nature of exponential math (this is why the number of potential chess games exceeds the number of atoms in the universe).

One of the reasons people get this and many other issues so wrong is that humanity is suffering from an affliction of technological hubris. Technology has driven many seemingly-miraculous changes to our lives over the past several centuries, but that does not mean there are no fundamental limits to what technology can do for us. It also doesn't help that many people don't understand how technology actually works - they just observe what appears to be exponential improvement in some areas, like compute, and then extrapolate that forward.

No amount of scientific and technological understanding will make energy generation more than 100% efficient (the law of conservation of energy makes it impossible). No amount of scientific knowledge will make it more energy efficient to carry a car than push it along the ground. These are fundamental physical limitations which technology cannot overcome, and there is nothing we can do about it. People have become too confident that technology can solve any problem, given enough time and will. 

A sci-fi rather than energetics mental framework has manifested in many of Musk's other views and business ventures, from hyperloop experiments (a niche solution for point to point routes at best, and a hugely wasteful and inefficient mode of mass transportation for everything else), to his enthusiastic, unchecked embrace of solar energy, and ambitious goals with respect to Mars/space colonization. Musk is talented, but no amount of talent can overcome fundamental energetic limitations. 

One of the issues that has puzzled and slightly unsettled Musk is the famed Fermi Paradox, which poses the question, if the universe is so big, there must be other intelligent alien species, perhaps billions of years ahead of us in technological development; they should have colonized the universe by now, so why haven't they and why can't we see them? If Musk and other philosophers came at the problem from the standpoint of economics and energetics - which people who discuss the Fermi Paradox seldom do, because they are Sci-Fi geeks - the problem might be somewhat less perplexing.

In my view, what people should do with the Fermi Paradox - but almost never do do - is simply begin by asking, is space colonization even technologically possible? It is simply assumed that with enough technology it could be done, but that could well be an unsafe assumption. Maybe even with perfect understanding of all the laws of physics, it simply wouldn't be possible to do it. Scientists and philosophers have proposed all manner of bizarre and exotic explanations, but in my view, the fact that it simply can't be done is by far the most likely explanation (though it may not perfectly explain why we can't detect radio waves).

One of the fundamental limiting factors for both life and civilization is energy. The ability of both to either survive and grow, or wither and die, is crucially dependent on it, and more specifically, on one key metric: the "energy return on energy invested" (EROI). If it takes an organism more energy to find food than the energy that food yields, it will starve to death. If the organism enjoys an energy surplus, it can grow and develop (in modern times, maybe outwards). 

From a civilizational standpoint, an energy surplus (beyond the calories we need for our bare survival) is necessary to allow us to manipulate the laws of physics to our advantage. The average citizen in developed countries indirectly consumes 200 times the their BMR (Basal Metabolic Rate) in energy to sustain their modern lifestyle, primarily delivered via fossil fuels. Without this energy, our standard of living would resemble Medieval peasants, at best (probably a lot worse, at least until the population was radially shrunk through war and famine). And the sustenance of this energy surplus requires we have methods of generating a high "energy return on energy invested". 

We have to expend energy, for instance, to find, drill and develop an oil well, transport the oil, refine it, etc. All of that takes energy. And for it to be worthwhile and sustainable, you need to get back more than 100% of the energy it took you to extract the energy. If it is less than 100%, you will run out of energy trying to procure energy, and your civilization will "starve" to death for lack of energy. For this reason, we will stop using oil long before we run out. We will stop using it when the EROI of finding, extracting, processing, and transporting it falls below 1 (or more economically competitive energy sources displace it).

For the modern oil industry, the energy return on energy invested averages about 10x. It has fallen from 20-30x over the past half century or so as we have depleted easier to recover reserves and have had to replace them with more expensive (and thus energy intensive) sources. This metric will continue to decline over time. If - for our energy sources/technologies overall - it ever falls below one, our civilization is doomed, irrespective of how technologically advanced we are.

This energy calculus applies to all forms of energy, including solar. While estimates vary, estimates for the gross EROI of solar panels seem to be about 4x, but this falls to closer to 1.6x with storage. That is also likely the EROI under fairly optimal conditions (solar panels are more efficient in some locations than others, and we build them in the most efficient locations first), and it is unlikely the storage metric includes more than the cost of dealing with night-and-day intermittency and short term weather variations. This low EROI reflects the significant energy that needs to be invested in extracting and producing all the raw materials that go into solar panels and batteries, as well as the energy invested in manufacturing and installing them, and the highly disbursed nature of solar energy. 

If this metric were to fall below 1x, no amount of technology, government subsidies, evangelizing or high-minded ambitions to decarbonize the world's energy system, would make it possible. It simply couldn't be done (or at least, we would have to consume more fossil energy installing solar panels and battery infrastructure than we would get out of them, which would make the entire exercise pointless).

When Musk thinks about solar, he thinks of it in terms of daily insolation from the sun (W/m2), and the amount of available land area. But those are only some of the constraints. The other constraints are the availability and cost of producing raw materials, including many rare metals that go into batteries etc, the economic and energy cost of which will go up exponentially as we try to scale solar and battery capacity to several orders of magnitude above where they currently are.

Musk recently said that China could "easily" power itself on solar. But again, he is thinking only in terms of land area and insolation. One of the biggest challenges with solar energy is not the daily fluctuations - night and day, and weather variations, though those fluctuations are already very costly to deal with. Instead, the main problem is with seasonal variations. Outside of equatorial regions, many of the world's most heavily populated regions receive only about one third of the solar energy during winter as they do during summer (that's why it's winter), while energy-intensive heating demand also significantly rises in the winter. Storing enough energy during the summer to get through the winter would require astronomical amounts of energy storage. It is not at all clear that the energy return on energy invested would be above 1 if we tried to make this happen (in this regard, the energy economics of wind are superior to solar, as it does not suffer this degree of seasonal variation in energy output).

To avoid large-scale seasonal storage, long distance transmission from hemisphere to hemisphere, or from equatorial regions, in theory could be feasible, but would come at a huge infrastructure cost, and entail significant transmission losses (which increase with distance), not to mention major geopolitical challenges. It is unlikely Europe, for instance, would want it's entire electricity network held hostage to stable geopolitical relations with North Africa.

Because storage on the scale needed to smooth seasonal variations is orders of magnitude uneconomic, self-reliance in China (and Musk was talking about China supplying its own needs through its own solar) would require that solar capacity be overbuilt by a factor of at least 3 relative to average annual energy availability, so that it could produce enough power at the low point of the year to meet peak energy requirements, coupled with there being enough redundancy to handle weather and other variations. That would require the commitment of truly massive levels of resources that would quite likely render the endeavour uneconomic in an energy sense (EROI < 1). The above estimate of a 1.6x EROI would decline to about 0.5x with a 3x overbuild. Even a doubling in total system efficiency would make it a marginal proposition, at best. Musk does not seem to have any real appreciation of these potential practical fundamental limitations (or at least hasn't admitted it publicly).

Now, I'll grant that it is possible we come up with creative technical solutions to address these problems. Maybe we find a way to install pumped hydro storage facilities at massive scale, for instance. We have also not yet reached the limits of our technical competencies in battery and solar production, and it is true that these are not fundamental limitations in the way the level of solar insolation is (although they do have fundamental minimum raw materials needs, and you can only reduce the energy inputs that go into raw materials procurement and solar and battery production so far via efficiency improvements). They are somewhat amenable to improved technical solutions. However, there is no guarantee that we will be successful, and saying that overcoming these challenges would be "easy" is foolhardy. It might be possible, at an extremely large cost, but it also might not be. We don't actually know.

This analysis is also relevant to Musk's universe-colonization ambitions, and indeed, to the Fermi Paradox referred earlier. Right now, our civilization - on earth - is facing an incredibly daunting challenge in trying to make our energy consumption sustainable - under ideal conditions. Doing so in space or on other planets will be orders of magnitude more difficult and energy intensive, and achieving any sort of EROI above 1x might well be impossible - even with perfect knowledge of the laws of physics. The reason this possibility is seldom considered is that humanity is currently afflicted with technological hubris.

In my view, by far the most likely answer to the Fermi Paradox is that colonizing the universe is simply not possible, because energy and other necessary resources are too sparsely distributed throughout the universe, and the EROI is below 1, such that any advanced civilization that attempts to do so in a self-sustaining/replicating way is doomed to fail. It might even be quite a bit worse than that - it could be the case that energy and resources are so sparse (including on earth) that it is difficult to sustain the type of high-technology civilization we currently have even one's home planet for more than a fleeting period of time - perhaps less than a millennia. Such civilizations may quickly run out of key resources, and then collapse back into more primitive, subsistence modes of living. Bleak? Yes, but not necessarily untrue (I'm not quite this pessimistic on humanity's prospects though).

Even on Mars - a planet that will be orders of magnitude easier for humanity to colonize than any other planet in the universe - the challenges are formidable and the odds of success are extremely low. The solar insolation received on Mars is only 0.4x that of Earth. If we can only barely make the EROI economics work on earth, how can we reasonably expect to do so on Mars, and particularly in an environment where dust storms can often engulf the entire planet and take solar panels offline for extended periods of time? Mars' atmosphere is also only 1% as dense as on earth - so forget wind power - and there are no hydrocarbons because there are no remains of dead plants and animals from past eons to dig up and burn for energy. Where is Mars going to get it's energy from? And a lot of it will be needed, because the average temperature on Mars is -60 degrees Celcius. Nuclear is possible, but unless there are large domestic uranium reserves, uranium rods will have to be imported from earth, and that could well be prohibitively expensive (energetically as well as financially). And what will Mars export back to pay for it (which it would have to do in the long run if it wanted to be "self sustaining")?

This is not to mention many other challenges. There is no magnetic field on Mars. Consequently, there is nothing to deflect the constant barrage of harmful gamma radiation emitted from the sun, which wreaks havoc with human DNA. Already small amounts of much-less-dangerous UV radiation routinely give people skin cancer on earth. Early settlers will likely be beset with cancers and have very short life expectancies, even ignoring everything else. We can't even cure cancer on earth. How are we going to stop it on Mars? Mars gravity is also only 0.375 of Earth's which will lead to many health problems. We will need vastly improved bioengineering technologies for the environment to be even remotely survivable long term. And there is very little oxygen on Mars (96% of their 1% atmosphere is CO2), and transporting it in canisters from earth, or trying to produce, collect and transport it in sufficient quantities on planet will require huge amounts of resources/energy.

Some of these problems may not be impossible to solve, but they could be, and at best the costs and energy requirements of doing so will be astronomically high, and most likely in excess of what is both economically and energetically feasible, and supportable by Earth's economy and resource endowment. And even if I'm wrong, all of this will only get us to Mars. 

The next leap will be several orders of magnitude more challenging again, and probably impossible, particularly because as Musk himself points out, you cannot escape Newton's Third Law in space propulsion - you need to power rockets with ejected matter, not pure energy (such as electricity stored on batteries; electric rockets are impossible). Distances are vast, which require very high speeds, and you need as much energy to slow down as you do to speed up. It is doubtful space colonization is energetically feasible even with perfect levels of scientific and technological understanding. If this is indeed the case, there is no Fermi Paradox.

I hope I'm wrong, and I don't necessarily think it's a bad idea to try, even if the odds of success are low, because it would be an incredible accomplishment for humanity if we could pull it off. But I'm not very optimistic, and perhaps much more relevantly to current day to day life and investing, I think investors ought to have a much more level-headed and realistic view on the limits of technology, and temper any Sci-Fi inspired optimism with a sound understanding of economic and energetic limitations.


*Post-publication clarification: Twitter discussions post-publication have revealed to me that my comment "in the not too distant future" here is unclear and subject to misinterpretation. This statement is relative in nature and is intended to be viewed in the context of Musk's argument we can extrapolate forward gains in computing power to the point of eventually simulating universes. It is not meant to be construed as an argument that we are very close to hitting the limits today in our ability to improve computing performance from a practical standpoint. 

There are still many avenues for improving compute performance, including improved architectures and moving away from general-purpose CPUs to more specialized chips optimized for specific tasks, and significant gains will likely continue for at least another decade or two (and maybe longer), and will have significant practical value. My point is that in the very long run, physical limits will eventually kick in, and you cannot extrapolate forward exponential gains in computing power indefinitely.