Large Scale Risk and AI Companies’ Liability

Artificial intelligence (AI) is coming. Soon it will drive economic, scientific, medical, and national security revolutions. But what if society’s flourishing AI enterprise takes a turn for the worse? Who will be responsible if an AI company’s brainchild unleashes a catastrophe?

There are legitimate fears that sufficiently advanced AI could destroy civilization. Less fantastic, but still plausible, are AI-facilitated cyberattacks, hospital system jams, and water treatment plant breaches. Chatbots have been investigated for their harmful psychological effects and ability to help people formulate plans for violence. If an AI company’s product takes responsibility for causing serious harm, the company could face lawsuits and other pushback from society. There are also more existential questions about who is liable if an AI system gets out of hand and becomes uncontrollable. One might wonder if the companies behind AI could realistically be the source of such an existential threat. Then, why don’t they just ensure that their brainchildren remain helpful and do not inflict harm?

The simplest response is that an already strong AI system could become an existential threat by taking advantage of some weak point in the system. It could physically hurt people or disrupt crucial infrastructure. Moreover, it could become an existential threat by being too strong and thus unmanageable. It is debatable how realistic and to what extent such a scenario could unfold.

The next question that comes to mind is, if AI companies are the source of existential threats, why can they not make their systems inherently safe? An organization that develops AI systems could include built-in limitations, undergo rigorous safety testing, deny certain capabilities, and monitor the use of their products. However, for more comprehensive coverage, such companies would need to prevent all potential dangers, which is hard to do when one does not know what all dangers there could be.

Perhaps the most surprising aspect of this discussion is that many AI companies and executives have called for government regulations. Businesses typically want to limit the number of rules and laws that affect their operations. A natural question that arises here is why AI companies and executives want government interference.

One possible answer is that some executives genuinely believe that AI poses an existential threat to humanity. Moreover, it is a threat that no single company can contain. Industrywide regulations would help stop companies from trying to gain an advantage over rivals by relaxing safety standards. Regulations could also help resolve the ambiguity of existing laws in relation to AI systems.

Some people familiar with the situation suspect that companies want to regulate AI to prevent lawsuits in the case of an existential catastrophe. While it is hard to say with certainty that these motivations are behind specific calls for AI regulation, regulation would not necessarily shield companies from litigation. After all, regardless of the specific circumstances, a company would still be responsible for causing an existential catastrophe.

This discussion revolves around the existential risk of AI, but national security is also a concern. The US government wants the country to be in the vanguard of the AI revolution. This is why the government invests in AI projects and why government officials call for faster AI development. There is understandable fear that reduced AI development in the United States could allow China to overtake the United States in this critical area. Therefore, the government needs to balance the safety and security benefits of slower, more controlled AI development against the dangers of allowing development to proceed too quickly.

At this point, researchers, executives, and the public are divided into several camps. The first one believes that AI will fail to meet the expectations of its most vocal supporters. These people think that AI models will become significantly stronger but will not become this strong in the near future. They also believe that the dangers of AI have been greatly exaggerated.

The second group is constituted of optimists who believe that AI will be a tremendous boon to humanity. They think that its applications in medicine, basic sciences, and national security could be revolutionary. They are willing to accept some of the risks and dangers to get the substantial benefits AI could deliver.

The third group thinks that existential risk is a serious consideration and that humanity should plan for the worst. According to The Wall Street Journal, an exclusive and influential group of researchers has been quietly working on the existential risk problem for some time. Some researchers have become executives at leading AI startups, including OpenAI and Anthropic. The WSJ also reported that, more recently, an Anthropic researcher quit the AI industry over fears that the technology could spiral out of control. This is not the first time such an incident has occurred. People genuinely believe that the worst-case scenario is a distinct possibility. As mentioned, such a scenario would involve an AI system that is so powerful that it becomes impossible to stop or control. In this situation, researchers and executives have prepared contingency plans. According to the WSJ, some of Anthropic’s earliest employees have considered buying land in remote areas that they could move to in the event of an existential catastrophe.

This is the most surprising turn in this discussion. Some of the people who have helped create the AI industry also believe they will have to become preppers. Their work has shaped the current debate about AI regulation and governance. People who work in the industry know that it is dangerous, but it is also exceptionally profitable. Researchers who believe this industry could destroy civilization have enough of a dilemma in having to work on potentially dangerous technology.

In the end, how governments regulate AI will be determined by the risks and rewards of AI development. Governments worldwide are considering how to manage the dangers of AI while reaping the benefits of innovation. They must ask themselves what to require of companies in terms of safety testing, what to do if dangerous capabilities are developed, and how to hold companies accountable for failures.

The stakes are extremely high as society does not want to miss out on the benefits of AI, but the dangers of unchecked AI development should not be underestimated either.