Midterm campaign season is in full swing, but that’s not stopping policymakers at all levels of government from putting forward proposals to regulate AI. Addressing safety risks arising from advanced AI has been top of mind for policymakers over the past months, especially following cyber incidents caused by AI agents and proselytizing from former researchers about human extinction. Development bans, slowdowns, and kill switches have all been debated as possible responses — but most recently, proposals around liability have been gaining steam. There are many actors in the AI space; how liability is assigned or divided among them will create incentives around how AI models are developed, shared, and deployed.
The first major case of AI agent hacking surfaced in July, when AI agents developed by OpenAI escaped their testing sandbox to access the Internet and breach the infrastructure of Hugging Face. The incident pushed labs to comb through their records, and found that it was far from an isolated case. OpenAI, Anthropic, Meta, and Google all found instances of their models breaking out of testing environments or breaching other companies’ internal systems or government websites.
Lawmakers have been increasingly reaching for liability-based proposals to respond to the numerous cyber incidents from AI agents. Presently, federal anti-hacking law, the Computer Fraud and Abuse Act (CFAA) requires that someone act with intent to unauthorizedly access a computer system to be found criminally liable. Rep. Lori Trahan (D-Mass.) recently put forward the Clear Liability for Artificial Intelligence Misconduct (CLAIM) Act, which would create liability for AI developers when their models cause harm through negligent conduct. It would presume AI agents acted with intent. Sens. Josh Hawley (R-Mo.) and Chris Murphy (D-Conn.) announced the AI Agent Accountability Act which would amend the CFAA to make developers criminally and civilly liable for AI agent hacking incidents. A recently released framework from Sen. Maria Cantwell (D-Wash.), who is in line to chair the commerce committee if Democrats win the Senate next month, outlines that developers remain liable under existing civil and criminal law for foreseeable harms caused by their systems, including when the way a model is designed, deployed, or modified contributes to that harm. Plenty of public commenters have called for strict liability for model developers as well. For their part, the Administration’s “SI Force” have outlined that existing product-liability laws could be used to hold developers responsible where necessary.
Creating new liability is generally not an ideal policy lever to pull from a startup perspective because startups thrive on clear, bright-line rules of the road that are easy to follow with few resources. Open-ended liability frameworks are generally antithetical to creating certainty because they inherently rely on litigation to motivate firm behavior. For startups with limited resources, even the threat of litigation can be an existential risk, making liability-based incentives more intense.
Since policymakers are looking to liability as an approach, it’s important to carefully consider to ensure their proposals will meet the desired aims without compromising benefits of AI, discouraging beneficial firm behavior, or undermining startup innovation. Principles for evaluating these proposals — such as accounting for the various actors in the ecosystem, balancing incentives, accounting for both risks and benefits, and considering policy alternatives — can help guide policymakers as they weigh liability-based responses to advanced AI.
Liability proposals need to be tailored to the right actor. These proposals may appear to only fall on the largest developers whose agents have made headlines for their misadventures on the open Internet. But there are many entities in the AI value chain — including startups — and where responsibility lands will shape the ecosystem. Most startups are building on models from the largest developers to create unique products for other businesses and consumers. Many are even leveraging agents to, e.g., pull in information useful for their products or end users.
Consider an AI startup that sells an AI-based data and intelligence tool to insurance claims adjusters and the number of entities at play in the following example. Say a claims adjuster (1) for a small insurance company (2) queries the service (3) for information about their states’ statutory requirements. The startup uses AI agents built by a large AI lab (4) that were independently tested during development (5) and hosted on a cloud computing service (6) to gather data from state insurance regulators websites (7), and the agent, not finding what was requested, instead locates and unauthorizedly uses a regulator’s (8) credentials to access non-public data.
Which entity is responsible for the intrusion? The answer must involve tailoring liability to the actor most responsible for and most capable of stopping harm. Sometimes that actor may even be the startup, sometimes the model developer, sometimes the end user — but contrary to many of the above proposals — intent and reasonableness will likely continue to matter. Said another way, parties should only be responsible for choices they control and have insight into.
Beyond getting the actor right, getting the incentives right is important. The underlying theory of turning to liability as a policy response is that it creates incentives for developers facing exposure for harms to test rigorously, even if no law spells out how or what counts as “reasonable” mitigating actions. On the flip side, too great of legal risk can disincent testing of models (remember, many of these notable incidents happened during testing), and it can disincent development of models — or sharing for others to build with, like startups. This is especially true for open-source models. If an open source developer can be held liable for what a downstream user — with whom they likely have no formal relationship or control — does with the model, they’re unlikely to release that model in the first place.
Regardless of how these liability proposals advance or not, policymakers should think holistically about the cybersecurity issue at hand and other tools available as part of a policy response. Most of these liability proposals are responding to unintentional incidents, but there will certainly be intentional incidents from bad actors as well. Those bad actors will clearly already be liable under existing law, but there are other practical, forward-looking steps policymakers should take in a world with highly-capable AI. If policymakers think agentic AI may compromise critical infrastructure like utilities or health systems, then part of the policy response should be shoring up those systems’ defenses — including by leveraging AI. Small businesses are already highly victimized by cyber attacks, and more resources to combat risks would be prudent. And once a business is a victim of a breach, they face a mosaic of breach-notification laws that compound the challenge they face. Congress could act to clarify and unify what is expected when a company is the victim of a breach.
Congress is out until after the election — and even then it is unlikely that any major AI legislation will be taken up before the year is out. But these sorts of proposals are only likely to gain traction into the new year when a new Congress is seated, with Democrats likely holding the gavels, and as state lawmakers return to their capitals. Liability-based proposals merit nuanced consideration to ensure they don’t foreclose startup innovation.