AI dog collars

I came across an interesting framework of AI accountability, in an article by Olivia Farrar in Harvard Magazine (not yet online). The following are reading notes.

First, Olivia brought up MyCity, which is an AI chatbot launched by New York City in 2024. As with other customer-service chatbots, the AI was trained on data about business rules and regulations, in order to answer people's questions about business rules and regulations. It probably answers quite a few questions competently but it's become infamous for jaw-dropping errors, such as claiming it's legal to fire workers for reporting sexual harassment, and claiming it's okay to serve cheese nibbled by a rat.

This has been my experience with various chatbots on several commercial software. Their footprints are everywhere, and their developers are so pushy that I eventually tried them out. So far, every chatbot of this type I've tested has disappointed. They don't seem to be able to answer even simple questions, much less perform simple tasks on my behalf. They sometimes also give ludicrous answers.

The article then addresses the important question of accountability. When an AI user is misled by AI errors, whose fault is it?

After all, nobody at City Hall wrote code instructing the chatbot to give any of these specific incorrect responses. The bad advice emerged from the AI model's own generated answers, as it repeatedly hallucinated legal "facts."

There is a lot to unpack in these sentences. The AI model is code written by someone, not someone at City Hall, but someone at the AI vendor. The generated answers are in one sense similar to predictions made by machine-learning or statistical models. Consider a model built to accept/deny credit card applicants. This model also makes mistakes. The chance of making mistakes can even be quantified by conducting appropriate experiments. For credit rating models, there are regulations aimed at preventing systematic discrimination.

A human may assign a task to an AI agent, which distributes work across thousands of specialized subagents... no human can easily reconstruct the chain of decisions and actions, making it difficult to assign legal liability.

Computer systems all have extensive logs. These agents also generate logs and "reasoning" traces. It's impossible for an outsider to figure out what's going on, but the AI vendors certainly can. What's the incentive for them to do so?

The BKC (Berkman Klein Center for Internet and Society) team calls their approach "canine agentic framework," inspired by medieval laws that governed dog owners. These laws distinguished between types of dogs: domesticated, dangerous owned aimals, wild foxes and wolves.

Domesticated dogs were considered safe.

Wild foxes were tolerated.

Wolves were deemed dangerous and beyond human control. They were actively eliminated.

BKC considers two factors: the risks posed by the AI, and whether a person or organization can be held accountable.

In the case of MyCity, the risks are low, but the companies and the government agencies can be held liable for fixing errors and compensating anyone harmed. These are managable risks.

By contrast:

An AI system that helps prioritize patients in an emergency room or controls an autonomous vehicle can lead to actions with life-or-death consequences, so hospitals and manufacturers can be held liable when something goes wrong.

I'm not sure if that's their opinion or the law. Regardless, I agree with this point of view. I just don't have the feeling that U.S. regulators are inclined to hold AI vendors responsible in the way just described.

An open-source, large language model that is copied and modified by thousands of developers might generate inaccurate coding advice or misleading information, but because no single organization maintains or deploys every version, responsibility is diffuse.

BKC likens these systems to "foxes" that can be tolerated.

The "wolves" are high-risk systems that have not clear accountable parties. They use the network of OpenAI agents that hacked into Hugging Face as an example. Surely, these systems could unleash devastating cybersecurity incidents. I don't buy the narrative that these are out-of-control agents that are acting on their own. They are written by developers, who, if they so choose, can shackle them.

... proposes that these types of AI systems carry digital "dog tags" or "collars": standardized metadata identifying their owner, identity, permissions, capabilities and authority.

Intriguing idea. It's not clear what's the point though. The harms may already have been devastating. And is this implying a world in which these owners can be held liable for the harms?