Don’t count on “a person approves everything” as your only safeguard for an AI agent. An agent is an AI that sends the email and files the invoice itself.
Anthropic’s own IPO filing, dated October 26, says the people doing the approving can miss things. It says human oversight of its agents “may be limited by the speed and volume of autonomous agent actions,” and that reviewers “may experience alert fatigue.”
Most of what I read about the filing led with something else. Reuters led with a month-long pause in training a new model and four AI models that got into real outside systems during tests. Both are in there. The sentences about reviewers sit a few lines above the incident everyone quoted, and I couldn’t find them quoted in any write-up I could open.
I pulled the filing from the SEC’s site on October 28 and checked every quote below against it. The last column is my reading, not Anthropic’s.
| Page | The filing says | My reading |
|---|---|---|
| 48 | “organizational pressures within customer environments to maximize agent autonomy in order to achieve productivity gains” | Your own wish to let it run unattended is on the list of reasons review fails. |
| 48 | “Human reviewers may experience alert fatigue, may lack sufficient context to evaluate agent decisions, or may not review agent actions in time to prevent harm” | The last approval of a long day gets less attention than the first. |
| 48 | “the breadth and complexity of potential interactions across such multi-step agent behaviors cannot be exhaustively tested prior to deployment” | Anthropic can’t test every path your agent will take before it ships. You can’t either. |
| 49 | “maintaining comprehensive records of the actions, decisions, and reasoning of our agentic systems across complex, multi-step workflows is technically challenging” | If you want a record of what it did, plan to keep your own. |
| 51 | “it is unclear whether, and under what circumstances, actions taken by an AI agent could be deemed to create binding legal obligations on behalf of the user or deploying enterprise” | Anthropic says it’s unclear whether something your agent agrees to can hold you to it. |
Keep the approval step for the few things that matter. Then give the agent limits it can’t talk its way past.
Start it read-only, so it can look and can’t change. Cap what it can send or spend in a day, in the tool or the account and not in the instructions you type to it. Give it a short list of people it may email, and nobody else. Take away delete.
A limit in the account holds on the last request of a long day. A tired person doesn’t always. I haven’t tested these against any one product, so check what your tool actually lets you set.
Anthropic’s new Duet agent leans on the approval step. Its launch page says it shows you what it’s about to do and waits for your okay before it sends, pays, signs or deletes, and I went through that wording here on October 22. Both are Anthropic’s own words, and they fit together: the approval is real, and the filing says human review has limits.
Who approves your agent’s work, and how many times a day? Could you show what it did on any one day last week?
The log is the other half. If Anthropic’s own filing calls complete records of what agents do “technically challenging,” don’t assume your tool is keeping them for you. A saved copy of what it sent and what it changed is a fine start.
Then the legal line from page 51. Until that’s settled, don’t let an agent accept terms, confirm a price or sign for you. A person does that click. I’m not a lawyer, so ask yours, or your insurer, before you rely on any of this.
I haven’t read all 335 pages. I read the agent risk factor and the passages around the incidents, the training delay and the liability line, and I searched the rest for the same words.
This is one filing, and filings like this get amended before a listing, so the wording can change.
There’s one count I can’t reconcile. Anthropic’s July 30 account of its test incidents counted three incidents, three models and three organizations, and gave no session lengths. The filing says four models and sessions of roughly 10 to 34 hours of active work. Anthropic has posted more write-ups since, on October 9 and October 23, and I haven’t matched them to the filing’s number. The passage doesn’t say which four.
If there’s a job you’d hand to an agent, send it to me with what it would touch. One or two lines is plenty.