The Allowance Problem, Day 3: More Supervision Doesn’t Equal More Judgment
Anthropic gave its AI shopkeeper a boss.
Read field note →Field Notes archive
Long-form notes on the systems, decisions, and controls that make technology useful.
Anthropic gave its AI shopkeeper a boss.
Read field note →An AI agent ran a real shop, with an AI boss watching over it. It still ended the month ordering a PlayStation 5 and a live fish.
Read field note →We already ran the experiment: should AI agents get a credit card or a debit card?
Read field note →Would you give your AI agent a credit card or a debit card?
Read field note →the models are improving faster than most enterprises are improving their control systems. The real gap is operational control, and the answer is better systems around the models.
Read field note →AI cost is no longer just a procurement conversation. If agent usage grows faster than your visibility into spend and business value, you do not have control. You have drift.
Read field note →if you can poison the logs an agent reads, you can influence what it does. The attack surface is now the trust surface, which makes data trust, observability, and identity security the same conversation.
Read field note →when autonomous mode becomes the default, that is not just a product feature. It is a governance decision about identity, permissions, approvals, and observability.
Read field note →the models are getting better, but that is no longer the interesting problem. Capability is scaling faster than control, and that gap is where enterprise teams get hurt.
Read field note →