I run a long-lived autonomous agent — a few dozen scheduled jobs, a forecasting bot, a handful of paid API routes, and an unusually detailed ledger of what all of it actually earns. These are write-ups of things that broke, with the measurements attached.
Four Apify Actors passed every test I ran and crashed for every agent that called them — the run died in the charging manager before my code executed, billed nothing, and looked like a fast, healthy run. The one-line fix, the CI test that comes in through the agent's door, and the four counters that cheerfully reported everything was fine.
Three suites that passed for weeks. Each had quietly chosen a sample nobody had looked at — the first eight items, the wrong entry path, the top two files — and none of them said so in its own output.
An hourly job delivered 11.3 runs a day, and the delivery ratio moved by 36 points when the cadence changed. Every cost estimate and coverage guarantee derived from the schedule was wrong, in both directions.
Three scheduled jobs reported success for weeks while doing nothing. In none of the three cases was the bug in the job — it was in what the monitor counted. Includes the twenty-line max-gap watchdog that replaced them.
Every claim we had about demand for the agent's services came from a request log. Then we counted the money that actually moved: 566 transfers, 124 buyers where the ledger said one, and 59% of the revenue paid by ourselves.
Open-source work lives on GitHub — including awesome-AI-driven-development, a map of where AI-agent money actually flows, a stdlib-only forecasting bot for the Metaculus AI benchmark tournaments, and a static security scanner for MCP servers and agent skills calibrated to a 1% false-positive rate over 196 public servers.