Anthropic's Playbook for Building with AI, Reviewed

Summary

Anthropic, the company behind the Claude AI models, verify: Aug 2024? launched its own training and resources website for Claude practitioners, with many free courses and tutorials. Among them are two I decided to complete (I have my certification), both on building software with AI: AI Fluency for builders, for individuals, and The AI-native SDLC playbook, for organisations. Having spent a year building our own framework for much the same job, we thought it would be interesting to put them under the microscope.

The short version: they are practical, and they agree with us on what matters. Let AI do the typing and keep the decisions with a person. Leave a written record at every stage. Enforce the rules that must always hold with software, not with the AI's good intentions.

We built ROME around those three ideas before reading a word of either course. Where we differ is in how much judgement can be handed to an AI. We come to that part at the end.

A word about ROME first

ROME stands for Requirements-Oriented Multi-agent Engineering. It turns a written description of what a business needs into working software using several AI agents, each with one job: one writes the requirements, one designs, others write the code, and one checks the work at the end of every stage.

An ordinary, non-AI program then refuses to let the project move on until that check has passed and the evidence is on file.

The person who wants the software, the sponsor, answers plain-English questions along the way and signs off on the design before anything is built. ROME stops at delivered, tested code.

That is the vantage point from which this review is written.

What AI Fluency is

AI Fluency is Anthropic's name for working with AI "effectively, efficiently, ethically, and safely." Two professors, Rick Dakan and Joseph Feller, developed it in 2023 and 2024 and published it under an open licence; Anthropic built the courses around it.

It has four parts, all beginning with D.

Description and Discernment form the inner loop you go round several times an hour: ask, look, ask better.

Delegation and Diligence are the outer loop, the bigger calls about whether to involve AI at all and whether to stand behind what came out.

The line that runs through everything is this:

"Delegating implementation is usually safe. Delegating judgment is usually not."

Let the AI do the work; do not let it decide whether the work was worth doing.

The courses add that most AI failures are really failures of description, discernment or diligence made earlier by a person. If the result is wrong, look upstream.

The framework is for anyone who uses AI at work. The builders course narrows that to "product people, designers, and makers": anyone who owns a piece of software from idea to real users.

You need not be a programmer, but you must be responsible for something.

The builders course

AI Fluency for builders is nine lessons, about three hours, with a quiz.

One exercise runs through the middle of it: a community health clinic wants patients to check the waiting time before they set off, and that is all you are told.

By the end, you have built it, tested it on a friend playing a patient with a sick child, and decided whether to ship it.

Three ideas are worth carrying around.

Where AI is strong, and where it is not

The course splits building into six steps and rates each.

Notice the shape.

AI is strongest in the middle, on the step that used to be slow and expensive, and weakest at both ends, on knowing who you are building for and on judging whether what you built is any good.

Hand over the middle; keep the ends.

The chain from a need to a line of code

Most AI training teaches better prompts.

The course says the prompt is only one link in a chain that runs from what the user said, to a requirement, to a specification, to the instruction you give the AI, to the tests that prove it worked.

A person translates at every link, and "AI cannot hear what the user did not say."

If the code works but the product does not, a link upstream broke; go back and find it.

Its sharpest line is:

"A passing test with an unhappy user means you described the wrong intent."

Five questions to ask of anything an AI builds

When an AI can produce a working program in minutes, "it works" stops being the bar.

The five questions run from the one a computer can answer to the one only a person can:

1. Does it work? On real data, not just the examples in the prompt.
2. Does it work well? Under load, with many users at once.
3. Is it the right thing? The user's actual need, not the literal words of the spec.
4. Is it good? Can someone complete the task without help?
5. Is it responsible? Honest about the AI's role, and nobody left out by default.

AI output usually passes the first and stumbles on the third.

That gap, between working and right, is where the builder earns their keep.

The closing rule is blunt:

"You own the outcome, not the output. 'AI wrote it' explains nothing and excuses nothing."

The playbook for organisations

The AI-native SDLC playbook is a different animal.

SDLC means software development lifecycle: the stages software passes through from idea to production.

The playbook is for the people who run that process at large companies, especially regulated ones, where nothing ships without the right person's approval.

It assumes you use Claude Code, Anthropic's AI programming tool, and can change how your organisation plans, reviews and releases software, not just how it writes code.

The argument is simple.

For decades the slow, expensive part was writing the code, and the whole process was built around that.

AI has made the writing fast, but approvals, reviews and handovers still run at human speed. The bottleneck has moved to either side of the coding, and the old controls no longer fit.

A security team sized for human output cannot review ten times the code. Either the queue builds or things ship unreviewed, and a regulated firm can accept neither.

The answer has four parts.

Turn the line into a loop.

Six stages in a row become a cycle with AI at every point and people above it, "instigating, directing, and governing."

A production fault does not wait for someone to raise a ticket; it is written up and re-enters the loop at the start.

Leave a paper trail at every stage.

Each stage ends by writing one file into version control, the system that records every change and who made it, and the next stage starts by reading it: a one-page statement of intent in the originator's own words, then a specification, a plan, the code and its tests, the review findings, and, if something breaks, the incident record.

Because each file carries a name and a timestamp, the chain of files is the audit trail.

Separate advice from rules.

A skill is a written instruction the AI is likely to follow.

A hook is a small program that runs before the AI acts and can stop it.

"The skill makes violations rare and the hook makes them close to impossible."

Any rule that must always hold needs a hook, and in regulated firms the hooks that matter live in settings engineers cannot switch off.

Put a named person at every gate.

A product owner accepts the intent, an engineer the plan, a code owner the change, a release manager the release.

The AI prepares everything up to that point and "cannot pass it."

Its review findings inform the decision; they never make it.

The twelve plays share five headings, and the last, how to measure whether it worked, is the one most guides skip.

Every play names one early indicator and one later one, and says where in the records to read it.

If you adopt nothing else, adopt the habit of naming the number.

What it is worth

For an individual, the builders course is worth an afternoon, and a second to do the clinic exercise properly.

Its value is not a technique but a vocabulary for the part of AI work that goes wrong quietly: accepting a result because it ran, without asking whether it was the right thing.

The strength table and the five questions are the kind of thing you find yourself using a week later without noticing.

For an organisation, the playbook is worth far more and costs more to act on.

It is not a course you finish but a set of changes you install, one at a time.

It answers the question most firms are stuck on: how do we let AI write most of our code without review either collapsing or becoming a rubber stamp?

A paper trail per stage, hooks for the rules that must hold, and a named human at each gate is the first answer we have seen that a compliance officer could sign.

One catch, which the course does not hide: the plays are Claude Code's own features arranged as a process.

The ideas travel; the recipes do not.

The view from ROME

We built ROME on three convictions, and Anthropic shares all three.

Every stage leaves a record the next one reads.

In ROME every phase produces named files, and one state file records which requirement led to which artifact.

The playbook's chain of committed files is the same idea, reached the same way: by watching an AI restate a requirement and lose a little each time.

The rules live in software, not in the AI's promises.

Our orchestrator's instructions say, in effect, you are an AI and may be wrong, so enforcement is not yours.

A plain program, the guard, refuses to let a phase complete until the evidence is on file.

The playbook's hooks are the same mechanism under another name.

An AI may never certify its own work.

The playbook stops an AI editing the tests while it fixes the code.

ROME learned that the hard way: our code-writing agents used to report which requirements they had satisfied, and in September we replaced that with a scan of the source itself, because a self-report is not evidence.

So far, so agreeable.

Two places would make an attentive reader of the builders course raise an eyebrow at ROME, so we will raise it ourselves.

The course rates understanding who you are building for as AI-weak and says keep it.

In ROME an AI agent does the requirements work.

Our defence: the agent does not decide what the sponsor wants, it decides what to ask, and every unanswered question stops the project until the sponsor answers.

The playbook does the same, with a product owner correcting the result.

The honest gap is that nothing in ROME yet confirms the agent heard every sentence of the brief.

We are building that check.

The course also rates deciding whether it is good enough to ship as AI-weak, and the playbook puts a person at every gate.

In ROME the routine gate decision is made by an AI agent.

We fence it: its approval counts only once the guard has confirmed every required piece of evidence exists and passes, and the decisions that cannot be undone, the architecture and the infrastructure, go to the sponsor in plain English before anything is built.

It is a different bet.

Anthropic scales by adding people at the gates; we scale by adding AI roles and asking one person to carry the calls a program cannot make.

Ours is the more automated; theirs is the easier to explain to an auditor.

One thing we are simply taking home.

Every play ends by naming a number.

ROME measures one thing well, how many requirements are covered by working code, and almost nothing about its own process.

While writing this we noticed that nothing runs our own test suite automatically and one test has been failing quietly since July.

The playbook would have caught that on day one.

Fair enough.

Who should take which

Both are free on Anthropic's Academy site, and both take less time than the meeting you would otherwise have about AI adoption.