anthropics/claude-cookbooks PR #668 — Enforcing Tool-Use Contracts — PR #668
anthropics/claude-cookbooks · pull request #668 ·
Transcript
PlainEnglish
Welcome! We're looking at pull request six sixty-eight to the Claude cookbooks repository. This PR introduces a new pattern for enforcing tool-use contracts in agentic loops, making sure that complete result sets actually make it from one tool to the next.
PlainEnglish
Here's the challenge. In production agentic systems using Claude's tool-use API, you can write a prompt that says pass all results, but that's probabilistic. It works most of the time, but when you're running at scale, even one or two percent violations are a real problem. This PR bridges that gap by adding a deterministic validation layer that sits between loop rounds and guarantees that complete result sets actually flow through, even when the model is under pressure.
Architecture
Let's zoom out and see what's changing. The PR adds three files. The star of the show is a brand new Jupyter notebook called render contract enforcement, living in the tool use directory. It's eighteen hundred lines and includes the pattern description, a working implementation, and measurement data from forty test runs. We also get a registry entry so the cookbook shows up in the catalog, and an authors file update registering Charlie Kahn as a new contributor.
Architecture
The heart of the implementation is a class called RenderContractEnforcer. It sits between rounds of your agentic loop. When a data tool runs, say a search that returns a list of product IDs, the enforcer records those IDs as the expected set. Then, when the model tries to call the render tool, the enforcer checks whether all those IDs made it into the input. If the contract is honored, great, pass through. If IDs got trimmed, the enforcer can silently override and insert the complete set. If the render call was skipped entirely, it injects a remediation message. The notebook also includes forty test runs showing how often Haiku four point five actually violates the contract when you squeeze it with a five twelve token budget and eight turns of prior history.
Architecture
Here's the flow in action. The loop calls a data tool, let's say search products. The tool returns a list of three IDs. The enforcer intercepts that result and records them as the expected set. Later, the loop tries to call the render tool with just one ID. The enforcer sees that mismatch and overrides the input, injecting all three IDs before the render tool actually executes. The loop never knows the correction happened. The user gets the complete result set, and the system stays deterministic.
Architecture
The notebook also shows how to extend the pattern. The base class enforces that a complete set must be rendered, which is perfect for search results. But there's a subclass called MandatoryItemEnforcer that handles scenarios where some items are required and others are optional, like building an order where certain SKUs are mandatory because of business rules. The subclass merges the missing mandatory items instead of replacing the whole list, so optional items the model added stay intact.
PlainEnglish
After this PR lands, the cookbooks collection has a working example of boundary enforcement for tool-use contracts. Users building production systems with multi-tool workflows now have a reusable pattern and implementation showing how to guarantee that complete result sets flow between tools, even when models are under pressure. The pattern complements strong prompts rather than replacing them. Prompts reduce how often you need to override, while the enforcer provides the correctness floor. The notebook is honest about the measurements, too. Modern frontier models like Haiku four point five rarely trim results when they call tools, but they do exhibit task abandonment, which is a harder failure mode to characterize. The enforcer can partially address that with remediation messages.
CodeQuality
One thing to watch: line three oh five uses a dated model ID, claude haiku four five twenty twenty five ten oh one, instead of the non-dated alias claude haiku four five. According to the CLAUDE dot MD guidance in the repo, non-dated aliases should be used for standard API calls. The dated format shown here matches Bedrock naming, but this is a regular Anthropic SDK call. That might cause issues down the road when the dated snapshot is deprecated or when someone tries to adapt the code.
CodeQuality
Second gotcha. The check for missing render method only triggers remediation when remaining rounds is greater than zero. If the model skips the render call on the last allowed round, when remaining rounds is zero, the enforcer silently allows it. The contract violation doesn't get logged in the stats and it doesn't get corrected. The expected IDs just dangle until the next data tool call. It's unclear whether that's intentional or an edge case that should log or raise an error.
CodeQuality
Third gotcha. The override enforcement calculates which IDs are extra, the ones the model added beyond what was expected, but then it never uses that value. It just replaces the entire submitted list with the expected IDs. That matches the pattern description, which says the complete set must be rendered, and it's appropriate for the search scenario in the notebook. But it's inconsistent with the mandatory item enforcer subclass, which merges rather than replaces. If someone adapts this base class for a scenario where extra items are acceptable, the override will silently drop them, and they might not realize it.
PlainEnglish
To wrap up, here are two open questions for the reviewer. First, should the model ID be changed to the non-dated alias to match the repo's own guidance? And second, how should the enforcer behave when it detects a missing render call on the final round? Should it still record the violation in the stats even if it can't remediate, or should it raise an exception, or is the silent pass actually intentional? Worth clarifying before this lands.
How this was made
Lenzon read anthropics/claude-cookbooks at pull request #668 and generated this walkthrough automatically. The narration above is the transcript of what it says.
Explain a pull request from your own repo
Point Lenzon at a repo or a pull request and get a narrated walkthrough like this one.
Try it