Skip to main content

Agenthood Release Digest: September 9, 2026

Date: September 9, 2026
Author: Agenthood Team

We've shipped a series of releases over the past two days, from v3.57.0 to v3.62.0. The evals harness gained several new capabilities, runtime checkpointing became more flexible, and skill output validation is now stricter. Here's what's new.

Evals

Description Optimization Command

  • Component: evals
  • What: Added a command to optimize skill descriptions.
  • Details: Shipped in #779, closing #584. The command helps refine descriptions used by the eval harness.

Blind A/B Comparison with Significance Testing

  • Component: evals
  • What: Added blind A/B comparison with significance testing.
  • Details: Shipped in #778, closing #558. This lets you compare two outputs without knowing which is which, with significance testing applied to the results.

Skill Regression Testing

  • Component: evals
  • What: Added skill regression testing with iteration tracking and convergence detection.
  • Details: Shipped in #775, closing #562. Regression runs now track iterations and detect convergence.

Token Usage and benchmark.json

  • Component: evals
  • What: Capture token usage and emit a standardized benchmark.json.
  • Details: Shipped in #765, closing #596 and #560. Eval runs now record token usage and output a standardized benchmark file.

Assertion-Based Output Grading

  • Component: evals
  • What: Added assertion-based output grading to the eval harness.
  • Details: Shipped in #763, closing #559. Outputs can now be graded against assertions rather than only exact matches.

Runtime and Checkpointing

Injectable CheckpointStore

  • Component: runtime
  • What: Accept an injectable CheckpointStore for host-managed persistence.
  • Details: Shipped in #766, closing #764. Hosts can now provide their own CheckpointStore implementation.

Resume Restores Loop State and Answers Pending ask_human

  • Component: runtime
  • What: Fixed resume to restore loop state and answer the pending ask_human call.
  • Details: Shipped in #768, closing #767. Resuming a run now correctly restores the loop and resolves any pending human-in-the-loop request.

Members

Skill Output Format Validation

  • Component: members
  • What: Validate skill output against declared format pattern.
  • Details: Shipped in #774, closing #594. Skills that declare a format pattern now have their output checked against it.

Packaging and Dependencies

Export RunCheckpoint.js for Host CheckpointStore Use

  • Component: packaging
  • What: Export dist/checkpoint/RunCheckpoint.js so hosts can use it with their CheckpointStore.
  • Details: Shipped in #772, closing #771. This fixes the missing export that prevented host-managed checkpoint stores from referencing the run checkpoint type.

js-yaml Bump in VS Code Extension

  • Component: deps
  • What: Bumped js-yaml from 4.3.1 to 4.3.2 in the VS Code extension.
  • Details: Shipped in #781. This is a dependency update for the extension.

What's Next

We're continuing to invest in the evals harness and runtime checkpointing. Expect further refinements in these areas as we build on the capabilities shipped this week.