Organize agent projects

An agent that matters deserves the same treatment as code: version control, code review, tests, and a deploy step. An agent project is a git repository where agent specs live as JSON files, and a docana.manifest.json tells the CLI which file maps to which agent in Docana.

Clone it and push in five minutes

Don't start from a blank folder. The template repository has the layout, a working example agent, an eval, and CI already wired. Copy, paste, follow:

# Get the project
git clone https://github.com/DocanaAI/docana-agent-template.git my-agents
cd my-agents

# Install the CLI and sign in
npm install -g @docana/cli
docana login

# Find your application id, then put it in docana.manifest.json
docana applications list

# Ship the example agent
docana push -y

That's a deployed agent, tracked in git. Everything below explains what you just used. (Prefer a repo of your own? Click "Use this template" on GitHub instead of cloning, and push to your org.)

The template also ships an AGENTS.md for AI coding assistants (Claude Code, Cursor, ...), so they follow the validate → diff → push workflow and the agent design rules from docana help ai instead of guessing.

A typical project looks like:

my-project/
├── docana.manifest.json    # Maps files to agents in Docana
├── agents/
│   └── main.agent.json     # Agent specs (export payloads)
├── routines/
│   └── daily-report.json   # Scheduled routines, if you use them
└── evals/
    └── eval-*.json         # Test cases, synced with the agent

The manifest

One docana.manifest.json per project. It names your entities, points each at a source file, and records which remote agent each file tracks:

{
  "apiVersion": "docana/v1",
  "project": {
    "name": "my-project",
    "applicationId": 42
  },
  "defaults": { "target": "main" },
  "targets": {
    "main": { "entities": ["agent:main", "routine:daily-report"] }
  },
  "entities": {
    "agent:main": {
      "type": "agent",
      "source": "agents/main.agent.json",
      "remoteId": "cmly7kpth0019x5c9dyn3hegk",
      "agent": {
        "pullEvalDir": "evals",
        "pushEvalFiles": ["evals/*.json"]
      }
    },
    "routine:daily-report": {
      "type": "routine",
      "source": "routines/daily-report.json",
      "remoteId": "cmohsze35003m21c9mu1jxfwc",
      "routine": { "agentRef": "agent:main" }
    }
  }
}

Most of it reads straight off the JSON: source points at the file, targets name groups of entities so one command deploys a coherent set, and defaults.target picks the group plain docana push / pull / validate operate on. Three things aren't obvious:

• remoteId is the entity's id in Docana. Set it after the first push; it's what lets docana pull and docana agents diff find the deployed version.
• applicationId on an entity overrides the project-wide one, for the odd agent that lives in a different application.
• agent.pushEvalFiles and agent.pullEvalDir round-trip test cases between the repo and the agent's test registry; the next section shows how.

Push, pull, validate

The generic commands read the manifest and accept a selector: an entity key, a target name, or a wildcard.

docana push -y                  # default target
docana push agent:main -y       # one entity
docana push --target main -y    # a named target
docana push "agent:*" -y        # every agent
docana pull                     # write deployed state back to your files
docana validate                 # schema-check every source file

docana push updates the agent's working version; users keep talking to the live version until you promote it. Add --publish to do both in one step, once evals pass:

docana push -y --publish

Evals live in the repo

With pushEvalFiles set, every push replaces the agent's test registry with the eval files in git. A pull request that changes a prompt can, and should, also change the tests, and reviewers see both together.

pullEvalDir completes the round trip: evals someone added in the UI come back to disk on docana pull.

Each eval file is one scenario:

{
  "name": "Stays concise",
  "enabled": true,
  "evalSpec": {
    "steps": [
      { "type": "user", "text": "What are your support hours?" },
      {
        "type": "smartAssertion",
        "expectedOutput": "A short answer that offers further help.",
        "evaluationCriteria": "Concise, no fabricated facts.",
        "minScore": 7
      }
    ]
  }
}

See Testing Agents for what goes into a good test case.

Dev and production

Don't test against the application your users are on. Two patterns that work well together:

• Separate applications: iterate against a dev application, and only push to the production one when evals pass.
• Side-by-side manifests: one manifest per environment, same source files, different applicationId and remoteIds:

docana push -y                                        # dev (docana.manifest.json)
docana push -y --manifest docana.manifest.prod.json   # production

Before promoting, see exactly what separates the two environments. docana agents diff takes two agents, so point it at the production and dev copies:

docana agents diff <prod-agent-id> <dev-agent-id>

It summarizes the changes in agent terms (reasonings added or modified, routing rule changes, nodes added or removed) before the full spec diff, and normalizes away the internal IDs that always differ between two copies of an agent.

CI

Because everything is files plus a CLI, CI is ordinary: validate on every pull request, eval before merge. The template ships a GitHub Actions workflow that does the first part: set DOCANA_API_URL and DOCANA_API_KEY as repository secrets and it runs docana validate "*" on each PR.

Next steps

• CLI: Everything else the docana command does
• Testing Agents: The eval concepts behind the test registry
• Routines: The scheduled runs a routine entity manages
• Docana MCP: Edit these same files with an AI coding assistant that knows your workspace