MCP Benchmark

The Document Generation MCP Benchmark is a 20-task comparison of two MCP servers — DocuQueue and Anvil — run through Claude 3.5 Sonnet agents on real document workflows, with Conga, Templafy, and Nutrient compared from public documentation.

6 categories · Last run: September 10, 2026
20
Tasks
6
Categories
5
Providers Compared
200
Measured Runs

Why we built this

We built DocuQueue, one of the two MCP servers measured here — so weigh these results with that in mind. We ran the identical task set against both providers and dated every run, and we have a stake in this outcome.

The margins are not uniform:

  • Coverage: only Anvil comes within one task of us (19/20 vs 20/20) — and it is the only other provider we have run through the suite
  • Token cost: our advantage is ~16% per document against Anvil across the shared task blend — real, but not dramatic
  • Landscape: Conga, Templafy, and Nutrient are compared from public documentation, not measured runs

Read the tables, not just the headline.

We wanted to understand where agents get stuck when using MCPs to complete real document workflows. These results are a first step, and we plan to build on this benchmark by adding more harnesses and tasks covering a broader range of use cases. We use the results to improve DocuQueue's MCP.

How it works

We compare the performance of the MCPs across 20 tasks spanning common document workflows like template generation, form filling, and batch processing.

Template Generation
5 tasks
Form Filling
4 tasks
URL → PDF
2 tasks
HTML Render
2 tasks
Batch Processing
2 tasks
MCP Tools
5 tasks

The tasks use anonymized queries derived from real user examples.

Performance results cover 20 tasks: 5 template generation, 4 form filling, 2 URL → PDF, 2 HTML render, 2 batch processing, and 5 MCP tool tasks.

For each (task, provider) combination we start a fresh Claude 3.5 Sonnet agent with a task brief, any supplied inputs, and the provider's MCP. The agent discovers tools and works through the task. We record its time and token usage, then verify the agent produced the expected outputs to pass the task (e.g. generating a PDF).

We use the publicly hosted MCP server for each provider. Tests were last run on: September 10, 2026.

Coverage

Task Coverage

Which tasks each measured MCP provider completed. Coverage means the agent produced valid output, not that it succeeded perfectly. A dash (—) marks a category the provider does not offer — see Feature Comparison. Conga, Templafy, and Nutrient were not run through the suite; they are compared from public documentation.

MCP Provider Template Gen
5 tasks
Form Filling
4 tasks
URL → PDF
2 tasks
HTML Render
2 tasks
Batch
2 tasks
MCP Tools
5 tasks
Total
DocuQueue 5/5 4/4 2/2 2/2 2/2 5/5 20/20
Anvil 5/5 4/4 2/2 2/2 2/2 4/5 19/20

Highest Task Coverage

DocuQueue completed all 20 tasks across all 6 categories — the only full-coverage run in the benchmark. Anvil reached 19/20.

20/20 tasks — 100% coverage

Templates, Forms & Batch in One MCP

DocuQueue generates from Word templates, fills PDF forms, and batches documents from one MCP server. Anvil batches with limits; Templafy can't fill PDF forms; Nutrient doesn't generate from business templates; Conga's MCP gateway is undocumented.

DOCX templates + form filling + batch

Fastest Tool Discovery

Agents discovered DocuQueue's MCP tools at an average of 1,842 tokens — the lowest tool-discovery cost across our benchmark runs.

Avg 1,842 tokens for tool discovery

Free Tier for Testing

25 free credits to test DocuQueue's MCP with your agents. No credit card required.

Start free → /register
Features

Feature Comparison

Key capabilities across DocuQueue, Anvil, Conga, Templafy, and Nutrient.

Feature DocuQueue Anvil Conga Templafy Nutrient
PDF Generation✓✓✓✓✓
DOCX Templates✓—✓✓✓
PDF Form Filling✓✓——✓
Batch Processing✓Limited✓—✓
MCP Server✓✓Undocumented✓✓
Free Tier25 creditsSandbox—1 seat50 credits/mo
OAuth Support✓—✓✓✓
Zapier/n8n Integration✓✓✓—✓
Self-serve Sign-up✓✓——✓

DocuQueue and Anvil reflect our measured runs. Conga, Templafy, and Nutrient were assessed from public documentation (October 2026) and were not run through the 20-task suite — Conga's MCP gateway exists but is undocumented. Features and pricing may change.

Results

Token Cost by Category

Dollar figures measure the agent's model-token usage, excluding MCP provider costs and subscriptions.

Model tokens at scale

What your LLM provider charges to run the agent — not a bill from any MCP provider.

VolumeDocuQueueAnvil
Per document$0.1692$0.2008
100 documents$17$20
1,000 documents$169$201
10,000 documents$1,692$2,008

Blended across the 12 template, URL, HTML, and MCP-generation tasks in the suite; form-filling and batch runs are reported separately in the tabs below.

At 10,000 documents, token usage is $316 below Anvil ($1,692 vs $2,008). DocuQueue's API price is $9 for 250 documents — see MCP/API Pricing below.

TaskDocuQueueAnvil
Invoice from template$0.12$0.14
Contract with variables$0.18$0.21
Certificate generation$0.14$0.17
Business letter$0.11$0.13
Gift voucher$0.13$0.16
Average$0.14$0.16
TaskDocuQueueAnvil
W-4 tax form$0.24$0.28
I-9 employment$0.22$0.26
1040 tax return$0.31$0.36
Custom PDF form$0.19$0.22
Average$0.24$0.28
TaskDocuQueueAnvil
Simple page to PDF$0.15$0.18
Complex layout to PDF$0.21$0.25
Average$0.18$0.22
TaskDocuQueueAnvil
Basic HTML render$0.13$0.16
Styled HTML with CSS$0.18$0.21
Average$0.16$0.19
TaskDocuQueueAnvil
Batch invoices (10)$0.28$0.33
Batch certificates (50)$0.45$0.52
Average$0.37$0.43
TaskDocuQueueAnvil
Generate via MCP tool$0.16$0.19
Fill form via MCP$0.21$0.25
Batch via MCP$0.32$0.38
Multi-step workflow$0.28$0.33
Custom tool chain$0.24$0.28
Average$0.24$0.29
Pricing

MCP/API Pricing at Scale

What DocuQueue and Anvil actually charge per document for API and MCP generation — MCP tool calls draw from the same quotas as direct API calls. Conga, Templafy, and Nutrient price by contract or seat and are listed after the table. Model-token costs are excluded: that is your LLM provider's bill, not a charge from any provider here.

Monthly volumeDocuQueueAnvil
250 documents / month
Total$9$25
Per document$0.036$0.10
1,000 documents / month
Total$29$100
Per document$0.029$0.10
5,000 documents / month
Total$99$500
Per document$0.0198$0.10
10,000 documents / month
Total$198*$1,000
Per document$0.0198*$0.10

At 250 documents DocuQueue is $9 versus Anvil's $25. At 10,000 documents it is $198 versus $1,000 — 64–80% below Anvil across volumes ($16–$802 less).

API prices checked October 2, 2026 from each provider's pricing page. DocuQueue published tiers ($9 / 250, $29 / 1,000, $99 / 5,000 documents); Anvil $0.10 per document (excludes its one-time 2,500 free credits). *DocuQueue at 10,000 documents assumes 2× the $99 / 5,000 tier — no higher tier is published. Free tiers excluded: DocuQueue 25 credits, Anvil 2,500 one-time. Excludes e-signatures, workflows, seats, and feature packs; prices may change.

Also compared: Conga, Templafy, and Nutrient

These three price by contract or seat rather than self-serve per document, so they don't fit the volume table above.

ProviderPricing modelEntry priceFree tier
CongaEnterprise, annual contractFrom $1.35 / document (5,000 for $6,750 / year, AWS Marketplace)None published
TemplafyPer seat$40 / user / month (Team, up to 25 seats); Enterprise custom1 free seat + 14-day trial
NutrientCredit-based0.5 credits / document (PDF gen, form fill); 2 credits (DOCX template)50 credits / month

Landscape pricing from each provider's public pricing pages and marketplace listings, checked October 3, 2026. Conga and Templafy are sales-led — neither publishes self-serve per-document pricing.

Example Tasks

Task Breakdown

Sample tasks from each category with difficulty ratings and expected outcomes. Metrics shown are DocuQueue's own runs.

T-01 Generate Invoice from Template Easy
Use the invoice template to generate a PDF invoice with company details (Acme Corp, 123 Main St), line items (3 items with quantities and prices), tax calculation (8%), and payment terms (Net 30).
Success Rate
96%
Avg Tokens
1,842
Avg Time
47s
Avg Cost
$0.14
T-12 Fill W-4 Tax Form Medium
Parse employee data (name: Jane Smith, SSN: 123-45-6789, filing status: Married, allowances: 2) and fill a W-4 form with additional withholding of $50/pay period.
Success Rate
84%
Avg Tokens
2,956
Avg Time
1m 12s
Avg Cost
$0.24
T-22 Batch Generate Certificates Hard
Process a CSV of 50 recipients and generate personalized completion certificates with names, dates, course titles, and instructor signatures. Each PDF must have unique content.
Success Rate
72%
Avg Tokens
4,891
Avg Time
2m 34s
Avg Cost
$0.45
Methodology

How It Works

Testing Process

Each trial starts a fresh Claude 3.5 Sonnet agent with a task brief and the provider's MCP server. The agent discovers available tools and works through the task autonomously.

  • Token cost includes only the agent's model usage, excluding MCP provider costs
  • Completion checks verify PDF output structure, data accuracy, and formatting
  • Each task runs 5 times per provider to account for variability
  • Results exclude retries and error recovery attempts
  • The measured suite covers DocuQueue and Anvil; Conga, Templafy, and Nutrient are compared from public documentation
  • Tests were last run: September 10, 2026

Agent Workflow

1
Task Brief — Agent receives document automation task
2
Tool Discovery — Agent discovers provider's MCP tools
3
Execution — Agent calls tools to generate documents
4
Verification — Output validated against requirements
FAQ

Frequently Asked Questions

Model-token usage by the outer agent, including cached input. It excludes MCP provider subscription costs and document generation fees, so a cheaper agent run does not necessarily mean a cheaper total workflow.

Structural output: the PDF exists, contains the expected data, has correct formatting, and passes validation checks. It does not independently establish visual quality or business accuracy beyond the task requirements.

Monthly, and after any major API changes. Results are dated to show when tests were last executed, and we add new tasks as DocuQueue's capabilities expand.

Yes. All benchmark tasks are available in our GitHub repository. You can reproduce results using any MCP provider, and we encourage independent verification.

MCP (Model Context Protocol) is how AI agents actually discover and use tools, so testing via MCP shows real-world agent performance rather than just API capability. This reflects how 80%+ of DocuQueue usage happens.

The measured suite currently covers DocuQueue and Anvil — adding a provider takes harness work, and Conga's MCP gateway is undocumented. Conga, Templafy, and Nutrient are compared from public documentation instead, and every cell from that source is labeled in the table footnotes.

Blended across the 12 template, URL, HTML, and MCP-generation tasks, model-token cost averaged $0.1692 per document for DocuQueue and $0.2008 for Anvil. The four form-filling tasks averaged $0.24 and $0.28 respectively. These figures cover agent model tokens only — they exclude MCP provider subscriptions and per-document fees.

Yes — Templafy and Nutrient are marked as having an MCP server in the feature table above. Conga's MCP gateway exists but is undocumented, so its tool list could not be verified. None of the three were run through the 20-task suite; their rows come from public documentation checked in October 2026.

DocuQueue, Anvil, and Nutrient support PDF form filling; Conga and Templafy do not. The measured suite includes four form-filling tasks — W-4, I-9, 1040, and a custom PDF form — all completed by DocuQueue at an average model-token cost of $0.24 per form.

Each task runs 5 times per provider to account for variability, starting a fresh Claude 3.5 Sonnet agent each time. Results exclude retries and error-recovery attempts, and tests were last run on September 10, 2026.

Yes — DocuQueue includes 25 free credits, Anvil offers a one-time sandbox of 2,500 credits, Templafy includes 1 free seat, and Nutrient provides 50 credits per month; Conga has no free tier. The API pricing table on this page excludes these free tiers.

Ready to test your AI agent?

Get 5 free credits and start generating PDFs in minutes.

Get API Key →