Recommendation for JSON & schema
JSON & Schema Output
Our top recommendation for JSON & Schema Output, based on the public evidence we track, is OpenAI: GPT-5.6 Luna.[1][2] Use as the automatic fallback in extraction pipelines when free tier models hit rate limits, since it was specifically chosen for validated record generation against a documented schema. Watch out: Watch for ordinal turn-tracking failures in debate coordination flows, where the model lost step sequencing compared to prior defaults. OpenAI: GPT-5.6 Sol is the next-ranked alternative.
About this recommendation
- Updated
- Sep 4, 2026
- Evidence through
- Sep 4, 2026
- Sources
- 11
- Revision
- v56
Decision audit
Why this result
Inspect the inputs and the computed order behind the recommendation.
Models screened
20
live candidates
Evaluation feeds
6
task-weighted
Winner coverage
42%
intended feed weight
Largest provider share
4 of 8
Anthropic
Sources evaluated
The task sets these weights before any model is scored.
| Evaluation feed | Weight | Winner result | Field measured |
|---|---|---|---|
| Structured-output evalunavailable | 40% | feed unavailable | 0/20 |
| LiveBench Data Analysis | 20% | #14 | 18/20 |
| Berkeley Function Calling | 15% | not measured | 3/20 |
| LMArena Text | 10% | #36 | 19/20 |
| price weight | 10% | 100/100 | 20/20 |
| OpenRouter usage | 5% | 100/100 | 20/20 |
Provider concentration
Each exact model is scored separately; provider identity is not a ranking input.
- Anthropic4 models
- OpenAI2 models
- Meta1 model
- Z.ai1 model
Decision table
Every published model is shown in computed order. Practitioner sources are distinct community threads, not the citations repeated in the prose below.
| Rank | Model | Relative score | Coverage | Practitioner evidence | Strongest measured reason |
|---|---|---|---|---|---|
| 01 | GPT-5.6 LunaOpenAI | 60 | 42% | 5 threads · 5 families · 2 cautions | #14 LiveBench Data Analysis · #36 LMArena Text |
| 02 | GPT-5.6 SolOpenAI | 59 | 42% | 1 threads · 1 families · 0 cautions | #3 LiveBench Data Analysis · #10 LMArena Text |
| 03 | Claude Fable 5Anthropic | 59 | 42% | 1 threads · 1 families · 1 cautions | #1 LiveBench Data Analysis · #1 LMArena Text |
| 04 | Claude Sonnet 4.6Anthropic | 58 | 42% | 2 threads · 2 families · 2 cautions | #15 LiveBench Data Analysis · #20 LMArena Text |
| 05 | Muse Spark 1.2Meta | 57 | 42% | 1 threads · 1 families · 0 cautions | #4 LMArena Text · #17 LiveBench Data Analysis |
| 06 | Claude Opus 4.6Anthropic | 54 | 42% | no linked practitioner threads | #2 LMArena Text · #35 LiveBench Data Analysis |
| 07 | Claude Sonnet 4.5Anthropic | 54 | 38% | 2 threads · 2 families · 1 cautions | #2 Berkeley Function Calling · #33 LMArena Text |
| 08 | GLM 4.6Z.ai | 51 | 38% | no linked practitioner threads | #3 Berkeley Function Calling · #57 LMArena Text |
Relative score combines normalized benchmark quality and signal coverage; independent practitioner evidence and freshness are bounded tie-breakers. It is an ordering score, not an absolute quality percentage. The writing model receives this order and cannot change it.
GPT-5.6 Luna is explicitly selected for planning and extraction workflows requiring validated JSON output, with fallback handling built around its reliability for schema-conforming responses.
Best when: Use as the automatic fallback in extraction pipelines when free tier models hit rate limits, since it was specifically chosen for validated record generation against a documented schema.
Tips
- Use as the automatic fallback in extraction pipelines when free tier models hit rate limits, since it was specifically chosen for validated record generation against a documented schema.
- Deploy for planning tasks where the model selection has been production-approved and environment variables already point to this endpoint.
Watch out for
- Watch for ordinal turn-tracking failures in debate coordination flows, where the model lost step sequencing compared to prior defaults.
- Account for judge skew when using Luna for evaluation: it asymmetrically promotes borderline citations to explicit, inflating strict-match metrics.
OpenAI: GPT-5.6 Sol scores 79.84% on LiveBench Data Analysis (#4 of 51), including table joining and reformatting tasks.
Best when: Consider only after reviewing the cited caution.
Claude Fable 5 leads LMArena overall rankings and scores 80.54% on LiveBench Data Analysis, yet community reports indicate structured output support was removed in recent versions.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Avoid for direct schema-constrained JSON generation: multiple users confirm Fable, Opus 4.7, and 4.8 dropped structured output support that existed in earlier versions.
Claude Sonnet 4.6 is the only Anthropic model in the Dify plugin with declared structured-output support, though the implementation currently falls back to prompt-based JSON without native schema validation.
Best when: Select when you need the only Anthropic model with explicit structured-output feature declaration in major integration frameworks, even if native handling is pending.
Tips
- Select when you need the only Anthropic model with explicit structured-output feature declaration in major integration frameworks, even if native handling is pending.
Watch out for
- Implement client-side validation: the Dify plugin declares the feature but contains zero handling for json_schema or output_format, so responses rely on prompt engineering and post-hoc parsing.
- Expect lower data analysis performance than top-tier alternatives: it ranks 18th of 51 on LiveBench Data Analysis.
Muse Spark 1.2 is deployed in production extraction pipelines as a second-stage validator to catch omissions and correct weakened statements in initial outputs.
Best when: Add as a validation layer after initial extraction to scan for missed list items and named properties, then correct weakened tensor notation against source text.
Tips
- Add as a validation layer after initial extraction to scan for missed list items and named properties, then correct weakened tensor notation against source text.
Watch out for
- Note benchmark limitations: it ranks 20th of 51 on LiveBench Data Analysis, below several alternatives for table reformatting tasks.
Anthropic: Claude Opus 4.6 ranks #2 of 144 on LMArena's overall text arena (Elo 1505), based on blind human preference votes.
Best when: Consider only after reviewing the cited caution.
Claude Sonnet 4.5 scores 73.24% on BFCL format-sensitive tool calling but requires strict mode enforcement to guarantee array types in schema outputs.
Best when: Enable strict: true in Bedrock Converse tool specifications to enforce constrained decoding and prevent type mismatches like string-where-array-expected in sections fields.
Tips
- Enable strict: true in Bedrock Converse tool specifications to enforce constrained decoding and prevent type mismatches like string-where-array-expected in sections fields.
Watch out for
- Without strict mode, anticipate schema violations: AWS documentation confirms non-strict tool use permits type mismatches and missing required fields.
- Confirm feature availability: unlike Sonnet 4.6, this version lacks structured-output declaration in the Dify Anthropic plugin.
Z.ai: GLM 4.6 scores 72.38% overall on BFCL (#3 of 32), measuring single-turn, multi-turn, web-search, memory, hallucination, and format-sensitive tool calling; mean latency was 4.34s.
Best when: Consider only after reviewing the cited caution.
Frequently asked
- What is the top-ranked model for JSON & Schema Output?
- OpenAI: GPT-5.6 Luna ranks first in the current evidence-weighted comparison. Use as the automatic fallback in extraction pipelines when free tier models hit rate limits, since it was specifically chosen for validated record generation against a documented schema.[1]
- What should I watch out for with OpenAI: GPT-5.6 Luna?
- Watch for ordinal turn-tracking failures in debate coordination flows, where the model lost step sequencing compared to prior defaults.[2]
Sources
- 1
“> *This was generated by AI during triage.* ## Parent #205 ## What to build A client module that takes document text and returns validated extraction records conforming to the schema in `docs/protocols/extraction-schema-by-document-type.md`. Two models: **Nemotron 3 Ultra (free)** as default, **GPT-5.6-Luna (~$4 for the full backlog)** as automatic fallback on rate-limit or unavailability. Via OpenRouter. Four things that already bit during evaluation and must be handled: 1. **Retry with backof…”
alexwolson · GitHub · Aug 6, 2026 - 2
“## Background While implementing #14 (renaming the Gemini-branded OpenRouter node), changing `OPENROUTER_MODEL`'s default from `google/gemini-3.5-flash-lite` to `openai/gpt-5.6-luna` broke 3 existing real end-to-end debate-coordinator tests. Investigation (running the real debate flow manually, see PR history for tickets #15/#16) found two distinct behavior differences from the prior default: 1. **Ordinal turn-tracking is unreliable.** Given a topic instructing "Step 1 (now)... Step 2 (on your…”
Weilun2022 · GitHub · Jul 28, 2026 - 3
“## 決定事項 + 追加作業（2026-08-17） - **planning モデルは gpt-5.6-luna で確定**（ユーザー承認済み）。`.env` の `LLM_MODEL` を切替済み - **分類レイヤの方針**: 専用小型モデルは置かない。実測 — 正規表現層ヒット時 2ms / LLMフォールバック（Luna）1.3s。分類コール自体のコストは 〜0.01円で無視可能、実コストは音声ループに直列で乗る 1.3s のレイテンシ → T-139 スコープに「単純ターンの分類+応答 1コール統合（または並行実行）」を最適化項目として含める。層構成は「regex(0ms) → Luna分類 → Luna生成」で維持 - **プロンプトチューニングUIを追加**: `apps/ai/eval/ui/index.html`（タスク分解/意図分類/チャットの3タブ、dialogues.json プリセット、operations テーブル・priority チップ・warnings の整形表示、raw JSON 折りたたみ、実行メモ付き履歴）。起動は `apps/ai/dev.…”
devacxx01 · GitHub · Aug 17, 2026 - 4
“## Amendment (from the parallel 70B session's findings) Two corrections that affect this spec's validity, both measured rather than assumed. **1. Judge skew is real and directional — comparisons must be Luna-vs-Luna.** Comparing gpt-5.5 and gpt-5.6-luna judgments over the same stored 70B generations, disagreements are asymmetric: 63 hedged→explicit versus 27 explicit→hedged. Luna promotes borderline citations to `explicit`, inflating `v_strict`. Observed consequence: the speech-act run's author…”
redagavin · GitHub · Jul 29, 2026 - 5
“I dont get why Opus 4.7, 4.8, and now Fable all stopped supporting structured outputs? Does no one else care about that? I find it incredibly useful to reliably pass LLM output directly to other APIs libraries”
coreylane · Hacker News · Jun 9, 2026 - 6
“Hi @Mabdurahman68! I'm [Dosu](https://go.dosu.dev/dosubot) and I'm helping the Dify team. Your analysis is confirmed by the code. `claude-sonnet-4-6.yaml` is the only model file in the Anthropic plugin that declares `structured-output` in its features list [[1]](https://github.com/langgenius/dify-official-plugins/blob/93f1deb4fa4d06fcf81f7843fd55a7fbd6ce2b27/models/anthropic/models/llm/claude-sonnet-4-6.yaml#L5-L11), while `claude-sonnet-4-5-20250929.yaml` and other models do not [[2]](https://…”
dosubot[bot] · GitHub · Aug 18, 2026 - 7
“Concrete instance filed separately: langgenius/dify-official-plugins#3671 `claude-sonnet-4-6` in `langgenius/anthropic` v0.3.26 declares `structured-output` in its model YAML while `models/llm/llm.py` contains no handling for it — zero matches for `json_schema`, `output_format`, `structured`, or `structured-outputs-2025-11-13`. That is what makes the branch above observable today: the declaration is sufficient to take the native path, the plugin never reads the schema, and the fallback has alre…”
Mabdurahman68 · GitHub · Aug 18, 2026 - 8
“Scores 77.95% on LiveBench Data Analysis (#18 of 51), including table joining and reformatting tasks.”
LiveBench Data Analysis · Benchmark · Jun 25, 2026 - 9
“## Problem Statement 当前 Entry Module 单阶段提取在 5 案例上均值 ~36.2（Sol 等效），距 flawless 档（≥40）差 4 分。剩余扣分第一来源是漏提取与陈述弱化：Skein1 的函子公理清单、RT 的 handle-slide 命名性质等以 prose 清单/命名性质形式存在的内容被跳过；部分已提取条目的平衡张量积记号被弱化、或幻觉出原文没有的边界条件。连续 4 个提示词单变量实验（v1.32–1.35）证明在同一容量内加规则只会搬分数，净零。 ## Solution 在现有工作流（MinerU → v1.31 提取 → 确定性合并 → 产物）后新增第二阶段：用 muse-spark-1.2 对初版产物做校验补漏。阶段二输入为初版产物 + 源文本全文，仅做两件事：1) 扫描源文本中以清单/命名性质形式存在但被初版遗漏的条目，补 1–3 条；2) 逐条比对初版中被评委判为弱化的陈述与源文本原文，逐字订正。不改提示词，不改窗口。 ## User Stories 1. As a benchmark operator, I want the…”
cyw6130 · GitHub · Aug 21, 2026 - 10
“Scores 76.46% on LiveBench Data Analysis (#20 of 51), including table joining and reformatting tasks.”
LiveBench Data Analysis · Benchmark · Jun 25, 2026 - 11
“## 背景 #111 で経路 B の応答を tool use で受け取るようにしたが、2026-08-22 23:09 の再投入 (2608.17950v1) で 12 ページ目がモデルの出力 `sections` が配列ではなく文字列で返り、`PageDecodeError` になった (#113 の再試行でも同じ)。 tool use の `inputSchema` は `strict` を付けないと型を保証しない。公式ドキュメントは「strict なしでは型の不一致や必須欠落が起こり得る」と述べ、解決策として `strict: true` (constrained decoding) を提示している。 Claude Sonnet 4.5 は Bedrock Converse (`bedrock-runtime`、cross-region inference 含む) で対象。 - [ToolSpecification API Reference](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime…”
tamaco489 · GitHub · Aug 22, 2026
Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.