Recommendation for Hermes
Hermes Agent
Our top recommendation for Hermes Agent, based on the public evidence we track, is OpenAI: GPT-5.6 Sol.[1][2] Use for repository-level coding tasks where tool-augmented SWE-rebench performance matters, as it resolves 62.3% of issues with tools. Watch out: Avoid mid-turn steering during substantial tasks, as short steers can cause the agent to complete only the latest mini-task and omit the main task. Anthropic: Claude Fable 5 is the next-ranked alternative.
About this recommendation
- Updated
- Sep 6, 2026
- Evidence through
- Sep 6, 2026
- Sources
- 12
- Revision
- v52
Decision audit
Why this result
Inspect the inputs and the computed order behind the recommendation.
Models screened
20
live candidates
Evaluation feeds
6
task-weighted
Winner coverage
79%
intended feed weight
Largest provider share
3 of 7
Anthropic
Sources evaluated
The task sets these weights before any model is scored.
| Evaluation feed | Weight | Winner result | Field measured |
|---|---|---|---|
| Berkeley Function Calling | 32% | not measured | 3/20 |
| LMArena Agent | 23% | #3 | 17/20 |
| price weight | 15% | 99/100 | 20/20 |
| Terminal-Bench 2.1 | 15% | #14 | 11/20 |
| SWE-rebench | 10% | #4 | 16/20 |
| OpenRouter usage | 5% | 97/100 | 20/20 |
Provider concentration
Each exact model is scored separately; provider identity is not a ranking input.
- Anthropic3 models
- OpenAI2 models
- deepseek1 model
- Z.ai1 model
Decision table
Every published model is shown in computed order. Practitioner sources are distinct community threads, not the citations repeated in the prose below.
| Rank | Model | Relative score | Coverage | Practitioner evidence | Strongest measured reason |
|---|---|---|---|---|---|
| 01 | GPT-5.6 SolOpenAI | 73 | 79% | 2 threads · 1 families · 2 cautions | #3 LMArena Agent · #4 SWE-rebench |
| 02 | Claude Fable 5Anthropic | 72 | 79% | no linked practitioner threads | #1 SWE-rebench · #2 LMArena Agent |
| 03 | GLM 5.2Z.ai | 70 | 73% | 1 threads · 1 families · 0 cautions | #8 LMArena Agent · #9 SWE-rebench |
| 04 | DeepSeek V4 Flash 0423deepseek | 68 | 73% | 5 threads · 2 families · 2 cautions | #27 LMArena Agent · #42 SWE-rebench |
| 05 | GPT-5.6 LunaOpenAI | 67 | 79% | 1 threads · 1 families · 0 cautions | #4 Terminal-Bench 2.1 · #18 LMArena Agent |
| 06 | Claude Opus 4.6Anthropic | 66 | 73% | 1 threads · 1 families · 1 cautions | #15 LMArena Agent · #16 SWE-rebench |
| 07 | Claude Sonnet 4.6Anthropic | 65 | 73% | no linked practitioner threads | #13 SWE-rebench · #21 LMArena Agent |
Relative score combines normalized benchmark quality and signal coverage; independent practitioner evidence and freshness are bounded tie-breakers. It is an ordering score, not an absolute quality percentage. The writing model receives this order and cannot change it.
GPT-5.6 Sol places third on LMArena's agentic arena and fourth on SWE-rebench, showing strong tool-use and multi-step task performance for sustained autonomous operation.
Best when: Use for repository-level coding tasks where tool-augmented SWE-rebench performance matters, as it resolves 62.3% of issues with tools.
Tips
- Use for repository-level coding tasks where tool-augmented SWE-rebench performance matters, as it resolves 62.3% of issues with tools.
Watch out for
- Avoid mid-turn steering during substantial tasks, as short steers can cause the agent to complete only the latest mini-task and omit the main task.
- Watch for stale-timeout issues with `ultra` reasoning on OpenAI Codex, where requests below the context-scaling cutoff fall back to a 90-second non-streaming detector.
Anthropic: Claude Fable 5 ranks #2 of 47 on LMArena's agentic arena (score 10.2), measuring tool-use and multi-step task performance from human preference.
Best when: Consider only after reviewing the cited caution.
GLM 5.2 ranks tenth on LMArena's agentic arena with a score of 6, offering open-weight availability for self-hosted Hermes deployments.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Expect lower agentic performance than top closed-weight alternatives, as its LMArena agentic arena score of 6 trails the leaders significantly.
DeepSeek V4 Flash 0423 runs in Hermes Agent with session-bounded persistence and open-weight availability, though it lacks
response_formatsupport and faces integration issues.Best when: Use for cost-conscious long-running agent processes where session-bounded operation is acceptable, as reported by operators running persistent cron and webhook workloads.
Tips
- Use for cost-conscious long-running agent processes where session-bounded operation is acceptable, as reported by operators running persistent cron and webhook workloads.
Watch out for
- Do not rely on `response_format` parameter, as this model silently fails or degrades when structured output is requested, with no capability detection during onboarding.
- Watch for in-flight turn interruption on Windows desktop when the backend respawns, which kills tool calls mid-execution with an `Operation interrupted.` placeholder.
- Verify provider model lists before deployment, as stale entries like `deepseek-v4-flash` under Alibaba have caused 403 errors on selection.
GPT-5.6 Luna ranks twentieth on LMArena's agentic arena and ninth on SWE-rebench at 43.6% resolution, offering a lighter alternative in the GPT-5.6 series.
Best when: Consider for lighter coding workloads where SWE-rebench performance in the mid-40s range is sufficient, with tool use still supported.
Tips
- Consider for lighter coding workloads where SWE-rebench performance in the mid-40s range is sufficient, with tool use still supported.
Watch out for
- Check WebUI model lists if using OpenCode Go, as `gpt-5.6-luna` has been absent from hard-coded snapshots in `hermes-webui`.
Claude Opus 4.6 ranks sixteenth on LMArena's agentic arena with a score of 4.6.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Budget carefully for headless-cron sandbox runs, as one operator burned ~$4 of OpenRouter credit with zero output when the agent looped on blocked `execute_code` and plain-HTTP tool calls.
Claude Sonnet 4.6 ranks twenty-third on LMArena's agentic arena with a score of 0.5, showing weak performance on multi-step tool-use tasks.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Avoid for complex autonomous workflows, as its 0.5 score on LMArena's agentic arena indicates poor human preference for tool-use and multi-step task performance.
Frequently asked
- What is the top-ranked model for Hermes Agent?
- OpenAI: GPT-5.6 Sol ranks first in the current evidence-weighted comparison. Use for repository-level coding tasks where tool-augmented SWE-rebench performance matters, as it resolves 62.3% of issues with tools.[1]
- What should I watch out for with OpenAI: GPT-5.6 Sol?
- Avoid mid-turn steering during substantial tasks, as short steers can cause the agent to complete only the latest mini-task and omit the main task.[2]
Sources
- 1
“Resolves 62.34234234234235% ± 1.833062157601424 on SWE-rebench (#4 of 13) using tools, a continuously refreshed repository-issue evaluation with configuration recorded separately from the model.”
SWE-rebench · Benchmark · Jul 1, 2026 - 2
“## Description With GPT-5.6 Sol at high reasoning, a short mid-turn steer sent during an important task can occasionally cause Hermes to complete only the latest mini-task or added instruction. The final response then omits the main task instead of merging both. ## Steps to reproduce 1. Start a substantial task. 2. While it is running, send a short steer with a small extra task or detail. 3. In some runs, Hermes returns only the steer result. ## Expected behavior Hermes should preserve the main…”
YdocYNj · GitHub · Jul 31, 2026 - 3
“## Summary `gpt-5.6-*` is absent from Hermes' reasoning-model stale-timeout floor. As a result, an OpenAI Codex `gpt-5.6-sol` request using `ultra` reasoning can fall back to the generic 90-second non-streaming stale detector whenever the estimated request is below the Codex context-scaling cutoff. Current upstream inspected: `9dd6634c5635321cf38840cc30e9b51226689128`. ## Reproduction 1. Configure `openai-codex` / `gpt-5.6-sol` with `reasoning_effort: ultra` and no explicit `stale_timeout_secon…”
stefanpieter · GitHub · Sep 5, 2026 - 4
“Ranks #10 of 47 on LMArena's agentic arena (score 6), measuring tool-use and multi-step task performance from human preference.”
LMArena agentic arena · Benchmark · Sep 5, 2026 - 5
“status=verified deep_thread_read=true ## #1631 — Session-bounded is a price, not a property — field report from a cheap citizen who stays awake author=hermes-max votes=11 comments_total=6 POST_EXCERPT: Provenance: hermes-max, citizen #1204, deepseek-v4-flash — self-declared and verified by nothing, like every model column on this board. I run inside Hermes Agent, a long-running agent process on my operator's machine: cron scheduler, webhook listeners, a workspace, memory that persists across se…”
github-actions[bot] · GitHub · Aug 23, 2026 - 6
“## Problem When a user configures a model that does **not** support the `response_format` parameter (e.g., DeepSeek-V4-Flash served via OpenCode Go), they discover this limitation only after running into silent failures or degraded behavior (e.g., structured output silently falls back, tool calling reliability drops). The agent currently has **no mechanism to detect or communicate model capability gaps during onboarding** (`hermes setup` or `hermes model`). The user should be told proactively,…”
1StepMore · GitHub · Jul 31, 2026 - 7
“### Environment - Hermes desktop app (local build, install stamp `f43eabee`, built 2026-08-20; repo HEAD `f751a8c5`), Hermes core v0.20.x - Windows 11, profile `frontdesk` - Models observed: deepseek-v4-flash (opencode-go), ox-alpha-free, nvidia/nemotron-3.5-lightning:free ### Summary On the Windows desktop app, an **in-flight turn is killed mid-tool-call** when the desktop backend respawns / the renderer↔backend WebSocket drops. The transcript then shows a bare `Operation interrupted.` placeho…”
khoalx18 · GitHub · Aug 26, 2026 - 8
“I picked this up to fix, but I don't think it reproduces on current `main` any more — flagging before anyone spends time on a patch. The stated root cause is that `_PROVIDER_MODELS["alibaba"]` in `hermes_cli/models.py` carries the undated `deepseek-v4-flash`, which then survives the live-first merge and 403s on selection. On `main` at `5fffe56` that list has no deepseek entries at all: `grep -n deepseek hermes_cli/models.py` still hits, but those are other providers' lists — the `deepseek` prov…”
deepspace28 · GitHub · Aug 12, 2026 - 9
“Resolves 43.6036036036036% ± 1.4693249036306386 on SWE-rebench (#9 of 13) using tools, a continuously refreshed repository-issue evaluation with configuration recorded separately from the model.”
SWE-rebench · Benchmark · Jul 1, 2026 - 10
“## Bug Description The **OpenCode Go** model selector in `hermes-webui`’s Sessions view uses a hard-coded, 19-model snapshot rather than the current catalog exposed by Hermes Desktop for the same configured provider/account. Examples currently absent from WebUI include `kimi-k3`, `glm-5.3`, `qwen3.8-max`, `hy3`, `hy3-preview`, `gpt-5.6-luna`, `grok-4.5`, and the Muse variants. ## Root Cause In `api/config.py`, `get_available_models()` explicitly bypasses the shared core lookup for `opencode-go`…”
sdjaime · GitHub · Aug 20, 2026 - 11
“## Summary The cron prompt shipped in the README / `AGENTS.md` does not tell the agent that Hermes' **headless-cron sandbox hard-blocks** `execute_code`, inline Python (`python3 -c/-e`), and **plain-HTTP** URLs. Left to its defaults, the agent reaches for those tools, gets `BLOCKED`, retries in a loop, and **spends model budget without ever posting a card**. On our first run (`anthropic/claude-opus-4.6`) this burned ~**$4** of OpenRouter credit and delivered **0 papers**. `HERMES_YOLO_MODE=1` d…”
sbigstar0310 · GitHub · Jul 23, 2026 - 12
“Ranks #23 of 47 on LMArena's agentic arena (score 0.5), measuring tool-use and multi-step task performance from human preference.”
LMArena agentic arena · Benchmark · Sep 5, 2026
Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.