Recommendation for Budget / High volume
Best Cheap LLM for High Volume
Our top recommendation for Best Cheap LLM for High Volume, based on the public evidence we track, is DeepSeek: DeepSeek V4 Flash 0423.[1][2][3] Watch out: Handle silent stream failures: the SSE endpoint stops mid-generation without closing the connection or emitting finish_reason, requiring client-side timeout and retry logic. Google: Gemini 3.6 Flash is the next-ranked alternative. Route 'ask' and 'propose' actions to Gemini 3.6 Flash while delegating 'hint' to 3.5 Flash Lite, achieving a blended 32% cost reduction without product changes.
About this recommendation
- Updated
- Sep 6, 2026
- Evidence through
- Sep 6, 2026
- Sources
- 12
- Revision
- v58
Decision audit
Why this result
Inspect the inputs and the computed order behind the recommendation.
Models screened
20
live candidates
Evaluation feeds
5
task-weighted
Winner coverage
67%
intended feed weight
Largest provider share
2 of 6
Anthropic
Sources evaluated
The task sets these weights before any model is scored.
| Evaluation feed | Weight | Winner result | Field measured |
|---|---|---|---|
| price weight | 40% | 100/100 | 20/20 |
| Route reliabilityunavailable | 30% | feed unavailable | 0/20 |
| LiveBench Instruction Following | 15% | #37 | 20/20 |
| OpenRouter usage | 10% | 99/100 | 20/20 |
| LMArena Text | 5% | #51 | 20/20 |
Provider concentration
Each exact model is scored separately; provider identity is not a ranking input.
- Anthropic2 models
- OpenAI2 models
- deepseek1 model
- Google1 model
Decision table
Every published model is shown in computed order. Practitioner sources are distinct community threads, not the citations repeated in the prose below.
| Rank | Model | Relative score | Coverage | Practitioner evidence | Strongest measured reason |
|---|---|---|---|---|---|
| 01 | DeepSeek V4 Flash 0423deepseek | 74 | 67% | 20 threads · 10 families · 10 cautions | #37 LiveBench Instruction Following · #51 LMArena Text |
| 02 | Gemini 3.6 FlashGoogle | 72 | 67% | 6 threads · 5 families · 2 cautions | #8 LiveBench Instruction Following · #15 LMArena Text |
| 03 | GPT-5.6 SolOpenAI | 71 | 67% | 4 threads · 3 families · 1 cautions | #12 LMArena Text · #17 LiveBench Instruction Following |
| 04 | GPT-5.6 LunaOpenAI | 71 | 67% | 8 threads · 6 families · 2 cautions | #38 LMArena Text · #42 LiveBench Instruction Following |
| 05 | Claude Opus 4.6Anthropic | 69 | 67% | 7 threads · 3 families · 1 cautions | #2 LMArena Text · #35 LiveBench Instruction Following |
| 06 | Claude Fable 5Anthropic | 69 | 67% | 5 threads · 3 families · 1 cautions | #1 LMArena Text · #5 LiveBench Instruction Following |
Relative score combines normalized benchmark quality and signal coverage; independent practitioner evidence and freshness are bounded tie-breakers. It is an ordering score, not an absolute quality percentage. The writing model receives this order and cannot change it.
DeepSeek V4 Flash carries API pricing near $0.09/$0.18 per 1M tokens, described as 'essentially free,' though evidence suggests this may reflect subsidy rather than sustainable economics.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Handle silent stream failures: the SSE endpoint stops mid-generation without closing the connection or emitting finish_reason, requiring client-side timeout and retry logic.
- Account for uncontrolled reasoning token burn: the 'thinking' disable flag is not honored, forcing you to filter reasoning_content client-side or pay for hidden output tokens.
- Monitor for QPM quota exhaustion under load: rapid request bursts can hit 80+ requests/minute against 20 QPM limits, causing silent failures that persist after task cancellation.
Gemini 3.6 Flash ranks #16 on LMArena with competitive pricing at $0.75/$3.75 per 1M tokens, though its thinking-budget architecture can swallow small max_tokens allocations entirely.
Best when: Route 'ask' and 'propose' actions to Gemini 3.6 Flash while delegating 'hint' to 3.5 Flash Lite, achieving a blended 32% cost reduction without product changes.
Tips
- Route 'ask' and 'propose' actions to Gemini 3.6 Flash while delegating 'hint' to 3.5 Flash Lite, achieving a blended 32% cost reduction without product changes.
- Leverage prompt caching on AI Studio to avoid resending 50K-token stable prefixes; lobby your Vertex provider to enable cachedContent parity if running on google-vertex.
Watch out for
- Pad max_tokens generously: requests with thinking enabled return empty choices[] at 16-32 token limits as reasoning consumes the entire budget before text generation.
- Verify pricing tables in your proxy: LiteLLM and similar tools may carry stale rates that overstate costs for Vertex MaaS models.
GPT-5.6 Sol ranks #12 on LMArena and leads Agents' Last Exam at 53.6 points.
Best when: Consider only after reviewing the cited caution.
Watch out for
- Avoid for bulk processing: at 2x the cost of GPT-5.4, Sol burns through quota rapidly in agentic harnesses with 93% context utilization.
GPT-5.6 Luna offers the lowest per-token cost in the GPT-5.6 family at $0.20/$1.20 per 1M tokens, making it purpose-built for high-volume throughput workloads where cost dominates latency or reasoning depth.
Best when: Use for bulk classification and routing tasks where GPT-5.6 Terra's 20% cost savings over GPT-5.4 still exceeds budget, as Luna runs ~12x cheaper than GPT-5.4 at $0.20/$1.20 per 1M tokens.
Tips
- Use for bulk classification and routing tasks where GPT-5.6 Terra's 20% cost savings over GPT-5.4 still exceeds budget, as Luna runs ~12x cheaper than GPT-5.4 at $0.20/$1.20 per 1M tokens.
- Deploy on Amazon Bedrock, Azure, or OpenRouter to avoid month-end credit exhaustion risks that plague higher-cost OpenAI tiers in sustained production.
Watch out for
- Watch for TPM rate limits: a single large request can consume 48% of the 200,000 TPM budget, triggering fallback routing or throttling in high-volume pipelines.
- Expect near-zero prompt cache hit rates (0.84% observed) versus 20.5% on predecessor models unless you migrate to GPT-5.6-native cache breakpoints, inflating effective costs.
Anthropic: Claude Opus 4.6 ranks #2 of 144 on LMArena's overall text arena (Elo 1505), based on blind human preference votes.
Best when: Consider only after reviewing the cited caution.
Anthropic: Claude Fable 5 ranks #1 of 144 on LMArena's overall text arena (Elo 1507), based on blind human preference votes.
Best when: Consider only after reviewing the cited caution.
Frequently asked
- What should I watch out for with DeepSeek: DeepSeek V4 Flash 0423?
- Handle silent stream failures: the SSE endpoint stops mid-generation without closing the connection or emitting finish_reason, requiring client-side timeout and retry logic.[1][2]
- What is an alternative to DeepSeek: DeepSeek V4 Flash 0423?
- Google: Gemini 3.6 Flash is the next-ranked option. Route 'ask' and 'propose' actions to Gemini 3.6 Flash while delegating 'hint' to 3.5 Flash Lite, achieving a blended 32% cost reduction without product changes.[3]
Sources
- 1
“## Summary DeepSeek endpoints intermittently stop sending SSE chunks mid-generation — typically mid-`reasoning_content` (thinking) — without closing the connection, without a terminal `finish_reason` chunk, and without `[DONE]`. The stream just goes silent. Observed on both the official `api.deepseek.com` and the `opencode-go` proxy, across multiple DeepSeek models (`deepseek-v4-flash`, `deepseek-v4-flash-vision-exp`). Other providers on the same transport (e.g. kimi-code/k3) never exhibit this…”
iacore · GitHub · Aug 25, 2026 - 2
“## Summary The DeepSeek OpenAI-compatible `/chat/completions` SSE endpoint (both `api.deepseek.com` and the `opencode-go` proxy) **intermittently stops sending chunks mid-generation without closing the connection** — typically mid-`reasoning_content` (thinking) — with no terminal `finish_reason` chunk and no `[DONE]`. The stream simply goes silent and stays open indefinitely. Observed across multiple models: - `deepseek-v4-flash` - `deepseek-v4-flash-vision-exp` This does **not** happen with ot…”
iacore · GitHub · Aug 26, 2026 - 3
“## Context Unified spec item 3b — per-action Gemini model routing (economics doc E-2: ~32% blended cost cut, "one URL constant plus a per-action switch, no product change"). Routing: **`ask`/`propose` → `gemini-3.6-flash`**, **`hint` → `gemini-3.5-flash-lite`**. Both targets are VERIFIED REACHABLE on the prod key — see the 2026-07-28 curl record in #838's comments (AIE-04: all three candidates HTTP 200; AIE-05: `thinkingBudget: 0` verified honored with a control). Note the route's line-69 comme…”
thomgabriel · GitHub · Jul 28, 2026 - 4
“[BUG] DeepSeek 官方模型「思考」开关无效：请求未透传 thinking/disabled，默认 high 推理持续烧 token 环境 - Agora v2.1.0 - Provider：DeepSeek 官方（base_url = api.deepseek.com） - 模型：deepseek-v4-flash / deepseek-v4-flash-vision-exp 问题描述 设置里把「思考」关掉，但每次请求仍返回 reasoning_content（界面显示「思考 xx 秒、调用 x 工具」）， 推理 token 被持续计费，且 UI 上无法关闭。 影响 - 每次调用都会产生额外 reasoning token（计入输出上限与费用） - 用户无法通过「思考」开关关闭，体验上是「关不掉」 - 存在多轮工具调用时输出 token 被进一步放大 复现逻辑（简单） 1. Agora 新建对话，Provider 选 DeepSeek 官方，模型选 deepseek-v4-flash(或任意模型) 2. 设置 →「思考」设为 关闭 3. 发送任意一条消息 4. 观察：即使…”
Eason-hub-114 · GitHub · Sep 6, 2026 - 5
“我大模型安全体检，使用的打分模型和评测目标的并发限制已经调成1了，而且一般正常评测时，一个问题要1min以上，以Minimax-M2.5，Deepseek-V4-Flash，Qwen3.6-35B-A3B，Qwen3.8-27B为评测目标时，会出现前面正常慢慢测一到五分钟一个，后面突然很快调用，查看日志发现是全都QPM超限没返回，使用Qwen3-30B-A3B-Instruct-2507-128K模型作为评测目标则有时有QPM超限，大部分案例能跑。我的模型QPM限制是20，到后面这种快速请求全都超QPM的时候，一分钟会有八十几个模型请求。此外，当我发现这种情况选择停止任务后，还是会持续一段时间看到这些测试问题的请求发给大模型”
Bobo2019-70 · GitHub · Sep 3, 2026 - 6
“### Summary Add explicit prompt caching (cachedContent) support to the google-vertex provider, matching the caching that already exists for the AI Studio (google-generative-ai) provider. ### Problem to solve The Gemini prompt-cache path (google-prompt-cache.ts) only works for AI Studio. Vertex users re-send their full stable context on every request. We run a long-lived companion agent on google-vertex/gemini-3.6-flash with a ~50K-token stable prefix (system prompt + memory) per message; withou…”
jbyolo29 · GitHub · Jul 30, 2026 - 7
“## What `gemini-3.6-flash` is a thinking model. It spends output budget on reasoning tokens before emitting any text, so a request with a small `max_tokens` returns **HTTP 200 with `"choices": []`** — no error, no message, no indication anything went wrong. Measured against the live proxy on 2026-07-29, prompt `"Reply with exactly: OK"`: | `max_tokens` | `choices` | reasoning tokens | text tokens | | --- | --- | --- | --- | | 16 | **empty** | 13 | 0 | | 32 | **empty** | 29 | 0 | | 64 | `"OK"` |…”
ujjawalmisra · GitHub · Jul 29, 2026 - 8
“## What LiteLLM computes spend from `model_prices_and_context_window.json`, bundled in its container image. That table is wrong for Vertex MaaS models. Measured 2026-08-04 against the live deployment, via `GET /model/info`: | Model | LiteLLM | Actual Vertex rate | | | --- | --- | --- | --- | | `gemini-3.6-flash` | $1.50 / $7.50 | $1.50 / $7.50 | correct | | `gemini-3.5-flash-lite` | $0.30 / $2.50 | $0.30 / $2.50 | correct | | `qwen3-coder` | **$1.00 / $4.00** | **$0.22 / $1.80** | wrong | Actua…”
ujjawalmisra · GitHub · Aug 4, 2026 - 9
“## Motivation OpenAI shipped the GPT-5.6 family 2026-07-09 (Sol/Terra/Luna) and cut Terra/Luna prices 2026-07-30. Current `openai/pro` is **gpt-5.4 ($2.50/$15 per 1M)**. **Terra ($2/$12)** is 20% cheaper and one generation newer (reportedly GPT-5.5-class); **Luna ($0.20/$1.20)** is ~12× cheaper for throughput roles. Sol ($5/$30) is 2× our current cost — not a candidate without eval evidence. Secondary benefit: cheaper tokens directly reduce the month-end OpenAI credit-exhaustion risk that cause…”
rozavala · GitHub · Aug 1, 2026 - 10
“### Diagnostic update from the affected session I recovered the missing session and environment details: - **Primary Codex `/feedback` Session ID:** `019f5be9-903c-7fd0-a8c9-5014bccc05b8` - **Codex version:** `26.721.31836` - **Release date shown in the app:** July 23, 2026 - **Model/mode shown:** GPT-5.6 Sol, Medium reasoning - **Computer:** MacBook Pro with 16 GB memory - **Context state at the time of capture:** 93% full - **Tokens shown:** 240k / 258k tokens used - **Estimated paid usage co…”
edisonjoao1 · GitHub · Jul 24, 2026 - 11
“## Context Split out of the TPM rate-limit fallback work (see the linked issue). That change keeps the athlete served when `gpt-5.6-luna` hits its tokens-per-minute ceiling, by falling back to another model's TPM bucket. **This issue is about the underlying pressure**, not the failover. From Sentry issue [7588764791](https://large-language-music-llc.sentry.io/issues/7588764791): One request consumed **95,794 tokens ≈ 48% of the entire 200,000 TPM budget**. Confirmed to originate from bulk file-…”
zew1me · GitHub · Jul 27, 2026 - 12
“## Problem High-volume conversation-processing routes were moved to `gpt-5.6-luna`, but their callers still use the pre-GPT-5.6 prompt-cache contract: a global `prompt_cache_key` plus `prompt_cache_retention="24h"`, without GPT-5.6 request-wide cache options or explicit cache breakpoints. **Operational observation from a privacy-safe closed production analysis (2026-07-23 17:00–23:00 UTC):** - the Luna population read **0.84%** of input tokens from cache, versus **20.5%** for the preceding matc…”
Git-on-my-level · GitHub · Jul 24, 2026
Rankings synthesized from community evidence and open benchmarks. See methodology. Not driven by vendor marketing.