Image Understanding

“I agree. It did very well on an extremely challenging task. I asked it to recognize and draw the very faint reflection of what I was wearing, visible in only a tiny black part of a very brightly lit poster behind glass. In addition, the poster itself also happened to contain similar clothing. You can see the reference images and its output in my writeup here: https: medium.com @rviragh gpt-5-6-sol-very-good-image-reco... While a human can focus on the reflection easily, this is an enormous chal…”

“### What version of Codex CLI is running? 0.146.0 ### What subscription do you have? Plus ### Which model were you using? gpt-5-6 Sol High ### What platform is your computer? Microsoft Windows NT 10.0.26200.0 x64 ### What terminal emulator and version are you using (if applicable)? Windows Terminal/PowerShell ### Codex doctor report ### What issue are you seeing? Compaction repeatedly preserves large image payloads, reducing available context In the latest compaction window (242), the preserved…”

“Ranks #5 of 68 on LMArena's vision arena (Elo 1292), based on human preference on image-understanding tasks.”

“Resolved model commandcode/gpt-5.6-luna does not support image input. Configure a vision-capable model for mo... same for muse spark 1.2 and other vision models”

“### Area Provider adapters ### What are you trying to accomplish? I need to route Gemini 3.7 Flash / 3.6 Flash multimodal video understanding requests from client agents (e.g. Hermes or custom AI agents) through the OpenCodeX local proxy, allowing requests that utilize Google's new **Agentic Video Understanding** protocol (). ### What prevents this today? Google has recently released Agentic Video Understanding ([Developer Guide](https://aistudio.google.com/learn/agentic-video-understanding-wit…”

“### Problem or Use Case Analyzing long-form video in Hermes currently relies on client-side frame sampling or multi-frame fan-out (as described in ), which causes high context token usage, heavy CPU bursts during Base64 encoding, and risks gateway payload timeouts on longer clips. Google AI Studio recently introduced **Agentic Video Understanding** for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite ([Official Developer Guide](https://aistudio.google.com/learn/agentic-video-understanding-with-g…”

“Opus 5 clearly frogmaxxed. gemini-3.6-flash runs 2 and 3 responded best to the royal portrait context.”

“The study used a temperature of 0.01. > "Thirteen food photographs were each submitted 495–561 times to four LLM vision APIs (GPT-5.4, Claude Sonnet 4.6, Gemini 2.5 Pro, Gemini 3.1 Pro Preview) using an identical structured prompt adapted from the iAPS automated insulin delivery system (26,904 total queries, temperature 0.01)"”

“Ranks #16 of 68 on LMArena's vision arena (Elo 1275), based on human preference on image-understanding tasks.”

“Discovered while working #2826 (default Codex imagegen to gpt-5.6-luna / low effort, configurable in settings). ## Problem / Goal Codex renders now carry a per-render reasoning `effort` (defaulting to `low`, overridable via `imageGen.codex.effort`). A media job's params snapshot could preserve the effort a failed job used, but the **Edit & Retry** flow has no way to inspect or change it, and the server has no way to clear it back to the default. #2826 deliberately left retry to fall back to the…”

“Resolved model commandcode/gpt-5.6-luna does not support image input. Configure a vision-capable model for mo... same for muse spark 1.2 and other vision models”

“Benchmark [Qwen3.8-27B](https://unsloth.ai/docs/models/qwen3.8) at Q4 on 2× RTX 5060 Ti with `llama-bench`, and use the run to settle how a 27B dense model should actually be split across two cards. ## Target | | | |---|---| | Model | [`unsloth/Qwen3.8-27B-GGUF`](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF) — 27B **dense**, vision + hybrid reasoning, 256K ctx | | Quant | `UD-Q4_K_XL` (17.92 GB), Unsloth Dynamic V3.0 preview — the 4-bit Unsloth's guide recommends | | GPU | 2× RTX 5060 Ti (16…”

“## Why Two workload shapes with opposite requirements are currently served by one always-on GPU: - **Sprinkled interactive traffic** (Discord, chat, vision, classifier, pi turns). Needs a resident model and low TTFT. Served well today by the 4090 running Qwen3.8-27B at 262K context. - **Clumped batch work** (`model-bench` sweeps, bulk classification, embeddings backfills, autonomous queue jobs). Latency-insensitive, preemption-tolerant, and wants a much larger model than the 4090 can hold. The…”

“Qwen3.8-27B is a 27B dense hybrid-attention model with a vision tower and an MTP draft head. It fits one H200 GPU at the full 262144 context, so it queues faster than any recipe that needs a whole node. | | Measured | | --- | --- | | One caller | 122.1 tok/s | | Concurrency 1024 | 3957.3 tok/s, still rising | | Context | 262144, the checkpoint maximum | | KV cache | 1,804,253 tokens, 6.88 full-length requests at once | | Hardware | 1 H200 GPU, TP1, no NCCL | Scope - New recipe at recipes/Qwen3.…”

“Qwen3.8-27B is a 27B dense hybrid-attention model with a vision tower and an MTP draft head. It fits one H200 GPU at the full 262144 context, so it queues faster than any recipe that needs a whole node. | | Measured | | --- | --- | | One caller | 122.1 tok/s | | Concurrency 1024 | 3957.3 tok/s, still rising | | Context | 262144, the checkpoint maximum | | KV cache | 1,804,253 tokens, 6.88 full-length requests at once | | Hardware | 1 H200 GPU, TP1, no NCCL | Scope - New recipe at recipes/Qwen3.…”

“Hello, vision is working fine for me, when I upload separated image file, however, when the VSCode browser tool is sending screenshot, getting this error: Launching command: `.\ninfer-serve.exe models\qwen3_8_27b_nvfp4.ninfer --model-id qwen3.8-27b --max-context 220000 --default-max-tokens 220000 --kv-dtype int8 --lm-head-draft --spec mtp --draft-tokens 3 --lm-head-draft --host 127.0.0.1 --port 8081 --cors --preserve-thinking --webui --max-pending-requests 50 --pending-timeout-ms 3000000 --visi…”

“If you use a good harness or add the right tools and plugins both the image and web search issues mentioned are non-issues. oh-my-pi (omp.sh) handles images for text models out of the box - as long as you have any vision capable provider enabled, it will be used when you paste images to a text model. Rather than let it guess I configured it to use MiniMax M3 for this task (as well as other utility tasks like code exploration & library functions). opencode has plugins that do the same thing, but…”

“That's right, but there are other recent open weights and relatively big LLMs that are multimodal, e.g. MiniMax-M3. With open weights LLMs, it is affordable to use many different models, each for whatever it is better. Moreover, for analyzing "UIs, photos, screenshots, etc." there are small models that can be run locally on smartphones or laptops, e.g. IBM granite-vision-4.1-4B, certain Google Gemma 4 variants and certain Qwen variants, whose output you can use as input for a big LLM, in order…”

“### Bug type Regression (previous MiniMax VL routing fix no longer covers the newly image-capable M3 catalog entry) ### Beta release blocker No ### Summary OpenClaw 2026.7.2-beta.5 (`ee929db`) marks `minimax/MiniMax-M3` as accepting `input: ["text", "image"]` while the provider uses MiniMax's Anthropic-compatible endpoint. OpenClaw therefore treats the active model as natively vision-capable and the `image` tool returns the image to M3 as `Loaded 1 image for direct visual inspection` instead of…”

“## Problem `dsh-vision-toolkit` (and upstream `vision_client.py`) hard-requires an OpenAI-compatible endpoint: it POSTs `{baseUrl}/chat/completions` with `image_url` content blocks. But some subscriptions expose their best vision models **only on the Anthropic protocol**: - **OpenCode Go** ($10/mo) serves `Qwen3.7 Plus` / `Qwen3.7 Max` / `Qwen3.8 Max` / `MiniMax M3·M2.7·M2.5` exclusively at `https://opencode.ai/zen/go/v1/messages` (Anthropic Messages format), while the cheaper vision models on…”