AI Agents

“Ranks #2 of 28 on LMArena's agentic arena (score 10), measuring tool-use and multi-step task performance from human preference.”

“Pi already emits good errors messages; I always see Claude Opus 4.8 correct itself in its next attempt when it gets a tool call wrong.”

“Here are the numbers from their bar chart: 1. SWE-bench Pro Model Score (%) GLM-5.2 62.1 GLM-5.1 58.4 Claude Opus 4.8 69.2 GPT-5.5 58.6 Gemini 3.1 Pro 54.2 2. Terminal-Bench 2.1 Model Score (%) GLM-5.2 81.0 GLM-5.1 63.5 Claude Opus 4.8 85.0 GPT-5.5 84.0 Gemini 3.1 Pro 74.0 3. NL2Repo Model Score (%) GLM-5.2 48.9 GLM-5.1 42.7 Claude Opus 4.8 69.7 GPT-5.5 50.7 Gemini 3.1 Pro 33.4 4. DeepSWE Model Score (%) GLM-5.2 46.2 GLM-5.1 18.0 Claude Opus 4.8 58.0 GPT-5.5 70.0 Gemini 3.1 Pro 10.0 5. ProgramB…”

“I'm using Kimi 2.7 and GPT 5.5 at home, Opus (4.8 I think) at work and I really don't see much difference honestly. Sure, Claude might be 90% correct and Kimi only 70% correct but does that matter when 90% isn't enough to make it work autonomously anyways? My workflow is just strict supervision of what's happening, I also edit the agents file with anything I see the model doing that I don't like. My sessions are also short, after any task which is completed, I just kill the session and start a…”

“I ran into a problem at work recently: we are given access to a bunch of models up to a full Claude Opus 4.8, but a monthly budget of 100k tokens. We are also given access to Gemini 3.5 Flash & 3.1 Pro with a daily budget of 50M tokens, but no tool calling. I'd love to hook Claude Code (or Pi) into the Gemini model, but the lack of tool-calling makes it quite difficult. I've been planning out how an intelligent router might be able to use a token-efficient tool-calling model (including a small…”

“Ranks #4 of 28 on LMArena's agentic arena (score 9.1), measuring tool-use and multi-step task performance from human preference.”

“Claude Sonnet 5 delivers close-to-Opus-4.8 agentic performance, and may be the cheaper way to run agents. Now you can run all of them together in one OpenAgents Workspace. No matter where your Sonnet 5 runs, bring them into the same room: Multi-Agent: 5 Sonnet in one thread. Or Sonnet 5 + Codex + Hermes + Cursor... Each sees others' work. Shared Files + Knowledge Base: every agent reads the same repo, docs and collaborates. Multi-Player: invite your team via one URL. Humans + agents, one thread…”

“In our coding evaluations, we found Sonnet 5 is more capable than Sonnet 4.6 (which was an underrated model itself), but is now faster and slightly cheaper. Sonnet 5's performance is comparable to GLM 5.2 in both one-shot coding and agentic ability. However, it's about ~20% less verbose than GLM 5.2 in average code submission sizes, and uses fewer reasoning tokens, which reduces the cost gap and suggests it writes cleaner code. In practice, Sonnet 5 ends up being 40% more expensive and ~2x fast…”

“Claude Sonnet 5 delivers close-to-Opus-4.8 agentic performance, and may be the cheaper way to run agents. Now you can run all of them together in one OpenAgents Workspace. No matter where your Sonnet 5 runs, bring them into the same room: Multi-Agent: 5 Sonnet in one thread. Or Sonnet 5 + Codex + Hermes + Cursor... Each sees others' work. Shared Files + Knowledge Base: every agent reads the same repo, docs and collaborates. Multi-Player: invite your team via one URL. Humans + agents, one thread…”

“the (imperfect) comparison having used both for planning and execution is that GLM5.2 is too jumpy and eager to do things, often to a fault (e.g. deploying using git when it shouldn't) while sonnet 5 was much lazier than any Claude model I have used has been, not adding an addendum to a plan that I asked for, then lying that it did when asked. Looking at the analysis[0] I don't think it's worth it for me. Maybe for others. Fable was certainly much better. [0]: https: artificialanalysis.ai model…”

“In our coding evaluations, we found Sonnet 5 is more capable than Sonnet 4.6 (which was an underrated model itself), but is now faster and slightly cheaper. Sonnet 5's performance is comparable to GLM 5.2 in both one-shot coding and agentic ability. However, it's about ~20% less verbose than GLM 5.2 in average code submission sizes, and uses fewer reasoning tokens, which reduces the cost gap and suggests it writes cleaner code. In practice, Sonnet 5 ends up being 40% more expensive and ~2x fast…”

“Ranks #6 of 28 on LMArena's agentic arena (score 7.7), measuring tool-use and multi-step task performance from human preference.”

“The only thing I've found impacting this, is when you specifically use the "Ultra" thinking reasoning effort, then codex adds a small part to the system prompt to further get the model to use sub-agents. Any other reasoning thinking effort than "Ultra" and this piece is no longer in the system prompt. Seemingly mostly a prompting thing it seems on the surface. GPT-5.5 (and maybe even GPT-5.4) already had (experimental?) support for sub-agents, remember using it even with -spark which I think wa…”

“We did the same thing. It took me a few iterations to get the library right, but overall I used approx 10 billion tokens here across a codex and Claude max plan The first iteration was a rewrite using codex goal but it ended up producing junk The second iteration was fable controlling gpt-5.5 agents to write the code with really strong acceptance tests. This pushed the library in the right direction and is basically what you see here today”

“I mean you can just ask them to do exactly that. Especially with GPT (5.5), I've been having a lot of issues with it just repeatedly stalling out. I had to build a quota monitoring skill so that it'd keep plowing forward until either the task was finished (in some way) or the quota budget was exhausted. I also had issues with the compaction. Codex seems to compact... weirdly, resulting in the agent becoming a newborn after each compaction event. Telling it to use a notes file is basically essen…”

“I mean you can just ask them to do exactly that. Especially with GPT (5.5), I've been having a lot of issues with it just repeatedly stalling out. I had to build a quota monitoring skill so that it'd keep plowing forward until either the task was finished (in some way) or the quota budget was exhausted. I also had issues with the compaction. Codex seems to compact... weirdly, resulting in the agent becoming a newborn after each compaction event. Telling it to use a notes file is basically essen…”

“Very. Fable 5 is incredibly efficient token wise, second only to GPT-5.5 and is far more affordable run-to-run than the pure input ouput costs would suggest. Task adherence, task inference, tool calling and task assessment are all significantly ahead of GPT-5.5, especially as the later strongly degrades the second compaction comes into the mix, I suspect because of OpenAIs obsessive optimisation of reasoning tokens into a hard to read (and thus also hard to compact) mess. Fable 5 meanwhile has…”

“I would be really interested in real life throughput. For an agentic chat situation, we are still on 5.4 - not because of the cost, but it's simply much faster than 5.5 with comparable results. Also we are using gpt-5.4-mini a lot for quick summaries, tldrs etc. In an ideal world we would upgrade 5.4 to 5.6 terra and 5.4 mini to 5.4 luna. But does somebody already have some measurements at least in terms of speed?”

“Ranks #9 of 28 on LMArena's agentic arena (score 5.9), measuring tool-use and multi-step task performance from human preference.”

“By domain, I really meant "tool calling" and "one-shot fluid intelligence" Anthropic models were the original leaders in tool calling and agentic work, even when other models felt significantly smarter in (Claude Sonnet 3.5 vs Gemini 2.5 Pro, for example). OpenAI models were the opposite, starting smart (more correct solutions on the first try) and got better at exploring and iterating with tools in 2026. The latest releases (Opus 4.5+ and GPT 5.4+) excel at both.”

“Yeah, I've run tests similar to this while evaluating gpt 5.4 vs claude 4.6 Claude is more likely to figure out workarounds and get things deleted if I tell it to delete stuff, so it performs much better in this benchmark and I prefer it. GPT is more likely to stop and prompt you "I got an error deleting this, should I try another way?", and since the operator gets more of these prompts, they'll hit continue more withut even reading it, so it ends up being more annoying for the operator and not…”

“> One of the operational headaches we didn’t predict was that large, advanced models like Claude Opus 4.7 or GPT-5.4 can sometimes spend quite a while thinking through a problem, and to our users this can make it look exactly like a hung job. I had the same problem in my recursive agent harness. It would always come back, but it could sometimes take up to 10 minutes depending. I fixed this by adding a required "purpose" argument to every tool and call return event. As the recursive evaluation p…”

“Ranks #8 of 28 on LMArena's agentic arena (score 6.4), measuring tool-use and multi-step task performance from human preference.”

“I’m biased to like it and I don’t. I find it’s way worse than opus 4.8, 5.6-sol:medium or glm 5.2. I’ve subscribed for supergrok anyway cause it’s a good deal but basically just use grok 4.5 for my explore agents commit agents and some smol tasks. I don’t trust it beyond that. Grok build and grok-build-0.1 are both supremely underwhelming. Many factors, one they often don’t “grok” tool use too well, ironically. They also didn’t function well as orchestrators, often getting confused, repeating t…”

“I’m biased to like it and I don’t. I find it’s way worse than opus 4.8, 5.6-sol:medium or glm 5.2. I’ve subscribed for supergrok anyway cause it’s a good deal but basically just use grok 4.5 for my explore agents commit agents and some smol tasks. I don’t trust it beyond that. Grok build and grok-build-0.1 are both supremely underwhelming. Many factors, one they often don’t “grok” tool use too well, ironically. They also didn’t function well as orchestrators, often getting confused, repeating t…”

“I’m biased to like it and I don’t. I find it’s way worse than opus 4.8, 5.6-sol:medium or glm 5.2. I’ve subscribed for supergrok anyway cause it’s a good deal but basically just use grok 4.5 for my explore agents commit agents and some smol tasks. I don’t trust it beyond that. Grok build and grok-build-0.1 are both supremely underwhelming. Many factors, one they often don’t “grok” tool use too well, ironically. They also didn’t function well as orchestrators, often getting confused, repeating t…”

“Interesting. I experimented with Grok 4 for openclaw when they made clear they wanted to bring claw users in the fold. It was (as expected) more verbally fluid than 5.5, but had real trouble with agentic tool calling - the model felt like it hadn't been trained to think of tool calling as one of its primary modalities. I'll give this a try, the speed and the benchmarks look good. In my experience, Grok slightly punches above its weight in language fluidity, and seems to not benchmaxx on coding,…”