I've tried Command Code (GOAT for $10.78) out for a month but decided to switch back to Opencode Go.
I burned 1.471B tokens it claims using up $47.23 / $70.00 of my usage, across 13,776 runs, with mostly their own harness. (Models I used: gpt 5.6 luna, muse spark 1.2 contributor, mimo v2.5, qwen 3.7 flash, minimax m3 (after free-ify))
I still have some time left to finish using it, probably I'll just start using some more expensive models instead of the top 8 cheapest models only.
First some advantages for command code:
- Desktop app is much better than opencode
- I personally think the usage how many tokens used etc is more clear than opencode because it just adds up to $70 instead of like $10 but each dollar is actually 7 dollars and this and that.
- Taste. Taste is great. Not having to tell it twice.
- Burst thinking. 10k tokens in 10s and just getting stuff done.
- Promos: I mostly used Mimo and gpt 5.6 luna, the price is amazing. Minimax free was great too.
- Way more models: Command Code has significantly more models than opencode go, opencode go only has 11, we have like 30+
But then the disadvantages:
- CLI gives up halfway through. Frequently, it would have a burst of thinking than freeze at <1000 tokens on the next turn, then stop there for a minute before continuing. Sometimes it would freeze for longer and just prompt me to type continue, which never did anything. Solution would be to exit and come back, then type continue.
- Using it over ssh means I can only use the CLI which as mentioned above which .... basically stops working after 10 minutes.
- Desktop app "Full access" Keeps asking me for permissions. That's just dumb. Same with --yolo I believe.
- Cache hit rate... 100% (actually 99.97%) on gpt 5.6 luna sounds amazing but makes me wonder, is it just re-reading context over an over more often than it needs to? Since cached read is literally re-read... sounds like token-inflation
- Having lots of models is useless if most of them are too expensive to actually use, and lots of them are either slow asf or never respond. Ox alpha has never generated a single token for me, I tried the first day of the period and every day and never got anything.
- Even the most reliable cheap model I can find (gpt 5.6 luna) still suffer from the above problems.
Suggested improvements:
- Better backend that actually responds before timeout
- Better retry scaling timing (1s -> 2s -> 4s -> 8s ...) instead of every 10s and give up
- Desktop app works for controlling remotes
- More transparency into what models are actually working.
So GUI, Taste, Desktop app, Promos and models are nice, but if the CLI breaks every 10 minutes and full access means nothing and half the models being useless, its still unusable.
not being able to tell it "go do this" and come back 3 hours later to see it done and instead seeing it done makes this just a no.
Goodbye for now, this was a brilliant idea, but I'm going to switch back to OpenCode go, where ssh is fine and models respond. I'll probably be back in half a year to see if these problems are fixed.
I think you are confusing the opencode harness and opencode go, and the command code hardness and the command code coast subscription. You can actually use the goat subscription with the opencode harness (and vice-versa). What your analysis shows is not really about the subscriptions at all.
Hey founder here. Have you filed any issues on these? I have a strong feeling that you used mostly cheap models like MiMo v2.5 that are super old and not good. No matter what we do they are just bad models when compared to DeepSeek flash or GLM flash.
We have also grown a lot. 307% MoM. Ox Alpha was free model with limited capacity and burned like 300 billion tokens. Judging the harness for a free model is a bit harsh.
The ssh issue you are talking about I personally use it like this and have never faced that issue. Have you filed a gh issue with repro? We would love to help.
We have made and continue to massive improvements. Please don’t base your opinion on free models. They are free. And we don’t optimize for free stuff so your experience can degrade on it.
No DeepSeek and GLM usage? It seems you might be missing out on all the good stuff we have.
We have worked hard to make cache best in the industry. In the docs there are benchmarks on every single top ten harness for read tool and ours is the most efficient. Even helped hermes save tokens based on our work.
Desktop app is alpha software. We have not even launched it yet. So I’m glad to hear you are already enjoying it. Next week is the beta launch.
Happy to help dig deep in the ssh issue you mentioned but need real high quality issue with repro to get it right
Small team. It’s not an easy task. Can’t code freeze the codebase. With new models coming out almost every day. 307% MoM growth. We are focused on the most prioritized workload. We have missed the deadline on open sourcing twice. It will happen when we are able to scrub through all the code base tbh. 90% of it’s done. But no one’s able to find enough free time.
No we can’t. That would go stale will be useless. It’s a giant monorepo. I wish six packages from it could live in an open source repository while still being in our monorepo. Git and GitHub suck at that. But we are super motivated to open source it. I think if I get a week distraction free I can build the sync and do it. It’s just a time thing. Should happen soon. Bear with us. We are open sourcing pieces of it starting next week.
last glm flash is too slow to do something, last qwen flash 130000 context. can You see what the matter is. I am especially interested in the situation with qwen 130,000 contexts is simply too little to utilize this model in any way.
I'm attempting to see if I can cleanly reproduce the issue if I manage to do so I'll file a issue on the GH repo.
Also. Cache hit rate per billion tokens shouldn't be the target. I can do even better: read the same 100k tokens 10,000 times and boom! 99.99% cache hit rate. (number of digits might be off but that's the idea). Makes me wonder if I actually needed 1.5B tokens to achieve what I did.
Deepseek and GLM on my existing workflows would kill my 5-hour usage limit far too fast to be usable. After the pricing changes on deepseek, I mostly switched to mimo v2.5. I did try both of them but didn't think I needed the extra bit of reasoning.
Cache hit rates I quoted above our from our 20T workload. On 1B it’s most certainly be 100%.
Cache hit rates speak to the harness and inference quality. Reading the same or different content doesn’t change cache hit ratios. It’s the harness doing odd things that changes already cached data. We are strictly append only log. It doesn’t matter if it’s a small Hi message or same log file being read (that you are assuming here which is never the case) in both cases cache is preserved. It’s when your tools or env is dealt poorly with your harness is when the cache breaks. It’s generally a sign of bad harness engineering or bad inference.
Compare TerminalBench 2.1 or SWE bench pro not .... cache hit rate.
If you can get lower cost, less time, less tokens and higher score than codex with gpt 5.6 luna on terminalbench 2.1, then sure, I have nothing to argue.
that's exactly why we tested if cache hit ratios and tool repairs coz our harness to be better on both, less costly per task, faster, overall cheaper per session, while solving the tasks better than others.
Command Code has best cache ratios, consumes lesser tokens, and cost per task is lower than OpenCode and Claude Code.
Is there somewhere I can actually test that and see clear definitions of "task" and possibly reproduce these results or just a GPT generated image?
EDIT: To the guy who downvoted me that image is gpt generated google verify openai and drag drop. There's no hard evidence ... actually no evidence at all of anything just bold claims
I did in fact complain 4 times on your discord every time it ended in the autobot telling me a reviewer will look at it and solve the problem.
as mentioned in the post, models I used: gpt 5.6 luna, muse spark 1.2 contributor, mimo v2.5, qwen 3.7 flash, minimax m3 (after free-ify) [in order of amount of usage in my opinion]
Discord is not meant for issues. It’s a discussion channel with thousands of messages daily. Please make verifiable issues on GitHub by running /feedback or cmd feedback.
I have an experience too: their harness is pretty good, it's have a very smart approach, and yes, maybe the most smart approach in the market because it is having the main goal: quality 👈 . Their harness is mature without any doubt... but it is so sloooow... imposible to work with it in real time if you have a real job with a lot of requirements from clients.
And that annoyd 5 hour limit's is something that is breaking any real work if you use a good model like Qwen3.8 max or Kimi 3, because the harness has a good approach but it is burning tokens so much and the rate limit is coming very fast,. So Goat is ok, but for cheap models, maybe for hobbiest users or somebody that can run a task for almost 45 minutes (that was my experience), it is ok. In a very honest position: I am not paying for a better plan for this slow service and if it is for the API KEY provider, well... it is the same than in any other service.
In another hand: I gave the same task to Opus-5 (Medium) on Claude Code and even that one was more faster and I not even near to use the 5 hours limir rate... also, the same task with Cursor (Grok-4.6 High) and Codex (Terra High) and them did the task in like 15 minutes or less and right. something is wrong here with their harness because the velocity, not the quality.
Finnaly: I used the desktop app if it something important to say. I couldn't to use the CLI version because gave some conflicts with dependecies in the dispositive of my office computer. I don't wanted use the time to fix it, I don't have that time.
I have 2 weeks more with the Goat plan, maybe I will try it again, but for $20 I can continue with Claude Code, even if Opus-5 is something that I really hate.
I hope that CommandCode could fix the velocity process. Grok Build is like the gold standard now about the velocity, it is so fast and the results are great (if you push the model, because their priority is not the quality and Grok become lazy with statics batteries and offer a clean code).
Thanks for sharing. Founder here. Which models did you use? DeepSeek and GLM are super good, fast, and cheap. We had scale issues last couple of days. Try DeepSeek now. You should feel the difference.
CLI harness is the most mature product we have. Our desktop is alpha software. Make sure you install with npm not bun or pnpm.
I’d love to see you use the CLI and reports any install issues so we can fix.
I phrased that wrong. I meant I tried using --yolo, and remember it somehow failing, either because the model failed to respond before timeout or hit a permission error. I wasn't paid to post this this is my honest critique.
To understand why I think cache hit rate could be bad is because cache read literally means for the model to re-read something and output text. you can artifically boost that by making the model read the same text over and over without a good reason for doing so.
Your website makes it look better but says only one concrete benifit: more models. but if those models burn your usage like a wildfire or never respond I see no reason to have them.
He means cache hit is just avg of how many tokens it read from cache, it can reach 99.9999% if it keeps reading and reading thr cache again and again in a loop too. Comparison based on cache hit alone is wrong
i have goat, my main harness is Claude Code with bespoke plugin. I only use command code mainly for Deepseek. I cant use the API with my claude code, thus I translate my plugin agent orchestration to command code. Deepseek with command code took ~1hr to fix geometric calculation on my framework, which with claude code that iteration usually done with 3-4 min either with Anthropic models or Deepseek models. My claude code also can use Deepseek API PAYG. So both command code harness and the subscription are not works as exptected at least as how it marketed.
i have no evidence and knowledge how claude code cache third party API, your screenshot are just numbers.
my claude code setup is not OOTB as you presumably how you tested with however method to produce those numbers. without domain specific consideration, those numbers are meaningless
just different screenshot with numbers and tells actually nothing.
my agents orchestration have mechanism to log sprints into formal, formatted docs. the exact same orchestration protocol used both at claude code and command code, nothing fancy only write summary, tracked progress, file changes, metadata such as time, agents/model, and commit message.
Claude Code (with any model, not just anthropic) would done it in a minute or 2.
Command Code will make model overthink and done it ~7m. 7 minute to log and write comit message is unacceptable.
I think command code cli is similar to pi in terms of cost on this plan but I've never actually measured the token burn despite having tried both so I can't give concrete numbers.
Yo tengo un tiempo usando CommandCode y nunca tuve esos problemas, solo llegué a tenerlos con Deepseek v4 Flash que entraba en bucles infinitos , pero por lo demás me va muy bien y con la suscripción Goat el uso que dan es grandísimo.
25
u/LoadElectrical2183 5d ago
I think you are confusing the opencode harness and opencode go, and the command code hardness and the command code coast subscription. You can actually use the goat subscription with the opencode harness (and vice-versa). What your analysis shows is not really about the subscriptions at all.