r/OpenSourceAI • u/conifer_v11 • 2d ago

Opensource Openrouter

Project: https://github.com/ConiferKit/use-conifer

Current routing options were charging 5% fees for byok plus provider fees (openrouter) or were built in heavy python packages with ecosystem restraints. I wanted to be completely free in terms of use, and not have to pay extra for tokens.

There's a maintained gateway with 100+ models from one api endpoint. I'm also talking to infra providers to get us access to pre-release models and discount prices. Everything is served at market price. BYOK is free, you can hook into self-hosted setups for free, and there's fallbacks + extra rate limits + server redundancy.

Took me ~3 months to build and looking for help maintaining!
Lmk any issues or feedback

7 Upvotes

100% Upvoted

9 comments sorted by

1

u/No_Elephant_7530 2d ago

Does the router account for prompt caching when it picks a model? Switching mid-session has burned me before, making me spend more than needed.

1

u/Michael_Jeffords 2d ago

Just started using for a harness that I had been building and was really surprised with results so thought I would leave a review. I had previously been using open router and aside from the fact that this is obviously a free option, I found none of my users would ever use the router haha. I’ve just been telling people to use balanced as from an eye test this seems to be the best of the 3? The main thing I appreciate is how easy it was to slot in took maybe 10 min. How much of a benefit is it to route cheapest/smartest versus balanced? Also are y’all making money from this? Curious what made you decide to opensource

1

u/Regular-Cat4711 2d ago

This is super cool! Going to try out, how up to date are yall with adding new models? Thinking of integrating with an app I’m building but want to be sure that as soon as new open source model comes out I can use it 

2

u/conifer_v11 2d ago

Better answer! We're getting pre-release deals to have models before they're public

But we're also constantly adding new models if we miss any you can ping us and we'll add it in a day (https://www.conifer.build/models/)

1

u/Calm-Landscape9640 2d ago

I joined. The future is using these frontier models that cost under $2 / million and using a better harness. So routing is the next logical piece to solve. Good work guys, cant wait to try it.

1

u/p3r3lin 2d ago

How do you compare to https://github.com/BerriAI/litellm ? When to choose either?

1

u/RogerAI-fm 1d ago

Or even rogerai, but I guess that is any person can be a provider.

1

u/nez_har 1d ago

Interesting project. Does this also work for oauth credentials?

I'm the maintainer of  https://github.com/VibePod/vibepod-cli and I've been thinking for some time that this sort of centralized endpoint might be useful, especially if you want to onboard a new agent.

How are you currently integrating it with Claude Code or Codex?

1

u/conifer_v11 9h ago

short answer: no oauth lane today. conifer is a gateway key (CONIFER_API_KEY / sk-conifer-…) plus optional byok where you paste an openai/anthropic/azure key in the console (azure also needs the resource url) — those get verified live, stored encrypted, and callers keep using the conifer key so they never see the upstream credential. there's no oauth token passthrough or "sign in with x and we'll refresh for vibepod" surface right now, so for a cli that needs user oauth you'd still own that flow. if the agent will take a custom openai-compat url, point it at https://api.conifer.build/v1 with the conifer key; that's the path that exists. i work on conifer; if oauth-as-a-credential-type is what you actually need for vibepod say what provider and we can be honest about whether it belongs on the roadmap vs stay out of band.