r/OpenSourceAI 15h ago

DeepSeek V4 Flash Vision-Exp let me remove an entire 27B routing model from my local AI studio

2 Upvotes

When DeepSeek V4 Flash Vision-Exp dropped, I was pretty excited, but not just because it added vision.

It actually let me delete part of the architecture of a project I've been working on.

I'm building DStudio, an open-source, local-first AI workspace around ds4. It has Chat, coding/knowledge-work agents, Design, research, local image generation/editing and video generation.

GitHub: https://github.com/sk8erboi17/DStudio

Until now I had a slightly ugly problem with multimodality.

The main model was running through ds4, while I was using Qwen3.8-27B Q8 as a separate visual/router model through llama.cpp.

The flow was roughly:

DS4 → unload/evict → Qwen3.8 router → unload → image worker → restore DS4

Qwen was responsible for looking at the prompt/source image and deciding whether the user wanted a new image or an edit, before dispatching the request to Ideogram 4 or HunyuanImage 3.

It worked, but on a 96 GB Apple Silicon machine the architecture was expensive.

The heavyweight models couldn't comfortably stay resident together, so DStudio had to manage memory leases, serialize the workers and repeatedly move between two inference stacks: ds4 and llama.cpp.

The router itself was becoming part of the latency problem.

With DeepSeek V4 Flash Vision-Exp, image pixels now go directly into the main ds4 model.

The model itself can understand the image/request and emit an explicit: generate or edit directive.

Then DStudio dispatches directly to: DeepSeek V4 Vision -> Ideogram 4 or HunyuanImage 3

The same idea now applies to the Design agent: instead of generating something and then loading a separate 27B VLM to inspect it, the selected model uses its own native vision encoder for the visual feedback loop.

I've also added the same native-vision path for GLM 5.3.

I still use llama.cpp in DStudio for a small Qwen3-Embedding-0.6B sidecar, so this isn't "I removed llama.cpp completely."

The important part is that llama.cpp is no longer sitting in the critical multimodal path just to run a 27B router.

I also changed PDF handling to be more explicit: text extraction/ranking stays lightweight, while actual pixel understanding is handled by the native multimodal model rather than silently spinning up another large VLM.

Recent commits have also added GLM 5.3 runtime support, persistent model-specific ds4 engine checkouts, expanded Cowork/Design workflows and a lot more regression/quality gates.


r/OpenSourceAI 9h ago

A virtual computer for AI Agents

Enable HLS to view with audio, or disable this notification

0 Upvotes

r/OpenSourceAI 12h ago

I looked into OpenHuman after it hit #1 on GitHub Trending — is this actually a different class of AI assistant?

0 Upvotes

OpenHuman caught my attention because the pitch sounds bigger than another LLM wrapper:

• persistent memory

• multi-agent orchestration

• workflows

• MCP/tools

• local models

• integrations

• desktop runtime

• research capabilities

So I went into the repository expecting to find out whether the architecture actually supports the marketing.

My current take:

There is something architecturally interesting here, but the “private/local ChatGPT” framing is too simplistic.

The interesting part is the separation between:

User

Persistent Memory

Agent / Orchestrator

Specialized Agents

Tools / MCP / Integrations

Workflows

New State + Memory

That's a fundamentally different abstraction from a conventional chatbot.

The memory system is particularly interesting: OpenHuman stores its Memory Tree locally in SQLite and exposes an Obsidian-compatible Markdown representation.

It also has explicit local-provider support through Ollama and LM Studio.

But here's the important caveat:

Local-first ≠ everything local.

The current project documentation says local AI is opt-in. The default experience can still use OpenHuman's hosted model routing and managed integration infrastructure.

And there are GitHub issues that make this distinction important rather than theoretical.

There have been reports/issues around:

• backend-dependent onboarding

• local-model reliability

• local voice/model setup

• orchestration stalls and worker leaks

• outdated documentation/positioning

• the gap between the “local-first” story and what is actually local by default

So I'm more interested in the architecture than the hype.

The question I'd like to discuss with people who have actually run it:

Does OpenHuman represent a useful new architecture for personal AI, or is it currently too dependent on managed infrastructure to really qualify as a user-owned AI runtime?

And for people running local AI:

Would you trust an agent with persistent access to Gmail/GitHub/Slack/Calendar if the memory is local but integration/model traffic can still pass through hosted infrastructure?

That's the part I think is worth debating.


r/OpenSourceAI 11h ago

FreshCtx 0.7.0: an Apache-2.0 Python guard that revalidates an AI agent’s evidence before it acts

2 Upvotes

I maintain FreshCtx, an open-source Python project for a specific AI-agent failure mode: an agent reads valid information, reasons from it, and then acts after that information has changed.

FreshCtx lets an application declare the evidence used during reasoning and revalidate it immediately before a consequential action.

The current release includes:

  • Agno 2.9 integration
  • LangGraph integration
  • OpenAI Agents SDK integration
  • A shared experimental pre-action contract
  • Async and bounded concurrent validation
  • Validation budgets and audit evidence
  • File, HTTP, SQLite, Postgres, MCP safe-reader and Stripe Subscription adapters

It is local-first, model-neutral, Apache-2.0 licensed, and has no account or telemetry requirement.

Repository:
https://github.com/Hyperwise-LLC/freshctx

I would particularly value feedback on the integration contract. Does a framework-neutral pre-action boundary belong in the core library, or should each framework integration remain completely independent?


r/OpenSourceAI 18h ago

Ciele: open-source (AGPL) platform for AI chat assistants that answer from your own content, self-hosted with one docker compose

Enable HLS to view with audio, or disable this notification

3 Upvotes

Demo video: https://www.youtube.com/watch?v=SoUEkM2Sjmw

I've been building Ciele, an admin console where an org builds and publishes its own AI assistants. They ship as embeddable chat widgets that answer only from content you feed them (crawled websites, uploaded files, curated FAQs) and cite the source of every answer.

What's in it:

  • RAG over Postgres + pgvector. An answer without a source doesn't ship.
  • A rule engine that runs before the LLM gets a say. Known question, exact answer. Or a button, an API call, an email, a handoff to a human.
  • Escalation to real help desks: email, phone, live chat, webhooks, with ticket forms and availability hours.
  • Conversation inbox, analytics, a kanban of answers someone flagged as bad, and alerts when an integration breaks.
  • Embed as a script floater or an iframe. There's also a CLI, a REST API and an MCP server.

Self-hosting is one docker-compose.yml (db, migrate, app, cron). bootstrap.sh generates every secret, including the JWTs it signs with the stack's own key. The crawler worker is an optional overlay. If you'd rather skip the terminal entirely, a desktop app stands up the whole local stack through a wizard.

You bring your own LLM provider keys. Nothing routes through my servers.

The two hardest problems so far: tenant isolation done entirely in Postgres row-level security (no where org_id sprinkled around, the database itself refuses cross-tenant reads), and making citations resolve to actual sources instead of opaque vector chunks. The second one took three rewrites.

It's open-core, so let me state the line plainly: this AGPL repo is the complete product. The paid part is only the managed cloud (hosting, plans, support). The boundary is documented and CI fails the build if enterprise code leaks into the mirror.

Stack: Next.js, shadcn/ui, Supabase, pgvector, Turborepo. AGPL. Self-host with docker compose, or there's a cloud version.

Repo: https://github.com/MattiaIppoliti/ciele
Docs: https://docs.ciele.app