If you are building RAG pipelines, agents, or data-extraction tools, you probably inject API responses or database rows directly into your LLM's context window. The problem? JSON is the standard for APIs, but it is notoriously terrible for LLMs.

You end up paying for thousands of useless structural tokens ({, ", ,, \n) which increases latency, drives up API costs, and eats into your context window limit.

I got tired of this and built two pure Python, zero-dependency micro-tools to compress structured data before it hits the LLM.

1. json-to-yaml-lite (The General Fix)

It’s a known trick that LLMs understand YAML just as well as JSON, but YAML consumes about 20-30% fewer tokens because it drops the quotes and brackets. However, standard libraries like PyYAML are massive, require C-bindings, and slow down serverless cold starts (AWS Lambda).

I built a purely AST-based micro-converter: * Token Efficient: Strips all unnecessary syntax while safely escaping edge cases (like strings with colons/newlines). * Zero Bloat: No external dependencies. Drops right into your pipeline. * Repo: Encephos/json-to-yaml-lite

2. json-to-toon-lite (The Heavy Compressor)

YAML is great, but if you are injecting an array of similar objects (e.g., 50 search results or users), repeating the keys every single time is still a massive waste.

TOON (Token-Oriented Object Notation) solves this by detecting uniform arrays and compressing them into a highly dense, CSV-like tabular format. * Massive Savings: Compresses uniform arrays like [{"id": 1, "name": "A"}, {"id": 2, "name": "B"}] into [2]{id,name}: 1,A | 2,B (saving up to 60% of tokens). * Safe Fallbacks: If the objects in the array have varying keys, it gracefully falls back to standard YAML bullet formatting. * Pure Stdlib: Again, zero dependencies. Just pure Python logic. * Repo: Encephos/json-to-toon-lite

Both tools are designed for devs who want to optimize their LLM API costs without pulling in massive frameworks. I’d love to hear your thoughts on data serialization for LLMs!

At some point, you've built this: a review UI that shouldn't have existed. Ours was a Retool app, stitched together on a weekend, because the extraction tool we were using handed us a confidence score and nothing else. No reason it was low. No way to route it. No path back into the pipeline once someone fixed it.

The problem we kept running into

A single confidence number isn't a workflow. It's a data point with nowhere to go. Most document AI tools stop there, so the developer ends up building the rest: a queue, a UI, some way to route flagged fields to a human, and a script to patch the correction back into the record because nothing does that automatically.

By the time you're done, you've built a second product just to make the first one usable.

Where the gap actually comes from

Review gets treated as an afterthought, something bolted on after extraction instead of built into the pipeline. So when a field comes out wrong, there's no context for why, no distinction between "the value is genuinely uncertain" and "the value depends on someone external," and no record of what changed if you correct it.

How we tried to close it with IDPForge

Flagged documents land in a queue, but they come with a reason attached. A trigger chip tells you exactly why the document is there, low confidence on a specific field, or a validation failure like line items not summing to the total. You're only shown the fields that were actually flagged. Everything that already passed stays untouched.

When you correct a value, the original isn't overwritten. It sits alongside your correction in a field ledger, so there's a record of what the model got wrong and what a human said instead.

Not every flagged field has an answer sitting on the page. Sometimes you're waiting on a vendor to confirm a PO number, or finance to approve an unbudgeted line. For that, there's Park, a separate action from correcting, so you're not tempted to guess just to clear your queue. Parking stops the clock on that document too, so it doesn't quietly wreck your team's handling-time numbers.

Submit a document, and it either delivers straight to its destination or goes through a verifier first if a coverage rule says it should. Either way, the correction doesn't vanish into a database somewhere. It's part of the record.

And every correction matters beyond that one document. We're already working on tightening the loop between what your reviewers fix and what the model does next time. That's not live yet, but it's where this is headed.

That Retool app we built years ago, this is basically it. Except it already exists, it's part of the platform, and nobody had to spend a weekend on it.

Has anyone built a traceable, high-recall “second brain”?
We’re working on a system that turns a large, messy archive — documents, notes, code, decisions, and historical versions — into useful and verifiable memory.
The problem we’re trying to solve goes beyond standard search or RAG.
We want the system to detect:
• duplicates and near-duplicates
• contradictions
• superseded information
• relationships between sources
• provenance behind every useful claim
…while minimizing the chance of missing relevant evidence.
The hardest tradeoff so far is coverage vs. reliability vs. cost.
We’re experimenting with things like sliced/partial reading, separate extraction and independent-review stages, mechanical validation, caching, and long-running workflows.
We’ve also started testing these ideas in shadow mode on real cases instead of relying only on isolated benchmarks.
I’d love to hear from anyone working on similar problems: high-recall RAG, e-discovery, systematic review, provenance-aware knowledge graphs, PKM/second brains, or long-running agent workflows.
A few things I’m especially curious about:
• How are you reducing cost without sacrificing recall?
• How do you represent contradictions and provenance?
• What do you automate vs. independently review?
• Which architectures actually held up once you moved beyond prototypes?
Happy to share what we’re learning as well. I’m particularly interested in comparing approaches with people who have already run into these problems at scale.

I built a React agent that executes a few tasks, such as executing a skill or tool and providing an answer. It’s connected to RAG. Now, if a user asks a query about where to go to RAG and answer it, that same single question takes about 40 seconds. I wonder why I need to send the full prompt when it only needs to hit the RAG pipeline. I need some help fixing this. If some of you are considering having a sub-agent, I think a sub-agent creates a split-brain problem, but it improves other things. Any comments or help would be appreciated.

It's Tuesday. You're integrating a new document type into your pipeline. By lunch, your Postman collection has four different auth headers, three different pagination styles, and one endpoint that hands you back snake_case while another insists on camelCase.

Nobody warns you about this part.

The problem we kept running into

Document automation isn't one step; it's four: parse the document, split and classify it, extract the fields you actually care about, and clean up what comes out the other end. Most tools out there are genuinely good at one of these. Maybe extraction. Maybe parsing. That's exactly why developers reach for them, and it's the right instinct.

The trouble shows up later. Once that one stage is wired in, you still need something for the rest of the pipeline. So you bring in another tool. Then another. Now you're not building a document pipeline, you're building a translation layer between three vendors who've never heard of each other, each with their own idea of what a "successful response" looks like.

Where that gap actually comes from

It's not that these tools are badly built. It's that nobody designed for the seams. Auth works stage to stage differently. Errors mean different things depending on which vendor threw them. Retry logic that works for the parsing API silently breaks against the extraction API's rate limits. You end up writing the same glue code three times, and it's the least interesting code you'll write all quarter.

How we tried to close it with IDPForge

We built IDPForge around one rule: everything from parsing to post-processing sits behind the same API surface. One auth token. One response shape, consistently cased, across every stage. One error taxonomy, so a 422 means the same thing whether the document failed at extraction or at classification. Retry and idempotency behavior that doesn't change depending on which part of the pipeline you're calling.

That's not a small design choice. It's the difference between assembling a pipeline out of parts that were never meant to talk to each other, and calling one thing that already knows how its own stages fit together.

We didn't build this because we guessed developers would want it. We built it because we spent years being the ones stitching pipelines together, and we got tired of writing the same glue code every time.

Same Tuesday, same new document type. This time, lunch isn't spent debugging auth headers.

Maintainer here. I built a small LangChain integration that lets an agent send and track faxes, and put it on PyPI as langchain-ictfax. Fax tooling is pretty thin in the LangChain ecosystem, so I figured it was worth sharing, and I would like a sanity check on how I shaped it.

What it gives an agent: - upload_fax_document: upload a PDF, TIFF or image and get a document id back - send_fax: send that document to a number and return a transmission id - get_fax_status: poll delivery status - list_faxes: list transmissions with their status

All four are bundled by an ICTFaxToolkit, so you pull them into an agent in a couple of lines:

from langchain_ictfax import ICTFaxToolkit
tools = ICTFaxToolkit.from_credentials(base_url=..., username=..., password=...).get_tools()

Being upfront: it is a client for an ICTFax server, so it needs a reachable ICTFax or ICTCore install and an API account to actually send. ICTFax itself is open source. I am not trying to sell anything here, what I am after is feedback on the LangChain side: does the toolkit shape read well, are the tool descriptions clear enough for an agent to pick the right one, and is returning raw ids the right call for tool outputs or would you expect richer objects?

pip install langchain-ictfax Repo: https://github.com/ictinnovations/langchain-ictfax

Happy to answer anything about the wiring.

I've been learning more about building production-ready AI agents with LangChain, and one thing that stood out to me is that giving an agent access to tools creates a very different problem than simply generating text.

For example, an agent might have access to:

- search()

- sendEmail()

- deleteUser()

- makePayment()

The question becomes: how do we make sure the agent only performs actions that it's actually allowed to perform?

I recently went through LangChain's middleware-based guardrail approach and wrote up a practical guide covering:

- Deterministic vs model-based guardrails

- PII detection and redaction

- Human-in-the-loop approval for sensitive tools

- beforeAgent guardrails

- afterAgent guardrails

- Custom middleware

- Combining multiple guardrails in one agent

One pattern I found particularly useful is:

User Request

↓

Input Guardrail

↓

PII Protection

↓

Agent

↓

Tool Call

↓

Human Approval (if required)

↓

Execute

↓

Output Guardrail

↓

Final Response

I also included TypeScript examples showing how these middleware components can be implemented.

I wrote the full guide here:

https://medium.com/@nayankunwar678/guardrails-in-langchain-a-practical-guide-to-building-safe-ai-agents-68a9d5783c9e

I'm curious how others are approaching this in real projects.

Do you mainly use:

1. LangChain's built-in middleware?

2. Custom middleware?

3. External guardrail systems?

4. A combination of these?

And where do you usually put your most important checks — before the agent, around tool calls, or after the agent?