Maintainer here. I built a small LangChain integration that lets an agent send and track faxes, and put it on PyPI as langchain-ictfax. Fax tooling is pretty thin in the LangChain ecosystem, so I figured it was worth sharing, and I would like a sanity check on how I shaped it.

What it gives an agent: - upload_fax_document: upload a PDF, TIFF or image and get a document id back - send_fax: send that document to a number and return a transmission id - get_fax_status: poll delivery status - list_faxes: list transmissions with their status

All four are bundled by an ICTFaxToolkit, so you pull them into an agent in a couple of lines:

from langchain_ictfax import ICTFaxToolkit
tools = ICTFaxToolkit.from_credentials(base_url=..., username=..., password=...).get_tools()

Being upfront: it is a client for an ICTFax server, so it needs a reachable ICTFax or ICTCore install and an API account to actually send. ICTFax itself is open source. I am not trying to sell anything here, what I am after is feedback on the LangChain side: does the toolkit shape read well, are the tool descriptions clear enough for an agent to pick the right one, and is returning raw ids the right call for tool outputs or would you expect richer objects?

pip install langchain-ictfax Repo: https://github.com/ictinnovations/langchain-ictfax

Happy to answer anything about the wiring.

Has anyone built a traceable, high-recall “second brain”?
We’re working on a system that turns a large, messy archive — documents, notes, code, decisions, and historical versions — into useful and verifiable memory.
The problem we’re trying to solve goes beyond standard search or RAG.
We want the system to detect:
• duplicates and near-duplicates
• contradictions
• superseded information
• relationships between sources
• provenance behind every useful claim
…while minimizing the chance of missing relevant evidence.
The hardest tradeoff so far is coverage vs. reliability vs. cost.
We’re experimenting with things like sliced/partial reading, separate extraction and independent-review stages, mechanical validation, caching, and long-running workflows.
We’ve also started testing these ideas in shadow mode on real cases instead of relying only on isolated benchmarks.
I’d love to hear from anyone working on similar problems: high-recall RAG, e-discovery, systematic review, provenance-aware knowledge graphs, PKM/second brains, or long-running agent workflows.
A few things I’m especially curious about:
• How are you reducing cost without sacrificing recall?
• How do you represent contradictions and provenance?
• What do you automate vs. independently review?
• Which architectures actually held up once you moved beyond prototypes?
Happy to share what we’re learning as well. I’m particularly interested in comparing approaches with people who have already run into these problems at scale.

Update from the FreshCtx maintainer: FreshCtx now protects the same pre-action freshness boundary across Agno, LangGraph, and the OpenAI Agents SDK.

The framework changes, but the failure mode remains the same: 1. An agent observes a file, API response, database record, approval, or other dependency. 2. It reasons from that evidence. 3. The dependency changes before the tool or action executes. 4. FreshCtx revalidates the declared evidence at the action boundary and returns CURRENT, STALE_SOURCE, STALE_REASONING, or UNVERIFIABLE.

Recent releases added: - Agno 2.9 pre-tool integration - LangGraph protected-node integration - OpenAI Agents SDK tool-boundary integration - A shared experimental pre-action contract - Async and bounded concurrent validation - Validation budgets and audit evidence - Semantic policy/configuration validation

The new policy example also distinguishes between raw-file changes and material field changes. A descriptive edit can remain CURRENT, while a change to a declared policy limit becomes STALE_SOURCE. Invalid or incomplete configuration becomes UNVERIFIABLE.

FreshCtx 0.7.0 is open source: https://github.com/Hyperwise-LLC/freshctx

I maintain the project. I’m looking for one specific kind of feedback: does the pre-action contract map naturally to where your framework executes tools, or would it force you to restructure the workflow?

I've been learning more about building production-ready AI agents with LangChain, and one thing that stood out to me is that giving an agent access to tools creates a very different problem than simply generating text.

For example, an agent might have access to:

- search()

- sendEmail()

- deleteUser()

- makePayment()

The question becomes: how do we make sure the agent only performs actions that it's actually allowed to perform?

I recently went through LangChain's middleware-based guardrail approach and wrote up a practical guide covering:

- Deterministic vs model-based guardrails

- PII detection and redaction

- Human-in-the-loop approval for sensitive tools

- beforeAgent guardrails

- afterAgent guardrails

- Custom middleware

- Combining multiple guardrails in one agent

One pattern I found particularly useful is:

User Request

↓

Input Guardrail

↓

PII Protection

↓

Agent

↓

Tool Call

↓

Human Approval (if required)

↓

Execute

↓

Output Guardrail

↓

Final Response

I also included TypeScript examples showing how these middleware components can be implemented.

I wrote the full guide here:

https://medium.com/@nayankunwar678/guardrails-in-langchain-a-practical-guide-to-building-safe-ai-agents-68a9d5783c9e

I'm curious how others are approaching this in real projects.

Do you mainly use:

1. LangChain's built-in middleware?

2. Custom middleware?

3. External guardrail systems?

4. A combination of these?

And where do you usually put your most important checks — before the agent, around tool calls, or after the agent?

Solo dev here. I've been bothered for a while by how expensive it gets to do GraphRAG ingestion with LLM extraction when you have real volume — you pay per token, it's slow, and sometimes the JSON comes back broken and you retry.

So I built a small non-autoregressive decoder (~37M params) that takes a sentence embedding (SONAR, Meta's multilingual space) and outputs the knowledge graph directly: entities + typed relations in a single pass. No text generation anywhere, so malformed output is structurally impossible.

The LlamaIndex part: there's a connector class, three lines to production:

extractor = CogitoGraphExtractor(checkpoints, "vocab-prose.json")
triples = extractor.extract(node.text)
extractor.to_neo4j(driver, triples, source=node.id_)

It takes plain text (whatever your pipeline hands it), returns string triples, and maps them to Cypher MERGE with per-edge provenance. There's also extract_batch so ingestion does one encoder call for N chunks.

Honest numbers, all on held-out data with the splits documented in the repo: tool-call structures 1.000 F1, python code 0.781, prose with entity candidates 0.827, open-vocab prose 0.651. Prose is the hard modality — the failed attempts (focal loss, LLM label distillation, char-level generation) are all in the changelog because I think negative results are half the value.

Everything was trained on a single RTX 5070 at home. Decoder heads are Apache-2.0; heads-up that the SONAR encoder itself is Meta's CC-BY-NC, migrating to BGE-M3 is next on the roadmap for a fully commercial-clean stack.

Repo: https://github.com/DeliVali/cogito-estella
Weights: https://huggingface.co/DeliVali/cogito-estella

Where I'd really value feedback (building this solo, so outside eyes matter a lot):

1. The candidate design: for prose, the extractor picks relations between entity candidates that come from the text + your existing graph nodes. Does that fit how your pipelines actually work, or do you need fully open extraction even at lower accuracy? (0.827 with candidates vs 0.651 open-vocab — that's the tradeoff)
2. The relation vocabulary is 60 verbs. When the sentence's verb isn't there, the model picks the nearest one ("approved" → "support"). Is that acceptable degradation for your use case, or is exact relation wording a dealbreaker?
3. What would make you actually try it? Be brutal — missing docs, the fairseq2 dependency, no async API, whatever. The friction you name is what I fix first.
4. If you run GraphRAG ingestion today: what does extraction cost you per 1M chunks, roughly? I want to check if my "1000× cheaper" math survives contact with real setups.

If you try it and it breaks, open an issue and I'll fix it fast — early issues are gold for me. And if anyone wants to benchmark it against their current extraction stack, I'll happily help set it up.

I spent about 6 weeks building an agent that drives our desktop app for regression testing. tools for screenshot, click and type, a vision model for grounding, a loop that keeps going until the goal is met. it demos quite well but then i ran the same 20 flows 10 times each and got roughly 12 percent variance in outcomes with nothing changed between runs. The agent sometimes takes a different route to the same end state, which is okay for an assistant and useless for a regression gate where the entire point is that identical input produces identical output. i'd built something that can operate the app but can't tell me whether the app changed.

Before I spend another 6 weeks on determinism i'd like to know whether i'm rebuilding something that exists. Bytebot and Goose are both further along than mine on the driving side and i assume they hit the same wall, but i can't find anyone writing about what they did after that. The dedicated QA models like Askui and Eggplant appear to solve it by making you write the steps explicitly and only using the model for perception.The specific thing i'm stuck on is whether you can get determinism out of an agent loop at all or whether the answer is to remove the loop.

Has anyone got an eval setup for this that isn't just running it 10 times and eyeballing the diffs?

Hi community, I am so confused . I have learnt ReAct loops , prompt engineering, Rag , Vercel AI Sdk few guadrails from the first principals and built minimal terminal agents .

Should I go to Langchain/LangGraph next or should I built some solid project from the things I have learnt and then move to next .

What projects should I build?

I have been trying a bunch of harness models and frameworks for the last month, and I realize most agent frameworks want you to rebuild your runtime around their harness.

We went the other way with AURA Harness: a thin membrane around loops you already run, plain Python, Ollama, LangGraph, whatever. You keep the body/runtime, while AURA records and optionally gates what crosses the boundary. Shouldn't this be what harness is about??

Local-first by default:

* Works offline with `agent_ref` \+ session IDs, no cloud identity required * `integrations/ollama/llama_loop.py,` stdlib HTTP against Ollama (`llama3.2:1b` is our dev default) * Optional verified operator identity if you need it later, not required for OSS/dev

What you get on close: JSONL spine, audit report, hash chain, `aura verify chain` for CI.

Loose coat = audit-only logging. Tight coat = rules/gates at egress when you wire tool paths.

Open source (Python): [github.com/ARPAHLS/aura](https://github.com/ARPAHLS/aura)

Happy to answer setup questions, especially Ollama related + “wrap my script, don’t replace it.”

Contributors more than just welcome, bunch of good first issues open <3

I built a React agent that executes a few tasks, such as executing a skill or tool and providing an answer. It’s connected to RAG. Now, if a user asks a query about where to go to RAG and answer it, that same single question takes about 40 seconds. I wonder why I need to send the full prompt when it only needs to hit the RAG pipeline. I need some help fixing this. If some of you are considering having a sub-agent, I think a sub-agent creates a split-brain problem, but it improves other things. Any comments or help would be appreciated.

At some point, you've built this: a review UI that shouldn't have existed. Ours was a Retool app, stitched together on a weekend, because the extraction tool we were using handed us a confidence score and nothing else. No reason it was low. No way to route it. No path back into the pipeline once someone fixed it.

The problem we kept running into

A single confidence number isn't a workflow. It's a data point with nowhere to go. Most document AI tools stop there, so the developer ends up building the rest: a queue, a UI, some way to route flagged fields to a human, and a script to patch the correction back into the record because nothing does that automatically.

By the time you're done, you've built a second product just to make the first one usable.

Where the gap actually comes from

Review gets treated as an afterthought, something bolted on after extraction instead of built into the pipeline. So when a field comes out wrong, there's no context for why, no distinction between "the value is genuinely uncertain" and "the value depends on someone external," and no record of what changed if you correct it.

How we tried to close it with IDPForge

Flagged documents land in a queue, but they come with a reason attached. A trigger chip tells you exactly why the document is there, low confidence on a specific field, or a validation failure like line items not summing to the total. You're only shown the fields that were actually flagged. Everything that already passed stays untouched.

When you correct a value, the original isn't overwritten. It sits alongside your correction in a field ledger, so there's a record of what the model got wrong and what a human said instead.

Not every flagged field has an answer sitting on the page. Sometimes you're waiting on a vendor to confirm a PO number, or finance to approve an unbudgeted line. For that, there's Park, a separate action from correcting, so you're not tempted to guess just to clear your queue. Parking stops the clock on that document too, so it doesn't quietly wreck your team's handling-time numbers.

Submit a document, and it either delivers straight to its destination or goes through a verifier first if a coverage rule says it should. Either way, the correction doesn't vanish into a database somewhere. It's part of the record.

And every correction matters beyond that one document. We're already working on tightening the loop between what your reviewers fix and what the model does next time. That's not live yet, but it's where this is headed.

That Retool app we built years ago, this is basically it. Except it already exists, it's part of the platform, and nobody had to spend a weekend on it.

If you are building RAG pipelines, agents, or data-extraction tools, you probably inject API responses or database rows directly into your LLM's context window. The problem? JSON is the standard for APIs, but it is notoriously terrible for LLMs.

You end up paying for thousands of useless structural tokens ({, ", ,, \n) which increases latency, drives up API costs, and eats into your context window limit.

I got tired of this and built two pure Python, zero-dependency micro-tools to compress structured data before it hits the LLM.

1. json-to-yaml-lite (The General Fix)

It’s a known trick that LLMs understand YAML just as well as JSON, but YAML consumes about 20-30% fewer tokens because it drops the quotes and brackets. However, standard libraries like PyYAML are massive, require C-bindings, and slow down serverless cold starts (AWS Lambda).

I built a purely AST-based micro-converter: * Token Efficient: Strips all unnecessary syntax while safely escaping edge cases (like strings with colons/newlines). * Zero Bloat: No external dependencies. Drops right into your pipeline. * Repo: Encephos/json-to-yaml-lite

2. json-to-toon-lite (The Heavy Compressor)

YAML is great, but if you are injecting an array of similar objects (e.g., 50 search results or users), repeating the keys every single time is still a massive waste.

TOON (Token-Oriented Object Notation) solves this by detecting uniform arrays and compressing them into a highly dense, CSV-like tabular format. * Massive Savings: Compresses uniform arrays like [{"id": 1, "name": "A"}, {"id": 2, "name": "B"}] into [2]{id,name}: 1,A | 2,B (saving up to 60% of tokens). * Safe Fallbacks: If the objects in the array have varying keys, it gracefully falls back to standard YAML bullet formatting. * Pure Stdlib: Again, zero dependencies. Just pure Python logic. * Repo: Encephos/json-to-toon-lite

Both tools are designed for devs who want to optimize their LLM API costs without pulling in massive frameworks. I’d love to hear your thoughts on data serialization for LLMs!

It's Tuesday. You're integrating a new document type into your pipeline. By lunch, your Postman collection has four different auth headers, three different pagination styles, and one endpoint that hands you back snake_case while another insists on camelCase.

Nobody warns you about this part.

The problem we kept running into

Document automation isn't one step; it's four: parse the document, split and classify it, extract the fields you actually care about, and clean up what comes out the other end. Most tools out there are genuinely good at one of these. Maybe extraction. Maybe parsing. That's exactly why developers reach for them, and it's the right instinct.

The trouble shows up later. Once that one stage is wired in, you still need something for the rest of the pipeline. So you bring in another tool. Then another. Now you're not building a document pipeline, you're building a translation layer between three vendors who've never heard of each other, each with their own idea of what a "successful response" looks like.

Where that gap actually comes from

It's not that these tools are badly built. It's that nobody designed for the seams. Auth works stage to stage differently. Errors mean different things depending on which vendor threw them. Retry logic that works for the parsing API silently breaks against the extraction API's rate limits. You end up writing the same glue code three times, and it's the least interesting code you'll write all quarter.

How we tried to close it with IDPForge

We built IDPForge around one rule: everything from parsing to post-processing sits behind the same API surface. One auth token. One response shape, consistently cased, across every stage. One error taxonomy, so a 422 means the same thing whether the document failed at extraction or at classification. Retry and idempotency behavior that doesn't change depending on which part of the pipeline you're calling.

That's not a small design choice. It's the difference between assembling a pipeline out of parts that were never meant to talk to each other, and calling one thing that already knows how its own stages fit together.

We didn't build this because we guessed developers would want it. We built it because we spent years being the ones stitching pipelines together, and we got tired of writing the same glue code every time.

Same Tuesday, same new document type. This time, lunch isn't spent debugging auth headers.