r/photogrammetry • u/prajwlshetru • 10h ago
context extraction from 2D PDF engineering drawings (P&IDs/Blueprints)? Looking for ideas on graph DBs & CV workarounds.
Hey everyone,
I am currently working on a pipeline to extract context from engineering drawings (like P&IDs and technical blueprints) stored in PDFs to feed into an LLM.
Multi-modal LLMs are great, but they aren't quite there yet for heavy-duty engineering files. Here are a few observations and hurdles I’ve hit so far:
- Multimodal LLMs lack precision: Standard vision models can read the overall drawing, but they miss critical component tags, small text, and fine details.
- OCR/Text extraction isn't enough: Pulling just the raw text from drawing tags helps with basic Quality Control (QC), but it completely misses the spatial classification and structural relationships.
- The Planned Workaround: I am planning to use classical Computer Vision (like Hough Transforms) for shape and icon detection. The goal is to extract the symbols and text, map their connections, and build a Graph Database to give the LLM structured context.
Has anyone successfully built a pipeline for this? I would love to hear your thoughts on a few things:
- Are there specific CV models or open-source frameworks you recommend for engineering symbol detection over traditional Hough transforms?
- How are you handling the layout/relationship parsing to build your graph edges?
- Any recommendations for structuring the final graph data so an LLM can query it efficiently via GraphRAG?
If you are working on something similar or have dealt with CAD/PDF parsing for AI, I'd love to hear your suggestions or brainstorm solutions!

