From 2D Drawings to 3D Models: Designing an AI Reconstruction Pipeline on MongoDB

Plenty of buildings exist only on paper. Small and mid-size building manufacturers still quote from 2D shop drawings, renovation firms inherit decades-old blueprints with no digital model, and archives everywhere hold plans for structures that were built long before BIM. Whenever one of these businesses needs a 3D model, to quote a job, visualize a change for a client, or plan a renovation, someone rebuilds the geometry by hand in a CAD tool. When the client asks to move a window or widen a wall, part of that manual work starts over, and on a competitive bid, those rework hours decide whether the quote goes out on time.

BIRD (Building Image Reconstruction and Dimensioning) is a research framework, developed and validated with complete building plans from an industry partner, that automates this conversion. It accepts the five 2D perspective drawings (front, back, left, right, and the floor plan) as input and produces a proportionate, dimensioned 3D model of the building envelope as output. In this post, we walk through how the technique works and how it can be designed to run as a production service on MongoDB Atlas, with Amazon Simple Storage Service (S3) for drawing storage, MongoDB Atlas Stream Processing to coordinate the pipeline, and MongoDB Search and MongoDB Vector Search for searching past project data.

How does BIRD convert 2D Drawings into a 3D building envelope?

Building plans are noisy. Annotations, dimension text, and symbols sit directly on top of the walls and openings that a reconstruction algorithm needs to trace. BIRD handles this with a dual-branch convolutional neural network: one branch produces a heatmap of genuine architectural line segments, the other produces a heatmap of the noise. Subtracting the noise map from the line map leaves a clean set of segments. A text recognition model then reads the measurements printed on the drawing and uses them to scale every segment to real-world inches. Finally, a novel wrapping algorithm folds the four elevation views around the floor plan, correctly handling angled walls and sharp corners, to produce the 3D envelope.

Figure 1. How BIRD converts 2D building plans into a 3D envelope. Figure adapted from this research paper.

Across the complete building plan sets evaluated in the research paper, from small residential structures to a multi-story apartment complex, the framework achieved a median accuracy of roughly 93% in terms of visually aligning with the expected final visualization. Because construction demands exact output, BIRD is open enough to allow for interfaces where a drafter could inspect the extracted lines overlaid on each drawing, for example, nudge any segment that was missed, confirm the dimensions, and then trigger the projection.

Storing BIRD output with MongoDB

Each processed perspective yields predicted line segments, filtered noise segments, recognized text boxes, extracted measurements, and a growing history of human corrections. The volume varies between buildings, and there may be instances where the drawing contains components that are irrelevant for a visualization: in the published evaluation, the count of irrelevant detected lines (e.g., extracted from an architectural symbol) per perspective ranged from 0 to 137. A simple rectangular elevation produces a handful of segments while a complex facade produces hundreds, and each correction event adds more.

That variability maps naturally into MongoDB’s document model. One JSON project document could hold everything about the build: the metadata, an embedded entry for each of the five drawings with its array of segments, and eventually the final 3D geometry. Every segment carries its coordinates, its real-world length, its type (wall, window, door), and its correction lineage. In a relational database, this becomes a segments table with millions of narrow rows, join-heavy queries for every screen of the review interface, and schema migrations each time the research team adds a new attribute to the model output. In MongoDB, the document a drafter reviews on screen would be the document stored in the database, and the schema evolves with the model.

The following example shows a simplified document containing one complete perspective view in a project document; the remaining views are abbreviated for clarity:

Every manual correction is recorded with a timestamp, so the system keeps an audit trail of what the model predicted versus what the drafter approved. For a manufacturer, this serves two purposes: production traceability when a dimension is questioned later, and a steadily growing set of corrected examples for retraining the model.

What a MongoDB Atlas reference architecture for BIRD looks like

The proposed deployment has three parts, explained below.

Figure 2. BIRD on MongoDB - a reference architecture.

Object Storage for the raw data: The raw 2K-resolution drawing images and the exported 3D model files reside in an S3 bucket. Atlas stores only the bucket and key references, plus all extracted data, keeping the database lean and the images priced for large binaries.

The inference service: BIRD can use a Python API. In production, it runs as a containerized service that pulls a drawing from S3, runs the dual-branch network and text recognition, and writes the resulting perspective document to Atlas.

Event-driven workflow: A drawing upload triggers inference, and a project advances to 3D projection only when all five perspectives are approved. The flow runs like this:

1. Upload: The drafter uploads the five drawings through the upload service. The service writes each image to S3 and one small record per drawing (project ID, view, S3 key) to the uploads collection in MongoDB.

2. Inference: The inference service watches the uploads change stream. For each new record, it downloads the image from S3, runs the BIRD models, and writes the extracted result (line segments, measurements, status) into that drawing's slot in the projects document. One project document holds all five drawings’ results.

3. Review: The drafter reviews each drawing's result in the interface, corrects segments, and approves. Every correction and approval is written to the projects document, and each approval increments its approvedViews counter.

4. Stream processing: MongoDB Atlas Stream Processing watches the projects change stream. When the fifth approval pushes approvedViews to 5, the processor writes one record to the projection_jobs collection.

5. Projection: The 3D projection service picks up the record from projection_jobs, runs the wrapping algorithm, uploads the model file to S3, and writes the final 3D segment back to the projects collection document.

At the end, the projects document holds everything about the build: results, corrections, approvals, and final geometry.

MongoDB Search can be used to search across the project history. We can also leverage Voyage AI models available through the Embedding and Reranking API on MongoDB Atlas. MongoDB Search provides full-text search over project metadata, client names, drawing annotations, and the free-text notes drafters attach during review, so a user can type "warehouse rafter detail" and land on the right project. MongoDB Vector Search enables semantic search: for example, at ingestion, voyage-multimodal-3.5, used to vectorize visual data such as figures and technical drawings, encodes each drawing image into a vector stored on the project document, while voyage-4 embeds the review notes for semantic text retrieval. When a new project arrives, the system embeds the incoming floor plan and runs a $vectorSearch to surface the most geometrically similar past projects and a re-ranker model, rerank-2.5 ranks the shortlist before the drafter sees it. Using these MongoDB features, a drafter quoting a residential design can start from the corrected model of a near-identical build instead of a blank screen, which directly shortens quoting time on repetitive work. The $vectorSearch stage also supports pre-filtering on indexed metadata fields to reduce the candidates used in the search pipeline.

Figure 3. MongoDB Search pipeline with Voyage AI models for embedding images and text.

Teams whose customers require self-hosted deployments are not locked out: MongoDB Search and Vector Search are generally available for MongoDB Enterprise Advanced and Community Edition, deployed as search nodes on Kubernetes, so the document model, change streams, $search, and $vectorSearch all carry over to self-hosted environments seamlessly.

What does this MongoDB architecture enable for manufacturers?

In the proposed approach, all the information and context live in one place: a single project document carries the extracted segments, the drafter's corrections, the approval trail, and the final 3D geometry, which gives the end user full traceability from uploaded drawing to exported model. The pipeline is event-driven end-to-end, with the stream processor doing the coordination, so there is no polling and no separate infrastructure to operate. The human review step is built in as well, and every correction the drafters make becomes labeled training data, so the system gets more accurate over time. And because the whole history is indexed with MongoDB Search and Voyage embeddings, each completed project makes the next quote faster.