3D Repo Runs MongoDB

If you’re an architectural or engineering firm, you’ve undoubtedly confronted the difficulty of managing and collaborating on 3D assets like CAD drawings.  The act of sharing massive files is hard but feasible, but it is significantly complicated by the inability to determine that you’re using the latest version.  For the CAD-inclined, there’s hope. Jozef Dobos, a doctoral student at University College London (UCL), has applied the geospatial indexing capabilities of MongoDB a version control system for 3D assets called 3D Repo.  Sponsored by Arup Foresight, the built environment innovation division of Arup Group Limited, a global design and business consulting firm with offices in over 30 countries, 3D Repo leverages the flexibility of MongoDB’s data model, not to mention its geospatial capabilities, to make collaboration on 3D assets easy.

The Problem

Whether an architectural firm or a product design company, collaboration on large (500GB+) 3D assets is a critical but difficult task, one not solved simply by sending links around to these assets. The maintenance of assets in a large 3D visualisation can involve authors that are numerous, geographically dispersed, and diverse in their skillsets and tools.

This presents problems including, but not limited to, maintaining consistency of the models and dealing with concurrent edits in the same part of a 3D scene. For example, in the industry today, each user loads a particular 3D scene into a modeling tool, modifies it, and then re-saves the entire file again, making any version tracking and sharing unnecessarily unclear. By the time a simulation is completed, the design might progress so rapidly that the results are often inapplicable. 

An SVN for 3D Assets

Looking for a better way to collaborate on 3D files, Dobos developed a unified and integrated framework, called 3D Repo, that supports collaborative editing and distribution of 3D assets.  Think of it like Subversion (SVN), but for 3D assets, not code.  

The 3D Repo framework tracks multiple revisions of 3D assets so that they can be integrated later on. It thus provides similar functionality to file-based revision control systems like SVN, but is built around MongoDB, thereby avoiding the constraints of a file-based system. The framework also supports distributed editing over the Internet and additional lightweight clients in web-browsers and mobile devices. Such an approach is expected to improve the engineering as well as public engagement and considerably reduce the costs of future industrial development.

Why MongoDB?

While there are other proprietary and open-source databases that support spatial data directly - e.g. Oracle Spatial and PostGIS for PostgreSQL - these alternatives focus on 2D geometry, not 3D models and their properties. They also do not support the type of revision histories that is commonly required in 3D asset management.

MongoDB, on the other hand, avoids rigid data models and is optimized for large read-write operations. It is therefore possible to store 3D models in a database and, due to MongoDB’s schema flexibility, also track other associated data such as semantic relationships and even individual revisions, as is the case in 3D Repo. 

As Dobos explains:

Basically, relational databases can in no way cut it. Rigid table structures are not suitable for highly diverse and large 3D data. In our case, a single 3D model or, better said, a “3D scene” is represented as a scene graph, where each node can be, and most of the time is, something different. It can be a node for a mesh, hence a large binary array, or a transformation, where transformation matrix is simply 16 numbers, or animation, bone, material, texture etc. Each of these is represented and stored differently. So key-value pairs are a match made in heaven for this type of data.

In MongoDB, 3D Repo assigns two collections (tables) per 3D scene, one for all the scene graph constituents and one for all the documents that belong to a revision history. Hence, each of these collections stores a directed acyclic graph, making the access implementation reusable. Once the data is in a database, access is implicitly supported in a distributed manner via a dedicated query language. 

Mostly simply stated, we don’t need any complicated graph traversal operations. We opted for MongoDB because of its efficiency with binary storage, BSON rather than ASCII (hence a clear winner over CouchDB, too), and due to its flexibility, not to mention its web-ready approach. MongoDB is proven to scale well, has sharding built-in and there is a thriving community to support it. Most of the design inspiration in our system actually comes from webinars by Wordnik and their experience.  We are really grateful for those!