Moving Mongoid In-House

Durran Jordan
September 30, 2015
#Business

One of the benefits of being part of an open-source community is that we have incredible 3rd-party contributions from our community. The more traction these tools gain, the more we need to be able to support them as part of the MongoDB ecosystem, which means being able to push bug fixes, schedule releases, and generally drive the product forward.

In this post we’ll discuss one of these projects, Mongoid, an ODM developed by the Ruby community.

What is Mongoid, and who uses it?

The Ruby community has a variety of developer tools for MongoDB. Developers who prefer to access the database via the driver level use the official MongoDB Ruby driver, and those who prefer to leverage an ODM like Mongoid instead. Mongoid provides an Active Record interface for working with documents as domain models in the application, as well as providing features such as a query DSL, schema constraints, and relational database style relations.

The Challenge

There were two major steps to bringing Mongoid in-house. The first was to make the core driver’s API flexible enough for an ODM’s needs. The second was to do an entire rewrite of the BSON library in order to make it both extensible and easily maintainable. The greatest challenge was making sure all of this happened with as little disruption as possible for both Ruby driver and Mongoid users.

The Strategy

We decided to rewrite the core BSON library in Ruby, separating it from the codebase and giving it its own release schedule. The focus of the rewrite was to make it more Ruby-like, following the patterns set forth by existing JSON and YAML implementations. Upon completion this was immediately included in Mongoid and Moped.

Once the BSON library was rewritten, the next step was to do a complete rewrite of the core Ruby driver. The driver was not only rewritten to meet the needs of Mongoid’s API, but it was also the first Ruby driver release to comply with the MongoDB driver team’s cross-platform driver specifications. With these enhancements, the new driver was made a dependency of Mongoid master and released with Mongoid 5.

The last steps were administrative, and included the migration of the Mongoid website at mongoid.org to the MongoDB ecosystem documentation, the migration of all issues into the official MongoDB Jira project, and the move of the repo to the official MongoDB Github organization.

Upgrading

Upgrading to Mongoid 5 requires minimal effort. The bulk of upgrading efforts will entail changing mongoid.yml, as all of the configuration options have changed with Mongoid 5. Also note that the references to session are now replaced with client, as the underlying entry point to the driver is now a Mongo::Client and not a Moped::Session.

The new mongoid.yml configuration can be found in the Mongoid 5 documentation.

The small number of API changes that need to be addressed are:

  • find_and_modify has been removed and replaced with 3 options: find_one_and_update, find_one_and_delete and find_one_and_replace.
  • text_search has been removed as it is now a $text option in a query from 2.6 on.
  • first and last no longer add an _id sort when no sorting options have been provided. In order to guarantee that a document is the first or last, it needs to now contain an explicit sort.

Lastly, support for MongoDB 2.2 and lower is removed to fall in line with the MongoDB versions that the core driver supports.

Deployments

The currently live mongodb.com is powered by Mongoid 5.

Support

We now officially support Mongoid 5, and will continue to support all future versions. Previous versions of Mongoid running on Moped are not supported. To report an issue or request a feature, please open a ticket at the Mongoid Jira Project.

Want more MongoDB? Read our white paper to learn about the latest features in MongoDB 3.0.
What's new in MongoDB 3.0

About the Author - Durran Jordan

Durran is a software engineer in the MongoDB integrations and Ruby driver teams, focused on developing 3rd party integrations with MongoDB and tools in the Ruby developer community, respectively. He is the original author of Mongoid and has been active in its development over the past 6 years.

← Previous

Introducing the 1.0 Perl Driver

The Perl Driver team is thrilled to announce the release of the v1.0.0 ("v1") MongoDB Perl driver! ###Goals of the Next-Generation Perl Driver In the next-generation MongoDB drivers announcement, we described why next-generation drivers follow a common set of behavior specification documents. Whereas previous drivers were written idiosyncratically, each driver now aims to deliver similar underlying behaviors through similar APIs, while still striving to be idiomatic for its language. For example, the v1 Perl driver implements the new MongoDB Driver CRUD API specification

September 24, 2015
Next →

Accelerating to T+1 - Have You Got the Speed and Agility Required to Meet the Deadline?

On May 28, 2024, the Securities and Exchange Commission (SEC) will implement a move to a T+1 settlement for standard securities trades , shortening the settlement period from 2 business days after the trade date to one business day. The change aims to address market volatility and reduce credit and settlement risk. The shortened T+1 settlement cycle can potentially decrease market risks, but most firms' current back-office operations cannot handle this change. This is due to several challenges with existing systems, including: Manual processes will be under pressure due to the shortened settlement cycle Batch data processing will not be feasible To prepare for T+1, firms should take urgent action to address these challenges: Automate manual processes to streamline them and improve operational efficiency Event-based real-time processing should replace batch processing for faster settlement In this blog, we will explore how MongoDB can be leveraged to accelerate manual process automation and replace batch processes to enable faster settlement. What is a T+1 and T+2 settlement? T+1 settlement refers to the practice of settling transactions executed before 4:30pm on the following trading day. For example, if a transaction is executed on Monday before 4:30 pm, the settlement will occur on Tuesday. This settlement process involves the transfer of securities and/or funds from the seller's account to the buyer's account. This contrasts with the T+2 settlement, where trades are settled two trading days after the trade date. According to SEC Chair Gary Gensler , “T+1 is designed to benefit investors and reduce the credit, market, and liquidity risks in securities transactions faced by market participants.” Overcoming T+1 transition challenges with MongoDB: Two unique solutions 1. The multi-cloud developer data platform accelerates manual process automation Legacy settlement systems may involve manual intervention for various tasks, including manual matching of trades, manual input of settlement instructions, allocation emails to brokers, reconciliation of trade and settlement details, and manual processing of paper-based documents. These manual processes can be time-consuming and prone to errors. MongoDB (Figure 1 below) can help accelerate developer productivity in several ways: Easy to use: MongoDB is designed to be easy to use, which can reduce the learning curve for developers who are new to the database. Flexible data model: Allows developers to store data in a way that makes sense for their application. This can help accelerate development by reducing the need for complex data transformations or ORM mapping. Scalability: MongoDB is highly scalable , which means it can handle large volumes of trade data and support high levels of concurrency. Rich query language: Allows developers to perform complex queries without writing much code. MongoDB's Apache Lucene-based search can also help screen large volumes of data against sanctions and watch lists in real-time. Figure 1: MongoDB's developer data platform Discover the developer productivity calculator . Developers spend 42% of their work week on maintenance and technical debt. How much does this cost your organization? Calculate how much you can save by working with MongoDB. 2. An operational trade store to replace slow batch processing Back-office technology teams face numerous challenges when consolidating transaction data due to the complexity of legacy batch ETL and integration jobs. Legacy databases have long been the industry standard but are not optimal for post-trade management due to limitations such as rigid schema, difficulty in horizontal scaling, and slow performance. For T+1 settlement, it is crucial to have real-time availability of consolidated positions across assets, geographies, and business lines. It is important to note that the end of the batch cycle will not meet this requirement. As a solution, MongoDB customers use an operational trade data store (ODS) to overcome these challenges for real-time data sharing. By using an ODS, financial firms can improve their operational efficiency by consolidating transaction data in real-time. This allows them to streamline their back-office operations, reduce the complexity of ETL and integration processes, and avoid the limitations of relational databases. As a result, firms can make faster, more informed decisions and gain a competitive edge in the market. Using MongoDB (Figure 2 below), trade desk data is copied into an ODS in real-time through change data capture (CDC), creating a centralized trade store that acts as a live source for downstream trade settlement and compliance systems. This enables faster settlement times, improves data quality and accuracy, and supports full transactionality. As the ODS evolves, it becomes a "system of record/golden source" for many back office and middle office applications, and powers AI/ML-based real-time fraud prevention applications and settlement risk failure systems. Figure 2: Centralized Trade Data Store (ODS) Managing trade settlement risk failure is critical in driving efficiency across the entire securities market ecosystem. Luckily, MongoDB integration capabilities (Figure 3 below) with modern AI and ML platforms enable banks to develop AI/ML models that make managing potential trade settlement fails much more efficient from a cost, time, and quality perspective. Additionally, predictive analytics allow firms to project availability and demand and optimize inventories for lending and borrowing. Figure 3: Event-driven application for real time monitoring Summary Financial institutions face significant challenges in reducing settlement duration from two business days (T+2) to one (T+1), particularly when it comes to addressing the existing back-office issues. However, it's crucial for them to achieve this goal within a year as required by the SEC. This blog highlights how MongoDB's developer data platform can help financial institutions automate manual processes and adopt a best practice approach to replace batch processes with a real-time data store repository (ODS). With the help of MongoDB's developer data platform and best practices, financial institutions can achieve operational excellence and meet the SEC's T+1 settlement deadline on May 28, 2024. In the event of T+0 settlement cycles becoming a reality, institutions with the most flexible data platform will be better equipped to adjust. Top banks in the industry are already adopting MongoDB's developer data platform to modernize their infrastructure, leading to reduced time-to-market, lower total cost of ownership, and improved developer productivity. Looking to learn more about how you can modernize or what MongoDB can do for you? Zero downtime migrations using MongoDB’s flexible schema Accelerate your digital transformation with these 5 Phases of Banking Modernization Reduce time-to-market for your customer lifecycle management applications MongoDB’s financial services hub

May 25, 2023