Perl Driver 0.46.1 Released

MongoDB
September 5, 2012
#Releases

This was originally posted to Mike Friedman’s personal blog

I’m happy to announce that after a long delay, version 0.46.1 of the Perl MongoDB driver has now been uploaded to CPAN, and should be available on your friendly local CPAN mirror soon.

This release is mostly a series of minor fixes and housekeeping, in preparation for developing a more detailed roadmap for more frequent releases down the line. Here’s what’s new so far:

Most of the distribution has been successfully transitioned to Dist::Zilla for automated building, tagging, and releasing to CPAN. This has vastly reduced the amount of effort needed to get releases out the door.

The behind-the-scenes algorithm for validating UTF-8 strings has been replaced with a more compliant and much faster version. Thanks to Jan Anderssen for contributing the fix.

Serialization of regexes has been improved and now supports proper stripping of unsupported regex flags across all recent Perl versions. Thanks to Arkadiy Kukarkin for reporting the bug and @ikegami for help with figuring out how to serialize regexes properly via the Perl API.

The driver will now reject document key names with NULL bytes, a possible source of serious bugs. Additionally, much of the distribution metadata has been cleaned up, thanks to the automation provided by Dzil. In particular, the official distribution repository and bug-tracker links now point to our GitHub and JIRA sites. Hopefully more bugs will now come in via those channels instead of RT.

Looking ahead, there is a lot of work yet to be done. I have prioritized the following tasks for version 0.47, which should help us moving forward to an eventual 1.0 release.

Eliminating the dependency on Module::Install
Significantly re-working the documentation to include better organization and more examples.
Additionally, much of the current documentation will be refactored via Pod::Weaver.
Replacing AUTOLOADed database and collection methods with safer generated symbols upon connection. Beginning with 0.48, these will have a deprecation warning added and will be removed entirely before the 1.0 release in favor of the get_database and get_collection methods. The docs will be updated to reflect this change.

I’m very excited about the future of MongoDB support for Perl, and looking forward to improving the CPAN distribution in concert with the Perl community!

Mike Friedman is the Perl Engineer and Evangelist at 10gen, working on the Perl Driver for MongoDB. You can follow his blog at friedo.com

← Previous

Motor: Asynchronous Driver for MongoDB and Python

Tornado is a popular asynchronous Python web server. Alas, to connect to MongoDB from a Tornado app requires a tradeoff: You can either use PyMongo and give up the advantages of an async web server, or use AsyncMongo, which is non-blocking but lacks key features. I decided to fill the gap by writing a new async driver called Motor (for “MOngo + TORnado”) , and it’s reached the public alpha stage. Please try it out and tell me what you think. I’ll maintain a homepage for it here , including basic documentation. Motor Source Documentation Status Motor is alpha. It is certainly buggy. Its implementation and possibly its API will change in the coming months. I hope you’ll help me by reporting bugs, requesting features, and pointing out how it could be better. Advantages Two good projects, AsyncMongo and APyMongo, took the straightforward approach to implementing an async MongoDB driver: they forked PyMongo and rewrote it to use callbacks. But this approach creates a maintenance headache: now every improvement to PyMongo must be manually ported over. Motor sidesteps the problem. It uses a Gevent-like technique to wrap PyMongo and run it asynchronously, while presenting a classic callback interface to Tornado applications. This wrapping means Motor reuses all of PyMongo’s code and, aside from GridFS support, Motor is already feature-complete. Motor can easily keep up with PyMongo development in the future. Installation Motor depends on greenlet and, of course, Tornado. It is compatible with CPython 2.5, 2.6, 2.7, and 3.2; and PyPy 1.9. You can get the code from my fork of the PyMongo repo, on the motor branch: pip install tornado greenlet pip install git+https://github.com/ajdavis/mongo-python-driver.git@motor To keep up with development, watch my repo and do pip install -U git+https://github.com/ajdavis/mongo-python-driver.git@motor when you want to upgrade. Example Here’s an example of an application that can create and display short messages: import tornado.web, tornado.ioloop import motor class NewMessageHandler(tornado.web.RequestHandler): def get(self): """Show a 'compose message' form""" self.write(''' ''') # Method exits before the HTTP request completes, thus "asynchronous" @tornado.web.asynchronous def post(self): """Insert a message """ msg = self.get_argument('msg') # Async insert; callback is executed when insert completes self.settings['db'].messages.insert( {'msg': msg}, callback=self._on_response) def _on_response(self, result, error): if error: raise tornado.web.HTTPError(500, error) else: self.redirect('/') class MessagesHandler(tornado.web.RequestHandler): @tornado.web.asynchronous def get(self): """Display all messages """ self.write(' Compose a message ') self.write(' ') db = self.settings['db'] db.messages.find().sort([('_id', -1)]).each(self._got_message) def _got_message(self, message, error): if error: raise tornado.web.HTTPError(500, error) elif message: self.write('<li>%s</li>' % message['msg']) else: # Iteration complete self.write('</ul>') self.finish() db = motor.MotorConnection().open_sync().test application = tornado.web.Application([ (r'/compose', NewMessageHandler), (r'/', MessagesHandler) ], db=db ) print 'Listening on http://localhost:8888' application.listen(8888) tornado.ioloop.IOLoop.instance().start() Other examples are Chirp, a Twitter-like demo app, and Motor-Blog, which runs this site. Support For now, email me directly if you have any questions or feedback. Roadmap In the next week I’ll implement the PyMongo feature I’m missing, GridFS. Once the public alpha and beta stages have shaken out the bugs and revealed missing features, Motor will be included as a module in the official PyMongo distribution. A. Jesse Jiryu Davis

September 5, 2012

Next →

Accelerating to T+1 - Have You Got the Speed and Agility Required to Meet the Deadline?

On May 28, 2024, the Securities and Exchange Commission (SEC) will implement a move to a T+1 settlement for standard securities trades , shortening the settlement period from 2 business days after the trade date to one business day. The change aims to address market volatility and reduce credit and settlement risk. The shortened T+1 settlement cycle can potentially decrease market risks, but most firms' current back-office operations cannot handle this change. This is due to several challenges with existing systems, including: Manual processes will be under pressure due to the shortened settlement cycle Batch data processing will not be feasible To prepare for T+1, firms should take urgent action to address these challenges: Automate manual processes to streamline them and improve operational efficiency Event-based real-time processing should replace batch processing for faster settlement In this blog, we will explore how MongoDB can be leveraged to accelerate manual process automation and replace batch processes to enable faster settlement. What is a T+1 and T+2 settlement? T+1 settlement refers to the practice of settling transactions executed before 4:30pm on the following trading day. For example, if a transaction is executed on Monday before 4:30 pm, the settlement will occur on Tuesday. This settlement process involves the transfer of securities and/or funds from the seller's account to the buyer's account. This contrasts with the T+2 settlement, where trades are settled two trading days after the trade date. According to SEC Chair Gary Gensler , “T+1 is designed to benefit investors and reduce the credit, market, and liquidity risks in securities transactions faced by market participants.” Overcoming T+1 transition challenges with MongoDB: Two unique solutions 1. The multi-cloud developer data platform accelerates manual process automation Legacy settlement systems may involve manual intervention for various tasks, including manual matching of trades, manual input of settlement instructions, allocation emails to brokers, reconciliation of trade and settlement details, and manual processing of paper-based documents. These manual processes can be time-consuming and prone to errors. MongoDB (Figure 1 below) can help accelerate developer productivity in several ways: Easy to use: MongoDB is designed to be easy to use, which can reduce the learning curve for developers who are new to the database. Flexible data model: Allows developers to store data in a way that makes sense for their application. This can help accelerate development by reducing the need for complex data transformations or ORM mapping. Scalability: MongoDB is highly scalable , which means it can handle large volumes of trade data and support high levels of concurrency. Rich query language: Allows developers to perform complex queries without writing much code. MongoDB's Apache Lucene-based search can also help screen large volumes of data against sanctions and watch lists in real-time. Figure 1: MongoDB's developer data platform Discover the developer productivity calculator . Developers spend 42% of their work week on maintenance and technical debt. How much does this cost your organization? Calculate how much you can save by working with MongoDB. 2. An operational trade store to replace slow batch processing Back-office technology teams face numerous challenges when consolidating transaction data due to the complexity of legacy batch ETL and integration jobs. Legacy databases have long been the industry standard but are not optimal for post-trade management due to limitations such as rigid schema, difficulty in horizontal scaling, and slow performance. For T+1 settlement, it is crucial to have real-time availability of consolidated positions across assets, geographies, and business lines. It is important to note that the end of the batch cycle will not meet this requirement. As a solution, MongoDB customers use an operational trade data store (ODS) to overcome these challenges for real-time data sharing. By using an ODS, financial firms can improve their operational efficiency by consolidating transaction data in real-time. This allows them to streamline their back-office operations, reduce the complexity of ETL and integration processes, and avoid the limitations of relational databases. As a result, firms can make faster, more informed decisions and gain a competitive edge in the market. Using MongoDB (Figure 2 below), trade desk data is copied into an ODS in real-time through change data capture (CDC), creating a centralized trade store that acts as a live source for downstream trade settlement and compliance systems. This enables faster settlement times, improves data quality and accuracy, and supports full transactionality. As the ODS evolves, it becomes a "system of record/golden source" for many back office and middle office applications, and powers AI/ML-based real-time fraud prevention applications and settlement risk failure systems. Figure 2: Centralized Trade Data Store (ODS) Managing trade settlement risk failure is critical in driving efficiency across the entire securities market ecosystem. Luckily, MongoDB integration capabilities (Figure 3 below) with modern AI and ML platforms enable banks to develop AI/ML models that make managing potential trade settlement fails much more efficient from a cost, time, and quality perspective. Additionally, predictive analytics allow firms to project availability and demand and optimize inventories for lending and borrowing. Figure 3: Event-driven application for real time monitoring Summary Financial institutions face significant challenges in reducing settlement duration from two business days (T+2) to one (T+1), particularly when it comes to addressing the existing back-office issues. However, it's crucial for them to achieve this goal within a year as required by the SEC. This blog highlights how MongoDB's developer data platform can help financial institutions automate manual processes and adopt a best practice approach to replace batch processes with a real-time data store repository (ODS). With the help of MongoDB's developer data platform and best practices, financial institutions can achieve operational excellence and meet the SEC's T+1 settlement deadline on May 28, 2024. In the event of T+0 settlement cycles becoming a reality, institutions with the most flexible data platform will be better equipped to adjust. Top banks in the industry are already adopting MongoDB's developer data platform to modernize their infrastructure, leading to reduced time-to-market, lower total cost of ownership, and improved developer productivity. Looking to learn more about how you can modernize or what MongoDB can do for you? Zero downtime migrations using MongoDB’s flexible schema Accelerate your digital transformation with these 5 Phases of Banking Modernization Reduce time-to-market for your customer lifecycle management applications MongoDB’s financial services hub

May 25, 2023