End of a Legacy : A Proposal to End-of-Life Our Generic Linux Tar Packages

Matt Lord
December 17, 2018 | Updated: February 1, 2019
#Community#Technical#Developer

TL;DR: We are planning to end distribution of the Legacy Generic Linux x64 Tar packages starting with MongoDB 4.2. We are seeking feedback from the community before a final decision is made.

Computing History

In a bygone era magnetic tape was a primary storage medium and the simple tar archive (tar) was the method and medium used to create bundles of files — A.K.A. a "tarball" — and archive them on the tape.

The fathers of Unix — Dennis Ritchie and Ken Thompson — at Bell Labs circa 1970
The fathers of Unix — Dennis Ritchie and Ken Thompson — at Bell Labs circa 1970

These tar files also became a convenient and common way to distribute software — which is precisely what we did at MongoDB for Unix-like systems early in our own history.

Modern Computing

While this was a wonderful time in the history of computing, a lot has changed. All popular operating systems (OS) now have native software package managers — APT, YUM, Zypper, and Pacman on Linux, Homebrew on macOS, Nuget/Chocolatey on Windows, and pkg on *BSD — and software vendors try and integrate well into those native OS ecosystems in order to provide a more seamless integration with other OS packages and to make the management of software stacks easier for operations and administration teams. There are myriad of reasons to use the native OS package managers, for example:

  • There's a central set of repositories of all software for the system.
  • All software can be easily discovered and managed with the same set of tools.
  • Packages can automatically manage and enforce any dependencies.
  • Packages can integrate with system security features such as SELinux.
  • Packages can integrate with system service management frameworks such as Systemd.
  • Packages can easily be kept up to date through automated upgrades.

It's for these reasons, and many more, that we wish to direct more of our available resources toward creating additional native OS packages and improving existing ones.

MongoDB Tar Packages

We have one specific legacy artifact around from the time when we did not integrate with these native OS package managers, nor even build tars for specific OS variants. This legacy artifact is our Legacy Generic x64 Linux Tar package — a legacy that we are planning to End-of-Life with MongoDB 4.2. The primary reasons for doing so are:

  • It has an opportunity cost because it takes up finite resources that can otherwise go toward providing MongoDB users with other value.
  • The generic nature of these packages make it so that we cannot dynamically link in other system packages within our build system, most notably OpenSSL and cURL (which is used for several MongoDB features).
    • This means that these binaries do not support SSL and should not be used in production.

However, tar packages do still provide user value and meet some use cases that native package managers can still struggle a bit with, for example:

  • Installing software without elevated system privileges.
  • Installing and maintaining many versions of a piece of software simultaneously.
  • Installing software within a single self-contained directory.

It's for these reasons that we will continue to provide tar (and zip) packages for specific OS variants — for example, RHEL 7 x86_64 — in order to meet these use cases. We believe that these packages will allow us to meet those (shrinking) use cases where tars are still a great solution, but without incurring the security, performance, and feature loss penalties that the Legacy Generic x64 Linux Tar package does.

Request for Comment

We wanted to announce our intentions well before the 4.2 GA release in order to give our users ample time to consider this change and provide us with feedback. We'd love to know what specific pains this may cause for you and how we might try and better address them. For example:

  • By adding official MongoDB, Inc. support for additional OS variants.
  • By working with OS package maintainers to add MongoDB packages where official MongoDB, Inc. provided ones do not exist.

So please let us know! You can comment here on the post, open a ticket in Jira, send an email to the user mailing list, grab me on the community slack channel, or send me a direct email. We need your input and look forward to some good discussions on the topic. This decision is not final and you as a valued MongoDB user have an opportunity to help drive our product direction.

THANK YOU for using MongoDB!

← Previous

MongoDB Stitch Functions – The AWS re:Invent Stitch Rover Demo

Using MongoDB Stitch functions to aggregate sensor data from our re:invent MongoDB Mobile/Stich Mars rover

December 17, 2018
Next →

Accelerating to T+1 - Have You Got the Speed and Agility Required to Meet the Deadline?

On May 28, 2024, the Securities and Exchange Commission (SEC) will implement a move to a T+1 settlement for standard securities trades , shortening the settlement period from 2 business days after the trade date to one business day. The change aims to address market volatility and reduce credit and settlement risk. The shortened T+1 settlement cycle can potentially decrease market risks, but most firms' current back-office operations cannot handle this change. This is due to several challenges with existing systems, including: Manual processes will be under pressure due to the shortened settlement cycle Batch data processing will not be feasible To prepare for T+1, firms should take urgent action to address these challenges: Automate manual processes to streamline them and improve operational efficiency Event-based real-time processing should replace batch processing for faster settlement In this blog, we will explore how MongoDB can be leveraged to accelerate manual process automation and replace batch processes to enable faster settlement. What is a T+1 and T+2 settlement? T+1 settlement refers to the practice of settling transactions executed before 4:30pm on the following trading day. For example, if a transaction is executed on Monday before 4:30 pm, the settlement will occur on Tuesday. This settlement process involves the transfer of securities and/or funds from the seller's account to the buyer's account. This contrasts with the T+2 settlement, where trades are settled two trading days after the trade date. According to SEC Chair Gary Gensler , “T+1 is designed to benefit investors and reduce the credit, market, and liquidity risks in securities transactions faced by market participants.” Overcoming T+1 transition challenges with MongoDB: Two unique solutions 1. The multi-cloud developer data platform accelerates manual process automation Legacy settlement systems may involve manual intervention for various tasks, including manual matching of trades, manual input of settlement instructions, allocation emails to brokers, reconciliation of trade and settlement details, and manual processing of paper-based documents. These manual processes can be time-consuming and prone to errors. MongoDB (Figure 1 below) can help accelerate developer productivity in several ways: Easy to use: MongoDB is designed to be easy to use, which can reduce the learning curve for developers who are new to the database. Flexible data model: Allows developers to store data in a way that makes sense for their application. This can help accelerate development by reducing the need for complex data transformations or ORM mapping. Scalability: MongoDB is highly scalable , which means it can handle large volumes of trade data and support high levels of concurrency. Rich query language: Allows developers to perform complex queries without writing much code. MongoDB's Apache Lucene-based search can also help screen large volumes of data against sanctions and watch lists in real-time. Figure 1: MongoDB's developer data platform Discover the developer productivity calculator . Developers spend 42% of their work week on maintenance and technical debt. How much does this cost your organization? Calculate how much you can save by working with MongoDB. 2. An operational trade store to replace slow batch processing Back-office technology teams face numerous challenges when consolidating transaction data due to the complexity of legacy batch ETL and integration jobs. Legacy databases have long been the industry standard but are not optimal for post-trade management due to limitations such as rigid schema, difficulty in horizontal scaling, and slow performance. For T+1 settlement, it is crucial to have real-time availability of consolidated positions across assets, geographies, and business lines. It is important to note that the end of the batch cycle will not meet this requirement. As a solution, MongoDB customers use an operational trade data store (ODS) to overcome these challenges for real-time data sharing. By using an ODS, financial firms can improve their operational efficiency by consolidating transaction data in real-time. This allows them to streamline their back-office operations, reduce the complexity of ETL and integration processes, and avoid the limitations of relational databases. As a result, firms can make faster, more informed decisions and gain a competitive edge in the market. Using MongoDB (Figure 2 below), trade desk data is copied into an ODS in real-time through change data capture (CDC), creating a centralized trade store that acts as a live source for downstream trade settlement and compliance systems. This enables faster settlement times, improves data quality and accuracy, and supports full transactionality. As the ODS evolves, it becomes a "system of record/golden source" for many back office and middle office applications, and powers AI/ML-based real-time fraud prevention applications and settlement risk failure systems. Figure 2: Centralized Trade Data Store (ODS) Managing trade settlement risk failure is critical in driving efficiency across the entire securities market ecosystem. Luckily, MongoDB integration capabilities (Figure 3 below) with modern AI and ML platforms enable banks to develop AI/ML models that make managing potential trade settlement fails much more efficient from a cost, time, and quality perspective. Additionally, predictive analytics allow firms to project availability and demand and optimize inventories for lending and borrowing. Figure 3: Event-driven application for real time monitoring Summary Financial institutions face significant challenges in reducing settlement duration from two business days (T+2) to one (T+1), particularly when it comes to addressing the existing back-office issues. However, it's crucial for them to achieve this goal within a year as required by the SEC. This blog highlights how MongoDB's developer data platform can help financial institutions automate manual processes and adopt a best practice approach to replace batch processes with a real-time data store repository (ODS). With the help of MongoDB's developer data platform and best practices, financial institutions can achieve operational excellence and meet the SEC's T+1 settlement deadline on May 28, 2024. In the event of T+0 settlement cycles becoming a reality, institutions with the most flexible data platform will be better equipped to adjust. Top banks in the industry are already adopting MongoDB's developer data platform to modernize their infrastructure, leading to reduced time-to-market, lower total cost of ownership, and improved developer productivity. Looking to learn more about how you can modernize or what MongoDB can do for you? Zero downtime migrations using MongoDB’s flexible schema Accelerate your digital transformation with these 5 Phases of Banking Modernization Reduce time-to-market for your customer lifecycle management applications MongoDB’s financial services hub

May 25, 2023