10gen will be at OSCON, the Open Source Convention, this week to meet with other open source enthusiasts. Stop by Booth 706 to meet 10gen Engineers and pick up your MongoDB T-Shirt.
Catch talks from 10gen and the MongoDB Community at OSCON this week:
- MongoDB - From Zero to Sharded with Shaun Verch, MongoDB Kernel Engineer, 10gen on Monday, July 22 at 1:30PM in Portland 252.
- Choosing A Shard Key In MongoDB with Shaun Verch, MongoDB Kernel Engineer, 10gen Wednesday, July 24 at 1:40PM in E143.
- Creating a User Journey for Your Open Source Community with Francesca Krihely, MongoDB Community Marketing Manager, 10gen, Thursday July 25 at 2:30PM in E144.
- MongoDB on Amazon Web Services: Operational Best Practices with Charity Majors, Engineer at Parse at 10AM Friday in E145.
Deadline 6: Powered by MongoDB
Thinkbox Software recently announced Deadline 6 , the newest version of their render software, which is powered by MongoDB. Deadline is a render management tool, used heavily in the visual effects, broadcast and architectural industries. Thinkbox’s Deadline team chose MongoDB as their database of choice for its JSON-based document structure and speed at handling connections. A bit of background on render management for those unfamiliar: when making a film, production teams create and render thousands to millions of individual frames for the layers of their shots. Computers process your image data for compositing blue screens, characters or rendering fire, smoke and other special effects. “There are a lot of products that make this happen,” says Chris Bond, founder and CEO of Thinkbox Software. “They all render, control and work differently depending on your OS platform.” In the past, filmmakers or graphic specialists would manually split these jobs up between a number of machines to distribute the amount of data for processing jobs. There are thousands of frames in a shot, and thousands of shots in a movie. Some rendering processes could take 24 hours, while others could take 20 minutes. There wasn't the virtualization platform that could distribute these tasks and manage these processes, so they were typically done manually or executed by scripts. Deadline solves this problem as a management queue for rendering farms. Deadline 6's Monitoring Dashboard Deadline currently supports over 40 applications and operating systems across the board, and breaks up shots and frames into tasks across different machines. Many of this year’s summer movies such as Man of Steel, Iron Man 3, Oblivion, and 300: Rise of an Empire, are made with Deadline 6 and used MongoDB as the backend. The team switched to MongoDB in order to improve create a tool that could run fast while operating on tens of thousands of tasks at a time. Up until Deadline 5, the team was using an XML, file-based storage system which was easy for rapid development, like JSON, and it was schema-less, so they didn't have to worry about tables. “Deadline was designed as a ‘serverless’ architecture.” said Ryan Russell, lead developer on Deadline 6. “Rather than having a central application telling its clients what to do, the clients would go to a central data store, based on the data in there, it would take the next step.” The client would go to the storage location where these "jobs" are housed and would pick up and frame and then pick up another frame. “It was a solid backend for our software, because Deadline could never ’go down’ unless our server hosting that data went down.” But a file-based system is hard to scale and the team needed a way to help Deadline scale and hit higher performance numbers. When searching for a new solution, one of the developers stumbled across CouchDB. “It matched our document-based system, and when we investigated further, the team kept seeing MongoDB come up in searches and blog posts.” For them, the 1 to 1 XML to JSON mapping was a winning feature for both document stores, but to get a better understanding, they decided to build a prototype with each data store to see which one performed better. MongoDB won out in the end. “MongoDB handled hundreds of connections at the same time without any issue. CouchDB had a hard time because of the RESTful API overhead. The improved performance with MongoDB created such a great experience for the end-user. That was a huge thing for us.” Select clients recently went through a 6 month a beta program with Deadline 6, and after using it in production, were amazed at how much faster their system was with MongoDB as the backend. In Deadline, MongoDB functions as a task queue, and controls the management of third party render processes. Each day, a large film set might render over 25-30 Terabytes of data. MongoDB handles the process management of each job and controls the data movement. To the team MongoDB “seems like something we can stand on and scale in many orders of magnitude more than we have before.” Look out for a film powered by Deadline and MongoDB this summer.
Accelerate Data Modernization with Infosys Data Model Converter
Are you in the process of migrating applications from a relational database to MongoDB? If so, you’re likely trying to best understand and decide how your enterprise data needs to be modeled. Our previous blog discussed how Infosys Data Services Suite can help enterprises move data seamlessly from legacy relational databases to MongoDB. But moving data is only one part of the puzzle. The more significant step is choosing the target data model, or schema design, a process that usually requires several hours of highly skilled talent. That’s why we created this follow-up blog to help you get started. Rethinking Schema Design Ultimately, schema design can be the difference between an inefficient, disorganized database and a strategic one that empowers the entire company. Schema design in MongoDB requires a change in perspective for data architects, developers, and database administrators. They have to: Rethink the legacy relational data model. This model flattens data into rigid two-dimensional tabular structures of rows and columns. The new data model is a rich and dynamic one with embedded sub-documents and arrays Rethink how the data platform works. In relational databases, it is extremely difficult to change the data platform as the application evolves. However, in MongoDB, the apps and APIs come first and the data platform dynamically accommodates the data Getting Schema Design Right Begin the schema design process by considering the application’s requirements. You’ll want to model the data in a way that leverages the flexibility of the document model. In schema migrations, it may seem easy at first to simply mirror the flat schema of the relational database in the document model. However, this negates the advantages enabled by the rich and embedded data structures of the document model. For example, data that belongs to a parent-child relationship in two RDBMS tables can be collapsed (embedded) into a single document in MongoDB. The application data access patterns should also drive schema design with a specific focus on: The read/write ratio of database operations and whether it is more important to optimize the performance of one operation over another The types of queries and updates performed by the databases The lifecycle of the data and growth rate of documents Simplifying Schema Design with Infosys Data Model Converter Infosys has developed a solution called Infosys Data Model Convertor that processes source relational schema and the above-mentioned signals as inputs and automatically provides target MongoDB schema suggestions. Infosys Data Model Converter is available as part of Infosys Modernization Suite which accelerates enterprises’ modernization journey. Each schema suggestion is accompanied by a detailed analysis report. The data modeler can use this as a starting point and iterate over the schema to arrive at the final MongoDB schema. The Infosys Data Model Converter reduces 50-60% of the effort typically spent in schema design. Key Features Boosts productivity by augmenting the migration of RDBMS to NoSQL database Saves time by automatically extracting schema, query and data patterns from an existing RDBMS Comprehensively analyzes the RDBMS entity relations, data and read-and-write patterns Applies a rich set of rules and generates a fully compliant NoSQL target state data model Offers flexibility by externalizing the rules for organization-specific customizations Connects and deploys the model to the target NoSQL platform with sample data Discover more ways in which Infosys can help you unlock value from modernization. Contact us for any modernization questions.