How Medtronic Manages Machine Data in MongoDB
While many think Big Data is all about “big,” the reality for most organizations is that data variety is a far thornier problem to tackle. Just ask Medtronic . Medical equipment maker Medtronic, perhaps best known for its pacemakers, offers devices and therapies that address more than 30 diseases. Last year the company served 9 million patients and this year the company announced that it serves a patient in some way every three seconds. In addition, Medtronic collects more than 30 million data samples about its devices every day. Matthew Chimento, principal test engineer and project manager at Medtronic, notes that more than 150 data collection and processing steps have been added to Medtronic’s manufacturing process in the last three years, and 40% of all of Medtronic’s stored data has been collected in the last two. Humans aren’t great at collecting data, but machines are, and “we have a lot of machines.” Now if only those machines all spoke the same language. Data Variety: Problem And Opportunity Unfortunately, with a proliferation of machines comes a proliferation of different data types. And while the media likes to talk about “Big Data” as if it were all about volume, companies like Medtronic realize Big Data is primarily a matter of data variety, as a NewVantage Partners survey discovered: Furthermore, for regulatory reasons, Medtronic must save device data for 10 years after the last implant of the device. Since those devices can last 20 years, some data is 30 years old, which means that Medtronic must contend with information spread across a multitude of obscure database systems, in a wide variety of formats. Does Your Data Speak MongoDB? To manage this data complexity, Medtronic turned to MongoDB. About two years ago, Chimento’s colleague, Jeff Lemmerman, a senior software engineer at Medtronic, heard about MongoDB. Intrigued by the NoSQL database and its potential to help Medtronic tame its ever-changing data requirements, Lemmerman launched a proof of concept, which “basically consisted of choosing one battery model that we manufacture.” When the battery goes through electrolyte fill – a step in the manufacturing process – all of the component data is loaded into MongoDB. “This is a very simple place to start,” he said. Lemmerman has high hopes for the next steps with MongoDB. He hopes to begin loading manufacturing data on every component Medtronic makes directly into MongoDB, and aggregating that data into a device-level view, and MongoDB’s data model will make that easy. “You’re trying to facilitate analysis across components, and you really want simple, fast queries … instead of doing those nasty joins that we saw in my relational example, I’m able to find the complete history for a component with a very simple query.” What Can MongoDB Do For You? Like Medtronic, your data changes constantly as business requirements change. And, like Medtronic and most enterprises, you likely use a relational database to manage that data. For reasons noted above, as well as here , an RDBMS is a poor fit for data that changes often or for applications that need to scale. We therefore invite you to check out the RDBMS to MongoDB Migration Guide to determine how best to migrate data from your RDBMS to MongoDB.
The Future Of Big Data On Display At MongoDB World This Week
Tens of thousands of organizations depend upon MongoDB, from the largest Fortune 500 enterprise to the smallest startup, and everything in between. But behind every company logo is a developer, and working alongside that developer is an operations professional. Ultimately, the success of organizations of any size depends upon these " kingmakers ." This week nearly 2,000 "kingmakers," along with the IT executives who love them and the system integrations, ISVs and IHVs who support them will gather in New York City for the inaugural MongoDB World . While MongoDB has hosted a range of events all over the planet, this is our biggest event yet, significantly overshadowing any other MongoDB event in size, scope and focus. I joined MongoDB nearly two years ago, and in that time I've seen the company more than double in size while the open source community around MongoDB has grown even more. In September 2012 I could never have guessed that we'd stage an event that would draw senior executives from Amazon, Citi, the U.S. Department of Veterans Affairs, Bosch, Verizon Wireless, Expedia, Sanofi-Aventis, Facebook and more. Even more impressive, these massive organizations would be joined on stage by cutting-edge startups like Hudl, Crittericism, Bugsnag, Cumulogic and others. The MongoDB community is its greatest strength, and this week we get to see that community in full force. If you haven't known whether to attend or not, you should. You will see how organizations of all kinds are building the future on modern data technologies like MongoDB and Hadoop. If attendance simply isn't possible, please see below for a way to participate through video. Working at MongoDB, I get to see the future written each day, app by app. But you're in an even better position: by using MongoDB, you get to invent the future. Please join me at MongoDB World - whether in person or by video - and become part of the biggest, fastest growing Big Data community on earth . To see all MongoDB World presentations, visit the [MongoDB World Presentations](https://www.mongodb.com/mongodb-world/presentations) page.
MongoDB And Teradata Join Forces To Make Big Data Smart
As enterprises increasingly depend on MongoDB to build and run modern applications, they need high-quality analytics solutions to match MongoDB's powerful data model. With the partnership Teradata and MongoDB just announced , they just got one. And it's exceptionally cool. With data analytics leader Teradata we've built a bi-directional connector that gives organizations interactive data processing at extremely fast speeds. Teradata's bi-directional QueryGrid connector allows Teradata customers to integrate massive volumes of JSON with cross-organizational data in the data warehouse for high performance analytics. Through the connector, MongoDB customers will have access to JSON that has been enriched by Teradata to support rapidly evolving applications for mobile, Internet of Things, eCommerce, social media and other applications. In other words, users will soon be able to easily connect MongoDB applications and analytics running on Teradata. The Future Is JSON For the past 40 years, enterprises have stored their data in the tidy-but-rigid tables and joins of relational databases. Given the explosion of unstructured data, however, enterprises need a more expressive, flexible way of describing and storing data. Enter JSON. MongoDB stores data in JSON documents, which we serialize to BSON . JSON provides a rich data model that seamlessly maps to native programming language types, and the dynamic schema makes it easier to evolve one's data model than with a system that enforces schemas like a relational database (RDBMS). Marrying MongoDB's operational database with Teradata's analytics platform a great way to bring together all of an enterprise's data. A Virtuous Cycle One way of thinking about the interaction between MongoDB and Teradata is to picture a crowd of people. MongoDB interacts with individuals within the crowd in real-time while Teradata looks for patterns within the crowd. With this connector, organizations can push their MongoDB data (website clicks, purchases, etc.) into Teradata, which runs queries against the data, looking for patterns. This intelligence is then pushed back to MongoDB, enriching the interaction with individual eCommerce buyers, mobile users, etc. It's a virtuous cycle, as Teradata describes on its blog . Here's what this looks like for an eCommerce application: By bringing the two together, an eCommerce vendor's interactions with its customers will continuously improve as their MongoDB-based application gets smarter and more tailored by Teradata analytics. Importantly, for enterprises that expect to use both relational databases and MongoDB, Teradata's JSON integration unifies relational and MongoDB data analysis. And, Not Or This last point is worth repeating. As much as enterprises might wish to shed their IT investments and start over, the reality is that they can't and won't, as a 2012 Gartner analysis found: By giving organizations an easy way to connect MongoDB's operational data with Teradata's enterprise data warehouse, the two organizations ensure existing and new data sources can coexist. By working closely together, MongoDB and Teradata give enterprises the best of a modern, operational database with a powerful analytics platform.
You Know What's Cool? 1 Trillion Is Cool
A million used to be cool. Then Facebook upped the ante to one billion. But in our world of Big Data, even a billion is no longer the upper end of scale, or cool. As I learned last night, at least one MongoDB customer now stores over 1 trillion documents in MongoDB. 1 trillion . That's cool. It's also far bigger than any other database deployment I've seen from any NoSQL or relational database, even from the simple key-value or columnar data stores that are only programmed to handle simple workloads, but to scale them well. That's what makes MongoDB über cool: not only does it offer dramatic, superior scale , but it does so while also giving organizations the ability to build complex applications. MongoDB delivers the optimal balance between functionality and performance, as this illustrates: Many systems are focused on nothing more than storing your data, and letting you access it quickly, but one and only one way. This simply isn’t enough . A truly modern database must support rich queries, indexing, analysis, aggregation, geospatial access and search across multi-structured, rapidly changing data sets in real time. The database must not trap your data and hinder its use. It must unleash your data . All 1 trillion documents of it. Want to see how major Global 2000 organizations like Bosch, U.S. Department of Veterans Affairs, Genentech, Facebook and many others scale with MongoDB? Easy. Just register to attend MongoDB World, June 24-25 in New York City. You can use my discount code to get 25% off: 25MattAsay.
Beyond NoSQL: A Modern Database Manifesto
There is no such thing as NoSQL. Not as we tend to think of it, anyway. While NoSQL was born as a movement away from rigid relational data models so web giants could embrace Big Data with scale-out architectures, the term has come to categorize a set of databases that are more different than they are the same. This broad categorization doesn’t work. It’s not helpful. While we at MongoDB still sometimes refer to NoSQL, we try to do it sparingly, given its propensity to confuse rather than enlighten. Deconstructing NoSQL Today the NoSQL category includes a cacophony of over 100 document, key-value, wide-column and graph databases . Each of these database types comes with its own strengths and limits. Each differs markedly from the others, with disparate models and capabilities relative to data storage, querying, consistency, scalability and high availability. Comparing a document database to a key-value store, for example, is like comparing a smartphone to a beeper. A beeper is exceptionally useful for getting a simple message from Point A to Point B. It’s fast. It’s reliable. But it’s nowhere near as functional as a smartphone, which can quickly and reliably transmit messages, but can also do so much more. Both are useful, but the smartphone fits a far broader range of applications than the more limited beeper. As such, organizations searching for a database to tackle Gartner’s three V’s of Big Data -- volume, velocity and variety -- won’t find an immediate answer in “NoSQL.” Instead, they need to probe deeper for a modern database that can handle all of their Big Data application requirements. Modern Databases For Modern Data One of these requirements is, of course, the ability to handle large volumes of data, the original impetus behind the NoSQL movement. But the ability to handle volume, or scale, is something all databases categorized as “NoSQL” share. MongoDB, for example, counts among its users those who regularly store petabytes of data, perform over 1,000,000 operations per second and clusters that exceed 1,000 nodes. A modern database, however, must do more than scale. Scalability is table stakes. It also must enable agility to accelerate development and time to market. It must allow organizations to iterate as they embrace new business requirements. And a modern database must, above all, enable enterprises to take advantage of rapidly growing data variety. Indeed the “greatest challenge and opportunity” for enterprises, as Forrester notes, is managing a “variety of data sources,” including data types and sources that may not even exist today. In general, all so-called NoSQL databases are much more helpful than relational databases at storing a wide variety of data types and sources, including mobile device, geospatial, social and sensor data. But the hallmark of a modern database its ability to allow organizations to do useful things with their data. Defining The Modern Database To count as a modern database, then, a database must meet three requirements. While relational databases are able to manage some of these requirements, and newer so-called “NoSQL” key-value or wide column data stores meet others, only MongoDB meets all three requirements. The database MUST scale . As data volume and velocity grows, so the database must grow too. It should scale horizontally and elegantly, without doing unnatural things to your application, in the cloud or on commodity hardware. Meeting the base requirements -- like having enough capacity to serve your customers -- should be a given. The database MUST adapt to change . The speed of business accelerates and your database must keep pace, enabling iteration. This means you must be able to process and mine new data sources and data types without the database breaking a sweat (or you breaking your back or budget). Your schema must flow from your application requirements, rather than forcing your application to fit a predefined, rigid schema. The database MUST unleash your data . Just storing data isn’t enough. You must be able to exploit the data, which particularly means you must be able to ask significant questions of your data. In part this means that the database must support rich queries, indexing, aggregation and search across multi-structured, rapidly changing data sets in real time. But it also means that it must support data for modern use cases including mobile, social, Internet of Things and other systems of engagement. Some relational databases can handle a few of these requirements, yet fail in the essential need to deliver scale and adaptability. Some newer databases, including so-called “NoSQL” key-value or wide column data stores, meet still other requirements, yet don’t give organizations the latitude to unleash their data. In fact, they constrain you to look up data by the key with which it was written unless you integrate external search engines and analytics nodes, which can create other problems. MongoDB: A Modern Database For Today's Business Needs But only one database today can deliver on each of these critical components of a modern database. Only one database offers orders of magnitude more productivity for developers and operations teams alike, while still delivering petabyte scale and lightning-fast performance. Only MongoDB, the modern database that tens of thousands of organizations depend upon to build and run today’s most demanding applications. To learn more about how MongoDB has enabled some of the world’s largest and most innovative companies to deliver applications and outcomes that were previously impossible, download our new whitepaper .
Looking beyond labels like relational and NoSQL
According to a new Dice.com salary survey , MongoDB ranks as one of top-10 most highly compensated technology skills. Indeed.com rates MongoDB as the second hottest job trend. And DB-Engines.com, which ranks over 200 databases on their relative popularity, MongoDB is now the fifth-most popular database in the world, this month surpassing IBM's DB2. All great, right? Maybe. Buried in the Dice.com data, as well as the Indeed.com data, is evidence of real confusion. For example, of the top-10 most highly compensated skills in Dice.com's survey is "NoSQL ." NoSQL is not a technology. It's not really something a developer can "know" in any real sense. NoSQL is a movement that describes a different way of modeling data but, as Basho founder Justin Sheehy correctly noted , there are as many differences among so-called NoSQL databases as there are similarities. As such, knowing Basho's Riak won't really help you understand MongoDB. Perhaps at a high, conceptual level, but expertise in one doesn't really translate into familiarity with another. They are different databases with different approaches. Employers looking for generic NoSQL skills need to think more deeply about what their application requirements are. Looking beyond relational databases for modern application requirements is a good start, but looking to generic "NoSQL" is not sufficient. Organizations should be looking for a modern database that dramatically improves developer productivity, encourages application iteration and enables a new wave of transformational applications in areas like Big Data , Internet of Things , mobile and more . That database is MongoDB. Is MongoDB "NoSQL." Sure. But it's much bigger than that ( based on what people search for on Google , many organizations already seem to understand this). MongoDB is the fastest-growing database in the world , not because it fits the NoSQL category, but because it significantly improves the productivity of developers and the organizations for which they work. So if you're looking to hire technology talent, you're far more likely to be successful hiring an experienced MongoDB engineer than a "NoSQL engineer." MongoDB, after all, is an actual database. NoSQL simply describes an important movement.
MongoDB Named InfoWorld 2014 Technology of the Year: It's A Matter Of Innovation
When it rains, it pours. Right on the heels of being named DB-Engines' 2013 Database of the Year and Linux Journal's Best NoSQL Database , InfoWorld has given MongoDB its 2014 Technology of the Year award , alongside Amazon Web Services and GitHub, among others. More than just point solutions to finite business problems, InfoWorld's list includes technologies that "point the way to the data centers, clouds, and applications of tomorrow. They’re the innovations that are changing the way we work and do business," as Doug Dineley, executive editor of InfoWorld’s Test Center, declares . Sometimes innovation is about lower costs. For example, one of the biggest advantages Hadoop brings is enabling data analytics on commodity hardware, as opposed to the expensive, proprietary solutions of yesterday. The real value of Linux, in its early years, was arguably less about product innovation and more a matter of helping enterprises transition away from expensive UNIX servers. MongoDB enables a different type of innovation. Yes, MongoDB is dramatically less expensive than licensing and running a proprietary relational database. But that's not what has made it the fastest-growing, most popular non-relational database (by a wide, wide margin). Instead, MongoDB is popular because it reinvents data management, enabling developers to write a new breed of application that is impossible, or exceptionally difficult, with a relational database. Part of this is a matter of simplifying data schema: And part of it is allowing the developer to focus on her application (pictured as a car in the graphic below), and not the unnecessary overhead of object relational mapping and upkeep on a rigid, relational schema: But the overall value is about enabling and enobling developers, giving them power to get work done for the line of business tasked with new marketing initiatives, optimizing business processes and more. Ultimately, then, MongoDB has won InfoWorld's 2014 Technology of the Year award because it brings innovation back to the data management market, something that has been sorely lacking for a long time.
MongoDB Named 2013 Database Of The Year: Why This Matters
In early December MongoDB was named the most popular database among Linux users . While a great commendation of the work MongoDB's development community has done, such accolades don't fully convey the breadth and depth of MongoDB's popularity. To get a more complete picture of just how successful MongoDB has been, it's important to factor in a number of different measures... ...which is precisely what the DB-Engines database ranking does. A Comprehensive View Of Database Popularity More than a simple popularity contest, DB-Engines aggregates data on a number of factors , including job creation, professional certifications, social media mentions, Google searches and more to yield a comprehensive view of a database's impact. For example, as nice as it is to have lots of people running Google queries on "MongoDB," it's far more potent to know that employers are hiring tens of thousands of MongoDB-experienced developers or that dramatically more industry professionals cite MongoDB as a technical competency than they do any other modern database: But synthesizing all three, along with information on technical discussions online, website mentions and more, gives a pretty complete view of a database's importance. MongoDB: 2013 Database Of The Year Which is why we're so happy that DB-Engines has named MongoDB the 2013 database management system of the year , based on MongoDB's growth. As noted by Solid IT, the company that compiles the DB-Engines ranking, "MongoDB is the database management system that gained more popularity in our DB-Engines Ranking within the last year than any other system." Importantly, MongoDB's gains weren't calculated by measuring a percentage gain, but rather an absolute rise in popularity. More than any other NoSQL database? Yes. But also more than Oracle, Microsoft SQL Server and every other open source or proprietary relational database. A Modern Database For Modern Applications That's pretty impressive, and speaks to the shift to a modern database to solve modern application requirements . As good as relational databases were for the neat-and-tidy data of yesteryear, they're often a poor fit for today's applications that depend upon unstructured or semi-structured data. This honor underscores something that is increasingly clear: MongoDB is very, very popular. Which of course has less to do with us and more to do with MongoDB's wonderful community of users and developers. MongoDB is an open-source database. Thank you for making it the industry standard for modern applications.
MongoDB Extends Its Lead As The Industry's "Best NoSQL Database"...Thanks To You
MongoDB has long been the industry's leading NoSQL database across a number of measures, and is also more popular than most most relational databases . This isn't news. What is news is how much MongoDB has cemented and extended its lead over the last year. This is readily apparent in Linux Journal's Readers' Choice Award for Best NoSQL Database. In 2013, MongoDB was named the " best NoSQL database " by 43.6% of survey respondents. The next nearest database garnered only 15.3% of votes: While impressive, it becomes doubly so in light of MongoDB's performance in the 2012 version of Linux Journal's Readers' Choice Award . Then, as now, MongoDB dominated its field. But in 2012 MongoDB won with 33.4% of the votes, with the next nearest competitor getting 22.3% of the votes: Enterprises are standardizing on a few good NoSQL database options, just as they did in the relational database market. It's simply too cumbersome to hope to embrace a dozen different databases to solve niche needs. MongoDB is clearly the enterprise standard for NoSQL. Not that we're resting on our laurels, or taking all the credit. That's not in our DNA. The MongoDB community deserves a huge amount of credit for consistently offering feedback - both positive and negative - that helps the MongoDB development community continuously improve an already exceptional database. This is not a solo effort. It's something we've earned together with you, and we thank you for making MongoDB such an excellent database.