MongoDB 3.2.5 is out and is ready for production deployment. This release contains only fixes since 3.2.4, and is a recommended upgrade for all 3.2 users.
Fixed in this release:
- SERVER-23274 Aggregate with out, then stepdown, out collection dropped.
- SERVER-23283 RangeDeleter does not log cursor ids correctly in deleteNow()
- SERVER-22964 IX GlobalLock being held while wating for wt cache eviction
- SERVER-22937 Retry catalog operations whenever possible
- SERVER-22831 Low query rate with heavy cache pressure and an idle collection
- SERVER-21681 In-memory storage engine not reporting index size
As always, please let us know of any issues.
– The MongoDB Team
Oxford Nanopore Technologies & MongoDB: Powering Real-Time Genetic Analysis with Docker, MongoDB, & AWS
Genetic analysis is entering the mobile age. Earlier this year scientific journal Nature published a paper showing how Ebola researchers in Guinea were able to analyse genetic material in hours, rather than the weeks it had previously taken. This increased speed meant doctors could better understand the spread of the disease. Then quickly develop strategies to stop it. The hardware that enabled the genetic analysis is the MinION , from UK-based Oxford Nanopore Technology . The stapler-sized MinION is the data-capture side of the analysis, but for the purposes of this article we’re interested in data processing and analysis. In particular how Oxford Nanopore has been able to build a fast, agile and powerful cloud-based platform that has the potential to deliver biological analyses to any scientist, at any time, anywhere in the world. The applications for this genetic analysis go far beyond the medical field and disease control. Oxford Nanopore is using technologies like MongoDB, Amazon Web Services, and Docker containers in its stated goal: “to enable the real-time analysis of any living thing, by any user, in any environment.” A Billionth of a Meter The MinION does its genetic magic through the use of nanopores. Each nanopore is just a billionth of a meter wide. The technology in the MinION threads the genetic material through the nanopores where tiny differences in each sample can be registered as electrical disruptions. If you want a more detailed explanation of nanopores, check out Oxford Nanopore Technologies’ website . DNA sequencing can be associated with predictive human questions alone, for example “what probability is there that this person will develop a specific disease?” But human genome research is just a part of the equation, and the portable nature of the MinION means it might be suitable for a more diverse range of questions: Is the soup I’m about to eat safe? What type of disease am I looking at? Where did this pathogen originate? How can we grow more resilient plants? Is this hospital ward clean? Crucially, these questions need to be answered quickly, and in a range of environments – from the science lab to the middle of the jungle. Three Billion Bases in the Cloud The cleverest sequencing tool in the world would be worthless if we were unable to process and understand the data it created. To deal with the volume and velocity of processing billions of lines of DNA, Oxford Nanopore Technologies built analysis capabilities offered by Metrichor , on powerful software that can scale seamlessly in the cloud. Richard Carter, Associate Director, Data Integration at Oxford Nanopore gave a presentation at MongoDB Days where he noted: “When we began building Metrichor services, it was clear our data would not fit in the neat rows and columns of a relational database. We needed a database that could look at our complex information in more flexible and dynamic ways. It was a straightforward decision to go with MongoDB. It’s robust, best of breed, and has the data modelling and analytics flexibility we required. We also observed the technology has an incredible community behind it, coupled with extensive documentation and training. All of which enable us to get productive with the technology much faster.” The DNA data is read locally onto the MinION and it’s then sent to an Amazon Web Services cloud. The findings are then analysed before the results are sent back to the user’s laptop or displayed in web reports. All of this is driven by, and stored in the non-relational database MongoDB. Docker containers are used to package, deploy and run the software across the cloud deployment. Carter also noted that: “The biology and hardware is the real trick, of course, but we needed power and scalability to run cloud based services as we wished.” There were other challenges the team had during development of their software. They had a technical goal and a number of ways they could reach it while keeping the focus on the biology. It was essential they had the freedom to experiment and make significant changes as they went along. “Happily, MongoDB supports an evolutionary approach to development.” explained Carter. “We were spinning up instances and working on the science almost instantly. The database got out of the way.” Carter’s team does not have a database administrator. They have found that MongoDB Cloud Manager is able to provide all the monitoring data needed to keep a deployment healthy. Features like simple, automated deployment across any cloud region, continuous backups, and telemetry visualisations also mean administration doesn’t monopolise the developers’ time. Giant Ideas Guinea is just one of the many places where researchers are using Nanopore’s data architecture for analysis. In fact, NASA will soon be using the MinION for testing biological molecules on the International Space Station. Regardless of the location, the combination of rigorous science and the power of cloud computing is ushering in a new way of understanding the world. Read more about MongoDB and its implementation on the AWS cloud platform. MongoDB on AWS: Guidelines and Best Practices About the Author - Mat Keep Mat is director of product and market analysis at MongoDB. He is responsible for building the vision, positioning and content for MongoDB’s products and services, including the analysis of market trends and customer requirements. Prior to MongoDB, Mat was director of product management at Oracle Corp. with responsibility for the MySQL database in web, telecoms, cloud and big data workloads. This followed a series of sales, business development and analyst / programmer positions with both technology vendors and end-user companies.
Australian Start-Up Ynomia Is Building an IoT Platform to Transform the Construction Industry and its Hostile Environments
The trillion dollar construction industry has not yet experienced the same revolution in technology you might have expected. Low levels of R&D and difficult working environments have led to a lack of innovation and fundamental improvements have been slow. But one Australian start-up is changing that by building an Internet of Things (IoT) platform to harness construction and jobsite data in real time. “Productivity in construction is down there with hunting and fishing as one of the least productive industries per capita in the entire world. It's a space that's ripe for people to come in and really help,” explains Rob Postill , CTO at Ynomia. Ynomia has already been closely involved with many prestigious construction projects, including the residential N06 development in London’s famous 2012 Olympic Village. It was also integral to the construction of the Victoria University Tower in Australia. Link to Podcast Episode Here “These projects involve massive outflow of money: think about glass facades on modern buildings, which can represent 20-30 percent of the overall project cost. They are largely produced in China and can take 12 weeks to get here,” says Postill. “Meanwhile, the plasterer, the plumber, the electrician are all waiting for those glass facades to be put on so it is safe for them to work. If you get it wrong, you can go in the deep red very quickly.” To tackle these longstanding challenges, Ynomia aims to address the lack of connectivity, transparency and data management on construction sites, which has traditionally resulted in the inefficient use of critical personnel, equipment and materials; compressed timelines; and unpredictable cash flows. To optimize productivity, Ynomia offers a simple end-to-end technology solution that creates a Connected Jobsite. Helping teams manage materials, tools, and people across the worksite in real time. IOT in a Hostile Environment The deployment of technology in construction is often fraught with risk. As a result, construction sites are still largely run on paper, such as blueprints, diagrams and models as well as the more traditional invoices and filing. At the same time, there is a constant need to track progress and monitor massive volumes of information across the entire supply chain. Engineers, builders, electricians, plumbers, and all the other associated professionals need to know what they need to do, where they need to be, and when they need to start. “The environment is hostile to technology like GPS, computers, and mobile phone reception because you have a lot of Faraday cages and lots of water and dust,” explains Postill. “You can't have somebody wandering around a construction site with a laptop; it'll get trashed pretty quickly." Enter MongoDB Atlas “On a site, you might be talking about materials, then if you add to that plant & equipment, or bins, or tools etc, you're rapidly getting into thousands and thousands of tags, talking all the time, every day,” said Postill. That means thousands of tags now send millions of readings on Ynomia building sites around the world. All these IoT data packets must be stored efficiently and accurately so Ynomia can reassemble the history of what has happened and track tagged inventory, personnel, and vehicles around the site. Many of the tag events are also safety critical so accuracy is a vital component and packets can't be missed. To address these needs Ynomia was looking for a database that was scalable, flexible, resilient and could easily handle a wide variety of fast-changing sensor data captured from multiple devices. The final component Postill was looking for in a database layer was freedom: a database that didn't lock them into a single cloud platform as they were still in the early stages of assessing cloud partners. The Commonwealth Scientific and Industrial Research Organisation , which Postill had worked with in the past, suggested MongoDB , a general purpose, document-based database built for modern applications. “The most important factor was that the database is event-driven, which I knew would be difficult in the traditional relational model. We deal with millions of tag readings a day, which is a massive wall of data,” said Postill. A Cloud Database Ynomia is using MongoDB Atlas , the global cloud database service, now hosted on Microsoft Azure. Atlas offers best-in-class automation and proven practices that combine availability, scalability, and compliance with the most demanding data security and privacy standards. “When we started we didn't know enough about the problem and we didn't want to be constrained," explained Postill. "MongoDB Atlas gives us a cloud environment that moves with us. It allows us to understand what is happening and make changes to the architecture as we go." Postill says this combination of flexibility and management tooling also allows his developers to focus on business value not undifferentiated code. One example Postill gave was cluster administration: "Cluster administration for a start-up like us is wasted work," he said. "We’re not solving the customer's problem. We're not moving anything on. We’re focusing on the wrong thing. For us to be able to just make that problem go away is huge. Why wouldn’t you?" Atlas also gives Ynomia the option to spin out new clusters seamlessly anywhere in the world. This allows customers to keep data local to their construction site, improving latency and helping solve for regional data regulations. Real Time Analytics The company has also deployed MongoDB Charts, which takes this live data and automatically provides a real time view. Charts is the fastest and easiest way to visualize event data directly from MongoDB in order to act instantly and decisively based on the real-time insights generated by event-driven architecture. It allows Ynomia to share dashboards so all the right people can see what they need to and can collaborate accordingly. “Charts enables us to quickly visualize information without having to build more expensive tools, both internally and externally, to examine our data,” comments Postill. “As a startup, we go through this journey of: what are we doing and how are we doing it? There's a lot of stuff we are finding out along the way on how we slice and re-slice our data using Charts.” A Platform for Future Growth Ynomia is targeting a huge market and is set for ambitious growth in the coming years. How the platform, and its underlying architecture, can continue to scale and evolve will be crucial to enabling that business growth. “We do anything we can to keep things simple,” concluded Postill. “We pick technology partners that save us from spending time we shouldn't spend so we can solve real problems. We pick technologies that roll with the punches and that's MongoDB.” When we started we didn't know enough about the problem and we didn't want to be constrained," explained Postill. "MongoDB Atlas gives us a cloud environment that moves with us. It allows us to understand what is happening and make changes to the architecture as we go. Rob Postill, CTO, Ynomia