MongoDB is conducting a research project to better understand our users. We are looking for database administrators who are willing to share their experience with databases and MongoDB. If you would like to participate, please fill out this questionnaire. We will be contacting selected participants for an hour-long interview.
More About the Research
We are looking for database administrators, located anywhere in the world, who are willing to talk to us about their daily task flows to help us improve the MMS experience. You can be a developer, a DBA or other operations professional who administer databases in your day-to-day. You don’t have to use MMS professionally nor do you need to manage MongoDB professionally. You must be fluent in English to participate.
This research entails an hour-long interview where participants will be asked about their workflows as database administrators, pain points encountered with monitoring tools, and other questions relating to their work. Interviews will be conducted either in-person or via Skype, depending on geography and personal preferences.
Personal information collected throughout the research will not be shared with any third party or used for solicitation. All participants will remain anonymous in all company reporting. Participants reserve the right to leave the study at any time. As a thank you for completing the interview, we’re offering $75 Amazon gift cards to participants.
If you are interested in participating, please fill out this screening questionnaire for an hour-long interview, and we will be in touch!
The Leaf in the Wild: MongoDB at MachineShop
Leaf in the Wild posts highlight real world MongoDB deployments. Read other stories about how companies are using MongoDB for their mission-critical projects. I had the chance to meet with John Cox, Senior Technology Director at MachineShop , who running their Internet of Services platform on MongoDB. MachineShop is one of many startups who are using MongoDB to power the Internet of Things and are changing the way developers and organizations engage data to garner insights and connect to their environments. Tell us a little bit about your company. What are you trying to accomplish? How do you see yourself growing in the next few years? MachineShop is an on-demand middleware service that simplifies the way organizations build applications, integrate systems and share data within an enterprise and its ecosystem. MachineShop is uniquely architected to connect with Internet enabled devices, systems and databases and offers an API-oriented approach to aggregating and managing services that engage, enrich, expose and manage data and their underlying sources. We offer Developers and Organizations access to rich tools, reports and analytics about their services and applications through the MachineShop Services Exchange – a customizable web-based portal that offers hundreds of discrete APIs and services based on the unique roles and permissions of users. What problem were you trying to solve? When aggregating disparate data sources to be processed by central business logic and served up through a standard RESTful API, we needed a database solution that can accommodate multi-structured data and gives us high-throughput. We also need something that’s easy to scale out as we add customers and ramp up data inputs exponentially. MongoDB has it all in spades. The fact that it’s super easy to spit everything out to our API in JSON is a [very nice] bonus. Was this a new project or did you migrate from a different database? What was it like to learn MongoDB Earlier iterations of MachineShop used a relational database, but the current product was build from the ground up on MongoDB. There was still a small learning curve for the team jumping into MongoDB. It was tiny, though. The prototype for the current product was built entirely in Ruby (Sinatra/Rails). The fact that we used the Mongoid ODM made the transition really easy to understand as a developer. There were a few things we had to get smart on quickly on system admin, but honestly it was fairly trivial. (Thank you!) Did you consider other alternatives, like a relational database or non-relational database? We considered a few alternatives. It became clear very quickly that we wanted to go with a NoSQL solution. Once we crossed that bridge, MongoDB was just an obvious choice. The barrier to entry was low – both in dollars and technical resources. There are a ton of folks working with it that made finding resources online and building relationships in the local community really easy. It’s really fun to work with great, new technology that’s constantly moving forward. It’s also nice to not be on an island trying to figure it out. Please describe your MongoDB deployment Right now we’re a pretty small footprint – 3 replica sets and that’s it. It’s fine for the moment. The plan is to move very soon to many shards across a lot of small instances. The idea is that striping the data buys us speed and it’s easy to scale out. We run Ubuntu on AWS for everything. We’re currently using MongoDB 2.4.6 in production. Are you using any tools to monitor, manage and backup your MongoDB deployment? If so what? We’re using MMS primarily for monitoring. We also use MongoLab for hosting our production database. They have some pretty good value-add service offerings that we use. We also monitor indirectly through our apps using Scout . Are you integrating MongoDB with other data analytics, BI or visualization tools like Hadoop? If so can you share any details We have a proof of concept in place with Hadoop for analytics as well as Storm for real-time processing and aggregation. In production we do fairly basic on-the-fly aggregations and MapReduce jobs with data from devices as well as API request metering. The ultimate goal is to make sure that it’s easy to bolt on common BI tools to allow customers to slice and dice however they like. How are you measuring the impact of MongoDB on your business? We’ve never measured anything like cost savings directly. With MongoDB we picked a direction and just started running. Using MongoDB never felt like cost us on any of the metrics you listed. It’s pretty much been smooth as silk. Had we not used MongoDB, I could definitely see where it would cost us in terms of engineering solutions to problems that we never encountered. What advice would you give someone who is considering using MongoDB for their next project Fear not! Dive in. When engineering solutions we need to make sure we’re using the right tool for any job that we do. MongoDB happens to be a great tool that can be the right one in a LOT of situations. It lets you move fast and treat your data as just data. It’s freaking fast, too. You don’t have to make so many decisions up front. You can experiment and move pieces around as needed. A couple of things I would recommend specifically: Make sure you have sufficient memory to store your working set (frequently accessed data and indexes). It’s just better. (Google “mongodb working set”) If you’re using some abstraction of data access, pay close attention to performance on aggregation. We ended up sidestepping some of the abstraction to gain performance in this area. MongoDB's dynamic schema and object-oriented structure make it a great fit for the Internet of Things. See how companies like Enernoc and Bosch are building a more connected world with MongoDB.
Australian Start-Up Ynomia Is Building an IoT Platform to Transform the Construction Industry and its Hostile Environments
The trillion dollar construction industry has not yet experienced the same revolution in technology you might have expected. Low levels of R&D and difficult working environments have led to a lack of innovation and fundamental improvements have been slow. But one Australian start-up is changing that by building an Internet of Things (IoT) platform to harness construction and jobsite data in real time. “Productivity in construction is down there with hunting and fishing as one of the least productive industries per capita in the entire world. It's a space that's ripe for people to come in and really help,” explains Rob Postill , CTO at Ynomia. Ynomia has already been closely involved with many prestigious construction projects, including the residential N06 development in London’s famous 2012 Olympic Village. It was also integral to the construction of the Victoria University Tower in Australia. Link to Podcast Episode Here “These projects involve massive outflow of money: think about glass facades on modern buildings, which can represent 20-30 percent of the overall project cost. They are largely produced in China and can take 12 weeks to get here,” says Postill. “Meanwhile, the plasterer, the plumber, the electrician are all waiting for those glass facades to be put on so it is safe for them to work. If you get it wrong, you can go in the deep red very quickly.” To tackle these longstanding challenges, Ynomia aims to address the lack of connectivity, transparency and data management on construction sites, which has traditionally resulted in the inefficient use of critical personnel, equipment and materials; compressed timelines; and unpredictable cash flows. To optimize productivity, Ynomia offers a simple end-to-end technology solution that creates a Connected Jobsite. Helping teams manage materials, tools, and people across the worksite in real time. IOT in a Hostile Environment The deployment of technology in construction is often fraught with risk. As a result, construction sites are still largely run on paper, such as blueprints, diagrams and models as well as the more traditional invoices and filing. At the same time, there is a constant need to track progress and monitor massive volumes of information across the entire supply chain. Engineers, builders, electricians, plumbers, and all the other associated professionals need to know what they need to do, where they need to be, and when they need to start. “The environment is hostile to technology like GPS, computers, and mobile phone reception because you have a lot of Faraday cages and lots of water and dust,” explains Postill. “You can't have somebody wandering around a construction site with a laptop; it'll get trashed pretty quickly." Enter MongoDB Atlas “On a site, you might be talking about materials, then if you add to that plant & equipment, or bins, or tools etc, you're rapidly getting into thousands and thousands of tags, talking all the time, every day,” said Postill. That means thousands of tags now send millions of readings on Ynomia building sites around the world. All these IoT data packets must be stored efficiently and accurately so Ynomia can reassemble the history of what has happened and track tagged inventory, personnel, and vehicles around the site. Many of the tag events are also safety critical so accuracy is a vital component and packets can't be missed. To address these needs Ynomia was looking for a database that was scalable, flexible, resilient and could easily handle a wide variety of fast-changing sensor data captured from multiple devices. The final component Postill was looking for in a database layer was freedom: a database that didn't lock them into a single cloud platform as they were still in the early stages of assessing cloud partners. The Commonwealth Scientific and Industrial Research Organisation , which Postill had worked with in the past, suggested MongoDB , a general purpose, document-based database built for modern applications. “The most important factor was that the database is event-driven, which I knew would be difficult in the traditional relational model. We deal with millions of tag readings a day, which is a massive wall of data,” said Postill. A Cloud Database Ynomia is using MongoDB Atlas , the global cloud database service, now hosted on Microsoft Azure. Atlas offers best-in-class automation and proven practices that combine availability, scalability, and compliance with the most demanding data security and privacy standards. “When we started we didn't know enough about the problem and we didn't want to be constrained," explained Postill. "MongoDB Atlas gives us a cloud environment that moves with us. It allows us to understand what is happening and make changes to the architecture as we go." Postill says this combination of flexibility and management tooling also allows his developers to focus on business value not undifferentiated code. One example Postill gave was cluster administration: "Cluster administration for a start-up like us is wasted work," he said. "We’re not solving the customer's problem. We're not moving anything on. We’re focusing on the wrong thing. For us to be able to just make that problem go away is huge. Why wouldn’t you?" Atlas also gives Ynomia the option to spin out new clusters seamlessly anywhere in the world. This allows customers to keep data local to their construction site, improving latency and helping solve for regional data regulations. Real Time Analytics The company has also deployed MongoDB Charts, which takes this live data and automatically provides a real time view. Charts is the fastest and easiest way to visualize event data directly from MongoDB in order to act instantly and decisively based on the real-time insights generated by event-driven architecture. It allows Ynomia to share dashboards so all the right people can see what they need to and can collaborate accordingly. “Charts enables us to quickly visualize information without having to build more expensive tools, both internally and externally, to examine our data,” comments Postill. “As a startup, we go through this journey of: what are we doing and how are we doing it? There's a lot of stuff we are finding out along the way on how we slice and re-slice our data using Charts.” A Platform for Future Growth Ynomia is targeting a huge market and is set for ambitious growth in the coming years. How the platform, and its underlying architecture, can continue to scale and evolve will be crucial to enabling that business growth. “We do anything we can to keep things simple,” concluded Postill. “We pick technology partners that save us from spending time we shouldn't spend so we can solve real problems. We pick technologies that roll with the punches and that's MongoDB.” When we started we didn't know enough about the problem and we didn't want to be constrained," explained Postill. "MongoDB Atlas gives us a cloud environment that moves with us. It allows us to understand what is happening and make changes to the architecture as we go. Rob Postill, CTO, Ynomia