Every month, we’ll be publishing the best community blog posts from the month. Here is the digest for July:
Every month, we’ll be publishing the best community blog posts from the month. Here is the digest for July:
- Mike O'Brien, node.js and python engineer at 10gen, wrote an overview of tips and tricks for using mongo, the MongoDB shell
- Jesse Jiryu Davis published an overview of Motor, the Asynchronous python driver for MongoDB.
- Kristina Chodorow wrote an overview of Shard Tagging, a new feature in the 2.2.0 series.
- Juhi Bhatia, a member of the Pune MongoDB User Group, wrote an article on how she improved on her Schema Design for Entrib’s EMG PaaS.
- Tobias Trelle, organizer of the Dusseldorf MongoDB User group, published a post on using GridFS with Spring Data.
- Rick Copeland wrote an overview of the Ming toolkit for Python with tips on how it can accelerate your development.
- Stephen Bronstein shared some tactics in continuous deployment in his blog post on Feature flipping with MongoDB and Node.js.
- 10gen interns, Samantha Ritter and Kaushal Parikh released their log visualization tool, Edda.
- Database as a service company ObjectRocket launched their blog with the post In Memory MongoDB concurrency testing on ObjectRocket
- Comsysto’s Johannes Brandstetter showcased how to use MongoDB to create a real-time Twitter Heatmap using MongoDB’s capped collections.
Want your blog post to be included in the next update? Tweet it out with the #mongodb hashatag or send it to us directly.
MongoDB at ShowClix
We recently spoke with Nate Good, Director of Software Engineering a t ShowClix , about ShowClix's technology stack and use of MongoDB. WHAT IS SHOWCLIX? ShowClix was founded in 2007 as a live event search engine and has evolved into the full-service event ticketing system. Millions of tickets, three offices and two cereal walls later, ShowClix is the preferred ticketing solution for thousands of performing arts theatres, live music venues, and festivals Ã¢â‚¬â€ù as well as museums, non-profit fundraisers, nightclubs, even circuses and rodeos. WHAT WERE SOME OF THE REASONS THAT LED YOU TO ADOPT MONGODB? We use MongoDB as our Swiss army knife of data stores. It supplements our primary relational database, filling in the gaps that are far better suited for MongoDB’s document oriented nature. The three main ways we use it are: Â· Membership Data: Our clients have the ability to import customer data into ShowClix to be used to authenticate against. The challenge is that each client may have a very different schema than the next. MongoDB’s schemaless nature lends itself very nicely to this problem by allowing us to store diverse data in a consistent and manageable way for our application. Â· Auto Complete: ShowClix uses an auto-complete or “suggest” feature in several areas of the application, which must quickly scan millions of records to provide relevant search results to our clients. MongoDB serves as the index for this feature and leverage its left-anchored regex index support to make things snappy. Â· Log Aggregation and Indexing: We use Scribe to aggregate the many types of logs across our many servers. However, once that data is aggregated, it sits in huge nasty flat files that can be a pain to navigate/search when they need to be accessed (grep/awk can only get you so far). We turned to MongoDB to help us make sense of our most important log files. These files are parsed and imported into MongoDB upon aggregation. This allows us to use MongoDB’s query engine to help us find the answers we need quickly. Additionally, we’re looking into using MongoDB’s newer aggregation framework and Scribe’s Thrift support to take this process one step further. YOU ARE USING MONGODB WITH A RELATIONAL DATABASE. HOW DO YOU DECIDE WHAT GETS STORED IN MONGODB VS THE RDBMS? For us, ACID compliancy, namely atomicity, is pretty important. We deal with financial transactions and finite resources like available seats in a venue. For this reason, much of our ticket reservation process lives in a traditional RDBMS. However, there are definitely areas where RDBMS falls short. The rigidness of RDBMS schemas can lead to some awkward “round peg in a square hole” situations. When we’re modeling data that is a little less predictable, MongoDB’s schemalessness is a natural choice. It provides the flexibility needed, with a feature rich query engine to boot. HOW IS THE DATA MODEL FOR YOUR MEMBERSHIP DATA DIFFERENT FROM HOW YOU WOULD HAVE MODELLED IT IN A RELATIONAL DATABASE? The data our clients collect on their members can vary drastically from one client to the next. Had we stuck with an RDBMS for modeling membership data, we would have had to choose from two awkward options: a) De-normalize the data into blobs and sacrifice the very features that make RDBMS’ great or b) Try to normalize the data and end up with a multi-table, key-value mess. With Mongo, we’re able to get the best of both worlds: a DB that models unique membership data in a natural, cohesive “document” and still provides a brilliant query engine. YOU MENTIONED THAT YOU ARE LOOKING FORWARD TO 2.2. WHAT TYPES OF ANALYSES ARE YOU PLANNING TO DO WITH THE NEW AGGREGATION FRAMEWORK? The 'GROUP BY’ clause in SQL is arguably one of its most useful features from an analytics standpoint. While we’re big fans of MongoDB’s map/reduce support, we’re very excited to see a more familiar analog to SQL’s aggregate support. We hope this will be a more natural transition into analytics in MongoDB for those on our team who are more accustomed to the SQL world. We’ll likely use it the most with our log dataset in mongo to get answers about user and system behavior. Want to learn more? Visit ShowClix to learn about their innovative event ticketing software for managing every aspect of ticketing operations. AndÃ¢â‚¬Â_ they're hiring ! Tagged with: showclix, search, tickets, MongoDB, Mongo, NoSQL, Polyglot persistence, 10gen
4 Common Misperceptions about MongoDB
One year ago, in the middle of the pandemic, Dev Ittycheria, the CEO of MongoDB, brought me on as Chief Technology Officer. Frankly, I thought I knew everything about databases and MongoDB. After all, I’d been in the database business for 32 years already. I’d been on MongoDB’s Board of Directors and used the products extensively. And of course I’d done my due diligence, met the leadership team, and analyzed earnings reports and product roadmaps. Even with all that knowledge, this past year as MongoDB’s CTO has taught me that many of my preconceived notions were just plain wrong. This made me wonder how many other people might also have the wrong impression about this company. And this blog is my attempt to set those perceptions straight by sharing my four major revelations of the last year. My first revelation is that MongoDB is not trying to become this generation’s relational database. For years I assumed that MongoDB basically wanted to be a better, more modern version of Oracle when it grew up. In other words, compete with the huge footprint of Oracle and other commercial RDBMSs that have been the industry archetype for so long. I was way off. The whole point of MongoDB is to leave all those forms of archaic, legacy database technology in the historical dust. This was never supposed to be an evolution, but instead a revolution. Our founders not only envisioned the world's fastest and most scalable persistent store, but also one that would be programmed and operated differently. The combination of embedded documents and structures combined with automatic high availability and almost-infinite distribution capability all add up to a fundamentally different way of working with data, building applications, and running those applications in production. Oracle and (SQL*Server, etc) still hang their hats on E.F. Codd’s 51-year old vision of rows and columns. To obtain high availability and distribution of data, you need add ons, options packages, bailing wire and duct tape. And you need a lot of database administrators. Not cheap. Even after all that, you’re still trailing the technological edge. This is how wrong I was. Our durable competitive advantages over these legacy data stores make competing with those products almost irrelevant. We instead focus on the modern needs of modern developers building modern applications. These developers need to create their own competitive advantage through language-native development, reliable deployments to production, and lightning fast iteration. And the world is noticing; just check out the falling slope of Oracle and SQL*Server and the rising slope of MongoDB on the db-engines website. Which brings me to my second revelation: MongoDB was built for developers, by developers. I always knew that MongoDB was exceedingly fast and easy to program against. One time while I was bored in a meeting (yes, it happens here as well!), I built an Atlas database, loaded it with 350MB of data, downloaded and learned our Compass data discovery tool, built-in analytics aggregation pipelines, and our Charts package, and embedded live charts in a web page. This took me all of 19 minutes, end to end. To build something like that for engineers , it just has to be built by engineers , ones that are free to focus on all the rough edges that creep into products as features are added. I was first exposed to software planning and management over 40 years ago, and my LinkedIn profile shows a pretty diverse tour around the industry. Now, one year in, I can emphatically state that engineering and product at MongoDB are both different and better than any company I’ve ever had the privilege to work at. Our executive leadership gives engineering and product broad brushstokes of goals and desired outcomes, and then we work together to come up with detailed roadmaps, updated quarterly, that meet those goals in the way we think best, with no micromanagement. And we’re not afraid of 3-5 year projects, either. For example, multi-cloud was more than three years in the making. Also unlike any other company I’ve been at, we embrace the creation and re-payment of tech debt, rather than sweeping it under the rug. We do this through giving our product and engineering teams huge amounts of context, delivered with candor and openness. And one more essential thing; we have an empowered program management team that improves processes (including killing them) as fast as we create them. In short, we paint the targets for our teams and let them decide how and when to shoot. They even design the arrows and bows. It’s true bottoms-up engineering. Our engineers feel valued and understood. And that, in turn, empowers them to develop features that make our customers feel valued and understood, like a unified query language, or real-time analytics and charting directly in the console, or multi-region/multi-cloud clusters where all the networking cruft is taken care of for you. And this brings me to my third revelation: MongoDB is built for even the most demanding mission critical applications. Fast? Yes. Easy? Of course. But mission-critical? That’s not how I saw MongoDB when I used Version 2 for a massive student data project 10 years ago. While it was the only possible datastore we could have chosen for the amount of data and the speed of ingestion and processing needed, it was pretty hard to set up and use in a 24 x 365 environment. MongoDB had gotten ahead of itself in the early 2010’s. There was a gap between our capabilities and the expectations of the market. And it was painful. Other databases had had more than 30 years to solidify their systems and operations. We’d had five. But with Version 3 we added a new storage engine, full ACID transactions, and search. We built on it with Version 4. And then again with Version 5, released this week at our .Live conference. I knew about all this progress intellectually of course when I joined, but not viscerally. I came to realize that the security, durability, availability, scalability, and operability our platform offers (of course in addition to all the features that developers love too) was ideal for architecting fast-moving enterprise applications. And I found the proof in our customer list. It reads like a Who’s Who of major global banks, retailers, and telecommunications companies, running core systems like payments, IoT applications, content management, and real-time analytics. They use our database, data lake, analytics, search, and mobile products across their entire businesses, in every major cloud, on-premises, and on their laptops. And that leads me to my fourth and final revelation. MongoDB is no longer just a database. Of course, the database is still the core. But MongoDB now provides an enterprise-class, mission-critical application data platform. A cohesive, integrated suite of offerings capable of managing modern data requirements across even the most sprawling digital estates, and scaling to meet the level of any company’s ambition, without sacrificing speed or security. Since the day I was first introduced to MongoDB’s products, I’ve had tremendous respect and admiration for the teams and their work. After all, I’m a developer, first and foremost. And it always felt like they “got” me. But had I known then what I know now, I would have jumped on this train a long time ago. In fact, I might have camped out on their doorstep with my resume in hand. And who knows? Maybe a bunch of people reading this will do just that, and have their own revelations about how fulfilling and exciting it can be to be at a great company, with a great culture, producing great products. I’ll write another letter a year from now, and let you know how it’s going then. In the meantime, please reach out to me here, or at @MarkLovesTech .