We are excited to announce that Cloud Manager now supports converting a replica set to a sharded cluster via Automation. This oft-requested feature will help you scale your deployment more easily. Here’s how:
- Head to your Deployment Page and use the “…” menu for an existing replica set to choose “Convert to Sharded Cluster”
- Enter in the requested details, like which hosts you want your mongoSes and config servers to be deployed on and on what ports.
- Click “Next” and then you can drag and drop processes as needed to tweak your deployment.
- Push “Review and Deploy” and then “Confirm and Deploy”
- Modify your application to reach out to the mongoSes rather than the replica set
Cloud Manager Automation has now converted your replica set to a sharded cluster. This does not mean that all of your collections and databases are sharded, though. Cloud Manager Automation does not issue
shardCollection commands to activate sharding for particular databases and collections, so issuing these and choosing a suitable shard key are left to you. Otherwise, you’re good to go, including adding additional shards via the wrench menu, if necessary.
MongoDB Debuts in Gartner’s Magic Quadrant for Data Warehouse & Data Management Solutions for Analytics
Why, you may ask, is MongoDB profiled in a research report dedicated to evaluating key trends and vendors in the data warehousing market? After all, MongoDB is designed to serve operational use-cases , including Internet of Things applications, customer data management, catalog and content management, mobile services and more. In fact, Gartner placed MongoDB as a Leader in its most recent Magic Quadrant for Operational Database Management Systems in recognition of its completeness of vision and ability to execute against requirements in the operational database market. While MongoDB is not a data warehouse, we believe its inclusion within Gartner’s latest DW/DMSA Magic Quadrant [available at no cost to eligible Gartner clients] reflects the growing demand from business users to accelerate speed-to-insight and turn analytics into real-time action. Whether that is to detect fraud during transaction processing, present relevant recommendations to shoppers as they browse an eCommerce store, or alert operators to the impending failure of a critical piece of manufacturing equipment, creating fast, actionable insight is accomplished by embedding real-time analytics into operational processes. Gartner calls this trend Hybrid Transactional/Analytical Processing (HTAP), and it is this specific capability, highlighted by users surveyed in Gartner’s research, that has driven MongoDB’s inclusion into the Magic Quadrant. Not only is this placement a first for MongoDB, it is also a first for Gartner. No other open source, non-relational database has ever been included in the DW/DMSA Magic Quadrant. Augmenting the Data Warehouse: Unlocking Real-Time Analytics Using traditional data warehousing platforms, the flow of data – starting with its acquisition from source systems through to transformation, consolidation, analysis, and reporting – follows a well-defined sequential process, as illustrated in Figure 1. Figure 1 : Data Flow in Traditional Analytics Processes Operational data from multiple source systems is integrated into a centralized Enterprise Data Warehouse (EDW) and local data-marts using Extract Transform Load (ETL) processes. Reports and visualizations of the data are then generated by BI tools. This workflow is predicated on a number of assumptions: Predictable Frequency. Data is extracted from source systems at regular intervals – typically measured in days, months and quarters. Static Sources. Data is sourced from controlled, internal systems supporting established and well-defined back-office processes. Fixed Models. Data structures are known and modeled in advance of analysis. This enables the development of a single schema to accommodate data from all of the source systems, but adds significant time to the upfront design. Defined Queries. Questions to be asked of the data (i.e., the analytical queries) are pre-defined. If not all of the query requirements are known upfront, or requirements change, then the schema is modified to accommodate changes. Slow-changing requirements. Rigorous change control is enforced before the introduction of new data sources or reporting requirements. Limited users. The consumers of BI reports and analytics are typically business managers and senior executives. Technology Foundations for Real-Time Analytics This workflow remains incredibly valuable, enabling businesses to run deep, historical analysis to monitor performance and inform business strategy. But it presents a significant “impedance mismatch” to the requirements presented by real time analytics: Eliminate latency. The frequency of data acquisition, processing and analysis must increase from days to seconds or less. Source data needs to be analyzed as it is generated by operational applications in order to provide the speed-to-insight demanded by the business. Moving data through an ETL pipeline to the data warehouse will not work for real time use-cases. Uncontrolled sources. Organizations need to harness data that is generated outside of their own firewalls – from location data, to web clicks, to sensors, to social media. The analytics team has no control over these data sources. Dynamic structures. Much of this data is rapidly changing with polymorphic, semi-structured or unstructured formats that do not map neatly to the fixed schema of traditional relational databases powering most data warehouses. Changing query patterns. It is impossible to predict the types of questions that will be asked of the data. Search, aggregations, geospatial analytics, and machine learning are just some of the tools now available to analysts as they explore new data sets and discover previously undetected trends. ”Big” volume. Data arrives faster, and in quantities that overwhelm traditional data management technologies. It means scaling out databases and analytics across commodity hardware, rather than the scale-up approach typical of most data warehouses. Wide consumption. Analytics now extends well beyond the management suite. Permeating through every part of the organization, analytics now need to be accessible to staff on the shopfloor, and consumed by operational applications to control real-time behavior. MongoDB augments the data warehouse by addressing the challenges above, enabling users to run analytics in real-time directly against their data: Rich data structures with complex attributes comprising text, geospatial data, media, arrays, embedded elements, and other complex types can be easily mapped to MongoDB’s JSON-based document data model. A dynamic schema means that each document (record) does not need to have the same set of fields. Users can adapt the structure of documents just by adding new fields or deleting existing ones, making it very simple to extend and evolve applications by adding new attributes for analysis and reporting. An expressive query language and secondary indexes allow fast and rich access to data, enabling complex analytics and search to be performed in place, without having to move the data to dedicated analytics infrastructure. Auto-shading allows MongoDB to partition and distribute large data sets across clusters of commodity servers in the data center or in the cloud. The latest MongoDB 3.2 release builds on these capabilities with advanced feature sets to enhance analytics: The MongoDB Connector for BI allows analysts, data scientists, and business users to seamlessly explore and visualize multi-structured data stored in MongoDB with industry-standard SQL-based BI and analytics platforms such as Tableau, Business Objects, and more. MongoDB Compass presents a simple-to-use, sophisticated GUI that allows any user to visualize and explore data with ad-hoc queries in just a few clicks – all with zero knowledge of the MongoDB query language. For data governance, document validation allows you to enforce checks on document structure, data types, data ranges, and the presence of mandatory fields. Dynamic lookup, new math operators and enhanced search allow richer analytics to be run against live, operational data Putting Real-Time Analytics to Work Some of the world’s largest and most innovative organizations are putting real-time analytics to work, creating operational efficiencies and building competitive advantage: Bosch uses MongoDB at the heart of its IoT Suite. Ingesting real-time telemetry data from millions of vehicles enables auto-manufacturers to deliver predictive maintenance schedules to their customers, and improve product design. The City of Chicago uses MongoDB to pull together millions of data points across its most crucial departments, providing real-time data analysis to city managers so they can better predict and allocate resources, respond quickly to emergencies, regulate traffic flow and uncover trends that would have otherwise been invisible. Media company BuzzFeed uses MongoDB to pinpoint when content is viewed, where it’s shared, and how it’s being consumed by its 400 million monthly website visitors. The system enables BuzzFeed’s employees to analyse, track, and display these metrics to writers and editors. The website of OTTO, Germany’s largest online retailer, generates some 10,000 events per second. Every click and hover of every mouse is stored in MongoDB , and real-time data analytics is used to provide unique and personalised web experiences to individual visitors. Hadoop and Spark: Building the Complete Data Analytics Platform Of course, its not just real-time analytics that is driving innovation in the data warehouse world – Apache Hadoop has emerged as a key part of the data management landscape. Some assumed Hadoop would replace the enterprise data warehouse, but that prediction was wrong. In fact, Hadoop is augmenting the data warehouse, in many cases, off-loading data and specific data transformation workloads from existing data warehouses to less-expensive commodity hardware in scale-out environments. Many organizations are harnessing Hadoop and MongoDB together using the MongoDB Connector for Hadoop , providing the ability to use MongoDB as an input source and an output destination for MapReduce, Spark, HIVE and Pig jobs. With this combination, users can create complete analytics and data management platforms: MongoDB powers the online, real time operational application, serving business processes and end-users Hadoop consumes data from MongoDB, blending its with data from other operational systems to fuel sophisticated analytics and machine learning. Results are loaded back to MongoDB to serve smarter operational processes. For example, Ebay handles user data and metadata management for its product catalog in MongoDB, and Hadoop for user analysis to provide personalized search & recommendations. Orbitz uses MongoDB for the management of hotel data and pricing, with Hadoop powering hotel segmentation to support building search facets. Pearson manages student identity and access control along with content management of course materials in MongoDB, and Hadoop for student analytics to create adaptive learning programs. The Rise of Spark No analytics discussion is complete without reference to Apache Spark – it has become one of the fastest growing Apache Software Foundation projects. With its memory-oriented architecture, flexible processing systems, and easy-to-use APIs, Apache Spark has emerged as a leading framework for real-time analytics, supporting streaming, machine learning, SQL processing and more. Unlike Hadoop which has to move all data into HDFS, Spark can directly work against data stored in any database, file system, or message queue. The MongoDB Connector for Hadoop provides a Spark plug-in , allowing Spark jobs to use MongoDB as both a source and a sink. A range of community-developed connectors are also available for MongoDB and Spark integration. Figure 2 : Modernized data architecture: MongoDB, Spark, and Hadoop Many organizations are already combining MongoDB and Spark to build new analytics-rich applications. A global manufacturing company has built a pilot project to estimate warranty returns by analyzing material samples from production lines. The collected data enables them to build predictive failure models using Spark Machine Learning and MongoDB. A video sharing website is using Spark with MongoDB to place relevant advertisements in front of users as they browse, view and share videos. A multinational banking group operating in 31 countries with 51 million clients implemented a unified real-time monitoring application, running Apache Spark and MongoDB . The bank wanted to ensure a high quality of service across its online channels, and needed to continuously monitor client activity to check service response times and identify potential issues. All log data is collected in Apache Flume before being persisted to MongoDB where Spark jobs then analyze that data to power real time visualizations and alerts of system health. MongoDB was selected due to high scalability, dynamic schema that can ingest and manage quickly changing log data, and a rich array of secondary indexes, allowing Spark job to efficiently filter and access only the slices of data that are needed to drive the analytics. This approach results in lower latency and higher analytical throughput. Putting it all Together If anyone ever tells you the data warehouse market was slow and boring, dominated by just a few mega-vendors, tell them they are wrong. With the adoption of modern technologies such as MongoDB, Hadoop and Spark, organizations are creating new classes of applications and analytics that offer the promise of unlocking new efficiencies, creating new business models and out-pacing competitors. And with MongoDB serving both operational and analytical use-cases, you can build those applications faster, with lower cost, complexity and risk. To learn more about real time analytics with MongoDB, Spark and Hadoop, read our white paper. Turning Analytics into Real-Time Action References: Gartner Magic Quadrant for Operational Database Management Systems , Donald Feinberg, Merv Adrian, Nick Heudecker, Adam M. Ronthal, Terilyn Palanca, and October 12, 2015. Gartner Magic Quadrant for Data Warehouse and Data Management Solutions for Analytics , Roxane Edjlali, Mark A. Beyer, and February 25, 2016. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Latinas in Tech: Andy Morales Coto
This spotlight is part of a blog series to amplify exceptional Latina talent in the tech industry. Through our partnership with Latinas In Tech, this article originally appeared on their site . Tell us about yourself, Andy. How did you get to where you are today? I’m originally from Costa Rica and have been living in NYC for the past six years. I’m a product designer, but I wasn’t always one: before coming to New York, I was working in multiple industries, as a game designer, a copywriter, and a digital marketer. But I guess most of that is just titles and places I come from, not really the way I got to be where I am. If I look more deeply, I would say that the moments that have led me to where I am today are a mixture of privilege and the fallout of self-discovery. I was born in an upper middle class family, the daughter of two public servants — a doctor and an engineer — and learned English pretty early on at their behest. I was able to go to private school my whole life, up until college, when I attended the University of Costa Rica, which is publicly funded by all Costa Ricans. I wouldn’t say I had a luxurious life growing up: there were certainly hand-me-downs from my sisters, but I also never had a problem buying a video game console if I wanted it — I’d just have to give up having a birthday party (and I did). Overall, I’d say my parents motivated me to follow my dreams, and would gladly take me to any classes I wanted (English, robotics, programming, drawing) from the time I was a little girl. In that sense, I always had a leg up, understood what was considered “excellence” in education, and pretty early on set my mind on studying abroad eventually. With that said, my comfortable life became, well, not comfortable at all when I came out at 19. College changed my life completely. Finally being able to understand who I was, I came out as queer to my very conservative parents, and the reception was extremely toxic. For the first time, I understood what it meant to not be able to afford a meal, or even a bus ticket. I walked miles to go to college several times, hell-bent on finishing my degree in communications (the closest thing to tech, I figured, without the toxicity of the homogeneity of computer science). Finally I graduated, but my whole perception of the world had changed: I became more empathetic and less judgmental of others, and I knew what depression and trauma were. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. I continued working for several years after graduating from college, did another degree in marketing while I worked, and finally got accepted into Parsons (NYC) on a scholarship to study transdisciplinary design. And here we are! Oh, also, and this is very important: I’m married to a lovely American and live with her and two fluffy tabby cats in Brooklyn. NYC is what I call home now (and probably forever). What inspired you to pursue a career in the tech industry? I think pretty early on I was in awe of technology, and I don’t just mean computers, but also cars, glasses, electricity, hammers. I’ve always admired anything that expands the possibilities of what a human can do. But my “aha moment” happened when I was 10 and accessed the internet at the University of Costa Rica. My mother was a teacher there and had access to connection before the rest of the country did. She’d sometimes let me use her computer, and I still remember using Netscape in complete fascination of what this meant for humanity: we would all be connected. That’s when it really clicked for me: I love this, I love computers. As a manager at MongoDB, what have been some of the most memorable and impactful projects you’ve worked on so far? I’m the most proud of the people I manage, and seeing them grow every day. My direct reports are infinitely more talented than I am in some ways, and I welcome that. I want to be surrounded by people more talented than I am, and they’re going to change the face of the design industry, I have no doubt. Watching them get better and better, lead projects of their own, and successfully navigate difficult stakeholder situations — well, it just puts a smile on my face! But, apart from that, a specific project I’ve enjoyed is Blue Sky, a yearly design-driven sprint that we do in conjunction with key stakeholders to create the “concept car” of the product I lead design for. This will be the second year we do Blue Sky, and we hope to use design thinking beyond the graphical user interface, partnering with product and engineering to imagine the future experience of MongoDB Realm in the CLI and the IDE. With each Blue Sky, design positions itself as a partner for our stakeholders, and our proposals coming out of the project tend to be implemented up to 75% of what we design. It’s exciting to become strategic partners in the direction the product will take. How has your culture (and/or other identity marker) shaped you as a leader? As a manager? Well, my culture is a mixture of queer culture, Costa Rican culture, and NYC culture. I think all of these shape me as a leader, because it means I am not a monolith as a person; I’ve learned to see the world through many different perspectives. Being able to compare and contrast how different cultures view or react to situations makes me self-aware, and puts me in a position where I strive to understand how others are reacting to situations, in the frame of their culture. I’d say this is empathy, which is a bit of a design cliche, but I actually think that it’s more than empathy — it’s vulnerability and sobering humility. Trust me, I wasn’t always super self-aware, but as I’ve gotten to know the world through different cultural lenses, I’ve realized that I have to be careful with how I help others be what they consider their very best. Whether it’s grappling with cultural expectations or navigating workplace biases, we fight through many challenges as Latinx women. What’s one you’re working through currently? I’m definitely sometimes worried about how I come off to my teammates, particularly those who are not Latin American. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. Unfortunately, as a woman and as a Latina, these can be seen as vapid qualities, symbols of weakness. Why is she so loud, so emotional, so open to talking? In the past, I’ve tried to cover this up by being serious, talking softly but more deeply, and avoiding vulnerable conversation; as I’ve grown older, I’ve realized that inhibiting those qualities hinders me at work, because it makes me feel miserable, and that I end up gaining more supporters in the long term by being as open-hearted as I am. I definitely think I have my upbringing in Costa Rica to blame for that: it is not the norm for women to be like that at work, but while I was growing up I certainly saw more female bosses be open and vulnerable. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. This, of course, sometimes brings some internal turmoil: Am I just not meant to be in this American culture? Am I borrowing from my Costa Rican experiences without giving back? There’s a certain sense of duty that you feel toward those who are in your home country, even if your current definition of home has changed (I consider myself more a New Yorker than anything else, by now). To be honest, I don’t have a solution to that sense of duty and loss, and I struggle with it pretty often. I deal with it by donating and helping others that want to chase their dreams in the USA, but I still struggle with it. It’s hard not to miss the place you grew up in. It’s a big piece of you, no matter where you go. Looking to the future, what inspires you, and what initiatives are you most excited about right now? I’m inspired by games, and I can’t wait to continue using playful design in every product I design. Tangentially, I design live action role playing (LARP) games, and I can’t wait to be able to play with my other designer friends again, hopefully at a house by the beach. What’s one piece of career advice you’ll never ever forget? One of my professors from grad school, Mathan Ratinam, told me once that throughout his career he learned that you are lucky if you get to choose a job for one of three reasons: you love the work, you love the mission, or you love the people. I’ve tried loving the work, and I’ve tried loving the mission, but let me tell you: if I don’t enjoy working with the people, I’m not going to be happy in the long term. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. How do you reset when you’re in a funk? I let myself cry/experience sadness first, I go to therapy (cannot stress this enough: if you can afford it, please go to therapy), and I practice Muay Thai. I just love kicking a bag and sweating the problems out, you know? Any podcasts or blog recommendations? I don’t really listen to podcasts or read blogs that often. I play games and I read books; those are my two sources of design inspiration. I’d say, if you can, play “Zelda: Breath of the Wild,” to see what the epitome of design is. Also, play any LARP from the Golden Cobra Challenge: http://www.goldencobra.org/ . You can print those for free and play them with people online. Bookswise, I’ve been reading Fall ; or, Dodge in Hell , by Neal Stephenson, but sometimes it hits too close to home. Is there anyone you’d like to shout out for their support along your career journey? My wife, Crystal Morales. She’s the best thing that has ever happened to me. She is the smartest career advisor I know, and the smartest person I know. Period. Mathan Ratinam, of course, whom I mentioned before. He has inspired me so many times and listened to me talk for hours on the phone. A champ. My friends who, during college, helped me get a meal when I couldn’t: Olalla, Edith, Diana (my best friend since then), Warren, Memo, MaJo. A big hug to them all. And my college teacher Andrea Alvarado, who understood the pains I was going through at home when I came out and, instead of failing me, gave me extra work to do, showing me that part of being compassionate is never being condescending. Andy is thriving as a lead product designer at MongoDB . If you’re ready to work with what sounds like an incredible group of people, here are three open roles you should check out! Product Manager, Server Sales Development Representative Lead Engineer, Docs Platform