In this blog post we will explore how to manage Mongos servers using Cloud Manager. Mongos (short for “MongoDB Shard”) is a routing service for MongoDB shard configurations that process queries from the application layer and determines the location of the data in the sharded cluster.
How to Add a Mongos server
For deployments under automation, it’s very easy to add a Mongos to your Sharded Cluster. Go to Deployments>Processes and click the wrench at the Sharded Cluster top level.
Next, scroll down to the MongoS settings. There you will see your current settings. Increase the # of Mongos processes to the desired number. For determining which host to deploy the new Mongos, here I chose Regular Expression and entered a string matching an existing host. Choose your desired port range for the Mongos. When you are finished click Apply.
Next review your changes. Here you see that the number of Mongos is increasing from 3 to 4. Confirm the host, port, and other details and click Click & Confirm.
How to Remove a Mongos server
Conversely, it’s also simple to remove a Mongos from your deployment if it’s under automation. From the Deployment>Processes view, change the filter to ‘Mongos’ as shown below. We must first Shutdown the process. Click the “…” menu for the target mongos and choose the Shutdown option. Review & deploy the change.
Once the target mongos is shut down go back to Deployment>Processes and select the ‘Mongos’ filter again. Click the “…” menu for the target mongos and this time choose ‘Remove from Cluster’
Finally, review your changes. Confirm that the number of Mongos processes is reduced and that the target host and server looks correct. Confirm & Deploy to complete the operation.
What’s New in MongoDB 3.2, Part 3: Tools and Integrations for Data Analysts and DBAs
Welcome to the final post in our 3-part MongoDB 3.2 blog series. In part 1 we covered the new storage engines and the use-cases they served. In part 2 we covered features designed to support mission-critical applications, including document validation and the enhanced replication protocol. In this post, I’ll provide an overview of new tools and integrations supporting data analysts, DBAs, and operations teams. Remember, you can get the detail now on everything MongoDB 3.2 offers by downloading the What’s New white paper . New Users With MongoDB deployed across a wider range of an organization’s application portfolio, data analysts, DBAs, and operations teams will need to integrate MongoDB within their existing processes and tool sets. MongoDB 3.2 allows analysts to support the business with new insights from untapped data sources, while DBAs and Ops teams are able to operationalize MongoDB alongside existing relational databases, protecting existing investments in management platforms and skillsets. Data Analysts and Scientists ### MongoDB Connector for BI Driven by growing requirements for self-service analytics, faster discovery and prediction based on real-time operational data, and the need to integrate multi-structured and streaming data sets, BI and analytics platforms are one of the fastest growing software markets. To address these requirements, modern application data stored in MongoDB can for the first time be easily explored with industry-standard SQL-based BI and analytics platforms. Using the MongoDB Connector for BI , analysts, data scientists and business users can now seamlessly visualize semi-structured and unstructured data managed in MongoDB, alongside traditional data in their SQL databases, using the same BI tools deployed within millions of enterprises. Figure 1: Uncover new insights with powerful visualizations generated from MongoDB MongoDB Connector for BI Implementation SQL-based BI tools expect to connect to a data source with a fixed schema presenting tabular data. This presents a challenge when working with MongoDB’s dynamic schema and rich, multi-dimensional documents. In order for BI tools to query MongoDB as a data source, the BI Connector does the following: Provides the BI tool with the schema of the MongoDB collection to be visualized. Users can review the schema output to ensure data types, sub-documents and arrays are correctly represented Translates SQL statements issued by the BI tool into equivalent MongoDB queries that are then sent to MongoDB for processing Converts the returned results into the tabular format expected by the BI tool, which can then visualize the data based on user requirements The BI Connector is available with MongoDB Enterprise Advanced. Watch the demo to see the BI Connector in action, and review the documentation to learn more. You can also download the BI Connector for evaluation Dynamic Lookup: Bringing Left Outer JOINs to MongoDB Applications get great efficiency from MongoDB by combining data that is accessed together into a single document. In contrast, a typical relational database schema scatters related data across scores of tables – e.g., a blog site that stores every tag, category, comment, author and callback as rows in separate tables from the blog post they’re associated with. Typically it is most advantageous to take a denormalized data modeling approach for operational databases – the efficiency of reading or writing an entire record in a single operation outweighing any modest increase in storage requirements. However, there are examples where normalizing data can be beneficial, especially when data from multiple sources needs to be blended for analysis. Consider a shopping cart, which presents two options for handling the order and product information: Include all data for an order in the same document Fast reads – one find delivers all the required data The order document contains the product details that were correct at the time the order was placed; the price of that product may change in the future but the order document remains an accurate representation of the order Consumes additional storage – the details of each product are stored in many order documents; this has become less of an issue as memory and storage prices have fallen Order document references product documents Space efficient – product details are stored just once Slower reads – multiple trips to the database If an attribute of the product (such as the unit price) changes in the future, any older order documents are then incorrect as they reference this newer version Extra application logic – an application must iterate over product IDs in the order document and then fetch the product documents MongoDB 3.2 introduces the ability to combine data from multiple collections by implementing left outer joins through the $lookup operator, which can now be included as a stage in a MongoDB Aggregation Framework pipeline . The new $lookup stage provides more flexibility in data modeling, and allows richer analytics to be run with higher performance and less application-side code. You can learn more from the documentation and see worked examples in this series of blog posts . Real-Time Analytics and Search MongoDB 3.2 extends the options for performing analytics on live, operational data – ensuring that answers are delivered quickly and simply, and are based on current data. Work that would previously have needed to be implemented in client code can now be performed by the database – freeing the developer to focus on building new features. Improved Aggregation The aggregation pipeline is a powerful way to perform complex analytical queries on MongoDB data. Aggregation pipeline stages allow manipulation of a "stream" of documents from a collection that can either be returned via a cursor to the client (similar to find ), or be stored in a new collection via a final $out stage. When analyzing very large data sets, it is frequently sufficient to look at a random sample of documents rather than all of the data. For example, if you wanted to compare the number of check-ins to coffee shops to those at bars, you can get a good approximation without searching through every single check-in. Previously, this sampling would have to have been implemented in the application, but MongoDB introduces the $sample operator, which can be included at any point in the aggregation pipeline. MongoDB documents can store arrays as well as simple values. While this feature is very expressive and powerful, without the corresponding ability to manipulate and filter arrays in documents during aggregations, their usefulness has been limited in an analytical context. New operators have been added to allow more flexibility when dealing with arrays: $slice , $arrayElemAt , $concatArrays , $isArray , $filter , and $min . New mathematical operators have been added for operations such as truncate, ceiling, floor, absolute, rounding, square root, logarithms and standard deviations. These operators can be used to move code from the client tier directly into the database, allowing higher performance with lower developer complexity. By combining the new and existing operators aggregation pipelines can be built to generate sophisticated results with a single query. Review the documentation to learn more about all of the aggregation pipeline enhancements. Improved Text Search Text searches on the data in MongoDB can either be performed in the database or by an external search engine. Performing the search within the database is more efficient and simpler to administer, so that is the preferred option whenever possible. MongoDB 3.2 increases the set of use cases that can be met with in-database text searches by adding support for case-sensitive searches, as well as additional languages including Arabic, Farsi, Urdu, Simplified Chinese, and Traditional Chinese. To provide support for these languages, MongoDB Enterprise Advanced provides integratation with Basis Technology Rosette Linguistics Platform (RLP) to perform normalization, word breaking, sentence breaking, and stemming or tokenization depending on the language. More information on text search enhancements can be found in the documentation . DBAs: MongoDB Compass MongoDB’s dynamic schema and rich document model make developers more productive, but they also make it difficult to explore and understand the underlying data and its structure – in particular for non-developers who aren't familiar with the MongoDB query language. The MongoDB Compass GUI allows users to understand the structure of data in the database and perform ad hoc queries against it – all with zero knowledge of MongoDB's query language. Typical users could include architects building a new MongoDB project or a DBA who has inherited a database from an engineering team, and who must now maintain it in production. They need to understand what kind of data is present, define what indexes might be appropriate, and identify if Document Validation rules should be added to enforce a consistent document structure. Until now, users wishing to understand the shape of their data would have to connect to the MongoDB shell and write queries to reverse engineer the document structure, field names and data types. Similarly, anyone wanting to run custom queries on the data would need to understand MongoDB's query language. MongoDB Compass provides users with a graphical view of their MongoDB schema by sampling a subset of documents from a collection. By using sampling, MongoDB Compass minimizes database overhead and can present results to the user almost instantly. Figure 2: Document structure and contents exposed by MongoDB Compass Querying Data As illustrated in Figure 3, a query can be built and executed by simply selecting document elements from the MongoDB Compass user interface. By selecting multiple values, sophisticated queries can be built. The query can then be executed at the push of a button and the results viewed both graphically and as a set of JSON documents. Figure 3: Interactively build and execute database queries with MongoDB Compass MongoDB Compass samples the database to provide a fast, interactive experience no matter how large the database. If the full results are needed then the query can be simply copied and pasted into a MongoDB shell window. MongoDB Compass is included with MongoDB Professional and MongoDB Enterprise Advanced. You can learn more about Compass in the documentation , and see it in action in our short 3-part demo series: Part 1: Introduction to Compass Part 2: Schema visualization Part 3: Visual query building To evaluate MongoDB Compass, head to the MongoDB Download Center . Operations Teams MongoDB Ops Manager and Cloud Manager are the best way to run MongoDB, reducing tasks such as deployment, scaling, upgrades and backups to just a few clicks or an API call. Operations teams can be 10-20x more productive using the Ops or Cloud Manager platforms. With the enhancements to both Ops and Cloud Manager in MongoDB 3.2, administrators can: Integrate MongoDB alongside existing Application Performance Monitoring platforms for global health visibility over the entire IT estate, all from a single pane of glass Drill down into any MongoDB-specific issues using Ops Manager’s granular monitoring of database telemetry, including new query profiler visualizations Use Ops Manager automation to initiate zero-downtime maintenance and upgrade activities, such as rolling out new indexes across a sharded cluster Create point-in-time, consistent snapshots of the database on standard network-mountable filesystems, and restore complete running MongoDB clusters from backup files. APM Integration: New Relic & AppDynamics Many operations teams use Application Performance Monitoring (APM) platforms such as New Relic and AppDynamics to gain global oversight of their complete IT infrastructure from a single management UI. Issues that risk affecting customer experience can be quickly identified and isolated to specific components – whether attributable to devices, hardware infrastructure, networks, APIs, application code, databases and more. The MongoDB drivers have been enhanced with a new API that exposes query performance metrics to APM tools. Administrators can monitor time spent on each operation, and identify slow running queries that require further analysis and optimization. In addition, Cloud Manager will provide packaged integration with the New Relic platform. Key metrics from Cloud Manager are accessible to the APM for visualization, enabling MongoDB health to be monitored and correlated with the rest of the application estate. Figure 4: MongoDB integrated into a single view of application performance As shown in Figure 4, summary metrics familiar to Cloud Manager users are presented within the APM’s UI. Administrators can also run New Relic Insights for analytics against monitoring data to generate dashboards that provide real-time tracking of Key Performance Indicators (KPIs). If the operations team needs finer grained telemetry, they can drill down into the 100+ system metrics maintained by Cloud Manager. For example, the new visual query profiler helps diagnose slow running queries, which can then be resolved by adding a new index and automatically deploying that across every node in the cluster. Query Performance Visualization: Enabling Fast and Simple Query Optimization The MongoDB database profiler collects fine-grained information that can be used to analyze query performance. However, the output could be difficult to parse, making slow running queries difficult to correct. In addition, the profiler had to be individually activated for each MongoDB instance, and the output manually aggregated from every node to provide a holistic view across the entire deployment. Delivered as part of Ops Manager and Cloud Manager, the new Visual Query Profiler provides a quick and convenient way for operations teams and DBAs to analyze specific queries or query families. The Visual Query Profiler (as shown in Figure 5) displays how query and write latency varies over time – making it simple to identify slower queries with common access patterns and characteristics, as well as identify any latency spikes. A single click in the Ops Manager UI activates the profiler, which then consolidates and displays metrics from every node in a single screen. Figure 5: Visual Query Profiling in MongoDB Ops Manager Index Suggestions & Automated Index Builds Further simplifying operations, the visual query profiler will analyze data it collects to provide recommendations for new indexes that can be created to improve query performance. Once identified, these new indexes need to be rolled out in the production system. In order to minimize the impact to the live system, the best practice is to perform a rolling index build – starting with each of the secondaries and finally applying changes to the original primary, after swapping its role with one of the secondaries. While this rolling process can be performed manually, Ops Manager and Cloud Manager can now automate the process across MongoDB replica sets, reducing operational overhead and the risk of failovers caused by incorrectly sequencing management processes. New Indexing Option: Partial Indexes Secondary indexes are one of the ways that MongoDB distinguishes itself from NoSQL databases – allowing applications to efficiently access their data in multiple ways. However, secondary indexes do come with a cost: Database writes will be slower when they need to update the secondary index Memory and storage is needed to store the secondary index Partial indexes balance delivering good query performance while consuming fewer system resources. For example, consider an order processing application. The order collection is frequently queried by the application to display all incomplete orders for a particular user. Building an index on the userID field of the collection is necessary for good performance. However, only a small percentage of orders are in progress at a given time. Limiting the index on userID to contain only orders that are in the “active” state could reduce the number of index entries from millions to thousands, saving working set memory and disk space, while speeding up queries even further as smaller indexes result in faster searches. By specifying a filtering expression during index creation, a user can instruct MongoDB to include only documents that meet the desired conditions. When performing the database find operation, the application should include the value being used for the filtering as well as the indexed value in order for the partial index to be used by the optimizer. Review the documentation to learn more. Additional Ops Manager Enhancements Beyond the functionality discussed above, a number of enhancements improve productivity of Ops teams and simply installation and management. Ops teams can now automate their database restores reliably and safely using Ops Manager and Cloud Manager. Complete development, test, and recovery clusters can now be built in a few simple clicks. Integrating with existing storage infrastructure, MongoDB backup files can now be stored on a standard network-mountable file system. Operations teams can configure backups against specific collections only, rather than the entire database, speeding up backups and reducing the requisite storage space. This enhancement is also available in the Cloud Manager platform. Installation and configuration of all application and backup components can now be made through the centralized Ops Manager UI, which also provides a single, at-a-glance dashboard for health monitoring. Enhancements to the backup architecture provide faster time to first database snapshot. Eliminating false alarms, maintenance windows can now be defined during which Ops Manager and Cloud Manager alerts will not be triggered. From APM integration through to profiler visualization with index suggestions and automated rolling index builds, through to platform simplification and automated cluster restores, Ops Manager can make your teams more productive. Next Steps This post wraps up our 3-part blog series. As you have seen, MongoDB 3.2 is a significant release of the world’s fastest growing database: New storage engine options, coupled with document validation, the enhanced replication protocol and sharding improvements extend the range of mission-critical applications MongoDB can serve. New tools such as the BI Connector, Compass, and Cloud Manager integration to APM platforms allow organizations to take advantage of MongoDB while protecting investments in existing frameworks and workflows. Remember, you can get the detail now on everything MongoDB 3.2 offers by downloading the What’s New white paper . Alternatively, if you’d had enough of reading about it and want to get your hands on the code now, then: Download MongoDB 3.2 today. Evaluate MongoDB Enterprise, along with the Connector for BI, MongoDB Compass and Ops Manager by heading to the MongoDB download center . To start using MongoDB 3.2 as quickly and efficiently as possible, bring in the experts. MongoDB’s consulting engineers can deliver a private training on 3.2 features tailored to your needs, then work with you to develop a customized upgrade plan for your deployment. Interested? Learn more About the Author - Mat Keep Mat is a director within the MongoDB product marketing team, responsible for building the vision, positioning and content for MongoDB’s products and services, including the analysis of market trends and customer requirements. Prior to MongoDB, Mat was director of product management at Oracle Corp. with responsibility for the MySQL database in web, telecoms, cloud and big data workloads. This followed a series of sales, business development and analyst / programmer positions with both technology vendors and end-user companies.
Latinas in Tech: Andy Morales Coto
This spotlight is part of a blog series to amplify exceptional Latina talent in the tech industry. Through our partnership with Latinas In Tech, this article originally appeared on their site . Tell us about yourself, Andy. How did you get to where you are today? I’m originally from Costa Rica and have been living in NYC for the past six years. I’m a product designer, but I wasn’t always one: before coming to New York, I was working in multiple industries, as a game designer, a copywriter, and a digital marketer. But I guess most of that is just titles and places I come from, not really the way I got to be where I am. If I look more deeply, I would say that the moments that have led me to where I am today are a mixture of privilege and the fallout of self-discovery. I was born in an upper middle class family, the daughter of two public servants — a doctor and an engineer — and learned English pretty early on at their behest. I was able to go to private school my whole life, up until college, when I attended the University of Costa Rica, which is publicly funded by all Costa Ricans. I wouldn’t say I had a luxurious life growing up: there were certainly hand-me-downs from my sisters, but I also never had a problem buying a video game console if I wanted it — I’d just have to give up having a birthday party (and I did). Overall, I’d say my parents motivated me to follow my dreams, and would gladly take me to any classes I wanted (English, robotics, programming, drawing) from the time I was a little girl. In that sense, I always had a leg up, understood what was considered “excellence” in education, and pretty early on set my mind on studying abroad eventually. With that said, my comfortable life became, well, not comfortable at all when I came out at 19. College changed my life completely. Finally being able to understand who I was, I came out as queer to my very conservative parents, and the reception was extremely toxic. For the first time, I understood what it meant to not be able to afford a meal, or even a bus ticket. I walked miles to go to college several times, hell-bent on finishing my degree in communications (the closest thing to tech, I figured, without the toxicity of the homogeneity of computer science). Finally I graduated, but my whole perception of the world had changed: I became more empathetic and less judgmental of others, and I knew what depression and trauma were. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. I continued working for several years after graduating from college, did another degree in marketing while I worked, and finally got accepted into Parsons (NYC) on a scholarship to study transdisciplinary design. And here we are! Oh, also, and this is very important: I’m married to a lovely American and live with her and two fluffy tabby cats in Brooklyn. NYC is what I call home now (and probably forever). What inspired you to pursue a career in the tech industry? I think pretty early on I was in awe of technology, and I don’t just mean computers, but also cars, glasses, electricity, hammers. I’ve always admired anything that expands the possibilities of what a human can do. But my “aha moment” happened when I was 10 and accessed the internet at the University of Costa Rica. My mother was a teacher there and had access to connection before the rest of the country did. She’d sometimes let me use her computer, and I still remember using Netscape in complete fascination of what this meant for humanity: we would all be connected. That’s when it really clicked for me: I love this, I love computers. As a manager at MongoDB, what have been some of the most memorable and impactful projects you’ve worked on so far? I’m the most proud of the people I manage, and seeing them grow every day. My direct reports are infinitely more talented than I am in some ways, and I welcome that. I want to be surrounded by people more talented than I am, and they’re going to change the face of the design industry, I have no doubt. Watching them get better and better, lead projects of their own, and successfully navigate difficult stakeholder situations — well, it just puts a smile on my face! But, apart from that, a specific project I’ve enjoyed is Blue Sky, a yearly design-driven sprint that we do in conjunction with key stakeholders to create the “concept car” of the product I lead design for. This will be the second year we do Blue Sky, and we hope to use design thinking beyond the graphical user interface, partnering with product and engineering to imagine the future experience of MongoDB Realm in the CLI and the IDE. With each Blue Sky, design positions itself as a partner for our stakeholders, and our proposals coming out of the project tend to be implemented up to 75% of what we design. It’s exciting to become strategic partners in the direction the product will take. How has your culture (and/or other identity marker) shaped you as a leader? As a manager? Well, my culture is a mixture of queer culture, Costa Rican culture, and NYC culture. I think all of these shape me as a leader, because it means I am not a monolith as a person; I’ve learned to see the world through many different perspectives. Being able to compare and contrast how different cultures view or react to situations makes me self-aware, and puts me in a position where I strive to understand how others are reacting to situations, in the frame of their culture. I’d say this is empathy, which is a bit of a design cliche, but I actually think that it’s more than empathy — it’s vulnerability and sobering humility. Trust me, I wasn’t always super self-aware, but as I’ve gotten to know the world through different cultural lenses, I’ve realized that I have to be careful with how I help others be what they consider their very best. Whether it’s grappling with cultural expectations or navigating workplace biases, we fight through many challenges as Latinx women. What’s one you’re working through currently? I’m definitely sometimes worried about how I come off to my teammates, particularly those who are not Latin American. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. Unfortunately, as a woman and as a Latina, these can be seen as vapid qualities, symbols of weakness. Why is she so loud, so emotional, so open to talking? In the past, I’ve tried to cover this up by being serious, talking softly but more deeply, and avoiding vulnerable conversation; as I’ve grown older, I’ve realized that inhibiting those qualities hinders me at work, because it makes me feel miserable, and that I end up gaining more supporters in the long term by being as open-hearted as I am. I definitely think I have my upbringing in Costa Rica to blame for that: it is not the norm for women to be like that at work, but while I was growing up I certainly saw more female bosses be open and vulnerable. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. This, of course, sometimes brings some internal turmoil: Am I just not meant to be in this American culture? Am I borrowing from my Costa Rican experiences without giving back? There’s a certain sense of duty that you feel toward those who are in your home country, even if your current definition of home has changed (I consider myself more a New Yorker than anything else, by now). To be honest, I don’t have a solution to that sense of duty and loss, and I struggle with it pretty often. I deal with it by donating and helping others that want to chase their dreams in the USA, but I still struggle with it. It’s hard not to miss the place you grew up in. It’s a big piece of you, no matter where you go. Looking to the future, what inspires you, and what initiatives are you most excited about right now? I’m inspired by games, and I can’t wait to continue using playful design in every product I design. Tangentially, I design live action role playing (LARP) games, and I can’t wait to be able to play with my other designer friends again, hopefully at a house by the beach. What’s one piece of career advice you’ll never ever forget? One of my professors from grad school, Mathan Ratinam, told me once that throughout his career he learned that you are lucky if you get to choose a job for one of three reasons: you love the work, you love the mission, or you love the people. I’ve tried loving the work, and I’ve tried loving the mission, but let me tell you: if I don’t enjoy working with the people, I’m not going to be happy in the long term. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. How do you reset when you’re in a funk? I let myself cry/experience sadness first, I go to therapy (cannot stress this enough: if you can afford it, please go to therapy), and I practice Muay Thai. I just love kicking a bag and sweating the problems out, you know? Any podcasts or blog recommendations? I don’t really listen to podcasts or read blogs that often. I play games and I read books; those are my two sources of design inspiration. I’d say, if you can, play “Zelda: Breath of the Wild,” to see what the epitome of design is. Also, play any LARP from the Golden Cobra Challenge: http://www.goldencobra.org/ . You can print those for free and play them with people online. Bookswise, I’ve been reading Fall ; or, Dodge in Hell , by Neal Stephenson, but sometimes it hits too close to home. Is there anyone you’d like to shout out for their support along your career journey? My wife, Crystal Morales. She’s the best thing that has ever happened to me. She is the smartest career advisor I know, and the smartest person I know. Period. Mathan Ratinam, of course, whom I mentioned before. He has inspired me so many times and listened to me talk for hours on the phone. A champ. My friends who, during college, helped me get a meal when I couldn’t: Olalla, Edith, Diana (my best friend since then), Warren, Memo, MaJo. A big hug to them all. And my college teacher Andrea Alvarado, who understood the pains I was going through at home when I came out and, instead of failing me, gave me extra work to do, showing me that part of being compassionate is never being condescending. Andy is thriving as a lead product designer at MongoDB . If you’re ready to work with what sounds like an incredible group of people, here are three open roles you should check out! Product Manager, Server Sales Development Representative Lead Engineer, Docs Platform