Jay Gordon
May 9, 2016 | Updated: March 7, 2017
#Cloud

Cloud Manager automation is an easy way to create new MongoDB deployments without having to do much of the underlying configuration work. This post is how you can get started using automation on your droplets.

A note: This does not provide details on configuring security for your system. Please review our documentation on configuring MongoDB with iptables on Linux.

Initial setup:

Let’s begin by determining what our use case and requirements are for our application. I think today I will start by working on a simple three node replica set using 1GB sized droplets. First ensure you’ve got credit and the appropriate ssh key you plan on working imported to your account.

First I’ve created the three droplets required, I am using Ubuntu 14.04 LTS x64, however feel free whichever of the supported versions we have agents for. Cloud Manager supports many different OSs and has a number of agents available. DigitalOcean has a Private Networking option, making it easy to configure your deployment securely. When creating these new droplets, it’s a good idea to enable this service.

Next, select a hostname, in this case we’ll use something able to identify where we are located and what we are doing:

Configuring the Droplets:

Your droplets will need a few post-configuration options for our automation to begin working correctly.

Hostname and Private IPs

SSH into your droplets and validate their hostname:

root@mongodb-1gb-nyc2-01:~# hostname -f
mongodb-1gb-nyc2-01

root@mongodb-1gb-nyc2-02:~# hostname -f
mongodb-1gb-nyc2-02

root@mongodb-1gb-nyc2-03:~# hostname -f
mongodb-1gb-nyc2-03

Review each host’s private IP:

root@mongodb-1gb-nyc2-01:~# /sbin/ifconfig eth1 | grep 'inet addr:' | cut -d: -f2| cut -d' ' -f1
10.128.32.63

root@mongodb-1gb-nyc2-02:~# /sbin/ifconfig eth1 | grep 'inet addr:' | cut -d: -f2| cut -d' ' -f1
10.128.32.64

root@mongodb-1gb-nyc2-03:~# /sbin/ifconfig eth1 | grep 'inet addr:' | cut -d: -f2| cut -d' ' -f1
10.128.32.66

Create hosts file entry for each site and then enter it into each server’s /etc/hosts/ file:

10.128.32.63 mongodb-1gb-nyc2-01
10.128.32.64 mongodb-1gb-nyc2-02
10.128.32.66 mongodb-1gb-nyc2-03

Our entire /etc/hosts file example for mongodb-1gb-nyc2-01:

127.0.1.1 mongodb-1gb-nyc2-01 mongodb-1gb-nyc2-01
127.0.0.1 localhost

10.128.32.63 mongodb-1gb-nyc2-01
10.128.32.64 mongodb-1gb-nyc2-02
10.128.32.66 mongodb-1gb-nyc2-03

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Test you can ping by name from a host:

root@mongodb-1gb-nyc2-01:~# ping mongodb-1gb-nyc2-02
PING mongodb-1gb-nyc2-02 (10.128.32.64) 56(84) bytes of data.
64 bytes from mongodb-1gb-nyc2-02 (10.128.32.64): icmp_seq=1 ttl=64 time=1.75 ms

Install the automation agent on all three:

Next, we install the automation agent to all three servers. Log onto your Cloud Manager Deployment, click the “Agents” tab and you’ll then find a tab titled “Downloads & Setttings.” Here you will be able to review the different agents available, in this case we will use the Ubuntu (12.X, 14.X) - DEB agent.

Once you click this link, you’ll see a set of instructions on how to install the agent, enter the appropriate API key information and how to prep your data directory for the MongoDB database files. To save time, you can repeat each step of these instructions in parallel across servers with the same OS, on Mac OS X I’ve found iTerm 2 useful for this feature along with ClusterSSH in Linux.

Install the package as instructions say and then start the agent across your three droplets:

root@mongodb-1gb-nyc2-01:~# sudo dpkg -i mongodb-mms-automation-agent-manager_2.7.1.1631-1_amd64.deb
Selecting previously unselected package mongodb-mms-automation-agent-manager.
(Reading database ... 86823 files and directories currently installed.)
Preparing to unpack mongodb-mms-automation-agent-manager_2.7.1.1631-1_amd64.deb ...
Unpacking mongodb-mms-automation-agent-manager (2.7.1.1631-1) ...
Setting up mongodb-mms-automation-agent-manager (2.7.1.1631-1) ...
Adding system user `mongodb' (UID 106) ...
Adding new user `mongodb' (UID 106) with group `nogroup' ...
Not creating home directory `/home/mongodb'.
Adding group `mongodb' (GID 114) ...
Done.
Adding user `mongodb' to group `mongodb' ...
Adding user mongodb to group mongodb
Done.
Processing triggers for ureadahead (0.100.0-16) …

Open the config file, add your API and Group information in the file as recommended:

sudo vi /etc/mongodb-mms/automation-agent.config

This file may look something like this, just paste in the key values given to you in the installer instructions on the Cloud Manager UI:

#
# REQUIRED
# Enter your Group ID - It can be found at https://cloud.mongodb.com/settings/group
#
mmsGroupId=


#
# REQUIRED
# Enter your API key - It can be found at https://cloud.mongodb.com/settings/group
#
mmsApiKey=

Save the file and close it up

Prep your data directory

The default location we normally like to recommend is /data. You may choose a different directory but it must be configured with much of the same values:

Prepare the /data directory to store your MongoDB data. This directory must be owned by the mongodb user.

sudo mkdir -p /data
sudo chown mongodb:mongodb /data

Start your agent (startup method may depend on your OS install):

And finally let’s start the agent!

sudo start mongodb-mms-automation-agent
mongodb-mms-automation-agent start/running

Monitoring Agent

Now that we have our automation agents installed, wait a few seconds and they should begin to report back to Cloud Manager (Deployment - > Agents):

As the red box says, we’re not quite ready to get started. We still need to install a monitoring agent, which we can do right here from Cloud Manager’s UI. Be aware, unlike the automation agent, the monitoring agent only requires installation on one server in your group. [You can review this detailed blog post on agents and how many are required for additional information on this subject] (http://blog.cloud.mongodb.com/post/120124825840/how-many-agents-do-i-need-and-why/).

First, go to the Server’s tab under Deployment. Your droplets should be visible here:

Since we only require this monitoring agent on one of the nodes, we can go ahead and just click the “…” and then select to install the monitoring agent. We’ll do this on mongodb-1gb-nyc02-01:

Click the yellow bar on the top to review deploy and allow the agent to install.

Automate your installation:

Now we are finally ready to configure our systems to run with MongoDB as a replica set.

Click Deployment and then find the green +ADD button, and select “New Replica Set”

This will bring up your replica set configuration setting within your Cloud Manager UI, you can then set your systems and MongoDB configuration here for your new deployment, then click apply:

Now we have a preview of our new replica set, check all looks right and then click Review & Deploy:

Now Wait…

Now wait and you should see your deployment begin, in but a few minutes you should then see your new DigitalOcean MongoDB Cluster in the Cloud Manager UI:

That’s it, MongoDB has been deployed on your droplets. At this time, it’s a good idea to begin the process of configuring your MongoDB authentication and security of your network. Once you’ve done these tasks, you can begin inserting your data and working to develop your application.

Jay Gordon is a Technical Account Manager with MongoDB and is available via our chat to discuss Cloud Manager at cloud.mongodb.com

← Previous

Make IoT a Reality at MongoDB World

Millions of people monitor their homes with “Smart Home” technology. A small genomic sequencer the size of a small cellphone helps health workers understand the spread of viruses like Ebola and stop the an epidemic in its tracks. One of the largest transformations in the twenty first century is the Internet of Things (IoT) - the availability for devices to process sensor data and to analyze this data for new insights in real time. IoT has been adopted across many industries, from home services to retail to industrial manufacturing. MongoDB World will showcase customers who have turned to MongoDB to tackle the complicated world of IoT. Learn more about how MongoDB is securing homes, tracking the spread of Ebola and more through IoT. Building and Scaling the Internet of Things with MongoDB at Vivint Three years ago Vivint started an ambitious initiative to make the Smart Home mainstream. Their new platform had to have ultimate extensibility, ultra-low latency, a wide array of functionality out of the gate, and it had to be ready to attach millions of devices in its first year of operation. With only a year to go from concept to customers, Nick Brown’s team at Vivint chose to built a brand new integration platform based around MongoDB. 8 million devices and 1 billion messages per day later, Nick will share best practices for architecting scaleable IoT platforms based on his experience. Building an IoT Network In this session, Dave de Groote will share how he and a team of developers built new IoT network for the largest Telecom in Belgium. You’ll learn about the system’s requirements and how they arrived at MongoDB. Dave will also walk you through some of the application design and schema tradeoffs they made, how they defined storage engine requirements, and the aggregation and indexing strategies that allow for read-intensive and high performance systems. Building the Internet of Living Things Oxford Nanopore Technologies uses MongoDB to help prevent the spread viruses by analyzing genetic material in real-time. The UK-based company has built a handheld device that can analyze any piece of genetic material in just a few hours, rather than the weeks it had taken previously. This speed is enabled by Oxford Nanopore’s MongoDB-powered cloud deployment that lets users access its genetic database anywhere in the world. Their device, MinIon, is being used in Guinea to track the outbreak of Ebola, and will soon be tested by NASA in the international space station. At MongoDB World, you’ll meet Richard Carter, Associate Director of Data Integration at Oxford Nanopore Technologies, who will teach you how they build the software behind the MinIon device to rapidly analyze complex genomic information. You’ll learn how they simplify operations with Cloud Manager and how they optimize their schemas for fast reads. The Device Mesh and the User Experience: A 3 Part Series on IoT and MongoDB Want IoT from soup to nuts? Join the Next Gen Apps track, you get three sessions that cover architecting, operationalizing, and deploying an app. The three-part session on IoT is led by a veteran team of MongoDB experts, Jake Angerman and Jay Runkel. In this series they’ll walk you through IoT architectures Complex analytical queries Data ingestion processes Efficient sharding techniques Native MongoDB analytics options Connecting to Spark and external BI sources Are you looking to build IoT applications with MongoDB? Join us at MongoDB World to learn the from the experts on analytics, operations and more. Early Bird ticket pricing ends May 13, so snag your tickets now below. Register for MongoDB World 2016

May 9, 2016

Next →

Three Ways Retailers Use Search Beyond the Ecommerce Store

When consumers think of retail search, the first thing that comes to mind is typically the search bar of an ecommerce website. This is for a good reason: a Salesforce commerce study shows that 87% of shoppers begin their shopping journey in the search bar, and Forrester has found that as many as 68% of shoppers would not return to a site that provided a poor experience. But retailers that exclusively focus on search capabilities in the context of ecommerce are missing out on huge benefits in customer experience and workforce efficiency. To drive fast application experiences, the querying and indexing of data sets is vitally important, and can be a game changer for easy performance optimization. Let’s explore some of the innovative ways that retailers are using search indexes to super-power their application experiences. Why search is important across retail organizations Large retail data sets, like product and customer data, are used by both customer-facing ecommerce or loyalty applications and internal use cases: inventory management, stock management, customer care, purchasing, supplier and vendor management, marketing and more. Customers using ecommerce search bars typically have an excellent “Google-like” experience with auto-complete, faceting, fuzzy matching, etc., but the retail workforce and back office staff often aren't given the same luxury. These internal teams are trying to work efficiently, but they are stuck using front ends powered by a traditional operational database with no search indexing capabilities. These teams are missing out on a search engine that is optimized for unknown or unpredictable workloads. Search indexing will speed up queries where the input is user-defined or might be searching across multiple fields. Let’s look at a comparison: FIgure 1: Database Query vs. Search Query For these under-served and often overlooked use cases, retailers need a quick and cost-effective solution to adding search, like MongoDB Atlas Search. Adding a search index to an application can be done in minutes, without creating operational complexity. MongoDB manages the spin up and management of the backend Apache Lucene 8 search engine, and the complex data and index synchronization activity. Figure 2: MongoDB Atlas: Integrated database and search The three most common use cases for retail The easy addition of search can optimize application performance and usability in the retail industry in three important areas: In-store workforce applications Back office inventory and assortment Customer servicing Figure 3: Example Search use cases in three retail industry areas In-store workforce applications Speed is important in workforce applications, because these interactions happen in real time. Think of an in-store customer spelling out a name in full at checkout for a grocery purchase to be added to a loyalty account. This could add five minutes to a checkout experience, disincentivizing the customer to engage with the loyalty program. Now imagine that the same checkout attendant can identify the customer by any number of data points, not only loyalty number, but also name, first line of address, email, etc., with faster lookup through auto-complete and fuzzy matching. A retailer that MongoDB works with does this auto-lookup in store with Atlas Search in 200-300 milliseconds for optimum customer satisfaction. Customers and staff also can have difficulty remembering or correctly identifying products. A DIY amateur or a new employee can’t be expected to know the exact name or product ID. This is a great use case for search indexing as we do not know the field in the document or the product attribute that we are querying against. MongoDB has customers that stock more than 150 million products. Strong typo tolerance makes life easier for everyone. Back-office inventory and assortment Flawless purchasing and stock management ensures brick and mortar and online stores get the right inventory at the right time to maximize sales and reduce wastage or deadstock. An operator responsible for distributing products into categories will define in which store shelf a product needs to be and adjust this depending on customer behavior and contractual changes with the supplier. Inventory applications will be used on a daily basis by every operator.. These are small internal applications that can have a huge impact on the overall business, but are often overlooked by large IT programs for budget or have a smaller IT team. These teams are adopting Atlas Search because they can get it up and running in as few as three weeks and fully integrated into their application without taking on more operational overhead. Customer servicing Long call center or chat conversations wait times have high operational costs and cause customer churn. It is vital to identify the customer as quickly as possible by the data they provide: order or customer ID, phone number, store address, etc. Retailers who have created a “Customer 360” across their customer relationship management and loyalty systems have created a large complex pool of data. The ability to run a single query to search across all available attributes makes it much faster to identify a customer. Search can also be used to optimize speed and accuracy of results for chat applications and chat bots who have to answer a large volume and variety of questions. This is a perfect use case for search with unpredictable user inputs. If answers to the questions can be searched across the entire knowledge base, speed and relevancy can be improved. MongoDB has retailer customers building chatbot applications for internal use cases like an IT team answering common questions, and external ones. For example, on the ecommerce homepage, a chatbot needs search functionality to be able to quickly do product lookup, customer identification, or make a suggestion. Quick and easy search implementation will add to the customer experience and reduce staff operations. Where your company could add search functionality It’s time to think beyond the ecommerce search bar. What are the search workloads within your company’s retail estate? Are there internal applications that have your frontline or back-office staff frustrated with inefficient lookups? Is the reason you’re not implementing search today the fact that it's a heavy lift to add an additional technical component to your architecture? These are the types of conversations that are driving adoption of Atlas Search across the retail industry, as businesses persevere in a tough macro-economic climate to do more with less. Adding vital functionality to applications without adding complexity is a win for the retailer, the workforce and the consumer. Want to learn about how MongoDB has integrated Search into the Atlas Developer Data Platform? Head to the Search solution page to explore more technical and in-depth resources.

March 29, 2023