GIANT Stories at MongoDB

Boosting JavaScript: From MongoDB's shell to Node.js

Dj Walker-Morgan

Node.js, mongo

Moving a script from MongoDB’s JavaScript-powered shell to Node.js offers a chance to get to use an enormous range of tools and libraries. Find out how to do this with only a few extra lines of code and how to then optimize the resulting script.

The Top 6 Questions From AWS re:Invent 2018

Lauren Schaefer

mongodb, aws

Hey there, MongoDB Community!

I'm Lauren Schaefer (@Lauren_Schaefer, linkedin.com/in/laurenjanece), MongoDB's newest developer advocate. I've only been on the job a couple of weeks, but I had the opportunity to travel to fabulous Las Vegas, Nevada, last week to speak with many of you at the MongoDB Atlas booth at AWS re:Invent 2018.

The people I chatted with complimented MongoDB over and over again. I heard things like, "The performance is great!" and "When I get to choose what database I use, I choose MongoDB" and "I love Mongo!"

People also asked me a lot of questions. I’ve compiled those questions into a list of the top 6 most frequently asked questions at AWS re:Invent 2018.

6. Are the socks different sizes?

My primary job at the booth was to give out socks. And I gave out a LOT of socks. Several people told me that they wear the MongoDB socks they received at last year's conference all the time. I even had people show me the MongoDB socks they were wearing.

Since I was giving out so many socks, one of the most common questions I received was, "Are the socks different sizes?" The socks were all one size, but they seemed to stretch to fit a variety of sizes--they’re built to scale!

5. What is Atlas?

To be fair, this question probably came up so frequently because I asked people, "Are you familiar with Atlas?" as I was handing them socks to which they commonly replied, "No. What is Atlas?"

You can think of MongoDB Atlas as MongoDB-as-a-service. Atlas is a fully managed, global cloud database. Atlas takes care of all the operations related to running a MongoDB database in production -- security, availability, upgrades, and patches -- so you can focus on your data and your app. You can get more details in the video below.

4. Can I see a demo of Atlas?

As you can probably imagine, people were pretty excited about Atlas when they heard about it, so they wanted to see a demo. We had experts on-hand ready to give demos. For those of you who weren't able to get a demo in-person, below is a demo of how to get started with Atlas.

3. Is Atlas new?

People were very excited about Atlas and many were surprised they hadn't heard of it before. A common question was, "Is Atlas new?"

No, Atlas is actually a little over two years old. It was officially announced at MongoDB World on June 28, 2016.

2. What is the Atlas pricing model?

Before people started to get too excited about Atlas, they wanted to know if there was a catch. They'd ask, "What's the pricing model?"

Atlas has a free tier so you can tinker and begin early development without paying a thing. You don’t even need to provide credit card information to get started. Once you exceed the free tier, Atlas is billed hourly based on how much you use. Check out the Atlas Pricing page for more details on the pricing model. The Atlas Pricing page also has a pricing calculator so you can estimate how much Atlas would cost for your particular use case.

1. Why would I choose MongoDB over Amazon DynamoDB?

Since we were at an Amazon conference, many people were curious about the differences between MongoDB and Amazon's DynamoDB.

You can get a detailed comparison of the two on the Comparing DynamoDB and MongoDB page. Some of the key points that resonated with people at the conference were:

  • MongoDB provides built-in document validation. Users can enforce checks on document structure, data types, data ranges, and the presence of mandatory fields. DynamoDB has limited support for different data types. As a result, developers must preserve data types on the client, which adds complexity and reduces data re-use across different applications. DynamoDB does not have native data validation capabilities.
  • MongoDB documents can be up to 16 MB in size whereas DynamoDB items or records can be up to 400 KB in size.
  • MongoDB provides for more flexible indexing and querying. For example, MongoDB indexes are consistent with data whereas DynamoDB indexes are sized and provisioned separately from data. Also, MongoDB allows for querying and analyzing data in multiple ways including single keys, ranges, faceted search, graph traversals, and geospatial queries. DynamoDB allows for key-value queries.
  • MongoDB can be deployed anywhere or as a service with MongoDB Atlas on Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure, so you are not locked into a particular vendor. DynamoDB is available as a service on AWS.

Summary

I had a blast meeting so many of you at AWS re:Invent, and I hope to meet many more of you at upcoming events! Give Atlas a shot and let me know what you think!

Building iOS and Android Apps with the MongoDB Stitch React Native SDK

Andrew Morgan

Technical, Cloud

Create React Native mobile apps for iOS and Android using the MongoDB Stitch React Native SDK

Stitch & Mobile Webinar Questions & Replay

Andrew Morgan

Cloud, Technical

Responses to questions from my recent MongoDB Mobile and Stitch webinar – also includes a recording to watch at your leisure.

Scaling to the Cloud With MongoDB Atlas and Pivotal Cloud Foundry

Sani Chabi Yo

mongodb

MongoDB Atlas is a fully-managed cloud database developed by the same people that build MongoDB. Atlas handles all the complexity of deploying, managing, and healing your deployments on the cloud service provider of your choice (AWS, Azure, and GCP).

An application deployed within Cloud Foundry can leverage MongoDB Atlas in a variety of ways:

  • App configuration: Manually create a cluster, then get the connection string which you will use in your application.
  • User-provided Service: This option extends the previous one and basically means that the connection string can be used to create what is called in Cloud foundry language a User-provided service which is simply a means to enable developers to use services that are not available in the marketplace.

However, these options have some drawbacks:

  • Manual provisioning
  • Poor self-service experience we are looking for
  • Can’t set quotas to control services usages

There is a third way, the ultimate holy grail in providing an amazing self-service experience is to use the Open Service Broker API project. This project is an open source effort sponsored by Google, IBM, Pivotal, Red Hat, SAP, and many others. The intent is to provide a simple set of API endpoints which can be used to provision, gain access to and managing service offerings. In our scenario, we will create a service broker that will be used by Cloud Foundry to provision MongoDB deployments in MongoDB Atlas.

Cloud Foundry exposes its services in the Marketplace for users to consume. The service manages the provisioning and de-provisioning of MongoDB deployments and provides the necessary credentials for applications.

Pivotal Cloud Foundry with MongoDB Atlas

Don’t We Already Have a PCF Tile for MongoDB?

For those that wish to deploy and manage their own MongoDB deployment within a PCF infrastructure, there is a PCF tile for MongoDB. While using this tile does allow you to deploy MongoDB within your infrastructure it does require setting up MongoDB Ops Manager in your environment as well as making sure you have enough resources to accommodate the MongoDB deployments.

By leveraging the Service Broker and MongoDB Atlas you Reduce management complexity/overhead as well as provide:

  • A true Marketplace experience
  • Support for many compliances (i.e: HIPAA, PCI, etc.)
  • Access more comprehensible deployment sizes (T-shirts)
  • Multi-Cloud deployment strategy
  • The flexibility to deploy into MongoDB Atlas’s 56 regions utilizing any of the 3 majors Cloud Providers (AWS, GCP, Azure)
  • End-users can take immediate advantage of any new capabilities of MongoDB Atlas.

Getting Started With the Service Broker for MongoDB Atlas

To get started, create a MongoDB Atlas account and create an Organization. You’ll also need to configure access to MongoDB Atlas API by creating an API Key. Please refer the to links given for full details.

To get the newly created Organization’s ID. This is simply done by clicking on the “Settings” menu on the left.

MongoDB Atlas Organization ID

The next step is to authorize PCF to communicate with your MongoDB Atlas account. If you have properly followed Cloud Foundry deployment recommendations, this part gets really easy. Below is a simplistic view of the recommended deployment topology.

MongoDB Atlas Organization ID

Without a NAT VM, you will require additional configurations, including configuring all Diego Cells with a static public IP which will then need to be added to Atlas IP Whitelist. This is not ideal and can get complicated very quickly, especially when your Network topology keeps changing.

Now, let add the NAT VM public IP to Atlas API whitelist. If you testing this using PCF Dev, then this will simply be your laptop public IP. In the Atlas interface, and the account view, there is an API Whitelist section, on the upper right side of that section, click on the “Add” button.

MongoDB Atlas API Whitelisting

Then add the NAT VM public IP as an entry

MongoDB Atlas API Whitelisting

The last step is to define the Atlas tiers that you want to make available to the developers and deploy the Service Broker for Atlas. Here is the GitHub link to work through the process.

Let’s Check Out the Developer Experience

Let's check what plans are available in the marketplace by running the following command:

Command

    
$ cf marketplace -s mongodb-atlas-aws 
    

Text Output


$ cf marketplace -s mongodb-atlas-aws                                                                                                              
Getting service plan information for service mongodb-atlas-aws as admin...
OK

service plan         description                                                                                                                 
aws-dev              Please use this for Dev (This is a multitenant environment)                                                               free
aws-qa               Please use this for Qa                                                                                                    paid
aws-prod             Please use this for any Production deploiement                                                                            paid
aws-global_cluster   Please use this for any Production deployment that requires global cluster. It includes 2 zones in US_EAST and US_CENTRAL paid

Here we can see the list of plans we have previously configured.

Now let request a MongoDB cluster for our Dev environment by issuing the following command:

Command

    
$ cf create-service mongodb-atlas-aws aws-dev dev-db
    

Text Output

                                                                                     
Creating service instance dev-db in org pcfdev-org / space pcfdev-space as admin...
OK

Create in progress. Use 'cf services' or 'cf service dev-db' to check operation status.

Since this is an asynchronous process, you can check the provisioning progress by running the following command:

Command

    
$ cf service dev-db
    

Text Output


Showing info of service dev-db in org pcfdev-org / space pcfdev-space as admin...

name:            dev-db
service:         mongodb-atlas-aws
tags:            
plan:            aws-dev
description:     MongoDB Atlas Service on AWS
documentation:   
dashboard:       https://cloud.mongodb.com/v2/5bf305dbf2a30bc3a008384f#clusters

Showing status of last operation from service dev-db...

status:    create succeeded
message:   
started:   2018-11-19T18:50:07Z
updated:   2018-11-19T18:52:14Z

There are no bound apps for this service.

What happened here is that a project has been created and its IP Whitelist configuration adjusted to allows connection from any application living inside PCF. Then the cluster was deployed inside that project according to the selected plan (i.e: tier). This process took about 1-2 minutes.

A dashboard URL is returned, which gives you access to the project within Atlas.

MongoDB Atlas Dashboard

You can also verify that the project IP Whitelist has been properly configured to allow communication from the PCF environment.

MongoDB Atlas Dashboard

Now as a developer, we might want to bind the newly created cluster to your existent application. This is achieved by “binding” your app to a service. You do that by running the following command:

Command

    
$ cf bind-service spring-music dev-db
    

Text Output


Binding service dev-db to app spring-music in org pcfdev-org / space pcfdev-space as admin...
OK
TIP: Use 'cf restage spring-music' to ensure your env variable changes take effect

In the command above I’m using the Spring Music app.

Once that command was issued, the service broker created a database user and granted it the Atlas admin role. A password was also randomly generated for that user and returned to the application, along with the connection string. The application can then just leverage that information to connect to the new cluster.

On the Atlas dashboard, you can check that a user has indeed been successfully created.

MongoDB Atlas Dashboard

Finally, we can verify in PCF that they have been successfully added to the application environment variables by issuing the following command:

Command

    
$ cf env spring-music 
   

Text Output


Getting env variables for app spring-music in org pcfdev-org / space pcfdev-space as admin...
OK

System-Provided:
{
 "VCAP_SERVICES": {
  "mongodb-atlas-aws": [
   {
    "credentials": {
     "database": "test",
     "groupId": "5bf305dbf2a30bc3a008384f",
 "mongodbUri": "mongodb+srv://79f7e2411058421aa5692fae44f0f0de-4xa0q.mongodb.net/test?retryWrites=true",
     "password": "FH0m92BeSDJj",
     "uri": "mongodb+srv://79f7e2411058421aa5692fae44f0f0de-4xa0q.mongodb.net/test?retryWrites=true",
     "username": "79f7e241-1058-421a-a569-2fae44f0f0de"
    },
    "label": "mongodb-atlas-aws",
    "name": "dev-db",
    "plan": "aws-dev",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "MongoDB",
     "Atlas",
     "AWS"
    ],
    "volume_mounts": []
   }
  ]
 }
}

Conclusion

With a service broker for Atlas, the world is truly yours. If you are a platform engineer leveraging Cloud Foundry in your enterprise, then you already know that an important part of your job is about bringing value into the platform and reduce complexity/redundancies as much as possible. A Service Broker for Atlas can help better deliver on that promise.

Get started with MongoDB Atlas today and try out service brokers for yourself!.

Reducing the Need for ETL with MongoDB Charts

Databases as we know them have been around for over 40 years. When they first came about businesses would often keep data in separate systems and separate formats. There were a variety of reasons for these decisions. One of the side effects of these separate data stores is the need to combine together to be able to perform data analysis. This led to the long-standing practice of ETL, or Extract, Transform, Load.

ETL is a process to extract data from a starting data source, transform the data in some fashion, then load it into another data store. Sounds simple enough, but in fact, there is a lot of work going on under the covers and a lot of steps and decisions to navigate. These additional steps reduce the speed at which we can get meaningful insights from our data. Further, they rely on many assumptions about transforming data into what is assumed to be the correct format for later consumption - without knowing very much about the business questions to be asked of this data down the road.

From Data Warehouses to the Cloud

Traditionally, enterprise applications have relied on performing ETL operations to move data into an enterprise data warehouse (EDW).

Traditional ETL Architecture
Traditional ETL Architecture

Creating a successful data warehouse can be a long, complicated, and expensive process. One of the technologies that have been created to help with the process is Apache Hadoop. Hadoop allows for the processing of massive amounts of data on commodity hardware with open source technologies. However, instead of simplification, the ETL and data warehousing landscape has only become more complex and cumbersome and the proliferation of tools combined with maturity and adoption issues have only increased the cost. Further, according to Gartner analyst Nick Heudecker, 85% of big data projects fail. Mostly due to the complexity of the process itself.

With the transition to the cloud many organizations are undertaking, ETL becomes even more complicated from a meaningful and timely data analytics standpoint. Moving data from one source to another takes time. Now there is hidden data transfer and compute costs and latencies to navigate. While some meaningful analytics can be performed on stale data, most modern analytics need to be as close to real-time as possible.

Issues With ETL

A few of the problems that we are faced with when setting up ETL processes are:

  1. Latency & Downtime - There is an inherent cost of moving data from point A to point B. Forty years ago, when ETL started, we were working with megabytes of data and not needing “instant” access. Today we’re dealing with terra or petabytes of data and needing real-time insight from that data.

    Moving data across the network isn’t free. On a 100 BaseT network, transferring one gigabyte of data takes 100 seconds. A terabyte takes 10,000 seconds or over two and a half hours. All assuming that it’s on a dedicated network that isn’t used by other applications. At ETL demands grow, data could easily be stale by many hours.

    We used to be able to schedule these transfers during “downtime” at midnight. However, in today’s global world, users are always online somewhere demanding instant access and insight. Downtime is simply no longer acceptable and latency has become the new downtime. Should suppliers on one side of the world suffer from poor performance just so executives on the other side of the world have up to date dashboards in the morning?

  2. Storage is cheap, labor is expensive - Data warehouses started at a point in time in which storage was expensive. In 1981, one gigabyte of data storage cost about $290,000. Today that cost is under $0.10. It was, therefore, important to transform and compress as much data as possible when storing to save costs.

    As storage costs have decreased, labor costs have gone the opposite direction. Having a good database administrator to design, manage, and maintain your data warehouse and ETL path is expensive. Storing raw data is frequently seen as a more economically viable choice.

  3. ETL is hard - ETL takes planning. Lots of it. And not just for your current load of data, but for what might happen to the load down the road. Additionally, ETL scripts can get long and complex.

    Bringing in data from a variety of sources, looping over them, adding logging, error handling, configurations are just the start. Determining how the data needs to be transformed can be complex, and fragile. What happens if data stored today as a string gets changed down the road? The process breaks and adjustments need to be made.

    Do you ever wonder why the first answer out of a DBA’s mouth is an emphatic “No!” when asked if something can be changed? One “simple” change can mean changing dozens or hundreds of lines of code. For these reasons and more, ETL requires planning for current and future data needs, loads, and shape.

  4. Are developers the right people to build the ETL pipeline? - Developers are great at many things, however, knowing about data storage and ETL pipelines aren’t often one of them. ETL design and implementation are typically best done by data engineers. While a developer may be able to get data through an ETL pipeline and into a data warehouse, generally speaking, it often isn’t done in the most efficient manner. Specialized data engineers should be responsible for these tasks. If you don’t have them on your team, this is another cost of ETL.
  5. Maintenance headaches - As the size and complexity of data, applications, and analytics requirements grow, so does ETL maintenance. Maintaining changes in data velocity, formats, connections, and features takes time. Many of these challenges may not be thought of at the start of a project, but lead to long-term maintenance needs.

Use MongoDB Charts to Avoid the Headache of ETL

Companies today still have data in a variety of systems. In certain instances, ETL is the only option to be able to perform visualization and analysis of your data. Or, perhaps, you’ve explored ETL but haven’t taken the steps needed to get your data ready for analysis because it’s overwhelming.

If you’ve leveraged MongoDB as your database, the need for ETL procedures has been dramatically reduced with the introduction of MongoDB Charts, now in beta. MongoDB Charts natively understands the MongoDB Document Model allowing for the rapid creation of data visualizations over your data.

With MongoDB Charts you can connect to your MongoDB server, assign user authorization policies to your reports, and easily generate visualization dashboards. With over a dozen different chart variations to choose from, stunning visualizations are just a few clicks away.

MongoDB Charts allows for data to be visualized without performing ETL operations, saving valuable time and resources. You don’t need to write any code or rely on third-party tools. Further, you still get to leverage the richness of the Document Model.

Conclusion

For those situations that you want to quickly access your MongoDB Data, MongoDB Charts is a terrific option. If you’re in a situation that requires multiple data sources to be analyzed, we offer the MongoDB Connector for Business Intelligence. If you are doing advanced analytics with Apache Spark, we have an option for that as well with the MongoDB Connector for Apache Spark.

For many roles in an organization, MongoDB Charts is a great tool for analyzing your data. There’s no need to go through the pain of the ETL process. It is the fastest way to build visualizations over your MongoDB Data, wherever it’s stored. On-premise or in the cloud hosted by MongoDB Atlas. Give it a try today!

Download MongoDB Charts and start visualizing your MongoDB data today.

Identity management application using Blockchain, MongoDB Stitch & MongoDB Atlas - Part 2

Pavel Duchovny

In part 1, we got started by introducing an application where digital identity is stored in a blockchain, focusing on the use case and the proposed system architecture. In this part, we will cover the implementation details and key takeaways.

Implementation

Modern and distributed applications require a modern and distributed data platform for streamlined development and rapid scale as the network grows. Storing the data locally is not reliable, performant or scalable enough, and here is where the advantages of the MongoDB Atlas cloud database and the MongoDB Stitch serverless platform really shine, providing the best foundation to build apps with a global reach.

MongoDB Stitch

MongoDB Stitch, MongoDB’s serverless platform, allowed us to utilize several major capabilities to improve our development speed, security, and scale. Here’s how:

  • Authentication models: Various authentication and user management features allow us to easily authenticate decentralized nodes in the blockchain network (using anonymous authentication). On the other hand, consumers of the network would authenticate with advanced security mechanisms to secure their data access. For simplicity our example uses email/password authentication:
            
    async function handleLogin() {
      const email = loginEmailEl.value;
      const password = loginPasswordEl.value;
      const credential = new UserPasswordCredential(email, password);
      
      try {
      
        await stitchClient.auth.loginWithCredential(credential);
        const user = stitchClient.auth.user;
        showLoggedInState();
        displaySuccess(`Logged in as: ${user.profile.data.email}`)
    
      } catch(e) {
        handleError(e)
      }
    }
            
        
  • Stitch Rules: Flexible and easy to define authorization rules that can be applied on collection, field, and document levels give us the ability to manage data access in very sophisticated and controlled ways. One of the most fascinating abilities is to automatically project fields such as “credit_score” and “assets_range” only if a user has permissions to view them. Stitch roles allow you to filter data based on a user’s preferences:
            
    "assets_range": {
                        "read": {
                            "%or": [
                                {
                                    "%%root.owner_id": "%%user.id"
                                },
                                {                  "%%root.identity_verification_log.transaction_details.assets_shared": "assets_range"
                                }
                            ]
                        }
                    }
            
        
  • Stitch Functions: Implement hosted, server-side logic that can govern process accuracy and validate data used within the data access rules. For example, approvedByMajority verification will return true when the block actually has a majority of votes from participating blockchain nodes:
            
    function(block){
      // Get collections 
      var nodes = context.services.get("mongodb-atlas").db("assets").collection("nodes");
      var pending_blocks = context.services.get("mongodb-atlas").db("transactions").collection("pending_blocks");
      
      // Retrieve the current block
      var doc = pending_blocks.findOne({index : block.index});
      if (doc)
      {
        // Retrieve approvals vs active nodes
        currentApprovals = doc.approvals;
        
        var numberOfActiveNodes = nodes.count( {"active" : true});
        var approvedNodes = nodes.count({"owner_id" : {"$in" : currentApprovals}});
        
        // See if majority has approved
        if (approvedNodes > (numberOfActiveNodes / 2))
        {
          return true;
        }
        else
        {
          return false;
        }
        
      }
      return false;
    };
            
        
  • Stitch Triggers & third-party services: Once a user’s identity has been validated, Stitch Triggers we notify parties about offers and promotions based on the data in the blockchain.

    With third-party service integrations, we can integrate information and security gathering services easily into the process.

Trigger pushOffer.

MongoDB Atlas

Note: The Atlas cluster must be of version 3.6 and above.

MongoDB Stitch is backed with an Atlas cluster which provides us with 4 key abilities:

  • Atlas provides data access and management with scalability, resilience, and worldwide distribution to conform with privacy regulations such as GDPR and HIPAA.
  • Rich query and analytical language with a built-in Stitch hybrid connection string. In particular, we have utilized MongoDB views built on top of the $graphLookup aggregation stage, which is a key capability to traverse and validate data structures such as Blockchains.
            
    db.createView("blockchain","pending_blocks",
                [    {
                        "$match" : {
                            "index" : 0
                        }
                    },
                    {
                        "$graphLookup" : {
                            "from" : "pending_blocks",
                            "startWith" : "$previousHash",
                            "connectFromField" : "hash",
                            "connectToField" : "previousHash",
                            "as" : "chain"
                        }
                    },
                    {
                        "$unwind" : "$chain"
                    },
                    {
                        "$sort" : {
                            "chain.index" : 1
                        }
                    },
                    {
                        "$group" : {
                            "_id" : "$_id",
                            "chain" : {
                                "$push" : "$chain"
                            }
                        }
                    },
                    {
                        "$project" : {
                            "chain" : 1,
                            "_id" : 0
                        }
                    }
                ]);
    
    db.pending_blocks.createIndex({index : 1},{unique : true});
    
            
        
  • Change Streams features are a tremendous game changer for event-driven applications. Any data changing events can be filtered within Atlas when only relevant notifications to the application watcher. One main use case for this is when nodes receive a notification about a new block generated by one of the network workers:
            
    // Configure change pipeline
    const pipeline = [
                   { $match: {'fullDocument.owner_id'  : {  "$ne" : client.authedId() },
                   "operationType" : "insert" } }
                 ];
            // Get collection and set a change stream
             var db = atlasConn.db('transactions');
             const changeStream = db.collection('pending_blocks').watch(pipeline);
            
        
    MongoDB Stitch Triggers make events even simpler to process.
  • Built-in TLS and enterprise security features allow us to enforce additional levels of access control, auditing, and encryption, layering upon the governance features of MongoDB Stitch and the Blockchain itself.

Conclusions & Key Takeaways

The world of blockchain in digital systems has a tremendous potential and I believe we are going to see some extremely innovative ideas that will thrive outside of the cryptocurrency space. MongoDB enables rapid innovation, developer productivity, and scale-out for applications like this one and many others – which all benefit from decentralized data control, trust and immutability.

Identity management application using Blockchain, MongoDB Stitch & MongoDB Atlas - Part 1

Pavel Duchovny

Our physical and digital identities intersect in a million ways in today's world. Sharing and gathering information with partners and third-party vendors to streamline business processes while keeping the information secure and genuine is challenging.

Last year was big for blockchain technology. Among its many use cases, blockchain unlocks opportunities to leverage its trust, distribution, and immutability to publish identity information among various entities while maintaining a clear cryptographic ledger. Sensitive information goes through a strict approval process and is encrypted to be accessed only by specific parties. Blockchain core concepts are out of scope for this blog post. If you wish to learn more about blockchains, please refer to the resources section at the end of this blog.

Use Case

As a proof of concept, we built a blockchain identity management application. We used the blockchain structure and concepts to save and publish digital identities for a fictional bank network. Using NodeJS we have built a network of nodes which are run by the various partners in the network, pushing and governing the production of blocks within the chain.

Figure 1 : Bank-side application which shows a login verification through the blockchain ( transparent to the user)
Figure 2. Identity information is published to the blockchain.
Figure 3. Nodes on the blockchain network approve the information/encryption signatures. Other approved parties can consume the information once it is approved.

The main idea is that data can be pushed to the blockchain network by Bank A (Figure 2), consumed and approved by the majority of the network, and appear as trusted information for Bank B or C (Figure 3). Whether the information is encrypted or in the clear depends on Entity A. Using this information, Banks B and C can push offers and promotions for those customers based on the blockchain network information without the need for a centralized registration authority.

Next Steps

That’s a wrap for Part 1 of this blog series. In Part 2 we’ll cover the implementation details and key takeaways.

Learn how we built a distributed identity management application on blockchain using the MongoDB Stitch serverless platform and MongoDB Atlas cloud database service for a financial services use case.

Visualizing Your Data With MongoDB Charts

Having data stored in a database is practically a given for today’s businesses. Customer information, order history, product pricing, IoT sensor data, and much more is being recorded for future use. However, just having the data stored isn’t enough to form a competitive market advantage. We must be able to analyze the data as well. There are many options to do so and in a variety of ways. If you have data that needs to be visually analyzed in MongoDB, MongoDB Charts is a terrific option.

Prior to MongoDB Charts, there were really three ways to visualize your MongoDB Data.

  1. Leverage the MongoDB Business Intelligence (BI) Connector in conjunction with third-party BI tools,
  2. Perform Extract-Transform-Load (ETL) operations and leverage third-party tools, or
  3. Write custom code and use charting libraries such as D3.js or Bokeh.

MongoDB Charts, currently in Beta, provides an easy way to visualize your data living in MongoDB. You don’t need to move your data to a different repository, write your own code, or purchase third-party tools. MongoDB Charts knows and understands the richness of the Document Data Model and allows for easy data visualization.

Further, MongoDB Charts allows for a secure way to create and share visualization dashboards with everyone, or just targeted team members. Similarly, the data source being used behind the scenes can be shared securely as well. For example, data for the Sales Department doesn’t have to be made available to Marketing unless needed. Very powerful and follows MongoDB’s design of security being a top priority.

After downloading the MongoDB Charts Docker image and following the installation instructions, we’re able to connect to a data source stored in MongoDB Atlas and start making visualization dashboards. Once connected to the MongoDB Charts server, there are three steps we need to take:

  1. Add a data source
  2. Create a dashboard
  3. Create our charts

Analyzing Airbnb Data with MongoDB Charts

I have set up a database with some Airbnb data from various cities. We’ll be exploring the dataset from Seattle, WA here, but feel free to explore others on your own. We need to get the connection string from the Atlas Cluster that has our data and connect to it in Charts.

Get URI from MongoDB Atlas

Add a Data Source

With our MongoDB Charts server running on localhost:80, we can log in and head to the Data Sources tab. We use the URI from Atlas (mongodb+srv://airbnbdemo:airbnb@airbnb-rgl39.mongodb.net/test?retryWrites=true) and select Connect. We’re next asked which data source we want to use from that cluster, I’ll select the seattleListingAndReviews from the airbnb database for this example. For permissions, I just want to keep everything private so I’ll accept the defaults and select Publish Data Source. Once published I can add an alias to the data source. I’ll call it Airbnb Seattle.

Note: The URI above contains a sample URI. You should connect to your own Atlas Cluster and use an authorized username and password.

Create a Dashboard

Next up is to create an actual dashboard to house our visualizations. In the Dashboards section choose New Dashboard and give it a name and description, like Ken’s Airbnb Dashboard. This will take me to where I can add charts to my dashboard.

Create a Chart

After clicking on the Add Chart button we can start building our visualization. We’ll want to choose the Airbnb Seattle data source from the drop-down. MongoDB Charts automatically determines which fields are available for exploration. For this exercise, I’d like to see which neighborhoods in Seattle have the most Airbnb properties and split them by property type. We’ll use the Stacked Bar chart for the type.

  1. For the X-Axis then, we’ll want the id field, aggregated by count.
  2. Assign X-Axis value to a MongoDB Chart
  3. Along the Y-Axis we’ll look at the address and the suburb. Notice that address is a subdocument here and that MongoDB Charts natively knows how to handle this type of data. I’d like to sort the suburb by aggregated value, in descending order, and limit our results to the top 20 suburbs.
  4. Assign Y-Axis value to a Stacked Bar chart
  5. Let’s add the property_type field as our series
  6. Assign a Series value to a Stacked Bar chart

Now we can name our chart, Properties by Location and save it. We’re then taken back to our dashboard where we can add other visualizations for further exploration.

Have a look at this short video to see some other visualizations being created from this same data source.

Conclusion

MongoDB Charts is an excellent new tool to visually explore your data. It has some great features for specific use cases, such as:

  • Ad hoc analysis of your data
  • Natively understands the benefits of the Document Data Model
  • Collaboration on projects is easy with user-based sharing and permissions
  • It’s intuitive enough for non-developers to use allowing for self-service data analysis

MongoDB Charts is the fastest way to build visualizations over your MongoDB data. I’d encourage you to download it and try it out today. Let me know what visualizations you come up with from the Airbnb dataset. I always enjoy seeing how people explore their data.

New to MongoDB Atlas — Get Started with Free Fully Automated Databases on Microsoft Azure

Leo Zheng

Release Notes, Cloud

We’re excited to announce that teams can now use MongoDB Atlas — the global cloud database for MongoDB — for free on Microsoft Azure. The newly available free tier on Azure Cloud, known as the M0, grants users 512 MB of storage and is ideal for learning MongoDB, prototyping, and early development.

The Atlas free tier will run MongoDB 4.0 and grant users access to some of the latest database features, including multi-document transactions, which make it even easier to address a complete range of use cases with MongoDB; type conversions, which allow teams to perform sophisticated transformations natively in the database without costly and fragile ETL; and updated security defaults (SHA-256 and TLS 1.1+).

Like larger MongoDB Atlas cluster types, M0 clusters grant users optimal security with end to end encryption, high availability, and fully managed upgrades. M0 clusters also enable faster development by allowing teams to perform CRUD operations against their data right from their browsers via the built-in Data Explorer.

Finally, free tier clusters on Azure can be paired with MongoDB Stitch — a powerful suite of serverless platform services for apps using MongoDB — to simplify the handling of backend logic, database triggers, and integrations with the wider Azure ecosystem.

At launch, the MongoDB Atlas free tier will be available in 3 Azure regions:

  • East US (Virginia)
  • East Asia (Hong Kong)
  • West Europe (Netherlands)

Creating a free tier is easy. When building a new Atlas cluster, select Azure as your cloud of choice and one of the regions above.

Next, select M0 in the “Cluster Tier” dropdown.

Then, give the cluster a name and hit the “Create Cluster” button. Your free MongoDB Atlas cluster will be deployed in minutes.

New to MongoDB Atlas? Deploy a free cluster in minutes.