Data in 2021: Four Predictions For an Uncertain Future
What a year it’s been. A global pandemic, a recession, and a U.S. presidential election unlike any in living memory made 2020 a tragic and tumultuous 12 months many want to forget, but can’t. Despite the uncertainty, looking back we can be sure of at least one thing: we’ve seen several years of digital disruption in a matter of months. The race to digitize as fast as possible, our “next normal”, has cut across all industries, accelerating several ancillary trends like cloud adoption, AI, and IoT. Ironically, one of the lasting effects of 2020’s profound unpredictability is just how certain we now are of the growing centrality of digitization, and therefore data, as the primary driver of business success, consumer demand, and even societal change in 2021 and beyond. As such, we asked several of MongoDB’s brightest minds to look ahead to the coming year and share their insights into how these trends in data management may play out. Petabyte-Scale Goes Mainstream The idea of “big data” isn’t new, and many firms have been working with petabyte, and even exabyte, sized data sets for some time. 2021, however, may just be the year that data finally goes “big” for everyone else. For many organizations, particularly those mid-sized and smaller, data management has until now been confined to the realm of terabytes. However, trends like the explosion of connected devices, the roll out of 5G, and the continuation of 2020’s headlong rush to digitize every aspect of business mean petabyte-scale data management is likely to become a reality for many more. And to paraphrase a famous saying: “Mo data, mo problems.” Keeping petabytes of data accessible and safe, while at the same time using it to meaningfully enrich a business, is an order of magnitude more difficult and complex than what many mid-sized enterprises are used to. Petabyte-scale data management demands stricter tolerances for uptime, scalability, and performance. In addition, the data is likely to be more distributed — on prem, in the cloud, and even across different clouds. Real-time analytics becomes a business necessity, as does taking advantage of features like automated tiering. The security and data privacy implications of holding that much data, and making it accessible to more people and connected “things,” mean petabyte-scale data management is also a business opportunity tinged with considerable financial and reputational risk. Data Privacy Continues to Be a Hot Button The coming year will further define the relationship between consumers and their data. In November, California voters approved the California Privacy Rights Act (CPRA). Along with enhancements to the already enacted CCPA (the California Consumer Privacy Act), the CPRA establishes an independent watchdog, the California Privacy Protection Agency, to enforce the CCPA now, and the CPRA when it comes into effect on January 1, 2023. There’s growing expectation that 2021 will also be the year the U.S. Federal government begins drafting a nationwide privacy law. With more states likely to follow California and enact their own CCPA-inspired privacy laws, and a new administration headed to Pennsylvania Avenue on January 20th, a national answer to the patchwork of state-based data privacy laws might finally see the light of day. An online ad for one of Apple's latest releases, a credit card Elsewhere, China and Canada are just two of several major world economies set to introduce new data privacy statutes, or overhaul existing laws over the coming 12 months. For businesses, 2021 is also set to be a landmark year for the emergence of data privacy as a competitive advantage. The latest indicator of this trend came in the closing weeks of 2020. In December, simmering tension between two of the largest and most influential companies on the planet spilled into open conflict when Facebook took out full-page advertisements in the New York Times, Wall Street Journal, and Washington Post declaring, “We’re Standing Up To Apple For Small Businesses Everywhere.” A full page ad Facebook took out in several national publications The ads were a response to changes in Apple’s iOS 14, which will prompt users to grant apps permission to gather data and track them as they move across other apps on their iPhone or iPad. That move will “break” parts of Facebook’s ad targeting system, among other things. Apple CEO Tim Cook has staked the company’s brand on becoming known as the big tech company that respects user privacy, in direct contrast to Facebook and other companies that rely heavily on customer data for their advertising-based business models. “You are not our product,” Cook said in an interview with ABC’s Diane Sawyer in 2019 . “Our products are iPhones and iPads. We treasure your data. We want to help you keep it private and keep it safe.” Make no mistake, Facebook vs. Apple is just one battle in a much larger conflict over data privacy and brand equity. No longer just a compliance challenge, the sanctity of customer data is now a business and brand burnishing advantage too. Real-time Analytics Becomes a Differentiator It’s one thing to ingest a lot of data, and quite another to put that data to use. As 2020’s digitization stampede continues, the next frontier for enterprises is to mine the information they collect for insights that drive personalized customer experiences—at scale and in real time. And to achieve this level of near-instantaneous insight and response, 2021 will be the year businesses focus their attention on moving to converged data platforms. Unlike the siloed databases of yesteryear, converged data platforms (otherwise known as translytical data platforms, like MongoDB !), combine transactional (System of Record), operational (System of Engagement), and analytical (System of Insight) workloads onto a single, unified data platform. A converged data platform allows businesses to exploit their mountains of data at the speed and efficiency consumers now demand, and all with lower complexity and risk. As business leaders seek an edge over their competition, those that prioritize real-time analytics, and move to a converged data platform, will pull further away from their peers. Not Every Cloud Has a Silver Lining From retail to recreation, hospitality to healthcare, moving data and operations to the cloud was already a right of passage on the way to digital transformation. The COVID-19 pandemic simply accelerated this move. But with speed, comes even greater risk , and embracing the cloud on an accelerated timeline is fraught with danger. Do it without proper planning—as in a simple “lift and shift” of your existing setup—and you may find the on-premise issues that currently hamper developer velocity and business agility simply follow you to the cloud. The COVID-19 pandemic has heightened the need for companies to adopt digital business models—and only cloud platforms can provide the agility, scalability, and innovation required for this transition. McKinsey, The Next-Normal Recovery Will be Digital Additionally, all the advantages the cloud affords, such as the ease of scaling your infrastructure, can quickly lead to more architectural silos and technical complexity if handled incorrectly. Our warning is that, with so many companies rushing their move to the cloud in 2021, many will fail to seize on its transformational benefits, and spend 2022 (and beyond) undoing bad architectural decisions.
Using MongoDB for Data Locality at Groupon
Groupon is a business built on place data. “The more data we have the better we can serve these businesses,” said Peter Bakkum in his talk on Using MongoDB for Groupon’s Place Data” at MongoDB San Francisco. “We know their data and we know their price point” The Merchant Data team at Groupon uses MongoDB heavily in its mission to create the most comprehensive database of places and merchants in the world. This team is a key part of Groupon's platform: the data is used for Salesforce CRM and public merchant pages, among other products, and has a direct impact on the business. This data makes it more efficient for salespeople at Groupon to prioritize their leads, helping them in negotiations, pick price points and create deal-structures. “Doing this at such a large scale, has, literally, a revenue impact of millions of dollars” says Bakkum. The Merchant Data team uses MongoDB throughout their Data system architecture: at the Input, processing and serving level. Watch Peter Baakum’s video from MongoDB San Francisco for more details on the Merchant Data team’s architecture.
What Are You Doing With Your Data?
Data presents new opportunities and challenges, and developers are at the forefront of these changes. At 10gen, we interact with users building new applications to support the growing need to collect, mobilize and analyze data for business goals. Gaming companies analyze user data to react to movements in real-time, keeping the users happy and winning, and increasing retention and payout. Manufacturing companies ingest data from sensors and location tracking machines to gather machine data, analyzed in real-time and batch contexts. Retail companies leverage structured and competitive data to optimize prices in real-time and to serve personalized product recommendations. Each of these represents novel ways to build a data-driven business. We love these stories and want to hear more from the developers who build these applications and are the driving force behind innovation. Take the MongoDB community survey and tell us how you’re using data in your applications to meet new data challenges. Complete the survey and enter for a chance to win an iPad Mini. This survey is platform agnostic, and is designed to get an understanding of the developer ecosystem as a whole. In the spirit of open source, we plan to share the results of the survey with the community in a report and on the 10gen blog. Stay tuned for the results.