3 Key Characteristics of Modernization

Steve Jurczak
November 28, 2022

Analyst and research firm TDWI released its latest report on IT modernization: Maximizing the Business Value of Data: Platforms, Integration, and Management. The report reveals the modernization strategies, objectives, and experiences of more than 300 IT executives, data analysts, data scientists, developers, and enterprise architects. Within the survey itself lies the deeper, fundamental question of what is IT modernization in today's digital economy? It's an important question because it gets at the heart of why organizations want and need to modernize in the first place. Considering the effort, expense, and risks of modernizing, there needs to be a compelling purpose guiding the process in order to keep it on track and ensure its success. By dissecting the TDWI survey questions and responses, we can deduce what the three key characteristics of modernization are.

#1: Modernization capabilities

If we were to examine the elements and components that comprise modernized architecture, we would get a sense of what modernization looks like but not the purpose behind its deployment. So instead, let's start by looking at the capabilities modern architecture enables so we can get a clearer view of its characteristics and why they matter.

Seventy-three percent of survey respondents reported that data democratization and self-service functionality are either extremely or very important. We've heard from numerous organizations that the task of managing data access at companies is slowing down innovation. Ben Herzberg, chief data scientist for data access company, Satori, recently told us, "The majority of organizations are still managing access to data in a manual way. Everyone is feeling the bottleneck. The data analyst who wants to do their job in a meaningful way just wants to understand what data sets they can use and get access to it fast." Getting access to data can be challenging without some sort of self-service data access capability. "Sometimes you have to go through three or four different teams to get access to data," Herzberg says. "It can take a week or two."

The TDWI report also indicated a long-standing trend toward easier, more intuitive experiences extending to data integration, data pipelines, data catalog interaction, and monitoring. Survey respondents' top priorities over the next 12 months support this trend. In addition to migrating and consolidating data in the cloud, they intend to prioritize the following key capabilities:

Enabling better data management for data science, AI, and ML
Supporting development and deployment of data driven applications
Supporting expansion in self service Business intelligence (BI) and analytics users
Unifying management of data across distributed systems

BI and analytics platforms remain one of the fastest growing software markets. The capabilities necessary to power these systems are in high demand: self-service analytics, faster discovery, predictions based on real-time operational data, and integration of rich and streaming data sets.

The survey responses also showed that handling an increase in data volume and the number of concurrent users are modernization priorities. And there's pressure to reduce data latency and increase the frequency of updates. The survey showed that one of the most challenging capabilities organizations are dealing with is enabling low latency querying, search, and analytics. Giving users the right data at the right time to answer business questions, solve problems, and innovate with data is critical today and it depends on these capabilities.

#2: Modernization outcomes

The capabilities organizations seek only serve their modernization goals as far as they enable specific outcomes. And it's outcomes that are ultimately driving modernization initiatives. According to the survey, the number one outcome organizations seek to bring about is gaining fuller value from the data they store and capture. Forty-six percent of respondents cited it as their top challenge.

Automating decision-making is another outcome organizations are seeking. Thirty-two percent of respondents rated automating decisions in operations in processes as very important. But it relies on the timely flow of insights into apps, one of the key capabilities identified earlier. Other key modernization outcomes cited in the survey include:

Increase efficiency and effectiveness
Generate new business strategies and models using analytics
Make faster decisions
Strengthen relationships via data sharing
Improve trust and data quality
Increase reuse and flexibility
Reduce costs
Provide authorized access to live data sets
Consolidate data silos

Developers in the survey said they were seeking to embed richer, personalized application experiences, with 52% saying they wanted seamless access to diverse data sets and sources. But first, they'll have to overcome several challenges that so far have proved difficult to solve. Sixty-eight percent of respondents said they face challenges processing streaming data and change data capture updates, and 64% struggle to integrate streaming with fast, high volume queries, and the same percentage said they struggle with combining historical and real-time analytics.

#3: Modernization platform

Modernized problems require modernized solutions. And the one most most commonly cited by respondents was a data platform, which they believe is the key to maximizing value from data. A data platform solves the issue of consolidating unnecessary data silos and ensuring access to data without the hassle of manual intervention or the risk of unauthorized access. Flexibility in the data platform is critical since data environments will continue to evolve, even after modernization milestones have been met.

A data platform is one of the key elements that comprise modernized architecture. The TDWI survey cited several other advantages of unifying distributed data within a data platform:

Simplifying and accelerating access
Discovering data relationships easier and faster
Creating a logical layer for single point of access
Unifying data governance
Reducing unnecessary data movement

Modernized architecture

Fifty-four percent of respondents said they were in the process of modernizing, and 29% were planning on doing so. The most frequently cited architectural feature by those modernizing or planning to was cloud migration from on-premises systems, with the goal being to change the dimensions of what was possible. But it wasn't just shifting to the cloud that respondents mentioned. The survey also indicated the prevalence of hybrid multi-cloud architectures as well, with data integration and management that span distributed data environments. Distributed architectures can lead to higher performance by putting data closest to where it's being used. It also solves data sovereignty issues by putting data where it's required to be due to regulatory jurisdiction. The report also mentions serverless architecture due to its pay-as-you-go computing model and improved business alignment. With serverless architecture, developers can build applications without thinking about infrastructure or traditional server management.

Read the full TDWI report, Maximizing the Business Value of Data: Platforms, Integration, and Management.

← Previous

10 years of MongoDB customers at AWS re:Invent

MongoDB has attended AWS re:Invent since its inception in 2012. A key reason for this is, of course, to help strengthen our partnership with AWS, which really began in 2015 and significantly expanded in March 2022 with a global, strategic collaboration agreement. But an even more fundamental reason for MongoDB's continued presence at AWS re:Invent over the years is the opportunity to engage with our many joint customers. Several MongoDB customers have been featured in re:Invent keynotes over the years. In fact, looking back at the customers AWS chose to feature in its keynotes, it's hard to find examples that are not MongoDB customers. Earlier this year, AWS celebrated 10 years of re:Invent by showcasing an equal number of "memorable customer moments" from the re:Invent mainstage. It was a great way to reaffirm AWS Leadership Principle #1 (Customer Obsession). It was also a great way to shine a light on the great things MongoDB's customers are doing. Rather than rewind on the many MongoDB customers spotlighted at re:Invent, let's look at those AWS called "most memorable" in its blog. All in on cloud Back in 2015, Capital One CIO Rob Alexander took to the re:Invent stage to discuss Capital One's "all in" approach to cloud. "We’re either using or experimenting with nearly every AWS service," Alexander said. What he didn't say, but which the company has been quite public about over the years, was how Capital One uses MongoDB in tandem with AWS services. A few months after Alexander's re:Invent comments, Capital One's Oron Gill Haus spoke at MongoDB World on Hygieia , the company's open source DevOps dashboard. Hygieia, built on MongoDB, provides the foundation for the company's attempts to reimagine banking. Haus detailed why MongoDB is so critical to Capital One's need to innovate quickly on customers' behalf, stressing how the variety and velocity of data makes MongoDB an ideal solution: We get data in from all different kinds of sources and formats, and we get it at different times. Now, what we have to do is predict the future and how you're planning on using the data. That's where traditional databases fall down. That's where you'll see MongoDB. We want to have the ability to find insights and be able to react quickly to those insights. Years later, Capital One advertises hundreds of jobs for those with MongoDB experience. (Hint: You may need to know how to roll back a MongoDB query for some of those jobs.) Capital One is doing impressive work with MongoDB, but it's not alone in its use of MongoDB for financial services. Goldman Sachs, Citi, Barclays, BBVA, Charles Schwab, FICO, HSBC, and Intuit are just a few MongoDB customers that have spoken publicly of how and why they use MongoDB. And, yes, some of these companies you may remember from the re:Invent main stage over the years. MongoDB to the Moon! Years before NASA Jet Propulsion Laboratory (JPL) took to the re:Invent stage (2016), the U.S. public agency was running MongoDB throughout NASA . By 2018, MongoDB was involved in the hugely interesting NASA Deep Space Network (DSN), a primary resource for communications and navigation for NASA's and partner agencies' interplanetary space missions. NASA had recently upgraded its decades-old infrastructure to base its modern Loading Analysis and Planning Software (LAPS) on Linux and MongoDB. LAPS, as a scientific paper details , "is responsible for long-term planning and forecasting, including studies and analysis of new missions, changed mission requirements, downtime, and new or changed antenna capabilities." Around the same time, and a key part of DSN operations, NASA was also looking for ways to improve the efficiency of operating antennas across the globe. The heart of this initiative was NASA's Link Complexity and Maintenance software (LCM), which stores all pertinent data in MongoDB. Hence, while it might not be accurate to say that MongoDB runs on the Moon, it would be true to say MongoDB helps NASA manage space missions to the Moon—and beyond. Can you hear me now? "[I]t’s just a massive moment for us at Verizon,” declared Hans Vestberg, chairman and CEO of Verizon, at re:Invent in 2019. He was talking about the company's partnership with AWS to deliver 5G network edge computing using AWS Wavelength. What wasn't said in the keynote, but that Robert Belson, Principal Engineer, Corporate Strategy, Verizon, explained , is that the vision was incomplete without MongoDB. “Verizon 5G Edge is a mobile edge computing platform, which embeds popular hyperscaler compute and storage, such as AWS Wavelength, at the edge of our 4G and 5G networks so application builders can extend existing workloads using the same popular services they know and love," he explained. “However, certain services, such as databases, are not natively supported, which is where [MongoDB] Atlas and Realm come into play by creating unprecedented flexibility for the developer and the end customer.” As we've described, Verizon decided that a comprehensive data platform was needed to make its 5G edge computing dream a reality. So Verizon integrated Atlas Functions with the Verizon edge discovery service to help direct 5G mobile clients to the topologically closest database instance across a customer’s edge deployment. In tandem, Verizon has overlaid a data persistence layer using MongoDB Realm, thereby enabling personalized experiences to extend to the network edge. Verizon is also using Atlas Device Sync and Realm to ensure the seamless synchronization of data between devices, the cloud and edge-of-network, online and offline. Customers love MongoDB + AWS Beginning to see a pattern here? While not every customer highlighted by AWS at re:Invent is a MongoDB customer, many are, including the few for which we've been able to provide some detail. Others include Epic Games, which runs its wildly popular game Fortnite on MongoDB ; or Volkswagen, which uses MongoDB throughout its web applications and in its Car Net service ; or Siemens, which runs MongoDB at the heart of its Monet system to provide monitoring, controlling, and remote management of field devices for advanced energy management services. This year while watching the various customers take the stage in re:Invent keynotes, keep in mind that they're also very likely a MongoDB customer, because customers that seek the agility and performance of AWS also tend to like how MongoDB's flexible data model enables them to do much more with their data. Interested in learning more? The best way might be to try fully managed MongoDB Atlas for free. You can get started now .

November 28, 2022

Next →

Retrieval Augmented Generation for Claim Processing: Combining MongoDB Atlas Vector Search and Large Language Models

Following up on our previous blog, AI, Vectors, and the Future of Claims Processing: Why Insurance Needs to Understand The Power of Vector Databases , we’ll pick up the conversation right where we left it. We discussed extensively how Atlas Vector Search can benefit the claim process in insurance and briefly covered Retrieval Augmented Generation (RAG) and Large Language Models (LLMs). MongoDB.local NYC Join us in person on May 2, 2024 for our keynote address, announcements, and technical sessions to help you build and deploy mission-critical applications at scale. Use Code Web50 for 50% off your ticket! Learn More One of the biggest challenges for claim adjusters is pulling and aggregating information from disparate systems and diverse data formats. PDFs of policy guidelines might be stored in a content-sharing platform, customer information locked in a legacy CRM, and claim-related pictures and voice reports in yet another tool. All of this data is not just fragmented across siloed sources and hard to find but also in formats that have been historically nearly impossible to index with traditional methods. Over the years, insurance companies have accumulated terabytes of unstructured data in their data stores but have failed to capitalize on the possibility of accessing and leveraging it to uncover business insights, deliver better customer experiences, and streamline operations. Some of our customers even admit they’re not fully aware of all the data in their archives. There’s a tremendous opportunity to leverage this unstructured data to benefit the insurer and its customers. Our image search post covered part of the solution to these challenges, opening the door to working more easily with unstructured data. RAG takes it a step further, integrating Atlas Vector Search and LLMs, thus allowing insurers to go beyond the limitations of baseline foundational models, making them context-aware by feeding them proprietary data. Figure 1 shows how the interaction works in practice: through a chat prompt, we can ask questions to the system, and the LLM returns answers to the user and shows what references it used to retrieve the information contained in the response. Great! We’ve got a nice UI, but how can we build an RAG application? Let’s open the hood and see what’s in it! Figure 1: UI of the claim adjuster RAG-powered chatbot Architecture and flow Before we start building our application, we need to ensure that our data is easily accessible and in one secure place. Operational Data Layers (ODLs) are the recommended pattern for wrangling data to create single views. This post walks the reader through the process of modernizing insurance data models with Relational Migrator, helping insurers migrate off legacy systems to create ODLs. Once the data is organized in our MongoDB collections and ready to be consumed, we can start architecting our solution. Building upon the schema developed in the image search post , we augment our documents by adding a few fields that will allow adjusters to ask more complex questions about the data and solve harder business challenges, such as resolving a claim in a fraction of the time with increased accuracy. Figure 2 shows the resulting document with two highlighted fields, “claimDescription” and its vector representation, “claimDescriptionEmbedding” . We can now create a Vector Search index on this array, a key step to facilitate retrieving the information fed to the LLM. Figure 2: document schema of the claim collection, the highlighted fields are used to retrieve the data that will be passed as context to the LLM Having prepared our data, building the RAG interaction is straightforward; refer to this GitHub repository for the implementation details. Here, we’ll just discuss the high-level architecture and the data flow, as shown in Figure 3 below: The user enters the prompt, a question in natural language. The prompt is vectorized and sent to Atlas Vector Search; similar documents are retrieved. The prompt and the retrieved documents are passed to the LLM as context. The LLM produces an answer to the user (in natural language), considering the context and the prompt. Figure 3: RAG architecture and interaction flow It is important to note how the semantics of the question are preserved throughout the different steps. The reference to “adverse weather” related accidents in the prompt is captured and passed to Atlas Vector Search, which surfaces claim documents whose claim description relates to similar concepts (e.g., rain) without needing to mention them explicitly. Finally, the LLM consumes the relevant documents to produce a context-aware question referencing rain, hail, and fire, as we’d expect based on the user's initial question. So what? To sum it all up, what’s the benefit of combining Atlas Vector Search and LLMs in a Claim Processing RAG application? Speed and accuracy: Having the data centrally organized and ready to be consumed by LLMs, adjusters can find all the necessary information in a fraction of the time. Flexibility: LLMs can answer a wide spectrum of questions, meaning applications require less upfront system design. There is no need to build custom APIs for each piece of information you’re trying to retrieve; just ask the LLM to do it for you. Natural interaction: Applications can be interrogated in plain English without programming skills or system training. Data accessibility: Insurers can finally leverage and explore unstructured data that was previously hard to access. Not just claim processing The same data model and architecture can serve additional personas and use cases within the organization: Customer Service: Operators can quickly pull customer data and answer complex questions without navigating different systems. For example, “Summarize this customer's past interactions,” “What coverages does this customer have?” or “What coverages can I recommend to this customer?” Customer self-service: Simplify your members’ experience by enabling them to ask questions themselves. For example, “My apartment is flooded. Am I covered?” or “How long do windshield repairs take on average?” Underwriting: Underwriters can quickly aggregate and summarize information, providing quotes in a fraction of the time. For example, “Summarize this customer claim history.” “I Am renewing a customer policy. What are the customer's current coverages? Pull everything related to the policy entity/customer. I need to get baseline info. Find relevant underwriting guidelines.” If you would like to discover more about Converged AI and Application Data Stores with MongoDB, take a look at the following resources: RAG for claim processing GitHub repository From Relational Databases to AI: An Insurance Data Modernization Journey Modernize your insurance data models with MongoDB and Relational Migrator

April 18, 2024