Predictions for AWS re:Invent 2017 (tl;dr: AI & IoT)

Peter Zawistowicz
November 7, 2017
This post is the second installment of our Road to AWS re:Invent 2017 blog series. In the weeks leading up to AWS re:Invent in Las Vegas this November, we'll be posting about a number of topics related to running MongoDB in the public cloud. See all posts here.

In just under two months, more than 46,000 technologists will descend on Las Vegas for this year’s AWS re:Invent. Ranging from seasoned members of the AWS community to the cloud-curious, re:Invent attendees should expect the conference’s sixth iteration to deliver the same parade of ecosystem partners, an extensive agenda focused on moving to (and being successful in) AWS cloud, and the inevitable announcement of a fresh batch of new AWS services.

In attempting to predict what this year’s re:Invent keynote will unveil, we’ll look at how the industry has changed since last November, as well as Amazon’s track record for debuting new products at past re:Invents.

Since last year’s conference, the two most significant shifts in the space are underpinned by the two largest trends of the moment: AI and IoT.

It is safe to assume that we will see an augmentation of AWS’s artificial intelligence and machine learning offerings next month. Last year’s conference brought us Lex, Polly, and Rekognition as Amazon made its entrée into advanced text, voice, and image processing. Widespread adoption of this flavor of artificial intelligence is still modest, so these releases may have been overshadowed by seemingly more relevant tools like Athena, which allows users to run SQL-based queries on data stored in S3. Nonetheless, the development of its AI portfolio is of strategic importance for AWS. Despite being the most popular public cloud, Amazon has faced increasing pressure from Azure and Google Cloud Platform. The latter has been able to differentiate itself among the early-adopter community primarily for its more mature AI offerings. To remain dominant over Google in the space, Amazon must prove able to keep up with the same pace of innovation in this sector.

The areas that appear most ripe for innovation from AWS this year are in voice, image, and video analysis. Already, we have seen success among e-commerce players when using text and image-based search to shorten their conversion cycles. In fact, Gartner reports that voice-based search is the fastest growing mobile search type. The opportunity to exploit users’ devices for image and voice-based search is evident in Amazon’s offerings (Alexa, Amazon iOS/Android app). Furthermore, the explosion of intelligent chat-based interfaces (Messenger, Drift, etc.) has increased the demand for a broader set of capabilities in natural language processing services like Lex. As a result, we should be prepared to see further enhancements to Lex, Polly, and Rekognition.

Video remains the one area of machine learning-based processing AWS has yet to touch. As their image analysis engines improve, the next logical step would be for the low-latency processing of video inputs. With the untold volume of video content being generated every day by ever-improving cameras, it stands to reason that organizations will want to turn that into insight and profit.

These first two predictions hint at another group of potential releases we could see from AWS next month. The development of extensible models for the analysis of text, voice, image, and video is predicated on the accessibility of high quality, low-cost microphones and cameras. While smartphones have supported these inputs for more than a decade now, the availability of WiFi and reliable cellular networks has increased the speed and frequency by which their outputs can be shared or uploaded for further analysis.

So, that brings us to our next theme: the Internet of Things.

Many analysts and skeptics have suggested IoT adoption is weak and its promises are over-hyped. Their skepticism is primarily centered on two ongoing challenges with IoT: 1) the lack of one or two emergent platforms on which IoT technologies can standardize and 2) the relatively limited ability for data from decentralized sensors to be analyzed at “the edge” rather than in a central cloud.

As with operating systems, media encodings, or network protocols, mass adoption of the technologies they support is typically preceded by one to three main players emerging as the default options. AWS entered the competition to build the winning IoT platform at re:Invent 2015 with its announcement of AWS IoT. All other major technology companies have made similar bids for dominance of this market. In addition, there are hundreds of venture-funded startups aiming to serve as a universal platform untethered from an existing “marketecture.” Nevertheless, the fact remains that no winner in this race has yet been crowned.

This remains a large opportunity and Amazon is well-poised with its existing portfolio of software and ecosystem of networking and hardware partners. AWS appeared to renew its commitment to capturing the IoT market at last year’s re:Invent with the debut of AWS Greengrass and Lambda@Edge. Greengrass allows for the running of Lambda functions on local, offline devices rather than in Amazon’s cloud. Lambda@Edge is one of AWS’s first forays into “edge computing,” allowing users to run low-latency and device-specific Node.js functions in their “edge locations”. Both releases mark a shift from centralized cloud computing to distributed edge computing—perhaps less comfortable for AWS, but necessary for sometimes-offline or time-sensitive IoT projects.

However, Greengrass was just the first step to enabling AWS users to better serve disparate, intermittently-connected devices. Notably, Greengrass still requires ML-powered data processing and analysis to take place in the cloud rather than locally (at the edge). Improvements in hardware technology may also prompt AWS to improve their on-device offerings and make services like S3 and DynamoDB available outside of their infrastructure to better store and process sensor data on the devices themselves. Similarly, we may also see devices become a more significant player in more seasoned services like Kinesis, enabling the local ingestion of data.

No matter what gets announced on the keynote stage this year, you can rest assured it will lead the conversation for the months that follow.

← Previous

Why Testim Left Google Firebase

Over the last three months we have migrated our service database from Firebase to MongoDB. In this post we will explore the reasons why we initially selected Firebase, the challenges we experienced as our needs evolved, and the reasons why we decided to make the transition.

November 6, 2017
Next →

Retrieval Augmented Generation for Claim Processing: Combining MongoDB Atlas Vector Search and Large Language Models

Following up on our previous blog, AI, Vectors, and the Future of Claims Processing: Why Insurance Needs to Understand The Power of Vector Databases , we’ll pick up the conversation right where we left it. We discussed extensively how Atlas Vector Search can benefit the claim process in insurance and briefly covered Retrieval Augmented Generation (RAG) and Large Language Models (LLMs). MongoDB.local NYC Join us in person on May 2, 2024 for our keynote address, announcements, and technical sessions to help you build and deploy mission-critical applications at scale. Use Code Web50 for 50% off your ticket! Learn More One of the biggest challenges for claim adjusters is pulling and aggregating information from disparate systems and diverse data formats. PDFs of policy guidelines might be stored in a content-sharing platform, customer information locked in a legacy CRM, and claim-related pictures and voice reports in yet another tool. All of this data is not just fragmented across siloed sources and hard to find but also in formats that have been historically nearly impossible to index with traditional methods. Over the years, insurance companies have accumulated terabytes of unstructured data in their data stores but have failed to capitalize on the possibility of accessing and leveraging it to uncover business insights, deliver better customer experiences, and streamline operations. Some of our customers even admit they’re not fully aware of all the data in their archives. There’s a tremendous opportunity to leverage this unstructured data to benefit the insurer and its customers. Our image search post covered part of the solution to these challenges, opening the door to working more easily with unstructured data. RAG takes it a step further, integrating Atlas Vector Search and LLMs, thus allowing insurers to go beyond the limitations of baseline foundational models, making them context-aware by feeding them proprietary data. Figure 1 shows how the interaction works in practice: through a chat prompt, we can ask questions to the system, and the LLM returns answers to the user and shows what references it used to retrieve the information contained in the response. Great! We’ve got a nice UI, but how can we build an RAG application? Let’s open the hood and see what’s in it! Figure 1: UI of the claim adjuster RAG-powered chatbot Architecture and flow Before we start building our application, we need to ensure that our data is easily accessible and in one secure place. Operational Data Layers (ODLs) are the recommended pattern for wrangling data to create single views. This post walks the reader through the process of modernizing insurance data models with Relational Migrator, helping insurers migrate off legacy systems to create ODLs. Once the data is organized in our MongoDB collections and ready to be consumed, we can start architecting our solution. Building upon the schema developed in the image search post , we augment our documents by adding a few fields that will allow adjusters to ask more complex questions about the data and solve harder business challenges, such as resolving a claim in a fraction of the time with increased accuracy. Figure 2 shows the resulting document with two highlighted fields, “claimDescription” and its vector representation, “claimDescriptionEmbedding” . We can now create a Vector Search index on this array, a key step to facilitate retrieving the information fed to the LLM. Figure 2: document schema of the claim collection, the highlighted fields are used to retrieve the data that will be passed as context to the LLM Having prepared our data, building the RAG interaction is straightforward; refer to this GitHub repository for the implementation details. Here, we’ll just discuss the high-level architecture and the data flow, as shown in Figure 3 below: The user enters the prompt, a question in natural language. The prompt is vectorized and sent to Atlas Vector Search; similar documents are retrieved. The prompt and the retrieved documents are passed to the LLM as context. The LLM produces an answer to the user (in natural language), considering the context and the prompt. Figure 3: RAG architecture and interaction flow It is important to note how the semantics of the question are preserved throughout the different steps. The reference to “adverse weather” related accidents in the prompt is captured and passed to Atlas Vector Search, which surfaces claim documents whose claim description relates to similar concepts (e.g., rain) without needing to mention them explicitly. Finally, the LLM consumes the relevant documents to produce a context-aware question referencing rain, hail, and fire, as we’d expect based on the user's initial question. So what? To sum it all up, what’s the benefit of combining Atlas Vector Search and LLMs in a Claim Processing RAG application? Speed and accuracy: Having the data centrally organized and ready to be consumed by LLMs, adjusters can find all the necessary information in a fraction of the time. Flexibility: LLMs can answer a wide spectrum of questions, meaning applications require less upfront system design. There is no need to build custom APIs for each piece of information you’re trying to retrieve; just ask the LLM to do it for you. Natural interaction: Applications can be interrogated in plain English without programming skills or system training. Data accessibility: Insurers can finally leverage and explore unstructured data that was previously hard to access. Not just claim processing The same data model and architecture can serve additional personas and use cases within the organization: Customer Service: Operators can quickly pull customer data and answer complex questions without navigating different systems. For example, “Summarize this customer's past interactions,” “What coverages does this customer have?” or “What coverages can I recommend to this customer?” Customer self-service: Simplify your members’ experience by enabling them to ask questions themselves. For example, “My apartment is flooded. Am I covered?” or “How long do windshield repairs take on average?” Underwriting: Underwriters can quickly aggregate and summarize information, providing quotes in a fraction of the time. For example, “Summarize this customer claim history.” “I Am renewing a customer policy. What are the customer's current coverages? Pull everything related to the policy entity/customer. I need to get baseline info. Find relevant underwriting guidelines.” If you would like to discover more about Converged AI and Application Data Stores with MongoDB, take a look at the following resources: RAG for claim processing GitHub repository From Relational Databases to AI: An Insurance Data Modernization Journey Modernize your insurance data models with MongoDB and Relational Migrator

April 18, 2024