Babu Srinivasan

4 results

Driving Innovation through Data Refractoring: Empowering Applications with MongoDB Atlas and AWS Modernization Strategies

In today's fast-paced digital landscape, businesses are constantly seeking innovative ways to stay competitive and deliver exceptional user experiences. However, many organizations find themselves hindered by the burden of outdated legacy applications, technical debt, and the subsequent impact on their customers. The need of the hour is to pivot towards a solution that can revolutionize the way businesses operate: application modernization. By addressing the challenges of legacy applications, tackling technical debt, and leveraging modernization strategies, organizations can enhance customer impact, streamline processes, and confidently embrace the future. In the journey toward application modernization, selecting the right database and database design plays a crucial role. Clients have the challenges of the correct method of refractoring, fear of touching the aged legacy code base, lengthy traditional modernization process, high uncertainty of business-as-usual (BAU) support, and difficulty to modernize the data layer. To overcome these challenges, organizations are turning to modern database solutions like MongoDB Atlas, a popular NoSQL database. MongoDB offers flexibility, scalability, and agility, making it an ideal choice for modernizing applications. Its schemaless design allows for dynamic changes without disrupting functionality, and it offers horizontal scalability to handle large amounts of data and growing user demands. Cloud-native databases, like MongoDB Atlas, further enhance modernization efforts. They provide fault tolerance, high availability, and elastic scaling, enabling organizations to design resilient and scalable systems. By leveraging MongoDB Atlas, businesses can optimize performance, adapt to changing needs, and ensure data availability. In the pursuit of application modernization, carefully evaluating existing database structures, and embracing MongoDB Atlas, cloud-native databases can unlock the full potential of applications. This enables improved performance, flexibility, and scalability while minimizing the limitations posed by structured databases. MongoDB Atlas MongoDB is an Amazon Web Services (AWS) Partner with multiple AWS Competencies including Data & Analytics, Financial Services, and Government. MongoDB Atlas on AWS is the leading unified developer data platform that accelerates and simplifies how you build with data. MongoDB Atlas gives you the versatility you need to build sophisticated applications that can adapt to evolving customer and market demands. An intuitive document data model and unified query API provide a first-class developer experience, delivered in a cloud-native developer data platform built for resilience, scale, and the highest levels of data privacy and security. Power transactions, search, and application-driven analytics seamlessly through an elegant and integrated data architecture. With MongoDB Atlas, developers can focus on building their modern applications without worrying about the complexities of infrastructure provisioning, database setup, and ongoing maintenance. It provides features such as automated backups, monitoring, and built-in security measures to ensure data integrity and reliability. MongoDB Atlas empowers businesses to leverage the full potential of MongoDB's flexible document-based data model while benefiting from the scalability, availability, and ease of use offered by a cloud-based database service. MongoDB on AWS MongoDB Atlas on AWS combines best-in-class operational automation, scalability, and resilience of cloud-native services. It is available across 27+ AWS regions and has deep integrations with multiple AWS services. It enables organizations to leverage the cloud infrastructure provided by AWS to deploy, manage, and scale their MongoDB databases seamlessly. With features like automated backups, monitoring, and integration with AWS services like Amazon Elastic Compute Cloud and Amazon Simple Storage Service (Amazon S3), MongoDB on AWS empowers businesses to optimize performance, ensure data availability, and easily adapt to changing demands. Accenture Data Factory Accenture Data Factory solution is a tri-party solution from MongoDB, Accenture, and AWS leveraging MongoDB Atlas and AWS services. It’s an industry-leading solution, with highly experienced delivery teams working on the nuances of the application and database migration. It accelerates the migration process and provides 24X7 support across 27+ AWS regions. The process includes the study of the current application architecture, decomposition of the legacy applications, and migration using the standard Re-imagine, Re-factor, and Modernization of applications. The Data Factory refocuses the customer’s perspective from cost to value throughout the cloud journey. The solution offering includes the Lift & Shift of self-managed databases, Legacy Modernization (Brownfield), and Modern Application Development (Greenfield) capabilities. Lift & shift This process of Lift & Shift is generally adopted in the customer-managed database. The Lift & Shift approach leaves the application architecture largely untouched, with the exception of migrating from self-managed on-premise infrastructure components into a modern, cloud-native / cloud-managed capability. Lift & Shift provides comparatively quick benefits, limited to infrastructure and support cost reductions, but since the process undertakes no functional improvement, the same technical debt is carried to the cloud environment. Legacy modernization The Legacy Modernization process is a brown-field method of re-imagining legacy monolithic applications to harness the true potential of the cloud. It modernizes the existing application by decomposing the existing application and transforming them to the latest microservices and event-driven architecture, leveraging some of the existing legacy code bases, wherever it’s possible. Accenture’s Refractory with Data offering operates inside this Legacy Modernization capability and helps unlock significant value trapped in legacy applications and codebases. See the detailed discussion below. Modern application development Modern Application Development provides a green-field approach where new applications are developed using modernized architectures like Microservices, Event-Based, Serverless, and others. For modern applications, NoSQL platforms, especially MongoDB Atlas on AWS provide the ideal solution for the data layer. A lengthy discussion of these advantages can be found in other whitepapers but can be summarized as follows: NoSQL solutions scale linearly and nearly indefinitely for minimal risk at a very low cost compared with typical RDBMS alternatives NoSQL solutions now offer the same ACID transaction guarantees and capabilities provided by RDBMS platforms – but also the modern architecture using a service layer for all database interactions reduces the critical complexity of ACID transactions at the data layer NoSQL solutions provide seamless support and integration with DevOps and CI/CD driven deployment pipelines. NoSQL platforms like MongoDB Atlas allow updates to schema and core architectures without the pain of a database outage to make schema updates, data table reloads, and planning for rollbacks. The Refractory with data appraoch to application modernization Refractory provides a systematic approach to define a clear path from the current state of legacy applications to the desirable state of modern cloud-native microservices with modern data architectures Refractory With Data follows a 4-step process: Decompose and decouple the legacy application with functional wrappers Improve and re-imagine code within each functional unit Arrange functions into containers with clear domain boundaries Re-architect legacy data structures (RDBMS) into domain-specific data components, typically in a NoSQL architecture ideal for Microservices Decompose and decouple the legacy application Decoupling the application's components promotes loose coupling, allowing for independent development and improved maintainability and Decomposing a legacy application involves breaking it down into smaller, more manageable components, enabling gradual modernization and enhanced scalability. There are several methods commonly used for decoupling and decomposing legacy applications. Some of the methods are described below. Functional decoupling Functional decoupling of a legacy application is a process that involves isolating and separating its individual functionalities into modular and independent components. By decoupling the functions, organizations can achieve improved flexibility, maintainability, and scalability. This approach allows teams to work on specific functionalities without impacting the entire application, enabling parallel development and faster iterations. Functional decoupling reduces the dependencies and interdependencies within the legacy application, facilitating easier updates, bug fixes, and enhancements. It also paves the way for adopting modern architectural patterns, such as microservices or event-driven architecture, ultimately leading to a more agile and future-proof system. Domain driven design (DDD) DDD emphasizes understanding and modeling the core domains within a legacy application. By identifying bounded contexts and defining domain-driven architectures, businesses can decompose the application into smaller, cohesive units focused on specific domains. This method enhances modularity, and maintainability, and enables teams to work autonomously on different domains. Improve and reimagine code The data factory solution will look into each of the functional units and adopt the modernization process either through the Microservice Architecture or Event-Based architecture. Microservices architecture Decomposing the legacy application into a microservices architecture involves breaking it down into smaller, independent services that communicate with each other via APIs. Each microservice focuses on a specific business capability, making it easier to develop, test, and deploy. This approach enhances scalability, and flexibility, and enables teams to work on different services simultaneously. It’s also following multiple microservice architecture patterns that are suitable for various business demands. Some of the patterns are described below. API Gateway pattern: The API Gateway pattern acts as a single entry point for client applications to interact with multiple microservices. It consolidates requests from clients, performs authentication and authorization, and routes requests to the appropriate microservices. This pattern helps to centralize cross-cutting concerns like security, rate limiting, and request/response transformations, simplifying the client's interaction with the microservices. Service registry pattern: The Service Registry pattern is used to dynamically discover and locate microservices within a system. It involves a central registry that keeps track of registered services and their network locations. Microservices can register themselves with the registry and retrieve information about other services at runtime. This pattern promotes loose coupling between services, as they can be added or removed without requiring explicit configuration between them. Circuit breaker pattern: The Circuit Breaker pattern helps to handle faults and failures in microservice interactions. It adds a layer of protection to prevent cascading failures in a distributed system. The circuit breaker monitors requests to external services and, in case of failures or unresponsiveness, temporarily halts subsequent requests to allow the failing service to recover. This pattern enhances fault tolerance, and resilience, and prevents the system from becoming overwhelmed by failing services. Event sourcing pattern: The Event Sourcing pattern involves capturing and storing all changes to an application's state as a sequence of events. Instead of persisting in the current state, the system maintains an append-only log of events. The events can be replayed to reconstruct the system's state at any given point in time. This pattern enables auditing, and scalability, and provides a historical record of how the application's state evolved over time. Saga pattern: The Saga pattern helps maintain data consistency and coordination across multiple microservices during complex, long-running transactions. It decomposes a single transaction into a series of smaller, loosely coupled steps or compensating actions. If any step fails, compensating actions are executed to undo the changes made by previous steps. This pattern allows for eventual consistency and prevents partial updates or inconsistencies in distributed transactions. Each of these microservice patterns addresses specific challenges and provides guidelines for designing resilient, scalable, and loosely coupled microservice architectures. Implementing these patterns appropriately can help organizations build robust and manageable microservice systems. Event-driven architecture (EDA) It is an approach where components of the legacy application communicate and react to events that occur within the system. Events can trigger actions and updates across various services, enabling loose coupling and scalability. EDA promotes responsiveness, extensibility, and the ability to handle complex, real-time interactions. Each of these decomposition methods offers a unique approach to modernizing legacy applications, and organizations may choose to adopt one or a combination of them based on their specific needs and constraints. The ultimate goal is to break down the monolithic nature of the legacy application, enabling agility, scalability, and improved software development practices. Arrange functions into containers Containerization and microservice architectures go hand in hand to enable scalable and efficient application development and deployment. Containerization involves encapsulating an application and its dependencies into lightweight, isolated units known as containers. These containers provide consistent environments across different platforms, ensuring that the application runs reliably, regardless of the underlying infrastructure. Microservice architectures, on the other hand, break down an application into smaller, independent services that can be developed, deployed, and scaled independently. By combining containerization with microservices, organizations can achieve greater agility, scalability, and ease of deployment. Containers provide a portable and consistent execution environment for microservices, allowing for rapid development, deployment, and scaling of individual services. This approach promotes modularity, and fault isolation, and enhances the ability to leverage cloud-native technologies, enabling organizations to efficiently build, deploy, and manage complex applications in distributed environments. Re-architect legacy data structures The refractory framework leverages MongoDB database schema design patterns to re-architect the legacy data structures. Each of these schema patterns satisfies a set of unique business requirements. Some of the patterns are described below: Embedded data model: This pattern involves embedding related data within a single document. It is suitable for one-to-one or one-to-many relationships where the embedded data is accessed and modified together. It improves read performance as the data is retrieved in a single document access, but updates can be more complex if the embedded data needs frequent modifications. Normalized data model: In this pattern, related data is stored in separate collections and linked using references or foreign keys. It is ideal for many-to-many relationships or scenarios where data updates are frequent. While it ensures data consistency and simplifies updates, it may require additional queries to fetch related data, potentially impacting read performance. Tree structure model: This pattern is suitable for hierarchical or tree-like data structures, such as categories, organizational charts, or comment threads. It uses a parent-child relationship to represent the hierarchy, allowing easy navigation and retrieval of hierarchical data. However, maintaining integrity and performing updates across the tree can be more complex. Polymorphic data model: This pattern handles scenarios where a field can accept different data types. It allows for storing different types of data within the same field, making the schema more flexible. This pattern can simplify the schema but may require additional logic to handle different data types correctly. Bucket pattern: The bucket pattern is used to optimize queries on large collections by partitioning the data into smaller "buckets" based on specific criteria, such as time ranges or ranges of values. It helps improve query performance by reducing the amount of data scanned in a query, but it requires careful planning and consideration of the query patterns. It's important to note that the choice of schema design pattern depends on the specific requirements of the application, such as data relationships, read and write patterns, performance considerations, and scalability needs. Understanding these patterns and selecting the most appropriate one can help optimize the performance and efficiency of MongoDB database operations. Customer references We adopted the refractory model with one of the leading North America Insurance firms for modernizing their application landscape. The customer needed to modernize a huge monolithic application delivering to the needs of the customer. But with the monolith architecture the insurance firm was challenged with the agile business changes and innovation of their product. Release management was limited to only two releases per year due to the complex regression testing of the massive application spanning the mainframe and multiple technologies. The solution was originally hosted on the Mainframe supported by an RDBMS database. Following the Refractory approach the application was modernized into Microservices on MongoDB Atlas on AWS. In the process, we also created a consolidated operational data layer to optimize the business layer of their application stack The developer productivity increased by more than 70% by taking advantage of MongoDB’s integration with agile-friendly deployment processes and the number of product releases also increased considerably to 5 major and 350+ minor releases in a calendar year. Recap With the Data Factory solution, you can deploy, manage, and grow your database on AWS for customer needs without any major disruption, enabling organizations to leverage the power of data. For Tri-party solutions contact: Minal Ahuja (AWS) For MongoDB private offers contact: Olivier Zieleniecki (MongoDB) For Accenture contact: Steve Meyer (Accenture)

June 27, 2023

Unmasking Deception: Harnessing the Power of MongoDB Atlas and Amazon SageMaker Canvas for Fraud Detection

Financial services organizations face growing risks from cybercriminals. High-profile hacks and fraudulent transactions undermine faith in the industry. As technology evolves, so do the techniques employed by these perpetrators, making the battle against fraud a perpetual challenge. Existing fraud detection systems often grapple with a critical limitation: relying on stale data. In a fast-paced and ever-evolving landscape, relying solely on historical information is akin to driving by looking into the rearview mirror. Cybercriminals continuously adapt their tactics, forcing financial institutions to stay one step ahead. The newest tactics often can be seen in the data. That's where the power of operational data comes into play. By harnessing real-time data, fraud detection models can be trained on the most accurate and relevant clues available. MongoDB Atlas, a highly scalable and flexible developer data platform, coupled with Amazon SageMaker Canvas, an advanced machine learning tool, presents a groundbreaking opportunity to revolutionize fraud detection. By leveraging operational data, this synergy holds the key to proactively identifying and combating fraudulent activities, enabling financial institutions to safeguard their systems and protect their customers in an increasingly treacherous digital landscape. MongoDB Atlas MongoDB Atlas , the developer data platform is an integrated suite of data services centered around a cloud database designed to accelerate and simplify how developers build with data. MongoDB Atlas's document-oriented architecture is a game-changer for financial services organizations. Its ability to handle massive amounts of data in a flexible schema empowers financial institutions to effortlessly capture, store, and process high-volume transactional data in real-time. This means that every transaction, every interaction, and every piece of operational data can be seamlessly integrated into the fraud detection pipeline, ensuring that the models are continuously trained on the most current and relevant information available. With MongoDB Atlas, financial institutions gain an unrivaled advantage in their fight against fraud, unleashing the full potential of operational data to create a robust and proactive defense system. Amazon SageMaker Canvas Amazon SageMaker Canvas revolutionizes the way business analysts leverage AI/ML solutions by offering a powerful no-code platform. Traditionally, implementing AI/ML models required specialized technical expertise, making it inaccessible for many business analysts. However, SageMaker Canvas eliminates this barrier by providing a visual point-and-click interface to generate accurate ML predictions for classification, regression, forecasting, natural language processing (NLP), and computer vision (CV). SageMaker Canvas empowers business analysts to unlock valuable insights, make data-driven decisions, and harness the power of AI without being hindered by technical complexities. It boosts collaboration between business analysts and data scientists by sharing, reviewing, and updating ML models across tools. It brings the realm of AI/ML within reach, allowing analysts to explore new frontiers and drive innovation within their organizations. Reference Architecture The above reference architecture includes an end-to-end solution for detecting different types of fraud in the banking sector, including card fraud detection, identity theft detection, account takeover detection, money laundering detection, consumer fraud detection, insider fraud detection and mobile banking fraud detection to name a few. The architecture diagram shown here illustrates model training and near real-time inference. The operational data stored in MongoDB Atlas is written to the Amazon S3 bucket using the Triggers feature in Atlas Application Services. Thus stored, data is used to create and train the model in Amazon SageMaker Canvas. The SageMaker Canvas stores the metadata for the model in the S3 bucket and exposes the model endpoint for inference. For step-by-step instructions on how to build the fraud detections solution mentioned above with MongoDB Atlas and Amazon SageMaker Canvas, read our tutorial .

June 21, 2023

How to Build Advanced GraphQL-based APIs With MongoDB Atlas and AWS AppSync Merged APIs

When businesses develop their own IT systems, sooner or later, the complexity of managing APIs becomes a challenge. Breaking down monolithic architectures into multiple microservices often results in a proliferation of APIs associated with each microservice. Each API, in turn, has versioning, leading to further fragmentation of the APIs and driving up maintenance costs. If your microservices diagram looks like a hairball you know you are living in API Hell. Conway’s Law , which states that systems mirror the communication structure of the organization, also applies to API development. Different teams build separate and, sometimes, overlapping APIs, further contributing to the fragmentation. While this is especially true for REST-based APIs , it's also a challenge for GraphQL -based APIs. GraphQL has emerged as a powerful tool for building flexible and efficient APIs that empower developers and elevate user experiences. AWS AppSync is the go-to service for customers looking to accelerate application development with serverless GraphQL and Pub/Sub APIs. AWS AppSync offers a managed GraphQL service with additional features and capabilities. It simplifies the development of scalable, real-time applications by seamlessly integrating with various data sources, providing offline support, enabling fine-grained authorization and security, and automating infrastructure management. By embracing AppSync, you can harness the full potential of GraphQL while leveraging the benefits of a comprehensive portfolio of services and products provided by AWS. MongoDB Atlas on Amazon Web Services (AWS) and AWS AppSync combined help developers build scalable, secure, and serverless applications. By seamlessly integrating MongoDB as a data source within AppSync, you're able to leverage MongoDB's flexible document model and AppSync's GraphQL-based querying to efficiently retrieve and manipulate data. And you can leverage AppSync's automatic scaling, ensuring optimal performance. This combined solution enables you to build high-performing serverless applications while simplifying application development. AWS AppSync recently added a feature called Merged APIs that allows you to compose multiple GraphQL source APIs into a single GraphQL API. Merged APIs give developers the ability to compose distinct APIs developed by different teams into a single, combined GraphQL schema. The merged API resolver function contains the logic to consolidate the source details. The resulting single GraphQL API can be cached for better performance. You can then present the unified API to the clients as a single API endpoint. AppSync Merged APIs combine MongoDB Atlas-backed APIs with other APIs, allowing you to enrich operational data residing in MongoDB Atlas with data coming from additional sources. You can serve data with a unified GraphQL schema across multiple data sources, including MongoDB. If you're interested in learning more about this powerful integration, check out our new tutorial that demonstrates two ways to combine MongoDB Atlas with AWS AppSync : leveraging the Drivers and the Atlas Data API. Both approaches work with the AWS AppSync Merged API as well. Checkout our tutorial on GitHub . Try out MongoDB Atlas on AWS ( Atlas GraphQL ) and AWS AppSync today. Sign up for MongoDB Atlas on AWS Marketplace Today

June 20, 2023

Modernize your GraphQL APIs with MongoDB Atlas and AWS AppSync

Modern applications typically need data from a variety of data sources, which are frequently backed by different databases and fronted by a multitude of REST APIs. Consolidating the data into a single coherent API presents a significant challenge for application developers. GraphQL emerged as a leading data query and manipulation language to simplify consolidating various APIs. GraphQL provides a complete and understandable description of the data in your API, giving clients the power to ask for exactly what they need — while making it easier to evolve APIs over time. It complements popular development stacks like MEAN and MERN , aggregating data from multiple origins into a single source that applications can then easily interact with. MongoDB Atlas: A modern developer data platform MongoDB Atlas is a modern developer data platform with a fully managed cloud database at its core. It provides rich features like native time series collections, geospatial data, multi-level indexing, search, isolated workloads, and many more — all built on top of the flexible MongoDB document data model. MongoDB Atlas App Services help developers build apps, integrate services, and connect to their data by reducing operational overhead through features such as hosted Data API and GraphQL API. The Atlas Data API allows developers to easily integrate Atlas data into their cloud apps and services over HTTPS with a flexible, REST-like API layer. The Atlas GraphQL API lets developers access Atlas data from any standard GraphQL client with an API that generates based on your data’s schema. AWS AppSync: Serverless GraphQL and pub/sub APIs AWS AppSync is an AWS managed service that allows developers to build GraphQL and Pub/Sub APIs. With AWS AppSync, developers can create APIs that access data from one or many sources and enable real-time interactions in their applications. The resulting APIs are serverless, automatically scale to meet the throughput and latency requirements of the most demanding applications, and charge only for requests to the API and by real-time messages delivered. Exposing your MongoDB Data over a scalable GraphQL API with AWS AppSync Together, AWS AppSync and MongoDB Atlas help developers create GraphQL APIs by integrating multiple REST APIs and data sources on AWS. This gives frontend developers a single GraphQL API data source to drive their applications. Compared to REST APIs, developers get flexibility in defining the structure of the data while reducing the payload size by bringing only the attributes that are required. Additionally, developers are able to take advantage of other AWS services such as Amazon Cognito, AWS Amplify, Amazon API Gateway, and AWS Lambda when building modern applications. This allows for a severless end-to-end architecture, which is backed by MongoDB Atlas serverless instances and available in pay-as-you-go mode from the AWS Marketplace . Paths to integration AWS AppSync uses data sources and resolvers to translate GraphQL requests and to retrieve data; for example, users can fetch MongoDB Atlas data using AppSync Direct Lambda Resolvers. Below, we explore two approaches to implementing Lambda Resolvers: using the Atlas Data API or connecting directly via MongoDB drivers . Using the Atlas Data API in a Direct Lambda Resolver With this approach, developers leverage the pre-created Atlas Data API when building a Direct Lambda Resolver. This ready-made API acts as a data source in the resolver, and supports popular authentication mechanisms based on API Keys, JWT, or email-password. This enables seamless integration with Amazon Cognito to manage customer identity and access. The Atlas Data API lets you read and write data in Atlas using standard HTTPS requests and comes with managed networking and connections, replacing your typical app server. Any runtime capable of making HTTPS calls is compatible with the API. Figure 1:   Architecture details of Direct Lambda Resolver with Data API Figure 1 shows how AWS AppSync leverages the AWS Lambda Direct Resolver to connect to the MongoDB Atlas Data API. The Atlas Data API then interacts with your Atlas Cluster to retrieve and store the data. MongoDB driver-based Direct Lambda Resolver With this option, the Lambda Resolver connects to MongoDB Atlas directly via drivers , which are available in multiple programming languages and provide idiomatic access to MongoDB. MongoDB drivers support a rich set of functionality and options , including the MongoDB Query Language, write and read concerns, and more. Figure 2:   Details the architecture of Direct Lambda Resolvers through native MongoDB drivers Figure 2 shows how the AWS AppSync endpoint leverages Lambda Resolvers to connect to MongoDB Atlas. The Lambda function uses a MongoDB driver to make a direct connection to the Atlas cluster, and to retrieve and store data. The table below summarizes the different resolver implementation approaches. Table 1:   Feature comparison of resolver implementations Setup Atlas Cluster Set up a free cluster in MongoDB Atlas. Configure the database for network security and access. Set up the Data API. Secret Manager Create the AWS Secret Manager to securely store database credentials. Lambda Function Create Lambda functions with the MongoDB Data APIs or MongoDB drivers as shown in this Github tutorial . AWS AppSync setup Set up AWS Appsync to configure the data source and query. Test API Test the AWS AppSync APIs using the AWS Console or Postman . Figure 3:   Test results for the AWS AppSync query Conclusion To learn more, refer to the AppSync Atlas Integration GitHub repository for step-by-step instructions and sample code. This solution can be extended to AWS Amplify for building mobile applications. For further information, please contact partners@mongodb.com .

November 23, 2022