Leverage real-time data insights to optimize prices and gain a competitive edge.
Use cases: Personalization, Gen AI
Industries: Retail
Products: MongoDB Atlas, MongoDB Atlas Collections, MongoDB Atlas Clusters
Partners: Google Cloud, Vertex AI, Cloud Functions
Solution Overview
Dynamic pricing, the art of adjusting prices in real time based on market conditions, is a crucial strategy for businesses aiming to maximize revenue and gain a competitive edge. To effectively implement dynamic pricing, a robust technological infrastructure is essential. This solution integrates MongoDB Atlas and Google Cloud Vertex AI to create a real-time dynamic pricing microservice. Vertex AI Notebooks and TensorFlow models utilize Google Cloud Pub/Sub for rapid data ingestion in order to analyze customer behavior and optimize pricing strategies. MongoDB Atlas serves as a flexible feature store for intricate pricing data, while Google Cloud's robust computational resources power complex calculations and hosting.
The outcome is a scalable and adaptable pricing system that delivers instant price adjustments based on the latest market intelligence. This integration enhances operational efficiency by effectively handling large datasets and complex pricing scenarios.
Reference Architectures
The architecture for a dynamic pricing microservice integrates MongoDB and Google Cloud Vertex AI to facilitate real-time data processing and responsive pricing adjustments. Google Cloud Pub/Sub handles the ingestion and distribution of customer behavior data, allowing for scalable and efficient message processing. This data is then cleaned and processed in Vertex AI Notebooks, where TensorFlow develops machine learning models to predict optimal prices based on patterns identified in historical data.
MongoDB Atlas serves as the central data repository and feature store, storing complex pricing data and supporting the machine learning models. The document model of MongoDB provides the flexibility needed to manage and update pricing data dynamically. Google Cloud Functions orchestrate the entire workflow, processing customer events, converting them into tensors, and ensuring that the model predictions are updated in real time in the MongoDB Atlas product catalog. In the architecture diagram, the blue data flow illustrates how customer event data is ingested into a Pub/Sub topic, leading to a push subscription that triggers the Cloud Function. This function transforms raw events into tensors and updates the predicted prices in the MongoDB product catalog.
This architectural approach allows for the isolation of raw event threads, enabling the development of various services that can react in real time for dynamic pricing or operate asynchronously for model training. By maintaining loose coupling between components, the system is resilient and avoids complete failures if one part experiences issues. Publishers and subscribers can independently continue processing their logic, ensuring a robust and flexible system that supports continuous operation and seamless updates.
Figure 1. Dynamic pricing architecture integrating different Google Cloud components and MongoDB Atlas as a feature store
Advantage of MongoDB for Modeling Events
MongoDB allows you to store tensors and related features together in a single collection to support event-driven architectures. This approach simplifies data retrieval and processing, but requires careful management because large documents can impact performance. This flexibility also facilitates integrated data analytics and real-time decision-making.
Figure 2. Data structure within the events collection representing a simulated user view of MongoDB white socks, including price information
For more-detailed instructions on how to build your own dynamic microservice, please refer to our blog post, Building a Dynamic Pricing Microservice with Vertex AI and MongoDB Atlas.
Build the Solution
The GitHub repository contains detailed instructions on how to build this solution.
Set up MongoDB Atlas
Sign in to MongoDB Atlas and create a new cluster.
Choose a region closest to your user base for optimal performance.
Configure security as follows:
Create database users with specific roles.
Enable IP access to secure the database connection.
Connect to your cluster using the connection string provided by Atlas for application integration.
Set up Google Cloud
Log into Google Cloud console and create a new project for your microservice.
Ensure that the necessary APIs are enabled for your project.
Install and initialize the Google Cloud CLI.
Authenticate with your Google Cloud account.
Set your project as the default.
Develop the microservice and model
Clone the repository and navigate to the dynamicPricing microservice directory.
Install required Python packages using pip.
Set up environment variables for MongoDB Atlas, Google Cloud credentials, and Pub/Sub.
Configure a Pub/Sub topic in Google Cloud.
Develop the pricing logic for the dynamicPricing service.
Use Vertex AI to create a new notebook for connecting to MongoDB and training the model.
Train a TensorFlow model by cleaning data, building the model, and training it.
Save and upload the trained model to Google Cloud Storage.
Register the model in Vertex AI for management and deployment.
Deploy a model to an endpoint
Deploy the Model.
Access the Vertex AI section in Google Cloud console.
Locate and select the model you want to deploy.
Click on "Deploy to endpoint" under the "Deploy & Test" tab.
Complete the deployment setup by configuring model settings and monitoring.
Retrieve Endpoint ID.
Go to "Endpoints" in Vertex AI and note the Endpoint ID for use in Cloud Function configuration.
Configure Cloud Function.
Create a new Cloud Function in Google Cloud.
Set the function's trigger to the Pub/Sub topic you created.
Implement code in the function to preprocess event data, invoke the Vertex AI model for predictions, and update MongoDB with the results.
Prepare Dependencies.
Ensure the requirements.txt file includes all necessary libraries for the Cloud Function.
Simulate Customer Events.
Use a provided script to generate synthetic customer behavior data.
Push this data to MongoDB and Pub/Sub to test the model and microservice integration.
Run the Simulation.
Execute the script to validate the complete end-to-end system.
Key Learnings
Centralized feature store: MongoDB serves as a feature store, acting as a centralized repository for storing, managing, and handling features for machine learning (ML) models. Its polymorphic capabilities use a single interface to represent various types of data, which allows you to incorporate new features and adapt pricing models. In the context of dynamic pricing, this capability facilitates the incorporation of new pricing factors or variables without causing disruptions to operations.
Scalability and efficiency: Google Cloud Pub/Sub can handle massive volumes of customer data efficiently, ensuring scalability for real-world applications.
Real-time price updates: Cloud functions trigger TensorFlow models to generate dynamic prices based on customer behavior. These generated prices are then inserted or updated (upserted) back into the product catalog collection in MongoDB, enabling real-time adjustments in the e-commerce application.
Authors
Francesco Baldissera, MongoDB
Sebastian Rojas Arbulu, MongoDB