Zomato Manages High-Volume Data and Delivers High-Speed Success

MongoDB
December 6, 2023 | Updated: December 8, 2023

Zomato is one of India’s largest consumer technology companies. The restaurant aggregator and food delivery operator provides restaurant information, menus and user reviews, and food delivery options from partner restaurants in over 1,000 Indian cities and towns.

With over 17.5 million customers, 220,000 restaurant partners and 350,000 delivery partners, Zomato deals with data in huge quantities. However, in 2017 Zomato became increasingly aware of the pain points of database management.

“Zomato was expanding at a remarkable pace,” says Abhishek Jain, senior software engineer, Zomato at MongoDB.local New Delhi earlier this year. “We realized that it would be prudent to migrate all our self-hosted clusters to a managed platform, which is when we integrated with MongoDB Atlas. As Zomato and its portfolios kept on expanding, so did our use cases for MongoDB.”

Using features such as index suggestions, geospatial queries and analytics nodes, MongoDB is now the driving force behind many of Zomato’s key operational systems:

  • The Order Tracking system covers live order statuses as well as live locations of delivery partners.

  • The Order Assignment system is one of the most critical components, and one of the most complex.

  • The Order Details system displays items and quantities for each order to ensure smooth fulfillment.

  • The Delivery Partner Expected Earnings feature combines key information such as a rider’s location, their distance from the partner restaurant and customer, and the type of transport used.

  • The Featured Restaurants system is what most people see when using Zomato app. A key marketing tool, it creates bespoke campaigns and offers, and suggests local restaurants to users.

  • The Delivery Partner Onboarding system simplifies the onboarding process by collecting and storing drivers’ names, addresses, driving licenses, vehicle type and more.

Another critical tool, the Delivery Partner Location system, is central to Zomato. With thousands of riders on the streets of India continuously publishing their locations at very high rates, flexible and responsive persistent storage is vital.

“We get hundreds of thousands of requests per minute,” he explains. “We need a database capable of handling this load without any issues or latency, even during extremely high-volume periods such as New Year’s Eve. Again, MongoDB was the answer.”

A final core element is Chat SDK, a central messaging hub that enables customers and delivery partners to stay in contact via email or text message and can include images and video.

“All this data is non-relational, so we need a NoSQL data store that provides strong consistency under heavy read and write workloads,” says Soumil Kanwal, Software Engineer at Zomato. “MongoDB's document-oriented database fits our requirements perfectly.”

He concludes: “We’re in the process of migrating features from other databases to MongoDB and building new features on top of those. MongoDB manages them all very comfortably.”

Check out the full presentation now

← Previous

阿里云数据库MongoDB版助力掌阅平滑上云，撬动数据红利

掌阅科技：深耕优质内容 优化数字阅读体验 掌阅科技股份有限公司成立于2008年9月，专注于数字阅读，是全球领先的数字阅读平台之一。 掌阅主营业务为互联网数字阅读服务及增值服务，同时从事网络原创文学版权运营，以及基于自有互联网平台的流量增值服务，服务覆盖自研产品、阅读服务、原创矩阵、网文出海、终端产品等五大方面。 掌阅的自研产品包括掌阅App、掌阅精选App、掌阅课外书App、得间小说App、阅爱聊等。其中，掌阅App是掌阅科技自研的数字阅读平台，拥有国学经典、严肃出版、原创文学、有声读物、漫画杂志等海量阅读内容，在业内率先实现了3D仿真翻页、护眼模式等技术的创新与引用，并在文档识别、转化、续读技术以及精装排版等方面形成了核心技术优势，处于行业领先水平。 业务挑战 数字阅读渐成主流 数据拓展需要数据管理同步升级 潜心数字阅读领域十余年，掌阅积累了海量阅读数据，如何做到这些数据的平滑升级和扩展、如何将这些数据充分利用起来并指导企业工作？是掌阅最大的痛点，也是掌阅多年来一直在研究和探索的问题。 依据阅读场景下的数据特点，掌阅在数据方面主要面对以下挑战： 双向数据 – 人和书这个关系相对复杂。如一个人喜欢哪些书、同时这些书又被哪些人同时喜欢，这在关键性数据库里需要建两张表，人一张表，书一张表。更为重要的是，要在两张表的数据关系中找到潜在读者、客户。这就需要一个能够有效管理非结构化数据的复合型索引。 字段的不确定性 – 在具体阅读的过程中，掌阅旗下不同APP要适应多重场景和运营活动，随着定位和画像的不同，数据标签随时更新、变化。传统的关系型数据库不再适合厘清这些不确定字段的数据信息。 数据升级 – 围绕掌阅的几大主营业务，涉及各个领域的APP矩阵不断扩大，数据量级已达到TB级，这种高并发海量数据需要数据管理的同步升级。 解决方案 阿里云数据库MongoDB版落地掌阅多元业务场景 大幅提升服务性能 2022年4月，掌阅科技在年报内显示将其核心服务进行云原生的改造，在大幅提升研发效率的同时，保证良好的服务可用性，持续提升服务性能30%以上。对客户端进行大幅包体瘦身和稳定性加固，崩溃率降低50%以上，冷启动打开速度提升30%以上，包体积降低30%以上。 这与掌阅选择阿里云数据库MongoDB版不无关系。掌阅平台研发部负责人张博表示，这主要是考虑到MongoDB在非结构化数据管理方面的显著优势，一是文档结构，字段可灵活扩展，可实现双向数据查询；二是支持Sharding、多节点容灾，实现海量数据的存储和可用性；三是事务支持，可实现多文档事务；四是阿里云的支持，为未来混合云架构打好基础。 张博还对阿里云数据库MongoDB版在云书架、书评、流水帐单、类积分管理四个业务场景中的落地实践进行了介绍。 云书架 – 每一位掌阅用户都会拥有一个个人书架，可以自助添加、删除书目。书、人之间会产生错综复杂的对应关系，而且这种对应数据呈持续变化，MongoDB介入其中进行动态管理。 书评 – 书评是阅读的一项“附加产品”。做书评是用户在掌阅APP上进行的重要交互行为，对每一本书、每一章节、每一句话，都可以上传个人感悟、评价。在平台后端形成了多事务多文档，这也是MongoDB善于处理的数据文档形态。 流水帐单 – 掌阅的流水数据很长一段时间内存在互联网数据中心 (IDC) 内的 HBase 系统 中，由于时间和历史原因，Hbase可维护性、可扩展性已到瓶颈阶段。作为掌阅与阿里云合作的第一个项目，掌阅将所有流水帐单从HBase迁移到MongoDB系统中，完成财务共享中心建设，突破瓶颈，轻松应对流水数据的不断增加。 类积分管理 – 积分机制是掌阅的一种重要“用户管理”方式，但多种积分活动相互交叉，积分规则各不相同，积分数值变化相对琐碎。大量积分数据在平台后端形成单事务多文档，MongoDB能够帮助实现积分管理清晰化、标准化。 客户价值 以数据为驱动 掌阅科技不断提升精细化能力 年报显示，掌阅科技2021年营收为20.7亿元，较上年同期的20.6亿元增长0.49%。其中，数字阅读平台收入占营业收入的比例分别为75.13%。可以说，以数据为驱动，掌阅不断提升精细化能力，运营效率进一步提高。聚焦在阿里云数据库MongoDB版为掌阅带来的重要价值，张博更喜欢用这样的三个词来概括：效率、成本、稳定性。 提高效率 – 在与阿里云和 MongoDB合作过程中，MongoDB技术团队会直接进入项目，并对重点问题全程追踪，通力解决问题，双方达成良性合作，提高工作效率。 加强稳定性 – 数据弹性大是云原生的特色能力。阿里云数据库MongoDB版为掌阅数据带来了稳定的数据弹性支持，尤其针对掌阅不同的运营活动、不同的爆款产品、不同的热点书目，随着数据量波动，MongoDB都能很好地帮助掌阅应对数据库扩容和缩容。 降低成本 – 效率的提高、稳定性的提高都可以直接带来成本降低。据掌阅统计，引入MongoDB系统后，运营成本节省10%以上。 客户证言 掌阅平台研发部负责人 张博 “阿里云数据库MongoDB版为我们提供了一系列操作工具，并作为我们研发平台上的重要组件，帮助我们在统一的、开放的研发环境下，不断提升研发成效。随着互联网行业格局的逐步成型以及数字阅读行业的市场竞争日益激烈，掌阅将与阿里云和MongoDB继续深度合作，变现技术红利，继续深化存量用户精细化运营和加大增量用户贡献。”

December 5, 2023
Next →

Building AI With MongoDB: Integrating Vector Search And Cohere to Build Frontier Enterprise Apps

Cohere is the leading enterprise AI platform, building large language models (LLMs) which help businesses unlock the potential of their data. Operating at the frontier of AI, Cohere’s models provide a more intuitive way for users to retrieve, summarize, and generate complex information. Cohere offers both text generation and embedding models to its customers. Enterprises running mission-critical AI workloads select Cohere because its models offer the best performance-cost tradeoff and can be deployed in production at scale. Cohere’s platform is cloud-agnostic. Their models are accessible through their own API as well as popular cloud managed services, and can be deployed on a virtual private cloud (VPC) or even on-prem to meet companies where their data is, offering the highest levels of flexibility and control. Cohere’s leading Embed 3 and Rerank 3 models can be used with MongoDB Atlas Vector Search to convert MongoDB data to vectors and build a state-of-the-art semantic search system. Search results also can be passed to Cohere’s Command R family of models for retrieval augmented generation (RAG) with citations. Check out our AI resource page to learn more about building AI-powered apps with MongoDB. A new approach to vector embeddings It is in the realm of embedding where Cohere has made a host of recent advances. Described as “AI for language understanding,” Embed is Cohere’s leading text representation language model. Cohere offers both English and multilingual embedding models, and gives users the ability to specify the type of data they are computing an embedding for (e.g., search document, search query). The result is embeddings that improve the accuracy of search results for traditional enterprise search or retrieval-augmented generation. One challenge developers faced using Embed was that documents had to be passed one by one to the model endpoint, limiting throughput when dealing with larger data sets. To address that challenge and improve developer experience, Cohere has recently announced its new Embed Jobs endpoint . Now entire data sets can be passed in one operation to the model, and embedded outputs can be more easily ingested back into your storage systems. Additionally, with only a few lines of code, Rerank 3 can be added at the final stage of search systems to improve accuracy. It also works across 100+ languages and offers uniquely high accuracy on complex data such as JSON, code, and tabular structure. This is particularly useful for developers who rely on legacy dense retrieval systems. Demonstrating how developers can exploit this new endpoint, we have published the How to use Cohere embeddings and rerank modules with MongoDB Atlas tutorial . Readers will learn how to store, index, and search the embeddings from Cohere. They will also learn how to use the Cohere Rerank model to provide a powerful semantic boost to the quality of keyword and vector search results. Figure 1: Illustrating the embedding generation and search workflow shown in the tutorial Why MongoDB Atlas and Cohere? MongoDB Atlas provides a proven OLTP database handling high read and write throughput backed by transactional guarantees. Pairing these capabilities with Cohere’s batch embeddings is massively valuable to developers building sophisticated gen AI apps. Developers can be confident that Atlas Vector Search will handle high scale vector ingestion, making embeddings immediately available for accurate and reliable semantic search and RAG. Increasing the speed of experimentation, developers and data scientists can configure separate vector search indexes side by side to compare the performance of different parameters used in the creation of vector embeddings. In addition to batch embeddings, Atlas Triggers can also be used to embed new or updated source content in real time, as illustrated in the Cohere workflow shown in Figure 2. Figure 2: MongoDB Atlas Vector Search supports Cohere’s batch and real time workflows. (Image courtesy of Cohere) Supporting both batch and real-time embeddings from Cohere makes MongoDB Atlas well suited to highly dynamic gen AI-powered apps that need to be grounded in live, operational data. Developers can use MongoDB’s expressive query API to pre-filter query predicates against metadata, making it much faster to access and retrieve the more relevant vector embeddings. The unification and synchronization of source application data, metadata, and vector embeddings in a single platform, accessed by a single API, makes building gen AI apps faster, with lower cost and complexity. Those apps can be layered on top of the secure, resilient, and mature MongoDB Atlas developer data platform that is used today by over 45,000 customers spanning startups to enterprises and governments handling mission-critical workloads. What's next? To start your journey into gen AI and Atlas Vector Search, review our 10-minute Learning Byte . In the video, you’ll learn about use cases, benefits, and how to get started using Atlas Vector Search.

April 25, 2024