Real-Time ESG Data Management

Wei You Pan
February 14, 2023 | Updated: June 26, 2023

ESG (Environmental, Social, and Governance) data collection and reporting has become a corporate priority, with over 96% of S&P 500 companies publishing sustainability reports in 2021, according to research from the Governance and Accountability Institute.

There are several factors driving the adoption and use of ESG data; ranging from consumer preference for companies with positive ESG information, to employees, who increasingly believe environmental, social, and governance metrics are important indicators when choosing an employer.

Many government bodies and regulators either have, or are considering, mandatory ESG data collection and ESG data reporting requirements for corporations under their jurisdiction. The European Union is taking the lead here, with several key pieces of legislation either already enacted, or coming soon. In the US, the SEC has also announced proposed rule changes for securities reporting, mandating companies make detailed climate-related disclosures in their filings.

In addition to companies that report on their own data, financial firms, including the private equity industry, use ESG data and research to weigh risks and identify opportunities for the companies they invest in.

Faced with growing scrutiny around ESG reporting and scoring, companies are struggling to meet ever more detailed and comprehensive reporting requirements. At the heart of the problem is the sheer volume and variety of data companies are expected to ingest and analyze to produce the scores that investors, consumers, and government entities demand. And with real-time data making its way into reports, ESG data management is becoming even harder.

ESG data collection and analysis

The volume and variety of ESG data makes collection and analysis difficult. The data collection problem can be broken down as follows:

Variety

Unlike financial datasets, which are mostly numerical, ESG metrics can include both structured and unstructured datasets, like an email or a media report. If a company wants to analyze satellite data to derive their own climate dataset, they may even need to analyze images and videos. Given these variables, companies need to employ a data model that can support many different types of data.

Velocity

As companies increasingly integrate real-time data sources into their ESG scoring systems, the velocity of data collected and analyzed increases exponentially. One example is loan due diligence in the financial sector. As customers demand faster loan approval turnaround times, financial institutions that currently rely on quarterly ESG data to make those decisions now need the information in real-time to instantly approve loans in an ESG compliant manner.

Volume

The increased variety of data sources, coupled with the growing velocity of data being collected leads to an increase in the sheer volume of data requiring analysis. Currently, ESG ratings and scores are derived from a blend of human judgment and model driven quantitative rating. But as the volume of data increases, along with the need for instant analysis of that data, real-time analytics and an increased use of AI/ML tools will become an ever greater part of ESG ratings and reporting.

On top of this, there are also no universally applicable ESG standards, leaving companies having to deal with multiple different standards, with different data requirements, depending on which jurisdictions they operate in.

Real-time ESG data analytics

Companies are increasingly incorporating real-time data into their ESG analysis, reporting, and scoring. Harnessing technologies such as cloud computing, AI, and machine learning, those that utilize real-time data can, for instance, instantly parse breaking news stories for ESG-related data on their investments, or incorporate up-to-the-minute satellite data into reports on a firm’s environmental impact.

The financial services industry in particular is taking a lead on integrating real-time ESG data into investment decisions.

Asset and fund managers use real-time data platforms that allow them to calculate up-to-date ESG scores to aid investment decisions and risk calculations. For example, a bank looking to invest in an electric vehicle company would be alerted to a breaking news story about a hazardous accident at the manufacturer’s battery plant, with follow up data from social media or analyst reports quantifying the size of the public reaction and the level of negative market sentiment around the accident.

MongoDB and ESG data management

MongoDB Atlas is an ideal data foundation for ESG platforms. MongoDB Atlas uses the document data model, giving users the ability to ingest data from almost any source, consolidate data from a number of siloed data sets, enable the easy search of that data, and with a few clicks, create customized views of the data without the need for additional ETL operations to other databases or tools.

MongoDB Atlas also future-proofs your ESG data platform with a flexible data schema that can easily adapt to rapidly changing ESG requirements and standards.

See why Hydrus chose MongoDB Atlas as the basis for its ESG reporting platform.

FAQ

ESG data definition

ESG (Environment, Social, and Governance) data comes from a growing list of sources, all of which help “score” a corporation based on how well positioned it is to handle the risks and opportunities presented by the environment, societal stakeholders, and corporate governance.

Environment - What are a company's greenhouse gas emissions? How about its stewardship over natural resources? And how well positioned is it to weather physical climate risks, like global warming, flooding, drought, fire etc.

Social - How does a company measure up against prevailing fair wage and employee engagement metrics? What impact does a company have on the communities where it operates?

Governance - How well is a company managed? How responsive is a company to shareholders? How accountable is leadership? What safeguards are in place to ensure transparency?

The growing interest around ESG data science and data analytics has prompted the rise of a new industry of ESG data companies and ESG data management software vendors.

What are the different ESG data sources?

ESG data may come from two primary sources; 'inside-out' and 'outside-in'.

Inside-out data is supplied by companies, used for analysis, and usually lags 6-12 months due to annual ESG-related disclosures.

Outside-in data is more regularly updated, sometimes even in real time. Most financial institutions, including banks who often have access to a lot of financial and company data from their customers, do not rely solely on their own data. ESG data analysis requires a broad range of inputs and data that the bank does not possess or can obtain even from their customers.

For example, a bank may want to assess the risk of flooding for a chip manufacturing company that has factories in several provinces in China. The bank would need to collect the flood data from the different operating locations in order to score the risk.

As banks don’t typically collect flood data themselves, the bank would purchase data from third-party climate data vendors. At this nascent stage of climate risk assessment within the banking industry, it is likely that the bank would not even attempt to collect the raw climate data and create the risk models to score the risk, relying instead on third-party risk scoring vendors.

The bank would then make use of these scores and combine in models which they have strong competencies eg. credit risk to come up with flood risk-adjusted credit risk scores for loan approvals.

Why is ESG data essential for investors?

ESG data is used by asset managers and investors for market analysis, supporting asset allocation and risk management, and in providing insights into the long-term sustainability of investments in various corporations.

← Previous

Honoring Black History Month: How These MongoDB Employees Defied the Odds

February is Black History Month. It’s a time to reflect on and celebrate the struggles and triumphs of the black community and remember the importance of elevating black voices. Each year at MongoDB, we ask members of our employee resource group BEAM (Black Employees At MongoDB) if they’d like to share a personal story about their experiences and what this month means to them. This year, hear from Administrative Assistant Rita Henderson and Regional Director Daniel Hawthorne to learn more about their journeys into tech. Rita Henderson: Breaking Down Barriers and Owning Technology for Social Justice As we celebrate Black History Month, I am grateful for those who have paved the way for us to have a voice and fight for our rights. I am reminded of the struggles and achievements of black leaders throughout history. The fight for equal rights and justice is ongoing, and technology plays a crucial role in this fight. It is important to empower and uplift underrepresented communities in the tech industry to create a more inclusive and equitable future. I am a proud member of the Afro-Latinx community from North Philadelphia. Growing up in a neighborhood called Badlands, I witnessed first-hand the impact and struggles of poverty, high crime rates, and drugs. I am the youngest of six children, with parents who worked two jobs to make ends meet. Despite my parents' hard work and dedication to provide for their children, life was still a struggle for my family. At the age of 17, after completing my junior year of High School, I became a teen mom. Unfortunately, society tries to shame young mothers, especially teen moms of color. Many people reminded me that teen pregnancy is closely linked to single parenthood and that growing up in single-parent families remains the largest factor in increased poverty among children. Me (middle) and my sibling with our dad. Yes, I photoshopped myself in. As a teen mom, I was determined to break through the barriers society placed on me. With $200 in my pocket, I moved my daughter and I to western Pennsylvania and enrolled in Indiana University of Pennsylvania. There, I earned my bachelor's degree in Criminology and studied the school-to-prison pipeline in black communities. After the murder of the young unarmed black teenager, Mike Brown, and the Ferguson uprising, my sister and I collaborated with organizers in the Ferguson community to launch a free technology program to empower community organizers, educators, and youth with skill sets to create technology tools for social and economic justice. Graduating from Indiana University of Pennsylvania. Pictured with my firstborn London Rae and my mom. I am influenced by the work of the Black Panther Party; specifically, the 10th Point of the party’s 10-Point Platform, “Community Control of Modern Technology”. For 45 years, the Black Panther Party included the right to learn, access, and control technology as a right. Huey said, "Knowing how to struggle is the essence of winning. Recognizing ills is fundamental; recognizing how to overcome ills is mandatory." That is why I believe it is critical for black and latinx people to understand the role technology plays in our society and the economy if we want to understand social justice and create tools for liberation. When I hear people talk about technology in black, latinx, and working-class communities, they often use it as a scare tactic. The fear of data and control and the feeling that technology is too advanced and that we lack the knowledge and tools to participate can be overwhelming. However, it is crucial for our community to claim our place in the tech world. We need to change our thinking and know there is a place for us, just like there is for anyone else. I am grateful for MongoDB's value "embrace the power of differences" and creating a platform where underrepresented communities can share their stories, bring their ideas to the forefront, and be heard in the tech industry. As we celebrate Black History Month, I am grateful for those who have paved the way for us to have a voice and fight for our rights. I am also thankful for the opportunities I have been given to make a difference in my community and empower others to do the same. With education and technology, we can continue breaking down barriers and striving for equality, justice, and liberation. In 2022, my partner and I welcomed our baby girl Lara Sky. Daniel Hawthorne: Building a Career as a Black Man in Tech Sales I was brought into the world with the odds against me, a black boy born in South Central Los Angeles in the 80s. However, I never felt that I was on my own. Throughout my entire life, God has choreographed my every step. At a very young age, my parents decided to move us to Austin, Texas where my grandparents were moving their church ministry. I was raised in Austin along with my two older brothers (Dante and Derrell), my younger sister (Amber), and my younger brother (Joseph). My siblings called me the “golden child” because I was a mama’s boy and kept to myself. The elementary and middle schools that I attended in Austin were fairly diverse, and I seldomly experienced racism. In the 7th grade, my family moved to a suburb of North Austin that wasn’t as diverse, and racist experiences became much more frequent. It was then that I began to acknowledge that being black brought different treatment. There were moments I embraced my blackness, but others where I was more focused on adapting myself into someone I thought those in my non-diverse environment wanted me to be. In middle school, the place to hang out was the Rec Center. I would run into kids from other schools, and we’d have the basketball gym to ourselves for a bit. Eventually, the older guys would take over the court, but I was good enough that I typically got to play with them. I remember observing them as they entered the gym. They’d be dressed in nice work clothes with Dell badges hanging from their shirts - the Rec Center was only five minutes from the Dell HQ - and that became an early image of what success looked like for me. In high school and college, I started my career in sales with a few small gigs. I enjoyed it because I was typically one of the top sellers no matter what I sold. I even sold women’s shoes at one point! After graduating with my M.B.A, I had no idea what my next move would be. But then, that image of success popped into my head. I focused my attention on getting a sales job at Dell. Despite not having any experience in tech, I knew I could excel. Who knew that 10 years after my days on the Rec Center courts, I would land my first job in tech. I joined the inside sales development team at Dell, and it was one of the most pivotal moments of my career. The job was intense. After a week of training, it was clear that I was the least technical in every room. But, I was determined to not let anyone outwork me. We were required to make over 100 outbound calls per day, but I quickly figured out how to achieve the true objective (10 scheduled virtual demonstrations in a week) in fewer calls. Through my efficiency, I helped form new standards and began to make a name for myself. Being in sales development wasn’t my end goal. I knew I wanted to get into outside sales, so I began building relationships with some of the Dell outside sellers I worked with. During a coaching session with one of my mentors, who was also a minority, he shared some guidance that I wasn’t ready for. He told me that if I truly wanted to be in outside sales, I needed to lose my earrings because professional men didn’t wear them. Even though he and I understood that earrings didn’t define me, his guidance was that being a person of color meant I was already playing from behind, and that I should exhaust all things within my control to create as level a playing field as I possibly could. This theme would continue throughout my career. Similar to when I was a kid in the non-diverse suburbs of Austin, as a black man in tech, I’ve felt heavy pressure to be a certain way to appease others. When I was first getting started, I hardly encountered sales folks that looked like me. I’d attend internal trainings and events where there might be one or two other black sellers out of 200+ people. In many ways, I felt that I was on an island and had to live through trial and error. I had a fear that being ‘too black’ would put me at an even greater disadvantage. I walked the line and was careful about what I said or did. I hardly engaged in extracurricular activities with co-workers, and when I did, I kept my guard up. So much of my energy and effort was exhausted into protecting my brand and trying to avoid negative stereotyping because of the color of my skin. I often think about how much more successful I could’ve been had I not felt obligated to focus on the things that never should’ve mattered. My wife and our two daughters at the apple orchards outside of St. Louis, Missouri. As I stated before, God has led my path in life. Numerous times when I was unsure of the next turn to make, He introduced someone to provide direction. I’m truly grateful for the people who may not have looked like me, but provided me with valuable coaching that helped guide my career in tech. I joined MongoDB to help customers with their data transformations, but I didn’t expect that I would go through a transformation myself. I’ve never felt more empowered to just be myself, and through that, I’ve reached new levels of individual and team accomplishments. I was a direct seller for my first two years with the company, and after receiving coaching from peers and leaders around me, I stepped into management a year ago. This wasn’t necessarily a milestone or goal that I had set out for myself, but I came to the realization that there was tremendous value in helping other sellers (and their families) achieve new levels of success. What better company to step into leadership than at MongoDB. Every company has employee resource groups nowadays, but the intentionality behind those groups at MongoDB is different. Our leadership team has leaned into those difficult, vulnerable discussions, sometimes simply to listen because they knew they didn’t have the answers. Even in those scenarios, they’d come up with relevant action that they could personally be responsible for. Despite the comfort zone I had created over the past 10+ years of watering down my blackness, our Sales team encourages individuality and has brought out the best version of me. It’s helped lift a giant weight off my back. I know I’m no longer starting from behind, and I don’t fear that folks are going to judge me. As I wrap-up my first year in sales leadership, I’ve noticed significant transformation in my personal development, and I’m excited that I get to continue taking on new challenges that will bring discomfort, but instill confidence that I can persevere. As we celebrate Black History Month, I think about the opportunity I have to expose other members of the black community to a profession in sales. Our experiences and our perspectives are highly valued and necessary in order to build a better tech-centric future. We’re passionate about cultivating a culture where people of all backgrounds, identities, and experiences feel valued and heard. Find your next career opportunity at MongoDB.

February 14, 2023
Next →

MongoDB: Powering Digital Natives

Today's rapidly evolving digital landscape is dominated by digital native companies, driving innovation . These are companies born in the digital age and who operate through digital channels with a business model enabled by technology and data. They are not only adept at using technology but are also reshaping the way software is developed and deployed. This article delves into the challenges and opportunities facing digital natives in modern application development, with a particular focus on the complexities of managing data. We’ll explore how the right data platform can empower your digital native organization to build high-quality software faster, adapt to changing market demands, and unlock the full potential of your business. Strong foundations: The four pillars of tech-fueled growth for digital natives Achieving explosive growth requires a strong foundation built on specific principles, which empower rapid scaling and success. Here, we explore the four key pillars that fuel tech-driven growth for digital natives: Product-market fit, fast: As a digital native, you must continuously ship and iterate products to achieve a quick product-market fit. This builds customer trust and captures opportunities before competitors can in an evolving market. Data and AI-driven decisions: You must leverage data to personalize experiences, automate processes, and guide product decisions. A robust data architecture feeds real-time data into AI models, enabling data-driven decisions organization-wide. Balance of freedom and control: Your developers must have the freedom to choose technologies, even as your organization maintains control over the infrastructure to manage risks and costs at scale. Selected technologies must integrate within your overall technology estate. Extensible and open technologies: You must explore disruptive technologies while maintaining existing systems. Freedom from platform and vendor lock-in enables quick adoption of innovations, from current generative AI capabilities to future technological advances. Data: The unsolved challenge in modern application development From cloud platforms and managed services to gen AI code assistants, advancements have transformed how engineering teams build, ship, and run applications: Agile methods and programmatic APIs streamline development, while CI/CD and infrastructure as code automate processes. Containerization, microservices, and serverless architectures enable modularity, while new languages and frameworks boost capabilities. Enhanced logging and monitoring tools provide deep application health insights. Figure 1: Tools and processes to maximize velocity. But none of these advancements address where developers spend most of their time— data . In fact, 73% of developers share time and again that working with data is the hardest part of building an application or feature. So why is data the problem? Traditionally, selecting a database, often an open-source relational one, is the first step in development. However, these databases can struggle with the characteristics of modern data: it’s high volume, unstructured, and constantly evolving. As applications mature and their data demands grow, development teams may encounter challenges with achieving scalability and maintaining service resilience. Some teams turn to NoSQL databases, but even then they find there are limited capabilities, pushing them back to relational databases. As the application gains traction, the business’s appetite for innovation grows, compelling development teams to incorporate an expanding array of database technologies. This results in an architectural sprawl, imposing on teams the challenges of mastering, sustaining, and harmonizing new technologies. Concurrently, the dynamic technology landscape undergoes constant evolution, demanding teams to swiftly adjust. As a result, self-contained, autonomous teams encounter these hurdles recurrently, highlighting the pressing need for streamlined solutions to mitigate complexity and enhance agility. Figure 2: The evolving tech landscape. Data sprawl: A major threat to developer productivity and business agility Data sprawl is slowing everyone down. The more systems we add, the harder it is for developers to keep up. Each new database brings its own unique language, format, and way of working. This creates a huge headache for managing everything—from buying new systems to making sure they all work together securely. It’s a constant battle to keep data accessible, consistent, and backed up across all these different platforms. Figure 3: Teams building on separate stacks leads to data sprawl and manageability issues across the organization It compromises every single one of the four outcomes your technology foundation should be providing, yielding the opposite results: Missed opportunities, lost customers: Fragmented development experiences consume time as engineers struggle with multiple technologies, frameworks, and extract, transform, and load mechanisms for duplicating data between systems. This slows down releases, degrades digital product quality, and impedes engineers from achieving product-market fit and effective competition. Flying blind: With your operational data siloed across multiple systems, you lack the data foundations necessary to use live data in shaping customer experiences or reacting to market changes. This is because you are unable to feed reliable, consistent, real-time data into your AI models to take action within the flow of the application or to provide the business with up-to-the-second visibility into operations. High attrition, high costs: Complex data architecture impacts development team culture, leading to siloed knowledge, inefficient collaboration, and decreased developer satisfaction. This complexity also consumes substantial resources in maintaining existing systems by diverting resources from new projects that are vital for business competition in new markets. Disruption from new technologies: Dependence on any one cloud provider can stifle innovation for development teams by restricting access to the latest technologies. Developers are confined to the tools and services offered by a single provider, hindering their ability to explore and integrate new, potentially more efficient, or advanced technologies. Speed: A unified developer experience for building high-quality software faster In today’s digital world, speed is king. Your customers expect seamless experiences, but clunky applications leave them frustrated. But traditional databases can be a bottleneck, struggling to keep pace with your ever-evolving data and slowing down development. The future of data is here, and it’s flexible: a data platform built for digital natives . It leverages a flexible document model, letting you store and work with your data exactly how you need it. This eliminates rigid structures and complex migrations, freeing your developers to focus on what matters—building amazing applications faster. Flexible document data models empower developers to handle today’s rapidly evolving application data ( 80%+ unstructured) that relational databases struggle with. MongoDB documents are richly typed, boosting developer productivity by eliminating the need for lengthy schema migrations when implementing new features. Developers get to use their preferred tools and languages. Through its drivers and integrations, MongoDB supports all of the most popular programming languages, frameworks, integrated development environments, and AI-code assistance tools. MongoDB scales! It starts small and scales globally. Built for elasticity and horizontal scaling, it handles massive workloads without app changes. Figure 4: A unified developer experience, integrating all necessary data services for building sophisticated modern applications Introducing MongoDB Atlas : a fully-managed cloud database built for the modern developer. It enables the integration of real-time data from devices with AI capabilities (through vector embeddings and large language models ) to personalize user experiences. Stream processing empowers constant data analysis, while in-app analytics provides real-time insights without needing separate data warehouses, all while automatically managing data movement and storage for cost-effectiveness. MongoDB Atlas simplifies database management with the following: Easy deployment via UI, API, CLI, Kubernetes, and infrastructure as code tools. Automated operations for cost-effective performance and real-time monitoring. MongoDB Atlas customer success stories: Development with speed, scale, and efficiency Delivery Hero Delivery Hero, a global leader in online food delivery, leverages MongoDB Atlas to power its rapid service. Founded in 2011, Delivery Hero now serves millions of customers in over 70 countries through brands like PedidosYa, foodpanda, and Glovo. Having replaced its legacy SQL database, Delivery Hero optimized operations and bolstered performance by using MongoDB Atlas. By leveraging MongoDB Atlas Search, Delivery Hero revolutionized its search functionality, ensuring a seamless user experience for its extensive customer base through simplified indexing and real-time data accuracy. MongoDB’s scalability has empowered Delivery Hero to manage over 100 million products in its catalog without encountering latency issues, enabling the company to expand its services while maintaining peak performance. This agility, coupled with MongoDB’s cost-effectiveness, has enabled Delivery Hero to swiftly adapt to evolving customer demands, solidifying its position in the fiercely competitive delivery market. MongoDB Atlas Search was a game changer. We ran a proof of concept and discovered how easy it is to use. We can index in one click, and because it’s a feature of MongoDB, we know data is always up-to-date and accurate. Andrii Hrachov, Principal Software Engineer, Delivery Hero Read the full customer story to learn more. Coinbase Coinbase, a prominent cryptocurrency exchange boasting 245,000 ecosystem partners and managing assets worth $273 billion , trusts MongoDB to handle its extensive data workload. As the company grew, MongoDB scaled seamlessly to accommodate the increased demand. To further improve performance in the fast-paced crypto world, Coinbase partnered with MongoDB to develop a system that significantly accelerated data transfer to reporting tools, reducing processing time from days to a mere 5-6 hours. This near real-time data access enables Coinbase to rapidly analyze trends and make informed decisions, maintaining a competitive edge in the ever-evolving crypto landscape. Watch Coinbase's full session at MongoDB.local Austin, 2024 to learn more. MongoDB: Your flexible platform for digital growth With MongoDB, you can freely explore, experiment, develop, and deploy according to your digital-native business needs. If you would like to learn more about how MongoDB can empower your digital-native business to conquer market trends, visit: Innovate With AI: The Future Enterprise Application-Driven Intelligence: Defining the Next Wave of Modern Apps AI-Driven Real-Time Pricing with MongoDB and Vertex AI

November 7, 2024