Upply is Using Data to Build a New Logistics Market – Here’s How

For most of us, a Logistics decision means choosing between fast and expensive, or slow and cheap, delivery when we buy something online. Behind that choice is a trillion-dollar industry that is one of the most complex and dynamic of the modern world. Whoever you give your delivery money to has to decide how to move your purchase, perhaps across half the planet, while dealing with customs, regulators, multiple transport companies and multiple possible routes - all on time and all while turning a profit.

This would seem a natural fit for automated data analysis, the perfect place for a highly-computerized market that combines all the data about a particular delivery requirement and produces an optimized result. Indeed some operations have been improved over the decades, such as Transport Management Systems (TMS). However, it was only addressed at an enterprise level. There was no holistic solution that would truly change the market.

A data-focused logistics start-up in France is hell-bent on changing all that.

Big Market, Bigger Problems

As Thomas Larrieu, Chief Data & Research Officer at Upply, the aforementioned start up, says, the sheer complexity, size and footprint of the global Logistics industry has slowed down the evolution of an automated, efficient market where information is shared fairly and smartly to help every player succeed. “All the market data is out there but the industry’s only been seriously dealing with it for five years. Nobody’s been drawing it together,” he says. “Big companies are reluctant to do anything that might upset their position, small companies can’t afford to invest in the analytics and data gathering needed. So the idea of Upply was born around three years ago. To bring data science and automation where it’s most needed. We are convinced that it will improve the whole industry.”

That meant building the database, analytics package and API to deliver clean market data straight to users. It’s the first time this has been done.

Building on long experience in the market, Upply started off by collecting a 50-million record global logistics dataset. “That sets the scope of the challenge. We have customs rules, we have prices, we have currencies, we have pounds versus kilos, metres versus feet, many kinds of vessels, trucks and on and on,” says Thomas Larrieu. In the lingo of logistics, the route a delivery might take is called a lane, and Upply’s first task was to characterize each lane according to all applicable data.

“That’s why MongoDB Atlas became interesting to us, for two things. We don’t know ahead of time what a user can put in the data, so we need to think about the data from a document aspect. We can add new stuff and new parameters MongoDB does that far better than the SQL world can.”

The second interesting thing? “Strong search, strong indexing.”

Making the Right Choices for Disparate Data

Once Upply had a handle on its industry data model, it started ingesting live information, generating and applying analytics and working out what to do with the results.

“We know our industry is facing many structural and time-related challenges. A lack of flexibility, a lack of visibility, and very complex operational and bidding processes with poor access to marketing information,” Thomas Larrieu says. “Almost every industry, either B2C or B2B, has online one-click price comparison. Not freight. We’re fixing that.”

For example, Upply recently extended its pricing analysis service with a feature called "Trends". This tool provides access to the historical rates on main trade routes and allows users to discover the future trends for the coming months.

That means designing systems to cope with half a million new records a day - from industry partners and open-source market feeds - aggregating, analyzing and providing everything about each lane on demand.

Leaders in Data, Become Leaders in the Market

On the back of that, Upply has become a market maker. “The freight market is not mature regarding trading services,” says Thomas Larrieu. “If you’re a big carrier you have much better access to a choice of intermediates or direct customers and can get more interesting rates. Smaller companies have to ask someone else to ship something from Shanghai to San Francisco, for example, and that third party will go and strike a much better deal with the actual carrier.”

With the marketplace on Upply.com that has been launched at the beginning of July 2019, he says, “we set the relationships between offer and demand, freedom of partners, customers, and suppliers, and more autonomy in the price paid or the price asked. The engine of the shipping market had almost seized up - we’re oiling it”.

For now, that marketplace is for French road freight, where Upply has the strongest relationships and the deepest knowledge. The plan is to use network effects to grow out to Europe and the US, based on the customers in France all having global reach and wanting similar services elsewhere once they’re comfortable with Upply at home.

The Right Choices Now Make Future Planning Easier

That future growth is implicit in Upply’s architecture. “We’re using a lot of big data tools like Databricks and Spark. We have different data providers with different types of files and formats. We put all this data on a single database over several servers with sharding. With Databricks backed by MongoDB on Atlas, we can spread the principles of Hadoop across one or 10 or 1,000 servers with tuned CPU and RAM requirements.”

Upply can provide benchmark analysis of up to 100,000 lanes in 12 minutes. One of the benefits MongoDB Atlas on Microsoft Azure is that this will be trivial to scale as demand grows, without incurring any management or configuration overheads.

“We are currently using neural network Bayesian tools like BSTS (Bayesian Structural Time Series) to help predict the effect of seasonal and other factors on the market. MongoDB is great for feeding polymorphic data into such systems”, says Thomas Larrieu. The extended future trends coverage, with look-ahead of to up to 24 months, “will definitely change the way market players access information.”

Right now, Upply’s creation of a clean API and online tools for getting market data is proving popular. “We’ve already got 2,000 users from 500 companies. Just one user has already analyzed 45,000 lanes on our platform,” explains Thomas.

Finally, creating greater knowledge promises more valuable results than mere profit. “We already provide an assessment of CO2 emissions for each analysis, and we’re working with partners to build much more detailed and up-to-date carbon emissions data.” With transportation accounting for 15 percent of global emissions, we may yet see a third option appear between cheap or fast when we click on that Buy button. Send it green. That’s an option we need, and one that the data makes possible.