Storing weather data

Working with Data
, , ,
1

Hello Everyone,

I’ve been looking around for quite some time now and can really seems to find a good solution to my problem and i hope that i’m not creating a duplicate or boring topics.

I’m working on a small app to manage my own farm and i have now collected weather information for every single hour since Jan 01 1979. Currently i have 382k doc in my collection and it’s growing…
My main plan now is to display daily weather and some graph displaying weather of the same day for all past years. i’m using node.js and graphql with apollo server for api and react native for the app.

To reduce the file size i grouped all hourly data in days. From 382k i’m down to 16k. But i’m still struggling on storing this data in an easy and efficient way so that i can make one query and get back a specific day of of each year without making al look query and also how to query dates properly.

I’m not really looking into how to code this but more like general opinion or suggestions.
I looked into finding similar problem/solution on the net but no luck so far.

in the meantime thanks for all the help

Kind Regards

M

2

You might be looking for Time Series Collections which were added in MongoDB 5.0 and they are built for this use case exactly. They even mention in at the top of the page :slight_smile:

image
image742×410 26.7 KB

Here is a weather TS collection example:

3

Have you tried MongoDB Charts?

If you use MongoDB Atlas, the Charts are also available for use in it. Also available for free/shared clusters.

By the way, you do not have to reduce your data size since your document size is relatively small enough. and if you use Charts after, you will be able to filter as you like later.

4

Many thanks, i had a look at it but i’ll look into this paying more attention.