Store 1M documents per day - Best Practices

I am building a trading system.
I have 40,000 different symbols, and 32 candles each symbol per day.
I would like to store a month of data. and the read actions to be fast (I’m querying symbol by symbol)

How do you suggest to design the DB? (collection for each symbol? collection for each day?)

1 Like

Can you share more details about your data and how you’re going to use it?

  1. Prettified example object, that represents your ‘symbol’
  2. Prettified example object, that represents 1 ‘candle’ for 1 ‘symbol’
  3. What data is more likely to change?
  4. What parts of the data will be requested more often?
  5. Are you planning to collect some statistical information based on ‘symbols’ and ‘candles’?
1 Like
  1. The symbol is just a string value inside the candle object
  2. An candle object example could be:
    “sym”: “MSFT”, // Symbol Ticker
    “v”: 10204, // Tick Volume
    “o”: 114.11, // Tick Open Price
    “c”: 114.14, // Tick Close Price
    “h”: 114.19, // Tick High Price
    “l”: 114.09, // Tick Low Price
    “t”: 1536036818784, // Tick End Timestamp ( Unix MS )
  3. no changes at all just adding candles each symbol every 15 mins.
  4. The candles will be requested symbol by symbol
  5. Not for now…