insertMany() and create() not working

Hi, I’ve had a detailed explanation posted here: node.js - Mongoose .insertMany and create function not working - Stack Overflow

In short, the create or insertMany is not working correctly if I run the function in the route. The create() only works correctly if I run node importDataToDb.js in the command line.

Based on the information provided, it seems that the issue is related to how the data is being imported to the database. Specifically, when running the importDataToDb.js script via the command line, the create() function works correctly, but when running it within the route, it does not.

This could potentially be due to a few different reasons, such as:

  1. Timing issues: It’s possible that the create() function is being called before the database connection is fully established. To ensure that the connection is established before calling the function, you could try using the once() method of the Mongoose connection object, which waits for the connected event before executing the callback function.

  2. Asynchronous behavior: The create() function is asynchronous, so it’s important to ensure that it is properly awaited or that a callback function is used to handle the result.

  3. Environmental variables: If the database connection string is stored in an environmental variable, it’s possible that it is not being properly accessed when running the script via the route. To ensure that the variable is accessible, you could try exporting it in your terminal before starting the server.

Here’s an example of how to use the once() method to ensure that the create() function is only called after the database connection is fully established:

const express = require('express');
const router = express.Router();
const mongoose = require('mongoose');
const Item = require('../models/Item');

const uri = process.env.MONGO_URI;
mongoose.connect(uri, { useNewUrlParser: true, useUnifiedTopology: true });

const db = mongoose.connection;
db.once('connected', () => {
  console.log('Connected to database');
  createItems();
});

async function createItems() {
  try {
    await Item.create([
      { name: 'Item 1', description: 'Description 1' },
      { name: 'Item 2', description: 'Description 2' },
      { name: 'Item 3', description: 'Description 3' }
    ]);
    console.log('Items created');
  } catch (error) {
    console.log(error);
  }
}

module.exports = router;

In this example, the createItems() function is only called after the connected event is emitted by the database connection. Additionally, the create() function is awaited to ensure that it completes before logging the success message.

I hope this helps, let me know if there’s still problems.

Thank you!
I made it work by wrapping the create() into an async function, something like this:

module.exports.getDownloadFile = async (req, res, next) => {
    async function createItems() {
        try {
            const receivedData= await downloadFile( url );
            
            const docs = await SomeDataModel.create(receivedData);
           
            return docs;
        } catch (e) {
            console.log("error import data", e);
        }
    }

    const docs = await createItems();
    res.status(200).json({
        status: "success",
        data: docs.length,
    });
};

This will also work if I put the data import logic into the server.js

const mongoose = require("mongoose");
const app = require("./app");
const { PORT } = require("./config");
const { SomeDataModel } = require("./models/someDataModel");
const { downloadFile } = require("./utils/downloader.js");
require("dotenv").config({ path: "./config.env" });

async function importToDB() {
    try {
        console.log("start importing...");
        const receivedData = await downloadFile(url);
        console.log(receivedData .length);
        await SomeDataModel.create(receivedData );
        console.log("Created");
    } catch (e) {
        console.log("error", e);
    }
}

mongoose.connect(`${process.env.DATABASE}`).then(() => {
    console.log("DB connected");
    importToDB().then(() => {
        console.log("finished importing...");
    });
});

app.listen(PORT, () => {
    console.log(`Express starts on port ${PORT}`);

Thank you again.
I have another question that might be off the topic, since the data has about 5k documents to be imported, this whole process might take a few seconds to finish, during this process the server is stalled, and I cannot perform any request until the data import is finished. What can I do to bypass this issue?