Writing Meaningful Tests

Test the Teaching Point, Not Just the Mechanics

Every code example exists to demonstrate a behavior:

• A find() example with a filter demonstrates filtering.

• An updateOne() example demonstrates modifying a document.

• An aggregate() example demonstrates a pipeline stage.

Your test proves that the behavior works as shown.

Weak versus Strong Tests

• A weak test proves the command ran:

  The example runs db.movies.updateOne(), and the test checks that the output is { acknowledged: true, matchedCount: 1, ... }.

  This example proves only that MongoDB accepted the command. If the $set syntax were wrong in a way that silently failed, this test would still pass.

• A strong test proves the teaching point:

  The example runs updateOne followed by findOne to verify the change. The test checks that findOne returns { title: "The Godfather", rated: "PG" }.

  This test proves the update took effect, which is what the reader learns from the example.

Levels of Test Confidence

Levels of test strength exist on a spectrum:

Aim for output tests at minimum. Use behavior tests when the example teaches a write operation or a side effect such as index creation or schema validation.

Acknowledgment-level tests are acceptable when the acknowledgment is the teaching point. For instance, if the example covers write concerns and the docs specifically discuss the acknowledgment format.

Common Patterns That Feel Like Tests but Prove Little

Over-permissive schema validation. A schema that asserts only a document count (and no required fields or field values) allows almost any output to pass validation, including results that have nothing to do with what the example teaches:

// Weak - proves 5 documents came back,
// but nothing about content
Expect.that(result)
  .shouldResemble("output.json")
  .withSchema({ count: 5 });
// Better - proves documents contain the fields
// the example queries
Expect.that(result)
  .shouldResemble("output.json")
  .withSchema({
    count: 5,
    requiredFields: ["title", "year", "genres"],
    fieldValues: { year: 2012 }
  });

Ignoring all the relevant fields. If you ignore every dynamic field, the assertion stops verifying anything meaningful:

// Weak - ignores the fields the example is about
Expect.that(result)
  .withIgnoredFields(
    "_id", "title", "rated", "year"
  )
  .shouldMatch("output.json");
// Better - only ignore truly dynamic fields,
// verify the rest
Expect.that(result)
  .withIgnoredFields("_id")
  .shouldMatch("output.json");

Testing the wrong output. If your example inserts data and then queries it, test the query result, not the insert acknowledgment:

// Weak - tests the insert, not the query
// that is the point of the example
const insertResult = await loadData();
Expect.that(insertResult)
  .shouldMatch("insert-output.json");
// Better - tests the query the example is
// actually teaching
const queryResult = await runPipeline();
Expect.that(queryResult)
  .shouldMatch("pipeline-output.json");

Considerations

Before you call your test done, verify the following information:

• The test catches a silent failure in the core operation. For example, if your update set the wrong field, the test fails.

• The assertion matches what the docs claim. For example, if the docs say "returns documents sorted by year," the test verifies the sort order using withOrderedSort().

• The test is specific enough that a completely different query would not pass. For example, if the example teaches how to filter by a specific field value, the test checks that the result contains the correct documents.

Capture Output

You do not always need to run the example manually. The test infrastructure can capture output for you.

Scenario 1: Read-Only Query Against Sample Data

Your example queries sample data and does not change anything. This is the simplest case. No need to reverse any changes.

// No need for any cleanup
it("finds movies by year", async () => {
  const result = await findMoviesByYear();
  Expect.that(result)
    .shouldResemble(
      "movies-by-year-output.json"
    )
    .withSchema({
      count: 8,
      requiredFields: [
        "title", "genres", "year"
      ],
    });
});

Considerations:

• Use your language's sample data utility so the test skips gracefully if the dataset is not loaded. See Add Tests for Examples for details.

• Use shouldMatch if the output is stable and predictable.

• Use shouldResemble with withSchema if you care only about shape. See the Choose Between shouldMatch and shouldResemble section for details.

Scenario 2: Writes to Sample Data (Must Clean Up)

Your example modifies documents in a sample database. You must undo exactly what the example did. Think of it as a "reverse transaction."

// After each test, restore the original value
afterEach(async () => {
  const db = client.db("sample_mflix");
  await db.collection("movies").updateOne(
    { title: "The Godfather" },
    { $set: { rated: "R" } }
  );
});
it("updates a movie rating", async () => {
  const result = await updateMovieRating();
  Expect.that(result)
    .withIgnoredFields("upsertedId")
    .shouldMatch("update-rating-output.json");
});

Considerations:

You need to check the sample data before writing your cleanup to save the original values. Query the document, write down the fields your example changes, and restore those exact values in teardown.

• For more cleanup patterns and strategies, see the Write Cleanup That Works section.

• To understand why cleanup matters, see the Idempotency section.

Scenario 3: Custom Data (Create and Destroy)

Your example does not use sample data. It creates its own database with its own collections and data. This is the cleanest pattern. You control all the data, so you can drop the whole database after the test.

// After each test, drop the custom database
afterEach(async () => {
  await client.db("iot_db").dropDatabase();
});
it("computes average temperature", async () => {
  const result = await computeAvgTemperature();
  Expect.that(result)
    .shouldMatch("avg-temperature-output.json");
});

Use shouldMatch

Use shouldMatch when:

• The output is predictable and stable across runs.

• You want readers to see realistic output in the docs.

• The exact values matter and are guaranteed to be consistent, such as when demonstrating an update result.

Expect.that(result)
  .shouldMatch("update-output.json");

Use shouldResemble

Use shouldResemble when:

• The output order or exact values can vary across environments, such as with vector search.

• You care only that the right number of documents came back containing the right fields.

Expect.that(result)
  .shouldResemble("find-output.json")
  .withSchema({
    count: 8,
    requiredFields: [
      "title", "genres", "year"
    ],
  });

When You're Not Sure

Start with shouldMatch. If the test is flaky (passes sometimes, fails sometimes), switch to shouldResemble.

Capture Real Output First

Never hand-write expected output from scratch. Always run the example first and copy the real output. Then, edit it to replace dynamic values. You can get the real output either by running the example by hand or by running a deliberately-failing test and copying the actual value from the error message. For details, see Capturing Output.

withIgnoredFields vs. Ellipsis in the Output File

Both handle dynamic values, but they work differently.

• withIgnoredFields("_id"): The field must exist in both actual and expected output, but the value comparison is skipped. Use this approach when the field is part of your example's story and you want readers to see it in the expected output:

  // Test ignores the value comparison:
  Expect.that(result)
    .withIgnoredFields("_id")
    .shouldMatch("output.json");

• Ellipsis in the output file: You can also use ellipsis patterns directly in the output file instead of withIgnoredFields(). The comparison engine supports three forms:

  • "..." as a field value means "this field exists but skip the value comparison." Use this for fields with dynamic values such as _id or timestamps:

    { "_id": "...", "title": "The Godfather" }

  • Standalone ... as its own line in the output file enables global ellipsis mode. This allows extra fields in the actual output that are not listed in the expected file. Use this when you want to verify specific fields but do not care what else the document contains:

    {
      "title": "The Godfather",
      ...
    }

  • "...": "..." as a key-value pair is an object wildcard that matches any object. Use this when a nested object can have any structure:

    {
      "title": "The Godfather",
      "metadata": { "...": "..." }
    }

Either approach (withIgnoredFields or ellipsis patterns) works. Use whichever reads more clearly in context.

Be Strategic About What You Hard-Code

For a quick reference of every supported wildcard pattern, see Grove Testing Cheat Sheet.

Output Format Notes

• Format must match your language's output. mongosh uses single quotes and unquoted keys ({ title: 'Argo' }). Driver suites typically produce standard JSON ({ "title": "Argo" }). Match what your example returns.

  Tip

  MongoDB Shell Consideration

  mongosh output uses single quotes, no trailing commas, and unquoted object keys. Your expected output file must match this style exactly.

• Array wrappers. Multi-document results from find() are wrapped in [...]. Single-document results from findOne() are not. Match what the driver or shell returns.

The Idempotency Test

Run your test twice in a row:

npm test -- -t "my test name" && \
  npm test -- -t "my test name"

If the test does not pass both times, something in your test (usually cleanup) leaves the database in a different state than it started in.

What Makes a Test Non-Idempotent

Anything your example does that persists after the test ends causes non-idempotency. For example:

• Modified documents: A field value differs from before.

• Inserted documents: New documents appear in the collection.

• Deleted documents: Expected documents are missing.

• Created indexes: Query plans change, output order shifts.

• Created collections or views: A createCollection call fails with "already exists" on the second run.

• Changed configuration: Validation rules, collation settings.

If your example does any of these and you do not undo them, the next run starts from a different state and produces different results.

Cleanup Achieves Idempotency

Every persistent side effect needs a matching undo. The goal is not "tidy up after yourself" in the abstract. The goal is to ensure that the next run sees the same database state as the first run.

Set Up Cleanup Before the Test Runs

If your test throws an error, the teardown hook still runs, but only if you set up your cleanup before the failure.

In driver suites, this is usually natural because cleanup lives in a framework hook (afterEach, tearDown, defer, @AfterEach, or [TearDown] depending on the framework) that registers before the test body runs. For the exact syntax per framework, see Grove Testing Cheat Sheet.

In mongosh, where cleanup is sometimes a per-test function, register it before calling Expect:

// mongosh pattern: register cleanup first
test("my test", async () => {
  // 1. Register cleanup
  currentCleanup = myCleanupFunction;
  // 2. Run the test
  await Expect.outputFromExampleFiles(...)
});

Cleanup Must Be Safe in Any State

Your cleanup might run when the database is in various states: the example succeeded, failed partway through, or did not run at all. Make each cleanup operation safe to run regardless of the database state:

// Node.js driver: each operation
// tolerates "not found"
afterEach(async () => {
  const db = client.db("sample_mflix");
  try {
    await db.collection("users")
      .dropIndex("email_1");
  } catch (e) {
    // Index may not exist if test failed
    // before creating it
  }
  await db.collection("users")
    .deleteMany({ testDoc: true });
});

# PyMongo: same idea
def tearDown(self):
    db = self.client["sample_mflix"]
    try:
        db.users.drop_index("email_1")
    except Exception:
        pass
    db.users.delete_many({"testDoc": True})

Consider What Other Tests Assume

Your cleanup does not serve only your test. It protects every test that runs after yours. Consider the following:

• If another test queries sample_mflix.movies, does it find unexpected documents that you inserted?

• If another test creates an index on users.email, does it fail because you left a duplicate index behind?

• If another test reads a specific document, does it see the original values or your modified ones?

Quick Cleanup Reference

Driver Suites: Multiple Steps in One Function

Driver examples are functions, so you can include setup and query logic in the same function. You can use Bluehawk's :remove: tags to exclude setup from the extracted snippet if it does not need to be included in the documentation:

// :snippet-start: avg-temperature
export async function avgTemperature() {
  const client = new MongoClient(
    process.env.CONNECTION_STRING
  );
  try {
    const db = client.db("iot_db");
    // :remove-start:
    await db.collection("sensors").insertMany([
      { sensorId: "s1", temperature: 22.5 },
      { sensorId: "s2", temperature: 25.3 },
      { sensorId: "s3", temperature: 18.7 }
    ]);
    // :remove-end:
    const result = await db
      .collection("sensors")
      .aggregate([
        {
          $group: {
            _id: null,
            avgTemp: { $avg: "$temperature" }
          }
        }
      ]).toArray();
    return result;
  } finally {
    await client.close();
  }
}
// :snippet-end:

MongoDB Shell Options

• Option A: Separate load and query files. The test runs them in sequence. Use this approach when both steps need to appear in the documentation:

  await Expect
    .outputFromExampleFiles([
      "topic/load-data.js",
      "topic/run-pipeline.js"
    ])
    .withDbName("iot_db")
    .shouldMatch("topic/output.sh");

• Option B: Comma operator in a single file. The harness captures only the last expression's result:

  // :snippet-start: update-and-verify
  (
    db.movies.updateOne(
      { title: "The Godfather" },
      { $set: { rated: "PG-13" } }
    ),
    db.movies.findOne(
      { title: "The Godfather" },
      { title: 1, rated: 1 }
    )
  )
  // :snippet-end:

  Note

  Comma Operator is MongoDB Shell-Specific

  The comma operator pattern applies only to mongosh. Driver suites do not need it because they run multiple operations in a single function.

The Problem

Your test creates an index. The test passes. You run it again, and now the index already exists. Either:

• The createIndex call returns a different result ("index already exists" instead of "created").

• Or, a subsequent query uses the index and returns results in a different order.

The Solution

Clean up indexes in both setup and teardown:

// Node.js driver: beforeEach + afterEach
const cleanupIndexes = async () => {
  try {
    await collection.dropIndex("email_1");
  } catch (e) { /* may not exist */ }
};
beforeEach(cleanupIndexes);
afterEach(cleanupIndexes);

# PyMongo: setUp + tearDown
def _cleanup_indexes(self):
    try:
        self.collection.drop_index("email_1")
    except Exception:
        pass
def setUp(self):
    self._cleanup_indexes()
def tearDown(self):
    self._cleanup_indexes()

Setup handles the case where a previous test run crashed before teardown ran. Teardown handles the normal case so the next test starts clean.

Read the Error Message

The comparison engine produces detailed diffs. Look for the following:

• "Expected N documents but got M": Your query returned a different number of results. Run the example by hand to see why.

• "Field X: expected Y but got Z": A specific value did not match. Check if your cleanup restores the original value.

• "Missing field X": The document structure does not match your expected output. Check the query projection.

Run the Example by Hand

Run the example outside the test framework. Compare the actual output to your expected output file. This usually reveals the mismatch immediately.

Check for Stale State

Run your test twice in a row. If the test passes once and fails the second time, your test is not idempotent. Cleanup is incomplete. Something your example changed is not reverted. See Idempotency for a list of things that might persist between runs.

Check for Index Side Effects

If your output order changed, an index might affect the query plan. Check what indexes exist on the collection.

Check the Generated File (MongoDB Shell Only)

When you get a syntax error in mongosh, the error message shows the generated temp file content. This reveals how the harness wrapped your code. Look for mismatched parentheses or unexpected printjson() wrapping.