> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

# Handle Overload Errors from Load Shedding

Overload occurs when your cluster receives more incoming operations than its resources can process, and it can cause a total or near-total outage on your cluster. Load Shedding, a feature of [Intelligent Workload Management](https://www.mongodb.com/docs/atlas/intelligent-workload-management.md#std-label-intelligent-workload-management) (IWM), allows Atlas to reject operations during prolonged overload. To configure Load Shedding for your cluster, see [Configure Intelligent Workload Management.](https://www.mongodb.com/docs/atlas/cluster-additional-settings.md#std-label-load-shedding-setting)

When Load Shedding rejects an operation, your application might see a new error for that operation with the `SystemOverloadedError` label. This error indicates that your cluster is overloaded and shedding operations.

If you use a [backpressure-aware client library](https://www.mongodb.com/docs/atlas/overload-errors.md#std-label-backpress-aware-drivers), the client library recognizes these errors and retries the ones that are safe to retry. Your application still controls how long to keep retrying and when to shed load. To learn more, see [Handle Overload Errors.](https://www.mongodb.com/docs/atlas/overload-errors.md#std-label-handle-overload-errors)

The `SystemOverloadedError` label on its own does not mean that the operation can be safely retried. To determine if an overload error is retryable, check for the following labels:

- `RetryableError` - the operation was not executed and is safe to retry

- `NoWritesPerformed` - the server rejected the operation before performing any writes

The following example shows an overload error that Atlas returns when Load Shedding rejects an operation:

```javascript
{
    "ok": 0.0,
    "errmsg": "Request rejected: ingress operation rate limit exceeded",
    "code": 463,
    "codeName": "IngressOperationRateLimitExceeded",
    "errorLabels": ["SystemOverloadedError", "RetryableError", "NoWritesPerformed"]
}
```

**Important:**

If an overload error does not include the `RetryableError` label, ensure that the operation is idempotent and wait before retrying to avoid contributing to overload. Use exponential backoff and jitter in your retry logic.

## Backpressure-Aware Client Libraries Versions

Backpressure-aware drivers and other client libraries automatically recognize overload errors with the `SystemOverloadedError` label and treat them as a signal of overload. If the error has a label that induces a retry, including the `RetryableError` label, the backpressure-aware client library automatically retries the operation with exponential backoff and jitter.

The following table lists the earliest client library versions that are backpressure-aware:

| Client Library | Earliest Backpressure-Aware Version |
| --- | --- |
| C Driver | 2.5 |
| C++ Driver | 4.6 |
| .NET/C# Driver | 3.11 |
| Go Driver | 2.9 |
| Java Sync Driver | 5.12 |
| Java Reactive Streams Driver | 5.12 |
| Kotlin Coroutine Driver | 5.12 |
| Kotlin Sync Driver | 5.12 |
| Node.js Driver | 7.6 |
| PHP Library | 2.5; requires `mongodb` PHP extension (PHPC) 2.5.0+ |
| PyMongo | 4.18 |
| Scala Driver | 5.12 |
| Ruby | 2.26 |
| Rust | 3.9 |

## Handle Overload Errors

Handle overload errors in your application even if you use a [backpressure-aware client library](https://www.mongodb.com/docs/atlas/overload-errors.md#std-label-backpress-aware-drivers). Backpressure-aware client libraries retry operations labeled as retryable, but your application decides how long to keep retrying and when to shed load by abandoning operations that the cluster continues to reject. If you are not using a backpressure-aware client library, also implement the error detection and retry logic yourself.

See the following procedure for examples of how to implement error detection and retry logic with exponential backoff to handle overload errors that are labeled as retryable:

Use the following procedure to implement utilities to detect overload errors and retry with exponential backoff.

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```python
   RETRYABLE_ERROR_LABEL = "RetryableError"
   SYSTEM_OVERLOADED_ERROR = "SystemOverloadedError"

   def is_system_overloaded_error(error):
       return isinstance(error, PyMongoError) and error.has_error_label(SYSTEM_OVERLOADED_ERROR)
   ```

2. Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculate_exponential_backoff()`. Adjust these values to tune the retry behavior for your application.

   ```python
   BASE_BACKOFF_MS = 100
   MAX_BACKOFF_MS = 10_000

   def calculate_exponential_backoff(attempt, base_backoff_ms=None):
       return random.random() * min(MAX_BACKOFF_MS, (BASE_BACKOFF_MS if base_backoff_ms is None else base_backoff_ms) * (2 ** attempt))

   def execute_with_retries(fn, max_attempts=2):
       for attempt in range(max_attempts):
           is_retry = attempt > 0

           if is_retry:
               base_backoff_ms = error.details.get("baseBackoffMS", None)
               delay = calculate_exponential_backoff(attempt, base_backoff_ms)
               sleep(delay / 1000)  # Convert ms to seconds

           try:
               result = fn()
               return result
           except Exception as error:
               is_retryable_overload_error = (
                       is_system_overloaded_error(error)
                       and error.has_error_label(RETRYABLE_ERROR_LABEL)
               )
               can_retry = is_retryable_overload_error and attempt + 1 < max_attempts

               if not can_retry:
                   raise
   ```

3. Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert()` or `find()` queries, with the `execute_with_retries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, consider the following query which fetches all users from the `users` collection and returns them as an array:

   ```python
   users = users_collection.find().to_list()
   ```

   You can modify this code to use the new `execute_with_retries` function:

   ```python
   users = execute_with_retries(lambda: users_collection.find().to_list())
   ```

1) Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```c
   const char *retryable_error_label = "RetryableError";
   const char *system_overload_error = "SystemOverloadedError";

   static bool
   is_system_overloaded_error(const bson_t *error_reply)
   {
      return mongoc_error_has_label(error_reply, system_overload_error);
   }
   ```

2) Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculate_exponential_backoff()`. Adjust these values to tune the retry behavior for your application.

   ```c
   const double BASE_BACKOFF_MS = 100.0;
   const double MAX_BACKOFF_MS = 10000.0;
   const int MAX_ATTEMPTS_DEFAULT = 2;

   static double
   calculate_exponential_backoff(int attempt, double base_backoff_ms)
   {
      unsigned int *seed = get_seed();
      double rand01 = rand_r(seed) / (double)RAND_MAX;
      return rand01 * BSON_MIN(MAX_BACKOFF_MS, base_backoff_ms * pow(2, attempt));
   }

   // get_base_backoff returns the base backoff to apply for an overload error. A server may attach a positive
   // `baseBackoffMS` to the error to replace the default base backoff.
   static double
   get_base_backoff(const bson_t *error_reply)
   {
      bson_iter_t iter;
      if (bson_iter_init_find(&iter, error_reply, "baseBackoffMS") && BSON_ITER_HOLDS_INT(&iter)) {
         int64_t base_backoff_ms = bson_iter_as_int64(&iter);
         if (base_backoff_ms > 0) {
            return (double)base_backoff_ms;
         }
      }
      return BASE_BACKOFF_MS;
   }

   // retryable_fn returns false and sets `error_reply` on error.
   typedef bool (*retryable_fn)(void *ctx, bson_t *error_reply);

   static bool
   execute_with_retries(retryable_fn fn, void *ctx, int max_attempts)
   {
      double base_backoff_ms = BASE_BACKOFF_MS;

      for (int attempt = 0; attempt < max_attempts; attempt++) {
         bool is_retry = attempt > 0;

         if (is_retry) {
            double delay = calculate_exponential_backoff(attempt, base_backoff_ms);
            usleep((useconds_t)(delay * 1000)); // Convert ms to microseconds
         }

         bson_t error_reply = BSON_INITIALIZER;
         bool ok = fn(ctx, &error_reply);
         if (ok) {
            bson_destroy(&error_reply);
            return true;
         } else {
            bool is_retryable_overload_error =
            is_system_overloaded_error(&error_reply) && mongoc_error_has_label(&error_reply, retryable_error_label);
            bool can_retry = is_retryable_overload_error && attempt + 1 < max_attempts;

            // Apply the server-requested base backoff, if any, to the next attempt's delay.
            base_backoff_ms = get_base_backoff(&error_reply);
            bson_destroy(&error_reply);

            if (!can_retry) {
               return false;
            }
         }
      }
   return false;
   }
   ```

3) Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert()` or `find()` queries, with the `execute_with_retries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following code defines a `do_find` function that fetches all users from the `users` collection and processes each result.

   ```c
   static bool do_find(void *ctx, bson_t *error_reply) {
      mongoc_collection_t *users_collection = (mongoc_collection_t *)ctx;

      // Find and process results:
      const bson_t *result;
      bson_t filter = BSON_INITIALIZER;
      mongoc_cursor_t *cursor =
            mongoc_collection_find_with_opts(users_collection, &filter, NULL, NULL);
      while (mongoc_cursor_next(cursor, &result)) {
         process_result(result);
      }

      // Check for error:
      const bson_t *error_reply_local;
      if (mongoc_cursor_error_document(cursor, NULL, &error_reply_local)) {
         if (error_reply) {
            bson_destroy(error_reply);
            bson_copy_to(error_reply_local, error_reply);
         }
         mongoc_cursor_destroy(cursor);
         return false;
      }

      mongoc_cursor_destroy(cursor);
      bson_destroy(&filter);
      return true;
   }
   ```

   The following code then executes this operation with retry logic:

   ```c
   int main(void) {
      mongoc_init();

      mongoc_client_t *client = mongoc_client_new("mongodb://localhost:27017");
      mongoc_collection_t *coll = mongoc_client_get_collection(client, "db", "users");

      // With retry:
      execute_with_retries(do_find, coll, MAX_ATTEMPTS_DEFAULT);

      mongoc_collection_destroy(coll);
      mongoc_client_destroy(client);
      mongoc_cleanup();
   }
   ```

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```javascript
   const RETRYABLE_ERROR_LABEL = 'RetryableError';
   const SYSTEM_OVERLOADED_ERROR = 'SystemOverloadedError';

   function isSystemOverloadedError(error: unknown): error is MongoError {
   return error instanceof MongoError && error.hasErrorLabel(SYSTEM_OVERLOADED_ERROR);
   }
   ```

2. Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any async operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculateExponentialBackoff()`. Adjust these values to tune the retry behavior for your application.

   ```javascript
   import { setTimeout } from 'node:timers/promises';
   import { MongoServerError } from 'mongodb';

   const BASE_BACKOFF_MS = 100;
   const MAX_BACKOFF_MS = 10_000;

   // Only an overload error that is also labelled retryable is safe to retry.
   function isRetryableOverloadError(error: unknown): boolean {
   return isSystemOverloadedError(error) && error.hasErrorLabel(RETRYABLE_ERROR_LABEL);
   }

   // The server may attach a positive `baseBackoffMS` to an overload error to replace
   // the default base backoff. A value of 0 means "use your own default".
   function getBaseBackoffMS(error: unknown): number {
   // `errorResponse` exposes the raw server reply. It was added in driver 6.5.0, so guard
   // for it -- on older drivers the property is simply absent.
   if (!(error instanceof MongoServerError) || error.errorResponse == null) {
      return BASE_BACKOFF_MS;
   }

   // `baseBackoffMS` is an int64 on the wire, so its runtime type depends on your BSON
   // options: `number` by default, `Long` under `promoteLongs: false`, `bigint` under
   // `useBigInt64: true`. `Number()` handles all three, and yields NaN when absent.
   const baseBackoffMS = Number(error.errorResponse.baseBackoffMS);
   return Number.isFinite(baseBackoffMS) && baseBackoffMS > 0 ? baseBackoffMS : BASE_BACKOFF_MS;
   }

   async function executeWithRetries<T>(fn: () => Promise<T>, maxAttempts = 2): Promise<T> {
   let lastError: unknown;

   for (let attempt = 0; attempt < maxAttempts; attempt++) {
      try {
         return await fn();
      } catch (error) {
         lastError = error;

         if (!isRetryableOverloadError(error) || attempt + 1 >= maxAttempts) throw error;

         const baseBackoffMS = getBaseBackoffMS(error);
         const backoffMS =
         Math.random() * Math.min(MAX_BACKOFF_MS, baseBackoffMS * 2 ** (attempt + 1));

         await setTimeout(backoffMS);
      }
   }

   throw lastError;
   }
   ```

3. Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert()` or `find()` queries, with the `executeWithRetries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, consider the following query which fetches all users from the `users` collection and returns them as an array:

   ```javascript
   const users = await usersCollection.find().toArray();
   ```

   You can modify this code to use the new `executeWithRetries` function:

   ```javascript
   const users = await executeWithRetries(() => usersCollection.find().toArray());
   ```

1) Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```rust
   fn is_system_overloaded_error(error: &mongodb::error::Error) -> bool {
       const SYSTEM_OVERLOADED_LABEL: &str = "SystemOverloadedError";
       error.contains_label(SYSTEM_OVERLOADED_LABEL)
   }
   ```

2) Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculate_exponential_backoff()`. Adjust these values to tune the retry behavior for your application.

   ```rust
   const DEFAULT_BASE_BACKOFF: Duration = Duration::from_millis(100);
   const MAX_BACKOFF: Duration = Duration::from_millis(10_000);
   const MAX_ATTEMPTS: u32 = 2;

   fn calculate_exponential_backoff(attempt: u32, base_backoff: Option<Duration>) -> Duration {
       let base_backoff = base_backoff.unwrap_or(DEFAULT_BASE_BACKOFF);
       let backoff = std::cmp::min(base_backoff * 2u32.pow(attempt), MAX_BACKOFF);
       let jitter = rand::random::<f32>();
       backoff.mul_f32(jitter)
   }

   fn is_retryable_error(error: &mongodb::error::Error) -> bool {
       const RETRYABLE_ERROR_LABEL: &str = "RetryableError";
       error.contains_label(RETRYABLE_ERROR_LABEL)
   }

   async fn execute_with_retries<F, R>(operation: F) -> mongodb::error::Result<R>
   where
       F: AsyncFn() -> mongodb::error::Result<R>,
   {
       let mut attempt = 0;
       loop {
           attempt += 1;
           match operation().await {
               Ok(result) => return Ok(result),
               Err(error) => {
                   let is_retryable_overload_error =
                       is_system_overloaded_error(&error) && is_retryable_error(&error);
                   let is_last_attempt = attempt == MAX_ATTEMPTS;
                   if !is_retryable_overload_error || is_last_attempt {
                       return Err(error);
                   }
                   let backoff = calculate_exponential_backoff(attempt, error.base_backoff());
                   tokio::time::sleep(backoff).await;
               }
           }
       }
   }
   ```

3) Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert()` or `find()` queries, with the `execute_with_retries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, consider the following query which fetches all users from the `users` collection and returns them as an array:

   ```rust
   use futures::TryStreamExt;
   let users: Vec<Document> = users_collection.find(doc! {}).await?.try_collect().await?;
   ```

   You can modify this code to use the new `execute_with_retries` function:

   ```rust
   use futures::TryStreamExt;
   let users: Vec<Document> = execute_with_retries(|| async {
       users_collection.find(doc! {}).await?.try_collect().await
   })
   .await?;
   ```

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```cpp
   constexpr auto k_retryable_error_label = "RetryableError";
   constexpr auto k_system_overloaded_error_label = "SystemOverloadedError";

   bool is_system_overloaded_error(mongocxx::operation_exception const& e) {
     return e.has_error_label(k_system_overloaded_error_label);
   }
   ```

2. Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `k_base_backoff`, `k_max_backoff`, and the exponential growth factor in `calculate_exponential_backoff()`. Adjust these values to tune the retry behavior for your application.

   ```cpp
   using milliseconds_t = std::chrono::duration<double, std::milli>;

   constexpr auto k_base_backoff = milliseconds_t(100.0);
   constexpr auto k_max_backoff = milliseconds_t(10000.0);
   constexpr auto k_max_attempts_default = 2;

   double random_01() {
     static std::mt19937 gen(std::random_device{}());
     static std::uniform_real_distribution<double> dist(0.0, 1.0);
     return dist(gen);
   }

   milliseconds_t calculate_exponential_backoff(int attempt, milliseconds_t base_backoff) {
     return random_01() * std::min(k_max_backoff, base_backoff * std::pow(2.0, attempt));
   }

   // get_base_backoff returns the base backoff to apply for an overload error. A server may attach a
   // positive `baseBackoffMS` to the error to replace the default base backoff.
   milliseconds_t get_base_backoff(mongocxx::operation_exception const& e) {
     if (auto const& error_reply = e.raw_server_error()) {
       auto const elem = error_reply->view()["baseBackoffMS"];

       if (elem && (elem.type() == bsoncxx::type::k_int32 || elem.type() == bsoncxx::type::k_int64)) {
         auto const base_backoff_ms = elem.type() == bsoncxx::type::k_int32
                                       ? static_cast<std::int64_t>(elem.get_int32().value)
                                       : elem.get_int64().value;

         if (base_backoff_ms > 0) {
           return milliseconds_t(static_cast<double>(base_backoff_ms));
         }
       }
     }

     return k_base_backoff;
   }

   using retryable_fn_t = std::function<void()>;

   void execute_with_retries(retryable_fn_t fn, int max_attempts = k_max_attempts_default) {
     auto base_backoff_ms = k_base_backoff;

     for (int attempt = 0; attempt < max_attempts; ++attempt) {
       auto const is_retry = attempt > 0;

       if (is_retry) {
         auto const delay = calculate_exponential_backoff(attempt, base_backoff_ms);
         std::this_thread::sleep_for(delay);
       }

       try {
         fn();
         return;

       } catch (mongocxx::operation_exception const& e) {
         auto const is_retryable_overload_error =
           is_system_overloaded_error(e) && e.has_error_label(k_retryable_error_label);
         auto const can_retry = is_retryable_overload_error && attempt + 1 < max_attempts;

         // Apply the server-requested base backoff, if any, to the next attempt's delay.
         base_backoff_ms = get_base_backoff(e);

         if (!can_retry) {
           throw;
         }
       }
     }
   }
   ```

3. Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert()` or `find()` queries, with the `execute_with_retries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following operation fetches all users from the `users` collection and processes each result. Without retry logic, an overload error ends the operation:

   ```cpp
   int main() {
     using namespace bsoncxx::builder::basic;

     auto instance = mongocxx::instance();
     auto client = mongocxx::client(mongocxx::uri("mongodb://localhost:27017"));
     auto users_collection = client["db"]["users"];

     auto cursor = users_collection.find(make_document());
     for (auto const& res : cursor) {
       process_result(res);
     }
   }
   ```

   To retry the operation, pass it to `execute_with_retries()` as a lambda:

   ```cpp
   int main() {
     using namespace bsoncxx::builder::basic;

     auto instance = mongocxx::instance();
     auto client = mongocxx::client(mongocxx::uri("mongodb://localhost:27017"));
     auto users_collection = client["db"]["users"];

     execute_with_retries([&]() {
       auto cursor = users_collection.find(make_document());
       for (auto const& res : cursor) {
         process_result(res);
       }
     });
   }
   ```

1) Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```csharp
   const string RetryableErrorLabel = "RetryableError";
   const string SystemOverloadedErrorLabel = "SystemOverloadedError";

   bool IsSystemOverloadedError(MongoException error)
   {
       if (error == null)
       {
           return false;
       }
       return error.HasErrorLabel(SystemOverloadedErrorLabel);
   }
   ```

2) Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BaseBackoffMs`, `MaxBackoffMs`, and the exponential growth factor in `CalculateExponentialBackoffMs()`. Adjust these values to tune the retry behavior for your application.

   ```csharp
   const double BaseBackoffMs = 100;
   const double MaxBackoffMs = 10_000;

   // Only an overload error that is also labelled retryable is safe to retry.
   bool IsRetryableOverloadError(MongoException error)
       => IsSystemOverloadedError(error) && error.HasErrorLabel(RetryableErrorLabel);

   // The server may attach a positive `baseBackoffMS` to an overload error to replace
   // the default base backoff.
   double GetBaseBackoffMs(MongoException error)
   {
       if (error is MongoCommandException { Result: { } result } &&
           result.TryGetValue("baseBackoffMS", out var value) &&
           value.BsonType is BsonType.Int32 or BsonType.Int64 &&
           value.ToInt64() > 0)
       {
           return value.ToInt64();
       }

       return BaseBackoffMs;
   }

   // Random.Shared requires .NET 6+. On earlier TFMs (e.g. net472) substitute
   // a thread-safe Random.
   double CalculateExponentialBackoffMs(int attempt, double baseBackoffMs)
       => Random.Shared.NextDouble() * Math.Min(MaxBackoffMs, baseBackoffMs * Math.Pow(2, attempt));

   async Task<T> ExecuteWithRetriesAsync<T>(
       Func<CancellationToken, Task<T>> fn,
       int maxAttempts = 3,
       CancellationToken cancellationToken = default)
   {
       for (var attempt = 1; ; attempt++)
       {
           try
           {
               return await fn(cancellationToken).ConfigureAwait(false);
           }
           catch (Exception ex)
           {
               // Overload errors raised during the connection handshake are wrapped in
               // MongoAuthenticationException.
               var error = (ex is MongoAuthenticationException authEx ? authEx.InnerException : ex)
                   as MongoException;

               if (!IsRetryableOverloadError(error) || attempt >= maxAttempts)
               {
                   throw;
               }

               var delayMs = CalculateExponentialBackoffMs(attempt, GetBaseBackoffMs(error));
               await Task.Delay(TimeSpan.FromMilliseconds(delayMs), cancellationToken)
                   .ConfigureAwait(false);
           }
       }
   }
   ```

3) Use the retry helper for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `Find()` or `InsertOne()` calls, with the `ExecuteWithRetriesAsync()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following query fetches all users from the `users` collection and returns them as a list:

   ```csharp
   var users = await usersCollection.Find(Builders<User>.Filter.Empty)
       .ToListAsync();
   ```

   The following code then executes this operation with retry logic:

   ```csharp
   var users = await ExecuteWithRetriesAsync(
       (ct) => usersCollection.Find(Builders<User>.Filter.Empty).ToListAsync(ct),
       cancellationToken: CancellationToken.None);
   ```

You can also implement the retry logic with [Polly](https://github.com/App-vNext/Polly), a .NET library for implementing operations reliability. The following steps show how to build a Polly resilience pipeline that applies the same overload error handling:

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label. The function also unwraps overload errors raised during the connection handshake, which the driver wraps in `MongoAuthenticationException`:

   ```csharp
   const string RetryableErrorLabel = "RetryableError";
   const string SystemOverloadedErrorLabel = "SystemOverloadedError";

   bool IsMongoSystemOverloadedError(Exception exception, out MongoException mongoException)
   {
       if (exception is MongoAuthenticationException mongoAuthenticationException)
       {
           exception = mongoAuthenticationException.InnerException;
       }

       mongoException = exception as MongoException;
       if (mongoException == null)
       {
           return false;
       }

       return mongoException.HasErrorLabel(SystemOverloadedErrorLabel);
   }
   ```

2. Define the resilience pipeline

   Build a Polly resilience pipeline that retries operations returning retryable overload errors with exponential backoff and jitter:

   ```csharp
   const double BaseBackoffMs = 100;
   const double MaxBackoffMs = 10_000;

   // The server may attach a positive `baseBackoffMS` to an overload error to replace
   // the default base backoff.
   double GetBaseBackoffMs(Exception exception)
   {
       if (exception is MongoAuthenticationException mongoAuthenticationException)
       {
           exception = mongoAuthenticationException.InnerException;
       }

       if (exception is MongoCommandException { Result: { } result } &&
           result.TryGetValue("baseBackoffMS", out var value) &&
           value.BsonType is BsonType.Int32 or BsonType.Int64 &&
           value.ToInt64() > 0)
       {
           return value.ToInt64();
       }

       return BaseBackoffMs;
   }

   var pipeline = new ResiliencePipelineBuilder()
       .AddRetry(new RetryStrategyOptions
       {
           // 1. Retry if SystemOverloadedError
           ShouldHandle = args =>
               IsMongoSystemOverloadedError(args.Outcome.Exception, out var mongoException)
                   && mongoException.HasErrorLabel(RetryableErrorLabel)
                   ? PredicateResult.True()
                   : PredicateResult.False(),
           // 2. Retry 2 times
           MaxRetryAttempts = 2, // (1 initial try + 2 retries)
           // 3. Exponential backoff with jitter, capped at 10sec, based on the
           //    server-supplied baseBackoffMS when present. AttemptNumber is 0 on the
           //    first retry.
           DelayGenerator = args =>
           {
               var baseBackoffMs = GetBaseBackoffMs(args.Outcome.Exception);
               var delayMs = Random.Shared.NextDouble()
                   * Math.Min(MaxBackoffMs, baseBackoffMs * Math.Pow(2, args.AttemptNumber + 1));
               return new ValueTask<TimeSpan?>(TimeSpan.FromMilliseconds(delayMs));
           }
       })
       .Build();
   ```

3. Use the pipeline for collection operations

   Execute operations with the pipeline. The pipeline retries any operation that returns a retryable overload error and surfaces all other errors immediately.

   For example, the following query fetches all documents from the `collection` and returns them as a list:

   ```csharp
   var documents = await collection.Find(Builders<BsonDocument>.Filter.Empty)
       .ToListAsync();
   ```

   The following code then executes this operation with retry logic:

   ```csharp
   var documents = await pipeline.ExecuteAsync(async token => await collection
       .Find(Builders<BsonDocument>.Filter.Empty).ToListAsync(token), CancellationToken.None);
   ```

1) Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```go
   const errSystemOverloadedError = "SystemOverloadedError"

   func isSystemOverloadedError(err error) bool {
       var lerr mongo.LabeledError
       return errors.As(err, &lerr) && lerr.HasErrorLabel(errSystemOverloadedError)
   }
   ```

2) Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `baseBackoff`, `maxBackoff`, and the exponential growth factor in `overloadBackoff()`. Adjust these values to tune the retry behavior for your application.

   ```go
   const (
       baseBackoff = 100 * time.Millisecond
       maxBackoff  = 10_000 * time.Millisecond

       errRetryableError = "RetryableError"
   )

   // serverBaseBackoff returns the base backoff that the server attached
   // to the error as "baseBackoffMS", or 0 if the server did not supply
   // one. A positive value replaces the client's default base backoff.
   func serverBaseBackoff(err error) time.Duration {
       // For command errors, "baseBackoffMS" is a top-level field of the
       // server response.
       var cerr mongo.CommandError
       if errors.As(err, &cerr) {
           if ms, ok := cerr.Raw.Lookup("baseBackoffMS").AsInt64OK(); ok {
               return time.Duration(ms) * time.Millisecond
           }
       }

       // For write errors, "baseBackoffMS" is a field of the
       // "writeConcernError" subdocument, not of the top-level response.
       var wex mongo.WriteException
       if errors.As(err, &wex) && wex.WriteConcernError != nil {
           if ms, ok := wex.WriteConcernError.Raw.Lookup("baseBackoffMS").AsInt64OK(); ok {
               return time.Duration(ms) * time.Millisecond
           }
       }

       return 0
   }

   // overloadBackoff returns the backoff duration for the given retry
   // attempt by doubling the base backoff once per attempt, capped at
   // maxBackoff.
   func overloadBackoff(base time.Duration, attempt int) time.Duration {
       d := base
       for i := 0; i < attempt && d < maxBackoff; i++ {
           d *= 2
       }
       if d > maxBackoff {
           d = maxBackoff
       }
       return d
   }

   // jitterDuration returns the input duration weighted by a
   // pseudo-random ratio in [0.0, 1.0). func jitterDuration(d time.Duration) time.Duration { return time.Duration(float64(d) * rand.Float64()) } // executeWithRetries executes the given function with retries if it // returns a SystemOverloadedError. func executeWithRetries[T any](
       ctx context.Context, maxAttempts int,
       fn func(ctx context.Context) (T, error),
   ) (T, error) {
       var result T
       var err error
       for attempts := 0; attempts < maxAttempts; attempts++ {
           isRetry := attempts > 0

           // The first attempt runs immediately. Every subsequent
           // attempt waits for an exponentially increasing backoff based
           // on the error returned by the previous attempt, with jitter.
           if isRetry {
               // Prefer the base backoff supplied by the server over the
               // client's default, if there is one.
               base := baseBackoff
               if serverBase := serverBaseBackoff(err); serverBase > 0 {
                   base = serverBase
               }

               sleep := time.NewTimer(jitterDuration(overloadBackoff(base, attempts)))
               select {
               case <-ctx.Done():
                   sleep.Stop()
                   if err == nil {
                       err = ctx.Err()
                   }
                   return result, err
               case <-sleep.C:
               }
           }

           result, err = fn(ctx)
           if err == nil {
               break
           }
           if !isSystemOverloadedError(err) {
               break
           }
           var lerr mongo.LabeledError
           if !errors.As(err, &lerr) || !lerr.HasErrorLabel(errRetryableError) {
               break
           }
       }
       return result, err
   }
   ```

3) Use the retry helper for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `Find()` or `InsertOne()` calls, with the `executeWithRetries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following operation fetches all users from the `users` collection:

   ```go
   ctx := context.Background()
   client, _ := mongo.Connect(ctx)
   coll := client.Database("db").Collection("coll")

   cursor, err := coll.Find(ctx, bson.D{})
   if err != nil {
       log.Fatal(err)
   }
   var users []bson.D
   if err = cursor.All(ctx, &users); err != nil {
       log.Fatal(err)
   }
   ```

   The following code then executes this operation with retry logic:

   ```go
   const defaultMaxAttempts = 2

   users, err = executeWithRetries(ctx, defaultMaxAttempts, func(ctx context.Context) ([]bson.D, error) {
       cursor, err := coll.Find(ctx, bson.D{})
       if err != nil {
           return nil, err
       }
       var res []bson.D
       err = cursor.All(ctx, &res)
       return res, err
   })
   if err != nil {
       log.Fatalf("Unhandled error: %v", err)
   }
   fmt.Printf("found %v\n", users)
   ```

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```java
   private static final String RETRYABLE_ERROR_LABEL = "RetryableError";
   private static final String SYSTEM_OVERLOADED_ERROR_LABEL = "SystemOverloadedError";

   // Only an overload error that is also labelled retryable is safe to retry.
   static boolean isRetryableOverloadError(final Throwable error) {
       return error instanceof MongoException
               && ((MongoException) error).hasErrorLabel(SYSTEM_OVERLOADED_ERROR_LABEL)
               && ((MongoException) error).hasErrorLabel(RETRYABLE_ERROR_LABEL);
   }
   ```

2. Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculateExponentialBackoffMs()`. Adjust these values to tune the retry behavior for your application.

   ```java
   private static final double BASE_BACKOFF_MS = 100.0;
   private static final double MAX_BACKOFF_MS = 10_000.0;

   private static double calculateExponentialBackoffMs(final int attempt, final double baseBackoffMs) {
       double jitter = ThreadLocalRandom.current().nextDouble(); // [0.0, 1.0) return jitter * Math.min(MAX_BACKOFF_MS, baseBackoffMs * Math.pow(2, attempt)); } // The server may attach a positive `baseBackoffMS` to an overload error to replace // the default base backoff. A value of 0 or absent means "use your own default". private static double getBaseBackoffMs(final Throwable error) { if (error instanceof MongoCommandException) { BsonDocument response = ((MongoCommandException) error).getResponse(); if (response.containsKey("baseBackoffMS")) { double baseBackoffMs = response.getNumber("baseBackoffMS").doubleValue(); return baseBackoffMs; } } return BASE_BACKOFF_MS; } static <T> T executeWithRetries(final Callable<T> body, final int maxRetryAttempts) throws Exception { if (maxRetryAttempts < 0) { throw new IllegalArgumentException("maxRetryAttempts cannot be negative"); } Exception lastError = null; for (int attempt = 0; attempt <= maxRetryAttempts; attempt++) { try { return body.call(); } catch (Exception error) { lastError = error; if (!isRetryableOverloadError(error) || attempt >= maxRetryAttempts) { break; } // Back off before the next attempt, using the server-supplied base backoff if any. double delayMs = calculateExponentialBackoffMs(attempt + 1, getBaseBackoffMs(error)); Thread.sleep((long) delayMs); } } throw lastError; } ``` 3. Use the retrier for collection operations Call the retrier from your application code. Wrap individual operations, such as `insertOne()` or `find()` calls, with the `executeWithRetries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately. For example, the following operation fetches a document from the `users` collection: ```java Document document = collection.find(Filters.empty()).first(); ``` The following code then executes this operation with retry logic: ```java private static final int MAX_RETRY_ATTEMPTS = 2; executeWithRetries(() -> collection.find(Filters.empty()).first(), MAX_RETRY_ATTEMPTS); ``` 1) Detect overload errors Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label: ```php function isSystemOverloadedError(Throwable $exception): bool { return $exception instanceof \MongoDB\Driver\Exception\RuntimeException && $exception->hasErrorLabel('SystemOverloadedError'); } ``` 2) Implement operation "retry" logic using exponential backoff and jitter Create a retrier function that wraps any operation you want to protect. The function does the following: - Retries only overload errors that are safe to retry. - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in the `$backoffMs` calculation. Adjust these values to tune the retry behavior for your application.

   ```php
   const BASE_BACKOFF_MS = 100;
   const MAX_BACKOFF_MS = 10000;

   function executeWithRetries(\Closure $closure, int $maxAttempts = 2)
   {
       for ($attempt = 0; true; $attempt++) {
           try {
               return $closure();
           } catch (\MongoDB\Driver\Exception\RuntimeException $exception) {
               if (
                   $attempt + 1 >= $maxAttempts
                   || ! isSystemOverloadedError($exception)
                   || ! $exception->hasErrorLabel('RetryableError')
               ) {
                   throw $exception;
               }

               $reply = null;

               if ($exception instanceof \MongoDB\Driver\Exception\CommandException) {
                   $reply = $exception->getResultDocument();
               } elseif ($exception instanceof \MongoDB\Driver\Exception\BulkWriteCommandException) {
                   $errorReply = $exception->getErrorReply();
                   $reply = $errorReply === null ? null : $errorReply->toPHP();
               }

               $baseBackoffMs = (int) (($reply->baseBackoffMS ?? 0) ?: BASE_BACKOFF_MS);

               $jitter = random_int(0, 2 ** 53) / 2 ** 53;
               $backoffMs = (int) ($jitter * min(MAX_BACKOFF_MS, $baseBackoffMs * 2 ** ($attempt + 1)));

               usleep($backoffMs * 1000);
           }
       }
   }
   ```

3) Use the retry helper for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `find()` or `insertOne()` calls, with the `executeWithRetries()` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following query fetches all users from the `users` collection and returns them as an array:

   ```php
   $users = $usersCollection->find()->toArray();
   ```

   The following code then executes this operation with retry logic:

   ```php
   $users = executeWithRetries(fn () => $usersCollection->find()->toArray());
   ```

1. Detect overload errors

   Create a function to identify server overload errors. The function checks for the `SystemOverloadedError` label. You can use this check to apply retry logic only to operations that MongoDB rejected based on the `SystemOverloadedError` label:

   ```ruby
   RETRYABLE_ERROR_LABEL = 'RetryableError'
   SYSTEM_OVERLOADED_ERROR = 'SystemOverloadedError'

   def system_overloaded_error?(error)
     error.respond_to?(:label?) && error.label?(SYSTEM_OVERLOADED_ERROR)
   end

   # Only an overload error that is also labeled retryable is safe to retry.
   def retryable_overload_error?(error)
     system_overloaded_error?(error) && error.label?(RETRYABLE_ERROR_LABEL)
   end
   ```

2. Implement operation "retry" logic using exponential backoff and jitter

   Create a retrier function that wraps any operation you want to protect. The function does the following:

   - Retries only overload errors that are safe to retry.

   - Waits longer between each attempt using [exponential backoff](https://en.wikipedia.org/wiki/Exponential_backoff) with [jitter](https://en.wikipedia.org/wiki/Jitter).

   **Note:**

   The following code uses arbitrary values for `BASE_BACKOFF_MS`, `MAX_BACKOFF_MS`, and the exponential growth factor in `calculate_exponential_backoff`. Adjust these values to tune the retry behavior for your application.

   ```ruby
   BASE_BACKOFF_MS = 100
   MAX_BACKOFF_MS = 10_000

   # The server may attach a positive `baseBackoffMS` to an overload error's
   # reply document to replace the default base backoff.
   def base_backoff_ms(error)
     document = error.respond_to?(:document) && error.document
     value = document && document['baseBackoffMS']
     value if value && value > 0
   end

   def calculate_exponential_backoff(attempt, base_backoff_ms = nil)
     rand * [ MAX_BACKOFF_MS, (base_backoff_ms || BASE_BACKOFF_MS) * (2**attempt) ].min
   end

   def execute_with_retries(max_attempts: 2)
     attempt = 0
     begin
       yield
     rescue StandardError => e
       raise unless retryable_overload_error?(e)

       attempt += 1
       raise if attempt >= max_attempts

       delay = calculate_exponential_backoff(attempt, base_backoff_ms(e))
       sleep(delay / 1000.0)
       retry
     end
   end
   ```

3. Use the retrier for collection operations

   Call the retrier from your application code. Wrap individual operations, such as `insert` or `find` queries, with the `execute_with_retries` function so that MongoDB retries overload errors with backoff and surfaces all other errors immediately.

   For example, the following query fetches all users from the `users` collection and returns them as an array:

   ```ruby
   users = users_collection.find.to_a
   ```

   The following code then executes this operation with retry logic:

   ```ruby
   users = execute_with_retries { users_collection.find.to_a }
   ```

### Guidelines for Safe Overload Retries

When retrying operations that failed due to an overload error, use the following guidelines to avoid contributing to overload and to increase the chances of successful retries:

- **Limit your retry attempts:** Use a maximum of two retries per operation. Higher limits reduce error rates but increase server load during overload, while fewer retry attempts can reduce server load but increase errors rates.

- **Apply selectively:** Use this pattern only for latency-sensitive or business-critical operations. For background workloads, log errors and retry at a higher level with longer delays.