Hi Shane, thanks for making me aware there is a ticket open for this.
In the meantime, for anyone reading this… the workaround will make my code not optimal, but instead of a generator of documents, I will use a list. Then I will create a reverse lookup dict (custom_id → document) before calling insert_many so if there is an OperationFailure I can look up the ID (extracted from the exception) and re-insert the oversized document in chunks. However, if there are multiple oversized documents, this is horrible because the OperationFailure only returns the last document ID that caused the error, not all IDs.
BTW, I tried checking the sizes of the documents before inserting but the bson.encode method is very slow. Not worth it for millions of documents.