I have recently updated the Mongo Go driver from v1.3.5 to v1.4.6 and I have noticed that the vendor directory pulled some 35.5k lines of code. After further inspection, I noticed that most of the updated code has nothing to do with Mongo driver, but its dependency on AWS SDK (mostly Credentials and Signer). Please note that I’m not using AWS to run my code. This giant dependency is caused by a rather simple need for some AWS auth utilities and I assume it can be avoided if you replace direct dependency with an interface wrapper and provide different implementations in different packages. That way, I would pull only what I need and have a cleaner update with no need to add dependencies I don’t intend to use. The other option is to simply reimplement a small chunk of the actual SDK MongoDB Driver needs to avoid any dependencies whatsoever because what you need really does not justify having the entire AWS SDK in vendors.
I’m not sure I fully understand the proposal. AFAIK, Go projects must declare all of their dependencies in the go.mod file and the language will install and build all of the dependencies when compiling the project. In the driver, we do not know at compile time if the application is using AWS authentication, so we have to declare the dependency and unfortunately all of that code gets pulled in even if the authentication mechanism is not used. Can you elaborate on your proposal to show how it would address this issue?
Let me try to elaborate my idea:
First, what made me open this topic: over 35k lines of code for SKD is a lot. That implies that there are probably some issues with the SDK codebase, but also that it should be avoided as a dependency as much as possible. I’m not familiar with all the details of the MongoDB Driver codebase, but I noticed that only an authorization subset of SDK is used by the driver (these two packages, to be more precise: github.com/aws/aws-sdk-go/aws/credentials and github.com/aws/aws-sdk-go/aws/signer/v4).
My initial proposal used to be to replace structures from these packages with interfaces defined on the root level and pass them around as interfaces. Provide different implementations of those interfaces in a separate package and use only the implementation you need. It’s similar to using interfaces to enable creating mock objects.
However, I dug into the code yesterday a bit more and noticed that driver is using an interface abstraction called d Authenticator and AWS authenticator is used in init method in auth package to register authenticator. That can simply be avoided by forcing the user to pass the entire Authenticator instead of passing only the name. That way, the root package (go.mongodb.org/mongo-driver/mongo) won’t depend on AWS and AWS SDK will be added to dependencies only if AWS Authenticator is used.
Did anyone check on this idea? If you think it’s viable, I’m willing to send a PR, but I first need your confirmation that the proposed changes are acceptable.
Thanks for the reminder. I looked into this a bit earlier but didn’t come up with anything conclusive. I’ll investigate more on Monday and respond here.
I looked over your proposal. The actual logic in the AWS authenticator is complex and it would not be reasonable to push the responsibility of maintaining that code onto the user. We could potentially put it in a separate package and add an Authenticator option to the ClientOptions type in the future, but removing the builtin support would be considered a backwards-breaking behavioral change, which we can’t do without a major version bump of the driver (e.g. v1.0 → v2.0). If you think this is worth considering, please file a ticket in our Jira Project and we will consider it at the time of our next major version bump.
Thanks for the response. I agree that the idea of not using AWS SDK is not a way to go (the purpose of SDK is exactly to avoid that). By the way, looks like that dependency is also out of date on the MongoDB driver.
However, I think that the idea of adding Authenticator to ClientOptions would be a good choice. Unfortunately, there is probably no way to avoid breaking backward compatibility. Despite that, it’s more than worth it. I will open a Jira ticket. If I can help you with development, please let me know.