There have been some recent messages popup around the depreciation of 3rd party services such as AWS and Twilio. I think this is going to cause people a lot of confusion so I would advise MongoDB to add some examples of using the AWS package as a dependency.
In my projects across multiple clients, I have utilised the 3rd party AWS service to do lots of things such as S3, SQS, SFN, SNS. I have therefore just explored how to implement this change.
The example below gives a sort of “hello world” on how to get the AWS SDK working.
The access key and secret are stored in realm using the secrets and values area.
The installed aws-sdk package is v2.737.0. More recent ones don’t seem to work yet and I found this out from @Drew_DiPalma post HERE
exports = async function(){
// Load the AWS SDK for Node.js
const AWS = require('aws-sdk');
// Set the AWS config
AWS.config = new AWS.Config();
AWS.config.accessKeyId = context.values.get("AWS_ACCESS_KEY");
AWS.config.secretAccessKey = context.values.get("AWS_ACCESS_SECRET");
AWS.config.region = context.values.get("AWS_REGION");
// Create S3 service object
s3 = new AWS.S3({apiVersion: '2006-03-01'});
// Call S3 to list the buckets
const buckets = await s3.listBuckets().promise()
return buckets
};
One thing I would like to know from the MongoDB realm team is what sort of overhead is this going to add to our functions? Is it going to slow them down considerably if we need to load in the AWS SDK? The 3rd party services worked well in my opinion and didn’t really add any overhead.
As I mentioned, it would be great if there could be some detailed documention with examples around these dependencies that people will be needing. Mainly around AWS and HTTP, so maybe an example of the library axios or node-fetch would be ideal.
Thanks for raising this query and I acknowledge it has been a while since you asked and I have good news to share
Please find the documentation link describing how 3rd party services can be replaced with npm modules.
One thing I would like to know from the MongoDB realm team is what sort of overhead is this going to add to our functions? Is it going to slow them down considerably if we need to load in the AWS SDK?
There should not be any difference in overhead between AWS service and aws-sdk. The service wraps the official Go SDK from amazon so under the hood, they are doing the same things.
I hope this helps. Please feel free to post a query should you run into issues.
I am seeing very similar results while working with the ‘aws-sdk’. Although, my results seem to be more random.
I have a simple function which signs URLs to upload to S3. In some cases the function returns in under 1 second and other cases it takes upwards of 15 seconds to return.
Could you try running the function with aws sdk version 3 like this and check if you get the same slow performance? const {S3Client, GetObjectCommand, PutObjectCommand} = require("@aws-sdk/client-s3");
Could you share the testing s3 file so we could try reproducing the same?
I’m unable to install that package. I get the following error info in the UI.
“failed to transpile node_modules/aws-crt/scripts/build.js. “aws-crt” is likely not supported yet. unknown: Unexpected reserved word ‘package’ (142:8)”
I did not specify a specific version. Just the package “@aws-sdk/client-s3”.
Thanks for sharing the result. This error happens if you click “Add New Dependency” to add it.
Could you try to use the old way (uploading node_modules.tar file) to install the dependency and let me know if the function still takes the same time?
Thanks a lot for the feedback, the “Add Dependency” should also work and the team is investigating that now.
When I try to install the module with the current version I receive the following error. Failed to install dependencies failed to transpile node_modules/aws-crt/scripts/build.js. "aws-crt" is likely not supported yet. unknown: Unexpected reserved word 'package' (145:8)
I receive this regardless of if I upload a node_modules.tar or if I install in the UI. The only version I can get to install is 2.737.0
I saw the deprecation yesterday too and I’m seeing increases of times on my replacement of context.http to node-fetch (v2) from 5-6ms to 800ms. (didn’t deploy the change after seeing the times) In my case I was just using the triggers + functions to send some slack notifications.
Even though it is a different use case the underlying problem might be similar.
As a quick update from the Realm engineering team – we’re actively working on tickets related to these performance issues and we expect to have a few significant improvements soon. While we have a date for removing 3rd Party Services, we are also going to continue monitoring usage and will be working with the community to make sure the deprecation date is fair, and extending if necessary.
Thank you for raising the query. This appears to be different than the aws-sdk issue. Could you please create a separate topic for this and share necessary details that can help us investigate.
exports = function(s3Path) {
const s3 = context.services.get("AWS").s3("ap-southeast-2");
const bucket = "my-bucket";
const presignedUrl = s3.PresignURL({
Bucket: bucket,
Key: s3Path,
// HTTP method that is valid for this signed URL. Can use PUT for uploads, or GET for downloads.
Method: "GET",
// Duration of the lifetime of the signed url, in milliseconds
ExpirationMS: 900000
});
return presignedUrl;
};