Hi, as a sysadmin I have to manage a web application based on a MongoDB database which has hugely grown over time. Miscellaneous files are stored in the database: images and documents. They are stored using GridFS in the fs.chunks collection, but it is now more than 4 TB in size.
By reading MongoDB’s documentation I see this actually is good practice, because this way files are easily replicated too, and because of easy access to file parts, etc., but as a sysadmin I see it as a nightmare for backups: I cannot perform incremental file backups, and a full backup (mongodump) of the collection takes many hours.
So, together with the developers I’m investigating migration paths to external file storage solutions.
I see from other posts here that storing a reference to the file in the DB, and then uploading the file elsewhere and storing the final URL as metadata, is a feasible solution. Any caveats?
Does anyone have a first-hand experience to share on this topic? Including suggestions on external storage solutions, where incremental backups are possible, and where authentication from the web application can possibly be used to allow file access only to authorized users.
Thank you very much for any info!