What a blast that was!? Thanks for the invite @Shane_McAllister… awesome job @Mark_Smith… If you’re reading this and want to skip around… I put this handy time index together.
0:00 - 13:48 - Introduction and Information about the Hackathon (Mike) 13:49 - 22:10 - Introduction to GDELT Dataset 22:11 - 25:46 - Launch a MongoDB Atlas Cluster 25:47 - 27:05 - Connecting to the MongoDB Atlas Database (Obtaining the connection string) 27:06 - 27:33 - Username and password information 27:49 - 28:40 - Network security information - IP Access List Entry 30:50 - 31:55 - Installing and using python venv 32:13 - 32: 55 - gdelttools install 32:55 - 33:30 - pip tip * don’t skip this 33:59 - 34:29 - Running gdelttools - getting help, obtaining master file with gdeltloader 34:30 - 36:11 - Master file (77mb) 36:58 - 37:32 - Looking at exports - last 20 files with gdeltloader --master --download --overwrite --last 20 --filter export 38:08 - 39:49 - Looking at the data from a GDELT export in TSV format 41:05 - 41:30 - Cloning the gdelttools repo - to get some additional helpful tools 41:31 - 42:26 - Looking at the gdelt_field_file.ff - formats and field types for the GDELT data 42:27 - 42:52 - mongoimport.sh - Script to import GDELT data into mongodb 42:53 - 45:33 - Modifying mongoimport.sh to remove hard reference to database name 45:34 - 45:54 - Running mongoimport.sh - importing data into your MongoDB Atlas cluster 45:55 - 48:42 - Viewing data in Atlas - Browse Collections 48:43 - 51:30 - Optimizing mongoimport 51:31 - Transforming the imported data into improved, document-style shapes using gdelt_reshaper.js
