#Day04 of #100DaysOfSecurity
Let’s keep on the Web Exploitation track, and look at Scavenger Hunt.
At first glance you may say to yourself, “Why, self! This looks EXACTLY the same as Day 3’s challenge. This will be a cinch!”
And indeed it starts the same way—with what even looks like the exact same web page!—but this one requires a bit more poking around.
Hint
For this one, you’ll need knowledge about other common files found on web servers, not just those embedded in the page itself.
Beyond that, read the clues the puzzle gives you carefully; each one contains a distinct hint to point you in the right direction.
Walkthrough
Just like yesterday’s challenge, you can start piecing the flag together by viewing source on the HTML and CSS files and looking at the code comments.
However, you’ll hit a wall when you get to the JS file. Instead of the comment giving you a part of the flag string like before, it will instead ask a cryptic question:
/* How can I keep Google from indexing my website? */
There is a Robots exclusion standard that exists as a means to communicate with (well-behaving, non-malicious) web crawlers about which areas of the website should and should not be processed or scanned.
An example file might look like the following, if it wanted to tell ALL robots not to scan the “private” directory:
User-agent: *
Disallow: /private/
(Ironically, Google’s own documentation states in bold, red letters: " Warning : Don’t use a robots.txt file as a means to hide your web pages from Google search results." 
A better approach is a noindex metatag, as that removes the page even if it’s linked to from somewhere else vs. crawled by Google.)
ANYWAY. Once you load that file, you’re given another piece of the flag, as well as another cryptic clue:
# I think this is an apache server... can you Access the next flag?
Apache is a very common web server, and this clue refers to an Apache configuration file that lets you make configuration changes on a per-directory basis, overriding the default Apache configuration found in httpd.conf. You can do things in there such as require a password to access the directory contents or re-write URLs.
Once you load THAT file, you’re given another piece of the flag, as well as another cryptic clue:
# I love making websites on my Mac, I can Store a lot of information there.
Unlike the others, this one isn’t actually a common file found on web servers… at least, not on purpose. 
Desktop Services Store files are found inside every directory accessed by macOS Finder, and they contain information about the containing folder, including what file names are inside it [!], which can be parsed and crawled by an attacker to find files they ought not have access to.
They are also the bane of many web developers’ existence, because they are dotfiles, which means they are hidden by default and thus easily accidentally committed to version control or uploaded to a web server. 
At any rate, throw that file name at the end of the URL and you’ve got the final part of your flag.