OSINT Gathering

The following is the process I’ve followed for gathering intel during security assessments I’ve participated in.  OSINT gathering shouldn’t require you to send any packets to your target organization and as such, can usually be done in advance to an assessment date.

People OSINT

1. Run theharvester and Maltego against the target domain (likely email domain) to generate a list of email addresses for importing into Recon-ng.
   • python theHarvester.py -d <targetDomain> -b all -v -f <outputFile>
   • import into Recon-ng using import/list
2. Run the following Recon-ng modules to check which users have been involved in any public credential leaks:
   • hibp_breach
   • hibp_paste
3. I use Recon-ng as the basis for storing the majority of the data gathered.  I’ll create a separate workspace for each assessment and start by uploading the results from step one and then run salesmaple to finish up the email address gathering.  I haven’t had much success with the Facebook or LinkedIn modules, so tend to do that research more manually.
4. For gathering intel on particular individuals (more for social engineering components of a security assessment), Michael Bazzell’s books and website have been invaluable.  Specifically his centralized search portal.  I highly recommend you at least watch the Free Video to get an idea of the power of some of the search tools in the portal.
   • The “Custom Search Tools” are what I’ve used the most.  The Facebook search tool can pull out all sorts of information that Facebook makes difficult to access if you are not friends with someone.  You do have to have to be logged into a Facebook account for this tool to work, and as a warning, whatever your account name is will likely show up on the target’s Facebook page suggesting that they become friends with you (and vice-a-versa); you may want to create an anonymous account (a good precaution for searches you make on any tools that require an account).
   • The Search Engines Tools is great for searching for a person’s name or other identifying information across a number of search engines (you do need to make sure you allow pop-ups if you want the results for each browser to be displayed in separate tabs, otherwise you can only search one at a time).
   • The Person Search, Username Search, Pastebin Search, Document Search, and others have been very useful.
5. An additional resource for researching a specific person is any database that your Library has access to.  I can login to several different premium background search databases through different public library portals free of charge.
6. Some premium services cost very little per month and are another resource (e.g., intelius.com, pipl.com, spokeo.com)
7. Some other resources for gathering information on individuals
   • Search for Amazon wishlists
   • wigle.net
   • echosec.net
   • articles published by local newspapers

Technology OSINT

1. Searching whois for netblocks belonging to the target organization is good place to start.  Be creative with how you search for the target entity’s name, often times the names are truncated or just different altogether.  You can use an asterisk (*) in your searches as a wildcard.  Recon-ng does a decent job at querying whois [whois_miner], provided you have a variety of company names to search by in your companies table.
2. Load any known domains into your Recon-ng db (or use modules to populate them) and run the following modules:
   • netblocks-hosts/reverse_resolve
   • google_site_web
   • yahoo_domain
   • baidu_site
   • bing_domain_web
   • hosts-hosts/resolve
   • pgp_search
   • salesmaple
   • census_2012 (dated, but could be useful)
   • xssed
   • xssposed
   • netcraft
   • ssl_san
   • punkspider
   • builtwith (this is best run if you redirect the output to a text file and then use some sed/awk magic to strip out the technologies in use, otherwise this data doesn’t get stored in the db)
   • vpnhunter
3. Maltego is a great resource.  Running any of their machines on various entities can provide a wealth of information.
4. Censys.io is another resource and relatively new.  It has an API and Recon-ng has a module for it, but it wasn’t working at the time of this post.  You can get scan data on your target without sending any packets.