Great reading from our friends at the CIA:

Not only are open sources increasingly accessible, ubiquitous, and valuable, but they can shine in particular against the hardest of hard targets. OSINT is at times the “INT” of first resort, last resort, and every resort in between.

https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/vol48no3/article05.html

Intelligence Gathering, Reconnaissance, Information Collection… No matter what you call it, it’s an important component of any security assessment project.

Intelligence Gathering:  The collection of intelligence both overt and covert to aid in the decision of a course of action.

Intelligence Gathering (IG) is often viewed and approached as the first step of an assessment project. A penetration tester will diligently scan the target’s website, gather DNS information, check Google for email addresses and they might even check SHODAN for exploitable hosts.

Unfortunately, this is often where the Intelligence Gathering stops. The assessor now has enough information to move on to the “Active Scanning” or “Exploitation” phases, suddenly ignoring that they will need to continuously perform IG on new information throughout an assessment.

… So what is is Intelligence Gathering at it’s core? There are a number of recognized disciplines within the scope of Intelligence Gathering. The most reconizable of these is Open-source intelligence (OSINT), or Intelligence Gathering performed on publicly available sources. In the Intelligence Community (IC), the term “open” refers to overt, publicly available sources (as opposed to covert or clandestine sources);

We often focus on OSINT, but there are others such as SIGINT and HUMINT that are often left untouched when assessing security of an entity since they may not be relevant, in scope, or within the control of the entity that commissioned the assessment.

The process can be difficult to scope – until you’ve gained enough information to capture your goal, you’ll continue to gather intelligence and analyze it, filtering it into a model of the target. “Enough” IG largely depends on the goals of the application for which its used. If you’ve not been successful at gaining your target, then you have more to do.

Performing Intelligence Gathering at scale can also be challenging. A small business or organization can consist of thousands of entities which may, or may not be relevant during an assessment. An enterprise, made up of thousands, if not millions of entities and the relationships between them is simply mind-boggling and impossible to process with traditional techniques. This is truly a “big data” problem.

Our mission is to make Intelligence Gathering and Analysis simple, and support the assessment efforts of security professionals.