OSINT Toolbox Talk: Streamline your OSINT activity, visualise Twitter activity and extract Instagram content

OSINT Tool Review

Streamlining your OSINT collection workflow with 'Scrummage'

Streamlining your OSINT collection workflow with 'Scrummage' Streamlining your OSINT collection workflow with 'Scrummage' https://github.com/matamorphosis/Scrummage

Every so often we showcase web applications which we believe has both long-term potential for development in addition to relevance for Digital Investigators from an OSINT collection perspective. In this OSINT Tool Review, we introduce ‘Scrummage’, a very impressive OSINT and Threat Hunting framework that has been developed with the aim to bring the ‘OSINT Framework’, an online repository of sites and sources that can be used for OSINT purposes, to life within an open-source web-based application.

So, in a nutshell, Scrummage is a centralised OSINT search platform that can search, collect and report data from a varied range of web and social media sites including:

  • Apple Store
  • Business Search
  • SSLMate
  • Craigslist
  • eBay
  • Emailrep
  • Flickr
  • Google Search
  • Have I Been Pwned (Accounts, Breaches, Emails and Passwords)
  • Hunter.io
  • intelX
  • ipstack
  • Naver Search
  • Odnoklassniki (Groups and Users)
  • Pinterest
  • Reddit
  • Shodan (IP addresses and Queries)
  • Tumblr
  • Twitter
  • Virus Total (Domains, File Hashes, IP Addresses and URLs)
  • VKontakte (Users and Groups)
  • Vulners
  • Windows Store
  • Yandex
  • YouTube

Each of the search capabilities listed above is executed by building a ‘Task’ within the Scrummage dashboard and then executing it. The ‘Task’ can be further configured to run according to a set number of frequencies and limits. The results from each of the ‘Tasks’ are then published across a variety of formats including PDF and JSON. Additionally, Scrummage can also be integrated with various third-party tools, enabling it to output search results into the following frameworks:

  • DefectDojo
  • ElasticSearch
  • Email
  • JIRA
  • Request Tracker Incident Response (RTIR)
  • Scrumblr
  • Slack Channel

Regarding the installation and deployment of Scrummage, this can be somewhat of a complex process depending on which installation method you choose. The framework currently supports Debian, RHEL and SUSE-based Linux distributions, though installation via these distributions is quite difficult. However, we are pleased to be able to say that Scrummage is compatible with Docker – offering users an easier way to install and deploy the framework. Once up and running via (http)localhost:5000 the user must look at the startup config log in order to retrieve the admin password to access the Scrummage Dashboard. Once logged in, the biggest hurdle facing users is obtaining the various APIs needed for most of the search modules – in most cases, web and social media sites such as Twitter, VKontakte and Odnoklassniki apply a high degree of due diligence with regards to API requests. What we like about the developer responsible for Scrummage is that they have developed a very useful list of instructions on how to obtain APIs for each of the search modules – this list can be accessed from this link: https://github.com/matamorphosis/Scrummage/wiki/The-Long-List-of-Tasks. Once the APIs are configured, Scrummage’s search, analysis and reporting capabilities are quite impressive.

It can be argued by many of our readers that Scrummage is limited with regards to its capability to allow users to easily configure their own custom search modules. Although it is possible to create custom modules by configuring the general.py and┬ácommon.py files within the framework’s libraries, this is far from ideal for less-capable Python users. Also, the framework’s analysis capabilities could be further developed to include link-based visualisation charts in addition to various options that can allow users to compile their own output report based on search results. Nevertheless, we believe that as Scrummage undergoes further development, the framework may become even more user-friendly and equipped with more comprehensive analysis and reporting features. In its current form, Scrummage is a very useful and highly recommended tool for Digital Investigators to have in their toolbox as it can automate multiple searches across a wide range of online sources and output potential digital evidence in JSON or PDF formats.

Overall, Scrummage is very impressive and displays lots of potential for Digital Investigators that require the capability to streamline their OSINT collection workflows.

Let's talk today Are you ready to begin discussing our range of training and capability development solutions?