Every so often we showcase web applications which we believe has both long-term potential for development in addition to relevance for Digital Investigators from an OSINT collection perspective. In this OSINT Tool Review, we introduce ‘Scrummage’, a very impressive OSINT and Threat Hunting framework that has been developed with the aim to bring the ‘OSINT Framework’, an online repository of sites and sources that can be used for OSINT purposes, to life within an open-source web-based application.

So, in a nutshell, Scrummage is a centralised OSINT search platform that can search, collect and report data from a varied range of web and social media sites including:

• Apple Store
• Business Search
• SSLMate
• Craigslist
• eBay
• Emailrep
• Flickr
• Google Search
• Have I Been Pwned (Accounts, Breaches, Emails and Passwords)
• Hunter.io
• intelX
• ipstack
• Naver Search
• Odnoklassniki (Groups and Users)
• Pinterest
• Reddit
• Shodan (IP addresses and Queries)
• Tumblr
• Twitter
• Virus Total (Domains, File Hashes, IP Addresses and URLs)
• VKontakte (Users and Groups)
• Vulners
• Windows Store
• Yandex
• YouTube

Each of the search capabilities listed above is executed by building a ‘Task’ within the Scrummage dashboard and then executing it. The ‘Task’ can be further configured to run according to a set number of frequencies and limits. The results from each of the ‘Tasks’ are then published across a variety of formats including PDF and JSON. Additionally, Scrummage can also be integrated with various third-party tools, enabling it to output search results into the following frameworks:

• DefectDojo
• ElasticSearch
• Email
• JIRA
• Request Tracker Incident Response (RTIR)
• Scrumblr
• Slack Channel

Regarding the installation and deployment of Scrummage, this can be somewhat of a complex process depending on which installation method you choose. The framework currently supports Debian, RHEL and SUSE-based Linux distributions, though installation via these distributions is quite difficult. However, we are pleased to be able to say that Scrummage is compatible with Docker – offering users an easier way to install and deploy the framework. Once up and running via (http)localhost:5000 the user must look at the startup config log in order to retrieve the admin password to access the Scrummage Dashboard. Once logged in, the biggest hurdle facing users is obtaining the various APIs needed for most of the search modules – in most cases, web and social media sites such as Twitter, VKontakte and Odnoklassniki apply a high degree of due diligence with regards to API requests. What we like about the developer responsible for Scrummage is that they have developed a very useful list of instructions on how to obtain APIs for each of the search modules – this list can be accessed from this link: https://github.com/matamorphosis/Scrummage/wiki/The-Long-List-of-Tasks. Once the APIs are configured, Scrummage’s search, analysis and reporting capabilities are quite impressive.

It can be argued by many of our readers that Scrummage is limited with regards to its capability to allow users to easily configure their own custom search modules. Although it is possible to create custom modules by configuring the general.py and common.py files within the framework’s libraries, this is far from ideal for less-capable Python users. Also, the framework’s analysis capabilities could be further developed to include link-based visualisation charts in addition to various options that can allow users to compile their own output report based on search results. Nevertheless, we believe that as Scrummage undergoes further development, the framework may become even more user-friendly and equipped with more comprehensive analysis and reporting features. In its current form, Scrummage is a very useful and highly recommended tool for Digital Investigators to have in their toolbox as it can automate multiple searches across a wide range of online sources and output potential digital evidence in JSON or PDF formats.

Overall, Scrummage is very impressive and displays lots of potential for Digital Investigators that require the capability to streamline their OSINT collection workflows.

As far as OSINT tools for Twitter go, there are several very powerful contenders such as Tinfoleak and Twint. Each has its own visual and analysis capabilities. However, in this OSINT Tool Review, we want to showcase another highly powerful Twitter-focused tool that has its distinct capability and best use case – ‘Twitwork’.

So, what is Twitwork? In short, it is a NodeJS-based tool that provides users with a fresh and minimalistic user interface from where Digital Investigators can visually monitor Twitter activity in real-time. The user interface has a search bar from where the user can input their desired search terms, this can be targeted towards a specific user by using ‘@’, a hashtag ‘#’, or free-text. Once the search is initialised, it will visually show associated tweets, retweets and quoted tweets. Going further, the user can also click on an individual result / node to view the tweet and filter the results by selecting / deselecting the tweet, retweet and quoted tweet toggles.

Installing and deploying Twitwork is very straightforward, users have the option of installing manually via NodeJS, or downloading and installing the pre-compiled executable file. The biggest hurdle facing users is obtaining a Twitter API to use the tool. Considering that Twitter applies a significant level of due diligence towards API requests, it is most certainly the most frustrating task that needs to be undertaken before using the tool. However, once an API has been obtained, configuring Twitwork is very self-explanatory.

During our test, we opted to see how much Twitter activity the tool could support. To do this, we focused the tool against hashtags associated with the coup de-tat in Guinea Conakry; knowing that the event was drawing a high volume of Twitter activity. Overall, the tool ran perfectly and a significantly large volume of tweets was processed without any issues identified. Needless to say, this tool comes with our highest recommendation for Digital Investigators that require the capability to monitor Twitter activity in real-time. However, the tool does have some drawbacks such as the fact that it doesn’t support multiple searches. We certainly believe that having a tabular feature that would enable Digital Investigators to switch between searches would be a huge welcome step. Also, our test of the tool found that it was unable to provide a report-based output; again, we believe that this is most certainly needed.

That said, this tool is very powerful, easy to install and quick to deploy. It can serve a variety of uses for Digital Investigators who require a real-time monitoring and visualisation capability. We can see several best-use cases for this tool such as monitoring the flow of disinformation, hate speech and radical content. As such, this tool comes highly recommended and is considered a must-have for every Digital Investigator’s toolbox.

There is a wide range of tools and scripts that can enable digital investigators to extract user content from Instagram profiles, the capabilities of such scripts vary considerably. However, in this OSINT Tool Review, we will showcase ‘Insta-Extract’ a lightweight Python-based script that provides an all-encompassed range of capabilities including:

• Extracting user information
  • User ID
  • Username
  • Name
  • Profile image URL
  • Biography
  • External URL
  • Private or public profile check
  • Business or personal account check
  • Follower and following user count
  • Business email, phone number and category name check
  • Most used tags and mentions
• Extracting media content including images and videos
• Extracting media content information
  • Image/video URL
  • ID
  • Accessibility check
  • Media type
  • Media content dimensions
• Extracting post information
  • Timestamp
  • Date and time of upload
  • Like metrics
  • Comment disabled/enable check
  • Comment metrics
  • Location data
  • Post caption
• Extracting and comparing follower and following data

We tested Insta-Extract extensively, and we were very impressed with its capabilities. Installation of the script is very easy as it does not require any prerequisite modules to be installed. The only main prerequisite needed is an active Instagram account and login information. Quite simply, the script can be downloaded directly from its Github repository and deployed straightaway.

However, we do see probable issues with regard to the long-term potential of Insta-Extract. The main issue in this regard is the fact that Instagram is very proactive in implementing changes to deter OSINT-focused scripts from extracting information from user profiles. Another issue that we identified is that the developers responsible for Insta-Extract have yet to address Instagram’s rate-limiting safeguards. Nevertheless, the developers have pointed out that this script remains in development, and we are confident that it will soon look at ways to bypass Instagram’s rate-limiting mechanisms.

Issues aside, Insta-Extract is very, very impressive when taking into account that it is lightweight, quick to install, easy to deploy and highly effective with regards to extracting user data. Having tested several Instagram-focused OSINT tools in the past, we do believe that Insta-Extract has far superior capabilities and is most certainly better composed with regards to installation and deployment. Overall, Insta-Extract comes highly recommended for Digital Investigators and OSINT’ers who require a quick and easy means to extract media, information and post data from Instagram profiles.