Another week, another OSINT Toolbox Talk! This week, we continue to wrap up three of the most effective OSINT tools reviewed during the course of last week. With Euro 2020 now over, our readers can set aside their woes and /or jubilations to learn about tools which can be used to either masquerade their Digital Investigation activity, analyse geolocation data associated with YouTube videos, and finally investigate usernames across over 2000 sources.
In this OSINT Toolbox Talk, we look closely at the capabilities presented to us from Maigret – a Python-based script that packs a lot of punch when it comes to searching usernames across a significant volume of online sources. Next up is YouTube Geolocation, a high-powered Ruby-on-Rails application developed by the same individual behind YouTube Metadata and YouTube Comment Suite – this tool comes highly recommended! Last but not least, we introduce Chaff – a Google Chrome extension that provides an additional layer of security through obscurity by creating fake internet traffic alongside normal browsing activity.
In one our previous OSINT Tool Reviews, we took a close look at a Command Line Interface and Graphical User Interface-based tool that has the capability to search across over 800 online sources (including social media). In this OSINT Tool Review, we will look at Maigret, another tool that can be used to investigate usernames.
Many readers will no doubt be curious with regards to the origin of the name Mairgret. As indicated by the tool’s developers, the name is based on Commissioner Jules Maigret is a fictional French police detective, created by Georges Simenon. His investigation method is based on understanding the personality of different people and their interactions.
Maigret is quite a powerful Python-based script that enables Digital Investigators and OSINT Analysts to conduct searches against usernames and detailing all available metadata obtained from webpages where a match is confirmed. The tool itself is based on the popular Python-based script Sherlock but is considerably far easier to deploy. According to Maigret’s Github repository, the tool currently supports more than 2000 websites, the full list of which can be found here. The most significant sites of which include various Tor and IS2P sites in addition to other domains associated with adult, dating, hacking and gaming forums. One point to note is that Maigret can also verify whether the username is associated with a Kik instant messaging application account. This is quite important since Kik is often associated with highly nefarious activity which includes the distribution of child sexual abuse material (of all categories).
In addition to being capable of extracting available metadata from profile matches, Maigret can also allow Digital Investigators to narrow searches by site categories and country codes. The tool also has an in-built captcha detection tool that can be used to bypass verification requests and can also retry searches in the event where search requests are timed out. Maigret is very flexible and can be installed and deployed via the popular Command-Line Interfaces Python or Docker. Additionally, the tool can be run through cloud shells and Jupyter notebooks such as Colab or Binder.
COMMAND-LINE INTERFACE OUTPUT
Search results are immediately indicated through the Command Line Interface. During this review, we used Python to conduct simultaneous searches against multiple target usernames. The search results which can be observed through the Command Line Interface vary depending on the website itself. For example, DeviantArt can provide a substantial amount of user information concerning account creation date and date/time of last activity. On the other hand, more privacy-conscious websites will offer little further information other than confirming the existence of an account that matches the username.
What is undeniably impressive about Maigret is its capability to output a report in PDF or HTML. The report will input a time / date stamp concerning when the search was undertaken in addition to high-confidence information concerning the target such as likely gender and location. The report goes further to provide you with details concerning each match that Maigret has discovered in addition to available metadata that it has managed to extract. What we consider to be most interesting is that Maigret also outputs a Mindmap file that can be opened with MindManager, enabling us to visualise the result of each search and its matches.
Maigret is a very powerful, yet easy-to-deploy and easy-to-use tool that has far-ranging capabilities. What impresses us the most is the large number of sources that it searches against in addition to its capability to output HTML, PDF and Mindmap reports for each search. Based on this, we thoroughly recommend the use of Maigret for Digital Investigators and Analysts that require the capability to conduct fast and efficient searches against a considerably large volume of web sources.
For the more Operational Security (OPSEC) aware Digital Investigators and OSINT Analysts, ‘Chaff’ is an effective Google Chrome-based extension that can add a layer of security through obscurity towards a digital investigation. Chaff’s is designed to make your network activity data less useful and deceive packet sniffers that may be monitoring your browsing behaviour. It does this by creating ‘fake’ network traffic within a single Google Chrome browser and redirect across various web pages. Essentially, this makes your internet browsing activity look less specific and disguise your activity amongst automatically-generated page visits.
Understandably, there may be concerns with regards to Chaff’s behaviour and the range of websites that it will redirect to. For added assurance, users can define the seed URLs for the extension to visit and also allow users to select one or more search engines that Chaff should use when searching for random phrases. The range of search engines includes Duck Duck Go, Google, Bing and Yahoo.
Additionally, users can direct Chaff to loop through bookmarks – this is relatively useful for Digital Investigators as they can build a range of seed URLs that appear more deceptively convincing rather than being completely random. Lastly, users can fine-tune the extension’s behaviour to match their browsing behaviour. The tuning capabilities that Chaff offers include:
- Altering the browsing speed by applying a maximum time between clicks
- Adjusting the length of search phrases which are based on existing phrases found from seed sources
- Set the maximum page load timeout – the time that Chaff will wait for a page to load
- Applying the site depth by specifying the maximum number of pages that Chaff will visit on a particular seed URL during a single session
- Set the total depth of a single Chaff session
For what it is and the capabilities it provides, Chaff is quite an effective tool that Digital Investigators and OSINT Analysts may consider should they need to apply an additional layer of security to their investigations. Whilst it can be argued that any digital investigations environment should have been prepared to an extent that significantly minimises the threat from packet sniffing; it may be the case that investigations taking place in lesser-secure environments will benefit from the capabilities that Chaff provides.
Following-on from our previous article which focused on extracting metadata from YouTube videos, playlists and channels; in this latest OSINT Tool Review, we look at another highly recommended tool that enables Digital Investigators and OSINT Analysts to extract geolocation data from YouTube videos, channels and playlists. ‘YouTube Geolocation’ is another Ruby-on-Rails-based tool developed by the same individual behind ‘YouTube Comment Suite‘ and ‘YouTube Metadata‘. Again, to use this tool, you will need to download and install Ruby; if you haven’t learnt how to do so by now, this is your chance!
YouTube Geolocation comes with a very wide range of capabilities that are of great use for Digital Investigations. According to the tool’s Github repository page, it indicates the following capabilities:
- Conduct channel searches and allow you to check all the uploads on a channel for geotags and displays them on a map.
- Conduct topic searches and allows you to check if any uploads found by regular searching or keywords have geotags.
- Conduct location searches and allow you to find videos with geotags within your chosen radius and timeframe.
- Export results to save your findings and use them elsewhere.
- Directly open results in the YouTube Metadata tool – enabling you to extract and analyse all associated metadata concerning the video and the author.
- Query API to share a search or linked custom search from your own site/tool
Deploying YouTube Geolocation is relatively straightforward. All that is required is some basic knowledge of installing Ruby and its associated developer tools on your local machine. From here, you can run Ruby through Command Prompt, navigate to the repository folder on your local machine and use bundle install to install the user interface, then run it and view it via http://localhost:4000.
Testing this tool was a huge pleasure, and the developer behind it (Matt Wright) certainly all the credit for his hard work. The user interface for the tool is flawless – though I did encounter a very minor issue with regards to the in-built map when it was used to query a considerably large volume of pins. That aside, the tool’s overall functionality is excellent, it enabled me to investigate several YouTube channels, visualise geolocations and export my results to a comma-separated value file. For some fun, I also conducted a geolocation search over my hometown of Llanfairpwll, North Wales, to see all of the available results – an interesting trip down memory lane. Fun aside, the capability to conduct geolocation searches is incredibly useful for any Digital Investigator or OSINT Analyst.
Overall, Matt Wright has developed an amazing tool that provides a great amount of capabilities towards Digital Investigations. As such, this tool comes highly recommended!