Firstly, we will begin by offering our sincere apologies to our readers for publishing this OSINT Toolbox Talk article far later than usual. As you will no doubt understand, much is taking place as a result of the ongoing military situation in Ukraine and this has resulted in us offering support to several organisations concerned. We would like to take this opportunity to express our solidarity with the people of Ukraine and to pay tribute to the brave men and women fighting for their country — Slava Ukraini!
Additionally, we would like to take this opportunity to direct our readers’ attention to a recent OSINT Workflow article that we produced which highlighted one of many examples of OSINT being applied to monitor Russian military activity. The OSINT Workflow article can be read here: https://os2int.com/toolbox/applying-effective-osint-to-geo-monitor-russian-military-activity/
Turning towards the focus of this latest OSINT Toolbox Talk article, we will begin by introducing the Maigret transform for Maltego, a highly effective utility that can be used to conduct username-based searches against over 2000 sources. Next, we show you the highly impressive capabilities of ‘Twayback’, a lightweight Python utility that can extract and save archived deleted Tweets. Lastly, we focus on ‘Serverse’ a very neat search engine that is designed to identify Discord servers of investigative interest by querying a wide range of individual Discord search utilities – this tool is most certainly a must-have!
Firstly, we would like to wish our readers a very happy New Year and to welcome you to our first OSINT Tool Review article for 2022!
In this article, we will delve into the world of Maltego. For the benefit of our readers who have yet to learn about Maltego, it is an OSINT and graphical link analysis tool that is used for collecting and connecting information for the purpose of creating intelligence and automating digital Investigations. Maltego is used extensively by several law enforcement organisations in addition to security analysts, investigative journalists, and researchers. Among the wide range of capabilities that Maltego has, an important one is the capability to create and integrate Transforms – small pieces of code that will automatically collect data from different sources and display the results within Maltego’s native link analysis chart. Several popular and regularly used OSINT Transforms can be integrated with Maltego from within the platform itself via the Transform Hub. However, many other external Transforms – such as Maigret – can be found on Github or Gitlab. With that in mind, we will now introduce the Maigret Transform – a very capable and effective utility that can be used to investigate usernames across over 2000 online sources including social media. Earlier in July 2021, we wrote about the Maigret utility that can be used within the Python command-line interface. However, the Maigret Transform brings a whole new and exciting range of benefits and capabilities to the Digital Investigator including the benefit of running the utility from a few simple clicks of a mouse button rather than invoking several commands within the command-line interface. Additionally, being a graphical link analysis tool, Maltego will neatly display the Maigret Transform’s results within a real-time and readable format.
So, what is Maigret? It is a highly effective Python-based utility that enables digital investigators to collect a dossier on a person by username only, and without the need of an API. The full list of sites that Maigret currently scans across can be accessed from this link. However, the primary websites that the utility will query include the following:
- Google Maps
- Google Plus (archived)
- YouTube (social media)
- Facebook (social media)
- Reddit (social media)
- VKontakte (social media)
- BongaCams (adult)
- Instagram (social media)
- Ebay (online marketplace)
- Twitter (social media)
- Odnoklassiniki (social media)
- ChaturBate (adult)
- Livejasmin (adult)
- Pornhub (adult)
- TikTok (social media)
- Xvideos (adult)
- xHamster (adult)
- Telegram (instant messaging and social media)
- Tinder (online dating)
The Maigret Transform for Maltego is undoubtedly a highly valuable utility for digital investigators. Integrating the Transform within Maltego is very easy. However, for the benefit of novice users, the developer of Maigret and the team at Maltego have each developed a series of comprehensive instructions that will most certainly help digital investigators get started with the utility. Once integrated and correctly configured, the Maigret Transform can be run within Maltego by using the drag-and-drop interface to place an ‘Alias’ Entity onto the graph window. The target username should then be indicated within the Entity, and then initiated by right-clicking on the Entity and selecting the Transform. From here, the digital investigator can visualise the results of the search in real-time, and within a very neat link analysis format. Going even further, the digital investigator now has the capability of using additional Maltego Transforms to further scrutinize and investigate the results of their initial search.
Like the command-line interface version of the Maigret utility, the Transform also scrapes and stores any metadata associated with search results. However, the Transform does not produce the Maigret custom report. That said, this feature is considered redundant when taking into account Maltego’s own reporting capability that allows Investigators to produce a PDF report complete with a snapshot of the link graph.
To bring this review to its natural conclusion, we at OS2INT must say that Maigret – when combined with Maltego – is a very powerful tool for digital investigators. However, the capabilities that lay behind the Maigret Transform are almost certainly driven by the automated data collection capabilities and neat graphical interface that Maltego provides. Over the course of the festive season, we ran several tests using the Maigret Transform within Maltego, focusing on several usernames associated with organized criminality in Scandinavia, the results of our tests uncovered several instances where the same usernames had been registered on other web pages. The outputs generated from the Transform were investigated even further by using additional Transforms. All-in-all, both the Maigret Transform and Maltego come with our highest recommendation – to the extent where our readers will see additional reviews of OSINT Transforms for Maltego throughout the course of the year. Stay tuned!
The capability to recover deleted Tweets and Twitter user data is a highly sought-after requirement for digital investigators, investigative journalists, and the like. When we consider how Twitter has become the number one platform used by political figures to communicate key messages, there is undoubtedly a requirement for political journalists, analysts, and commentators to use Twitter-focused OSINT tools that can identify inconsistencies in those messages. Additionally, in several well-reported instances worldwide, journalists have identified historical Tweets posted by politicians that identify instances of anti-Semitic, anti-Islamic, and homophobic speech – a simple Google search will reveal such instances. For law enforcement digital investigators, historical Tweets and Twitter user activity has – on many occasions – been used as evidence in criminal cases, especially those involving hate speech, online harassment, and sexual abuse.
To that end, we will now present ‘Twayback’, a lightweight Python utility that can be used to identify and extract archived deleted Tweets and Twitter user activity. What makes this utility even better for users is that it does not require the Twitter API – instead, it will query the Wayback Machine to identify archived deleted Tweets, then save them as an HTML file. The key capabilities of this tool include:
- Download some or all of a Twitter user’s archived deleted Tweets (of course, this is dependent on whether the Wayback Machine has archived those Tweets!)
- Extract deleted Tweets, retweets, and replies to an HTML file
- Apply a custom time range to allow digital investigators to narrow their search for deleted Tweets between two dates
- Differentiate between accounts that are active, suspended, or deleted
- Indicate whether a target Twitter user’s Tweets have been excluded from the Wayback Machine.
Installing, configuring, and deploying the tool is very straightforward. For Windows OS users, the developers of Twayback provide an executable file that can be used to launch the utility within the command-line interface. Alternatively, users can clone the repository directly from Github and install the tool by invoking the necessary command pip install -r requirements.txt within the command-line interface. And that is it! With the utility now installed, users can use it by invoking the command
twayback.py -u USERNAME (e.g. os2int). To apply the custom time range, the command should be appended with
-from YYMMDD -to YYMMDD.
The utility will output archived deleted Tweets in a custom folder within the utility’s root directory – each Tweet contained within its own respective folder. Depending on your system, the extracted Tweets may be saved as a generic FILE extension – this can be quite frustrating as that means that the user will have to append all of the extracted Tweets with a
.html extension. That said, for what the utility is designed to do – it is simply fantastic.
To put the utility to the test, we ran it against the Twitter accounts for UK Prime Minister Boris Johnson and the leader of the Labour Party Sir Keir Starmer (to highlight our political neutrality of course!). The results of our test revealed a wide range of deleted Tweets from each of the aforementioned individuals spanning several years – for a political journalist, this is very likely to be pure gold dust! It goes without saying, if we wanted to view the archived deleted Tweets from each of the two individuals manually, that would involve a lot of time spent searching through the Wayback Machine and identifying those that have been archived. With Twayback, it does all of the hard work for us by automating the entire process. However, as the developer behind the tool has rightly pointed out, there are some considerations that every user should be aware of:
- The quality of the extracted Tweets can vary drastically depending on how the Wayback Machine has archived them. Of course, this isn’t ideal, though it is most certainly through no fault of the developer.
- Again, depending on how the Wayback Machine has archived the Tweets, you may or may not be able to extract embedded images. Videos most certainly cannot be extracted.
- If a Twitter account is suspended or deleted, this can affect the number of Tweets that can be extracted.
- The custom date range does not reflect when the Tweets were made, but rather when they were archived. This means that a Tweet from 2020 may have only been archived today.
Overall, we at OS2INT really love this tool based on its simplicity with regard to installation and deployment. At the same time, we believe that this tool can save investigators and journalists an incredible amount of time and streamline their workflows quite effectively. On that note, this tool comes highly recommended!
For the benefit of our readers who are unaware, Discord is an instant messaging platform that focuses on group chats. It was launched in 2015 and has about 300 million registered users. Discord allows its users to create public and private communities known as servers, where they can communicate using text messages, video and voice calls. Like Twitch, Discord is primarily used by gamers, though there are now many servers on it for a variety of discussions and activities other than gaming, this has inevitably resulted in the platform being used for illegal purposes and hate speech. In response to a growing number of servers being set up to spread hate speech, Discord said in a blog post in April 2021 that it had removed 3.2 million accounts for “spammy behaviour” in the second half of 2020. Prior to this, an investigation by the New York Times revealed the 2017 Unite the Right rally in Charlottesville had primarily been organised on Discord, with alt-right leader Richard Spencer and editor of the neo-Nazi website The Daily Stormer, Andrew Anglin, regularly using private servers to discuss strategy.
Of note, early QAnon influencers Tracy Diaz, Coleman Rogers and Paul Furber, created a Discord server to communicate strategies and plan for mainstreaming the conspiracy theory from the fringes of the internet. Following the Capitol riots, left-wing media collective Unicorn Riot revealed chat logs of 18 private Discord servers, members of which had attended the riots and discussed strategy as the attack raged on. Several of the chat logs were primarily used by QAnon adherents. Another server, “The Donald”, was banned by Discord in the aftermath of the 6 January attack. The server’s suspension was due to “overt connection” to the similarly named online forum “The Donald”, Discord said. The Donald online forum was involved in organising the riots, and calls to violence against members of Congress were easily found in the days leading up to the siege.
So, it goes without saying, Discord is an interesting source of open-source data and information which could have intelligence / investigative value.
However, the first obstacle we will undoubtedly encounter is the lack of tools and / or resources that can be used to pinpoint Discord servers that we may want to investigate – cue ‘serverse’, a search engine that will provide Digital Investigators with the capability to quickly and effectively find Discord servers based on keywords. When accessing serverse, users are presented with a clean and very user-friendly search engine interface from where they can input the search keyword parameters, then implement the search.
Once the search has been implemented, users will be directed to a Google and Yandex search results page which will show any Discord servers matching the keywords that have been queried. From here, Digital Investigators can proceed to investigate further. For many of our readers, they will no doubt be wondering whether they can invoke operator searches – the answer to this is yes, absolutely! Serverse supports AND / OR operator searches. Using the AND operator will only return results matching all keywords whilst using the OR operator will return results matching any of the provided keywords. However, the team behind serverse rightly point out that advanced search operators should be avoided where possible. The reason for this is because serverse works by using advanced operators to query a large number of Discord search sites including disboard.org, top.gg, disforge, and dicordland. Also, we should point out that to use this resource effectively, pop-ups need to be enabled on Google Chrome by following these steps:
- Open your Chrome web browser and navigate to the site you want to allow pop-ups on.
- Click the lock icon to the left of the address bar.
- In the pop-up menu, click the drop-down next to Pop-ups and redirects, and change it from Block to Allow. You can also do this by clicking Site Settings and scrolling down to Pop-ups and redirects.
So, what do we especially love about serverse? First, this resource does not require any installation or major configurations aside from enabling popup within your browser. With serverse, users can access the search engine straight away and begin their searches! Secondly, there are absolutely no prerequisites to use servers; however, we would highly encourage our readers who are unfamiliar with Discord to take the time to understand the platform and why it is a highly under-rated resource when it comes to digital investigations.
Most certainly, we at OS2INT firmly believe that serverse will help Digital Investigators and OSINT Analysts streamline their investigation workflow by enabling them to effectively and efficiently search for, and identify Discord servers of investigative interest. This was proved to us when we tested serverse over the course of the week by conducting a wide range of searches for Discord servers likely being used by UK-based far-right users to spread conspiracies in addition to others potentially being used for more criminal activities. Overall, we cannot fault serverse for its search capability to make Discord-focused searches much quicker and easier.