OSINT Toolbox Talk: Geo-Monitoring Telegram users, scraping Twitter data, and investigating Yandex accounts

OSINT Toolbox Talk

Geo-Monitoring Telegram users, scraping Twitter data, and investigating Yandex accounts

Concluding another fortnight of OSINT tool testing, we present to you, our valued readers, the latest OSINT Toolbox Talk article. By far, this is our best article yet as we introduce to you three very effective tools. The most impressive of the tools tested was Telegram Nearby Map, a fantastic NodeJS-based application that enables Digital Investigators to visualise and monitor Telegram users over any given location in the world. Most certainly, this tool comes with our highest recommendation! Next on our list of tools is TweetScraper, a lightweight Python utility that enables users to scrape Tweets from specified Twitter accounts – of course, this tool can only be used in conjunction with the Twitter Developer API. However, we believe that it is most certainly a tool worth using. Lastly, we introduce YaSeeker, another Python utility that enables Digital Investigators to search against Yandex users and retrieve results with regards to activity on Yandex-associated services. As we feel that Yandex is an under-looked resource with regards to Digital Investigations, we do feel that YaSeeker deserves quite a lot of credit for its capabilities.

Stay tuned for our next OSINT Toolbox Talk article ahead of Christmas! Once again, we will present to you the latest and most effective OSINT tools for Digital Investigations!

Geo-monitoring Telegram user activity with 'Telegram Nearby Map' Geo-monitoring Telegram user activity with 'Telegram Nearby Map' https://github.com/tejado/telegram-nearby-map

While we often discover OSINT tools and utilities through our own research, credit for this OSINT Tool Review article should be given to multiple experts within the OSINT field who brought this tool to our attention. Additionally, it isn’t every day where we are overly impressed with various OSINT utilities, though we can confidently say that Telegram Nearby Map is in a league of its very own.

So, what is Telegram Nearby Map? It is a NodeJS-based application that has been developed by a very talented Github user. The application uses the Telegram API to query the Telegram library and extract the position of nearby users and plot them onto an OpenStreetMap interface. The application allows Digital Investigators to query users positioned within a specified distance of a given position. Once the application is set to run, it will initiate a search every 25 seconds and plot new and existing users onto the map and provide Digital Investigators with the distance of the user in relation to the specified position. One very important note to mention here is that the application can only discover Telegram users that have activated the ‘nearby’ feature within their Telegram app – by default, this feature is deactivated.

Installing the application is not so straightforward, users will first need to create an API key for their own Telegram account and then input the necessary keys within the tool’s config.js file. Once downloaded, the application can be installed via the NodeJS command-line interface by invoking npm install. Once all the required dependencies are now installed and configured, the application can be deployed by invoking the command npm start within the same NodeJS command-line interface. At first run, the command-line interface will request the user to input the phone number associated with their Telegram account in addition to the login code used by Telegram to authenticate logins. With the application now running, it can be accessed through the browser via the address http://localhost:3000.

So, with the application fully functioning, we put it to the test by setting the tool to visually show us active Telegram users located within a 3km radius of the Poland-Belarus border – specifically within the vicinity of the Polish border town of Ku┼║nica. After running the application to search over this area for a period of several hours, the results showed us a flurry of likely migrant activity taking place on the Belarus side of the border in addition to the movements of other users taking place within the 3km radius of the border. All-in-all, the results were simply fantastic!

However, we know that the geolocations of the extracted data is not 100% accurate based on a wide variety of factors including whether the Telegram users’ devices are connected to the internet via local WiFi or via their mobile service providers. Additionally, elevation and local conditions (whether the location is positioned within a rural or built-up area) can also determine the accuracy (or lack thereof) in this regard. However, with three or more geolocations of a single user identified within close proximity to each other, this information can be used to determine the location of that user to a greater degree of accuracy.

Our test of Telegram Nearby Map to monitor the developing situation on the Belarus – Poland border is just one of many examples where this application can prove to be valuable. For example, it can be used to monitor activity within the immediate area of a terrorist incident or track users within a location associated with organised criminal activity. However, as great as the tool is, it does have limitations. For example, it can only locate up to 100 users per search and Telegram applies a rate-limit with regards to the number of searches undertaken, this means that long-term monitoring cannot be achieved. Additionally, we also felt that tool would greatly benefit from providing users with the option to download extracted geolocation data across several file types including CSV and GeoJSON. On the same note, it would also be beneficial to allow users to create a Google Earth-compatible file such as a KML or KMZ that will allow Digital Investigators to visualise collected data on Google Earth and apply various techniques to obtain a more accurate location for extracted user locations.

All that said, Telegram Nearby Map is a very powerful application that has great potential use cases for Digital Investigators. Although the application may very well be difficult to install for any novice NodeJS user, deploying the application and running it to search for users over any given area is incredibly straightforward. Most certainly, this application deserves a five-star based on its capability and should also be a key utility within every Digital Investigator’s toolbox.

Scraping Twitter posts and follower counts with 'TweetScraper' Scraping Twitter posts and follower counts with 'TweetScraper' https://github.com/Durhamster/TweetScraper

Undoubtedly, there is an abundance of OSINT tools focused on extracting data from Twitter, each tool has its own unique set of capabilities and outputs. In this latest OSINT tool review, we will present ‘TweetScraper’, a lightweight Python script that can be used to easily scrape from single and multiple Twitter accounts in addition to querying the overall number of followers for given users. As is the case for the majority of Twitter-focused OSINT tools, Digital Investigators who intend to use TweetScraper will require a Twitter Developer Account in order to obtain the prerequisite keys needed to deploy the tool.

Obtaining a Twitter Developer Account is best described as a tedious process due to Twitter’s due diligence process that can take up to a week from start to finish. However, once the developer account has been created and the keys have been obtained, installing and deploying TweetScraper is incredibly easy. As is the case with the majority of Python-based scripts, the prerequisite modules are installed by invoking pip install -r requirements.txt. Thereafter, TweetScraper can be called by invoking python app.py, resulting in a series of options that the user can select:

  1. Scrape a single account
  2. Scrape a list of accounts
  3. Get the follower count for a list of accounts
  4. Check a list of handles to see if any are suspended, private, or incorrect

Scraping individual accounts can be achieved directly within the command-line interface simply by indicating the target account that you want to scrape. For multiple accounts, the user should list the target accounts within the handles.txt file located within the same directory – the file should then be called from within the command-line-interface. By default, the tool is configured to scrape the first 500 tweets, though this can be easily changed within the app.py script. Lastly, the tool is also configured to pause in-between account scrapes, this is designed to reduce the risk of users exceeding Twitter’s rate limiting mechanisms. In terms of output, TweetScrape provides each scrape with a Microsoft Excel file (.xlsx) with details of each tweet including creation date, account name, Twitter handle, URL, number of likes, number of retweets, and the overall tweet content.

All-in-all, TweetScraper is a nice and simple tool that can be used by Digital Investigators to quickly extract tweets and neatly output them in a readable format. Whilst there are other Python-based scripts and utilities that can offer a more integrated range of features such as exporting data to SQL and sentiment analysis, TweetScraper is unique based on its simplicity. Unfortunately, like other Twitter-focused OSINT tools, obtaining a Twitter Develop Account in order to use TweetScraper can be a frustrating process. That said, with the necessary prerequisites obtained, deploying TweetScraper is incredibly easy and its output is quite effective.

Investigating Yandex accounts with 'Ya Seeker' Investigating Yandex accounts with 'Ya Seeker' https://github.com/HowToFind-bot/YaSeeker

Yandex is understandably an under-looked resource with regards to Digital Investigations, especially since most people tend to rely on western-based email and digital services providers such as Outlook, Google, and to a lesser extent Yahoo. That said, there are many reasons as to why searches should be extended towards Yandex, especially when such investigations may be focused on a suspect with links to Russia or Belarus.

With all that said, we should now introduce our readers to ‘Ya Seeker’, a very lightweight and simple Python-based script that enables Digital Investigators to run a search against a username or Yandex email address and retrieve a range of account information including:

  • Full name
  • User photo
  • The user’s gender
  • Yandex UID
  • Yandex Public ID
  • Linked social accounts
  • User activity (count of reviews, comments; subscribers and subscriptions)
  • Account features (whether the account is verified, banned, deleted, etc.)

Additionally, the account will query several Yandex services in order to identify whether the target account is subscribed to such services. The type of services include:

  • Music
  • Collections (a service where users can share ideas and inspiration – akin to Pinterest)
  • Bugbounty (where users can report vulnerabilities within Yandex’s services)
  • Reviews (akin to Google Reviews)
  • Q / Znatoki (a question and answer service)
  • O (a Russian classifieds service)
  • Zen (a recommendations service)
  • Market (a service to sell items)
  • Messenger

Installing ‘Ya Seeker’ is very straightforward through the command-line interface. Deploying the tool is just as easy by invoking python3 yaseeker.py <your target username / email address>. What we particularly like about this tool is that no API is required in order to conduct searches. That said, some of the service-based searches require users to login into Yandex, retrieve the session cookies within Netscape format, then save the cookies into a file named cookies.txt within the tool’s directory. To sum up this article, ‘Ya Seeker’ is a very nice simple tool that can be used to obtain information concerning Yandex users and it comes with our recommendation.

Let's talk today Are you ready to begin discussing our range of training and capability development solutions?