Imagery geolocation – current challenges

Long gone are the days where we could rely on most images containing EXIT geolocation data. The reality is that almost all social media platforms will automatically strip media content of EXIF data. However, some social media platforms including VKontakte will instead serve EXIF geolocation data within their own mapping interface on a person’s profile. But, if we are required to investigate and analyse imagery that neither has EXIF geolocation data or is served within any such mapping interface, we must our own analytical techniques to geolocate images. Is this an ideal solution? Far from it! For starters, the application of analytical techniques to geolocate images will inevitably be a time-consuming process. But, what we can do is reduce the amount of time spent to analyse images – one solution we can turn to achieve this is GVision.

What is GVision and when should I use it?

Developed by GONZOsint, GVision is a reverse image search utility that is installed and run as a Python script and delivered in the browser in a very neat application interface. The utility uses the Google Cloud Vision API to detect landmarks and identify their locations. The use of the Google Cloud Vision API is significant in this case as it is a machine learning-powered service that enables us to detect objects, faces, text and logos that exist within an image. However, it should be pointed out that very start that the Google Cloud Vision API will not be able to geolocate every single image, nor will it be able to detect every single landmark within an image. However, it is a good service that can be used to perform a quick reverse search and geolocation analysis of an image. By using GVision, it is possible to use its user interface to quickly drag and drop files and analyse them efficiently.

How to get started with GVision

Before we get started with the installation and configuration of GVision, we must first register with the Google Cloud Platform Console and enable the Google Cloud Vision API. But – here is the catch – using the Google Cloud Vision API is free up to 1000 queries per month, any queries over this limit will incur a fee. Also, to acquire the API, each user must input their billing details. Then, we must create a service account inside the Google Cloud Platform Console and download a private key in JSON format.

Installing and configuring GVision

This process is incredibly straightforward – and GONZOsint has done an incredibly great job in keeping this utility nice and simple to not only use, but to also get running. Arguably, the most frustrating process involves the acquiring of the necessary API key – but that is certainly something that is outside of GONZOsint’s control.

To get install the tool, users can simply git clone the project’s repo from Github and simply use the command-line to install the necessary prerequisites by invoking pip install -r requirements.txt. At this point, we can now invoke streamlit run gvision.py to serve the utility locally via port 8501. The utility can now be run directly from the browser through the given IP address and port number.

Next, we need to upload the JSON file containing the Google Cloud Vision API to the GVision interface by clicking ‘Upload a config file’ and selecting the JSON file in question. Once done, GVision will authenticate the API key.

Analysing and geolocating images

Once again, we will point out that GVision and the Google Cloud Vision API applies landmark detection in order to identify prominent features within images and geolocate them. The success rate is not 100%. However, as we indicated at the start, manually geolocating images is a time-consuming process and GVision can enable analysts to save time and streamline their workflow accordingly.

So, what kind of results do we get from GVision. As shown in the images, we uploaded some pictures taken from Russian social media of Wagner Group fighters and Wagner Group-aligned proxies and the results were quite impressive. Firstly, the reverse image search capabilities provided us a list of URLs where there same image has been used online. Yes, it can be argued that an ordinary reverse image search will achieve the same – but sometimes, it is better to get a list of URLs so that we can undertake our own verification.

Where the Google Cloud Vision API is able to match an image of a landmark, GVision will show the geolocation of the image through a built-in OpenStreetMap interface. At the same time, GVision will also indicate the latitude, longitude value and location nearby name for the landmark as well.

Next, GVision will also show the detected entities on the image. The type of entities returned is very impressive. For example, one image of a named Wagner Group fighter returned the following web entities:

[
"Tank",
"Motor vehicle",
"Russia",
"Внештатный",
"Mercenary",
"Wagner Group",
"Tree",
"GRU",
"Museum"
]

At the same time, it correctly matched an image to an online database showing the same Wagner Group fighter in addition to visually similar images.

Sp, the big question is whether GVision is a useful tool to have in our OSINT Toolbox – absolutely it is. Whilst we must point out that the Google Cloud Vision API is not advanced enough to detect all landmarks – it does a damn good job when it does find a match! Secondly, the entity detection capabilities are also very impressive and can certainly give OSINT’ers a great amount of context to an image.