The Growth of Open Source Intelligence (OSINT): Opportunities and Challenges for Governments and Organisations


In their new blog, Research Fellow University of Bologna, Associate Professor and Senior Research Fellow, Oxford Internet Institute and Professor, Oxford Internet Institute share their insights on the growth of OSINT and reflect on the opportunities and challenges for governments, defence and civilian organisations. The academics also highlight the importance of effective governance and oversight to ensure the future reliability of intelligence practice.

The Growth of OSINT

Most data online is open source, meaning interested parties can freely access them with relatively low technical requirements. In the right setting, such data can become open source intelligence (OSINT), i.e. publicly available information exploited for intelligence purposes. Over the years, OSINT has become an integral part of intelligence practice, with technological progress delivering new collection methods and creating new intelligence sources, such as satellite images, social media, public records, and digital currencies. Indeed, many estimates place OSINT at around eighty per cent of all the intelligence material used by law enforcement agencies.

Prominent Cases of OSINT

Recently, there have been many prominent cases of digital OSINT used as valuable intelligence by both state agencies and activist networks. For example, in 2018, a group of OSINT analysts was able to featured in one of the most famous Islamic State propaganda videos. This was achieved by comparing the terrain, vegetation and building characteristics shown in the video frames against satellite images from Google Earth. More recently, the use of OSINT in the ongoing Russo-Ukrainian conflict has impacted contemporary warfare by outsourcing parts of the targeting cycle to civilians. The widespread use of smartphones amongst citizens has enabled military personnel to exploit intelligence gathered by the civilian population and shared on social media to obtain approximate location estimates for enemy combatants. This has raised significant concerns about the extent of military surveillance of civil society, and the risks of involving the civilian population in military operations.

The Role of AI in OSINT

As with many other aspects of contemporary society (and perhaps even more so), Artificial intelligence (AI) has affected the retrieval and analysis of OSINT data. As computational power becomes cheaper and algorithms more sophisticated, more data can be acquired and processed in almost real-time. Moreover, AI can also assist the analysis phase of the OSINT cycle, valuable intelligence based on pre-trained models and thus countering the problem faced by intelligence analysts. For instance, many hours of video footage can be automatically scanned and labelled for future use, while long documents can be translated and summarised by AI algorithms. Social media can be mined to assess opinions about a given topic or to geolocate a particular person based on the content they post. Queries can then be run on a database, and the relevant intelligence is returned to the analyst. As AI becomes more integrated into every day OSINT analysis, it becomes clear that those who can train state-of-the-art AI algorithms can fully exploit the goldmine of publicly available data and produce better intelligence products.

In a 1904 article titled, geographer Halford John Makinder laid down the foundations of geopolitical theory as a struggle over the control of the Heartland, a strategically dominant and resource-rich area roughly corresponding to contemporary Russia. Extending this analysis to the intelligence domain, AI becomes the through which the timely collection and analysis of open-source intelligence is made possible and provides an to the analyst. Paraphrasing Makinder’s famous quote, , thus breaking the monopoly of intelligence collection and analysis held by intelligence agencies and projecting it into the social dimension. In , we discuss the role AI has played and will play in the context of OSINT and focus on the emerging area of research known as the Governance, Ethical, Legal and Social Implications (GELSI) of OSINT. Indeed, we are the first to reference and address this GELSI framework in an academic paper. We do so by systematically searching for GELSI-themed articles within the relevant literature and providing a thorough review of the existing scholarship.

Concerns and Issues

As it turns out, current concerns focus on the (individual) and (societal) issues around adopting AI algorithms for OSINT analysis. At the micro-level, we find privacy issues relating to how user data is handled during investigations, considering anonymisation and increased regulation as potential solutions to privacy breaches. Some attention is also given to the development of privacy-friendly software, which would restrict the ability to link citizens to their online personas. At the macro level, the focus is on the role of OSINT in shaping society as a whole. The emergence of activist networks and other non-state actors as key players in the OSINT landscape brings up two related phenomena. First, the democratising force of open-source information, which increases the potential for citizen oversight of government activities. Second, the hidden threat of misinformation disguised as grassroots activism. Indeed, malicious actors may try to poison the well of OSINT analysis by deliberately providing misleading information, which could then lead to doxing of innocent citizens. Thus, the need to verify sources and cross-reference key data points becomes essential, even more so as intelligence collection becomes automated and risks including potentially inaccurate material.

The Asymmetry in AI Tools

Finally, as we have hinted above, integrating AI tools into OSINT analysis creates an which favours the actors with easy access to more advanced algorithms, richer datasets, and computational power. These asymmetries are likely to manifest between the public and private sectors and between state and non-state actors. In these conflicts, those who can exploit the informational pivot better will prevail and determine the future of intelligence practice. It is hard to tell which transformation will have the heaviest impact on OSINT practice. However, as more phases of the intelligence cycle become automated, it is clear that a suitable framework for validating AI-powered OSINT must be developed to address reliability, transparency and oversight issues. If not, the opacity of traditional intelligence will become even worse, and it will be hard to disentangle facts from misinformation and unilaterally determine responsibility for intelligence decisions.

Download their full paper ‘Open Source Intelligence Analysis in the Age of Artificial Intelligence’, published in AI & Society.

About the authors

Mariarosaria Taddeo is an Associate Professor, Senior Research Fellow and Programme Director of the DPhil in Information, Communication and the Social Sciences at the OII.

Luciano Floridi’s research areas are the philosophy of Information, information and computer ethics, and the philosophy of technology.

Riccardo Ghioni is Research Fellow, Centre for Digital Ethics (CEDE), Department of Legal Studies, University of Bologna. His research interests include: AI machine learning and causal inference.

This website uses cookies. By continuing to use this site, you accept our use of cookies.  Learn more