PostCog: A tool for interdisciplinary research into underground forums at scale

Abstract

Underground forums provide useful insights into cybercrime, where researchers analyse underlying economies, key actors, their discussions and interactions, as well as different types of cybercrime. This interdisciplinary topic of study incorporates expertise from diverse areas, including computer science, criminology, economics, psychology, and other social sciences. Historically, there were significant challenges around access to data, but there are now research datasets of millions of messages scraped from underground forums. The problems now stem from the size of these datasets and the technical nature of methods and tools available for data sampling and analysis at scale, which make data exploration difficult for non-technical users. POSTCOG has been developed to solve this problem. We first provide a survey of prior work into underground forums; this was used to understand the requirements and functionalities valued by researchers, and to inform the design of a data exploration tool. We then describe POSTCOG, a web application developed to support users from both technical and non-technical backgrounds in forum analyses, such as search, information extraction and cross-forum comparison. The prototype’s usability is then evalu- ated through two user studies with expert users of the CRIMEBB dataset. POSTCOG is made available for academic research upon signing an agreement with the Cambridge Cybercrime Centre.