Go wide, go deep: Quantifying the impact of scientific papers through influence dispersion trees

Abstract

Despite a long history of the use of 'citation count' as a measure of scientific impact, the evolution of the follow-up work inspired by the paper and their interactions through citation links have rarely been explored to quantify how the paper enriches the depth and breadth of a research field. We propose a novel data structure, called Influence Dispersion Tree (IDT), to model the organization of follow-up papers and their dependencies through citations. We also propose the notion of an ideal IDT for every paper and show that an ideal (highly influential) paper should increase the knowledge of a field vertically and horizontally. We study the structural properties of IDT (both theoretically and empirically) and propose two metrics, namely Influence Dispersion Index (IDI) and Normalized Influence Divergence (NID) to quantify the influence of a paper. Our theoretical analysis shows that an ideal IDT configuration should have equal depth and breadth (and thus minimize the NID value). We establish the superiority of NID as a better influence measure in two experimental settings. First, on a large real-world bibliographic dataset, we show that NID outperforms raw citation count as an early predictor of the number of new citations a paper will receive within a certain period after publication. Second, we show that NID is superior to the raw citation count at identifying the papers recognized as highly influential through 'Test of Time Award' among all their contemporary papers (published in the same venue)