skip to main content

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, February 13 until 2:00 AM ET on Friday, February 14 due to maintenance. We apologize for the inconvenience.


Title: Challenges in Measuring the Internet for the Public Interest
ABSTRACT The goal of this article is to offer framing for conversations about the role of measurement in informing public policy about the Internet. We review different stakeholders’ approaches to measurements and associated challenges, including the activities of U.S. government agencies. We show how taxonomies of existing harms can facilitate the search for clarity along the fraught path from identifying to measuring harms. Looking forward, we identify barriers to advancing our empirical grounding of Internet infrastructure to inform policy, societal challenges that create pressure to overcome these barriers, and steps that could facilitate measurement to support policymaking.  more » « less
Award ID(s):
2131987 1724853
PAR ID:
10465582
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Journal of Information Policy
Volume:
12
ISSN:
2381-5892
Page Range / eLocation ID:
195 to 233
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The goal of this paper is to offer framing for conversations about the role of measurement in informing public policy about the Internet, the barriers to gathering measurements, public policy challenges that are creating pressure for reform in this space, and recommended actions that could facilitate gathering of measurements to support policy-making. 
    more » « less
  2. One foundational justification for regulatory intervention is that there are harms occurring of a character that create a public interest in mitigating them. This paper is concerned with such harms that arise in the Internet ecosystem. Looking at news headlines for the last few years, it may seem that the range of such harms is unbounded. Hoping to add some order to the chaos, we undertake an effort to classify harms in the Internet ecosystem, in pursuit of a more or less complete taxonomy of harms. Our goal in structuring this taxonomy can help to mitigate harms in a more systematic way, as opposed to fighting an endless defensive battle against whatever happens next. The background we bring to this paper is on the one hand architectural—how the Internet ecosystem is actually structured—and on the other hand empirical—how we should measure the Internet to best understand what is happening. If everything were wonderful about the Internet today, the need to measure and understand would not be so compelling. A justification for measurement follows from its ability to shed light on problems and challenges. Sustained measurement or compelled reporting of data, and the analysis of the collected data, generally comes at considerable effort and cost, so must be justified by an argument that it will shed light on something important. This reasoning naturally motivates our taxonomy of things that are wrong—what we call harms. That is where we, the research community generally, and governments should focus attention. We do not intend this paper as a catalog of pessimism, but to help define an action agenda for the research community and for governments. The structure of the paper proceeds "up the layers'', from technology to society. For harms that are closer to the technology, we can be more specific about the harms, and more specific about possible measurements and remedies, and actors that could undertake them. One motivation for this paper is that we believe the Internet ecosystem is at an inflection point. The Internet has revolutionized our ability to store, move, and process information, including information about people, and we are only at the beginning of understanding its impact on society and how to manage and mitigate harms resulting from unregulated commercial use of these capabilities. Current events suggest that now is a point of transition from laissez-faire to regulation. However, the path to good regulation is not obvious, and now is the time for the research community to think hard about what advice to give the governments of the world, and what sort of data can back up that advice. Our highest-level goal for this paper is to contribute to a conversation along those lines. 
    more » « less
  3. In January and April 2021 we held the Workshop on Overcoming Measurement Barriers to Internet Research (WOMBIR) with the goal of understanding challenges in network and security data set collection and sharing. Most workshop attendees provided white papers describing their perspectives, and many participated in short-talks and discussion in two virtual workshops over five days. That discussion produced consensus around several points. First, many aspects of the Internet are characterized by decreasing visibility of important network properties, which is in tension with the Internet's role as critical infrastructure. We discussed three specific research areas that illustrate this tension: security, Internet access; and mobile networking. We discussed visibility challenges at all layers of the networking stack, and the challenge of gathering data and validating inferences. Important data sets require longitudinal (long-term, ongoing) data collection and sharing, support for which is more challenging for Internet research than other fields. We discussed why a combination of technical and policy methods are necessary to safeguard privacy when using or sharing measurement data. Workshop participant proposed several opportunities to accelerate progress, some of which require coordination across government, industry, and academia. 
    more » « less
  4. Personalization on digital platforms drives a broad range of harms, including misinformation, manipulation, social polarization, subversion of autonomy, and discrimination. In recent years, policy makers, civil society advocates, and researchers have proposed a wide range of interventions to address these challenges. This Article argues that the emerging toolkit reflects an individualistic view of both personal data and data-driven harms that will likely be inadequate to address growing harms in the global data ecosystem. It maintains that interventions must be grounded in an understanding of the fundamentally collective nature of data, wherein platforms leverage complex patterns of behaviors and characteristics observed across a large population to draw inferences and make predictions about individuals. Using the lens of the collective nature of data, this Article evaluates various approaches to addressing personalization-driven harms under current consideration. It also frames concrete guidance for future legislation in this space and for meaningful transparency that goes far beyond current transparency proposals. It offers a roadmap for what meaningful transparency must constitute: a collective perspective providing a third party with ongoing insight into the information gathered and observed about individuals and how it correlates with any personalized content they receive across a large, representative population. These insights would enable the third party to understand, identify, quantify, and address cases of personalization-driven harms. This Article discusses how such transparency can be achieved without sacrificing privacy and provides guidelines for legislation to support the development of such transparency. 
    more » « less
  5. On 16-17 December 2020, CAIDA hosted the 11th interdisciplinary Workshop on Internet Economics (WIE) in a virtual Zoom conference. This year our goal was to gather feedback from researchers on their experiences using CAIDA’s data for economics or policy research. We invited all researchers who reported use of CAIDA data in these disciplines. We discussed their successes and challenges of using the data, and how CAIDA could help these fields via Internet measurement and data curation. To avoid Zoom fatigue, we had a conversation-focused rather than presentation-focused workshop. Research topics we discussed included: Internet data for macroeconomics; connectivity and its effect on economic interdependence; effects of the EU’s new GDPR on internet interconnection; measuring corporate cyber risk; measuring work-from-home trends; measuring the economic value of open source software; and more generally how to best support evidence-based policymaking. 
    more » « less