skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Validity Concerns in Research Using Organic Data
With the advent of computing technologies, researchers across social science fields are using increasingly complex methods to collect, process, and analyze data in pursuit of scientific evidence. Given the complexity of research methods used, it is important to ensure that the research findings produced by a study are robust instead of being affected significantly by uncertainties associated with the design or implementation of the study. The field of metascience—the use of scientific methodology to study science itself—has examined various aspects of this robustness requirement for research that uses conventional designed studies (e.g., surveys, laboratory experiments) to collect data. Largely missing, however, are efforts to examine the robustness of empirical research using “organic data,” namely, data that are generated without any explicit research design elements and are continuously documented by digital devices (e.g., video captured by ubiquitous sensing devices; content and social interactions extracted from social networking sites, Twitter feeds, and click streams). Given the growing popularity of using organic data in management research, it is essential to understand issues concerning the usage and processing of organic data that may affect the robustness of research findings. This commentary first provides an overview of commonly present issues that threaten the validity of inferences drawn from empirical studies using organic data. This is followed by a discussion on some key considerations and suggestions for making organic data a robust and integral part of future research endeavors in management.  more » « less
Award ID(s):
1760059 1850605
PAR ID:
10547001
Author(s) / Creator(s):
;  ;  
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
Journal of Management
Volume:
46
Issue:
7
ISSN:
0149-2063
Format(s):
Medium: X Size: p. 1257-1274
Size(s):
p. 1257-1274
Sponsoring Org:
National Science Foundation
More Like this
  1. Large-scale organic data generated from newspapers, social media, television, and radio require an expertise in infrastructure management, data collection, and data processing in order to gain research value from them. We have developed text analytic research portals to help social science researchers who do not have the resources necessary to collect, store, and process these large-scale data sets. Our portals allow researchers to use an intuitive point and click interface to generate variables from large, dynamic data sets using state of the art text mining and learning methods. These timely variables constructed from noisy text can then be used to advance social science research in areas such as political science, economics, public health, and psychology research. 
    more » « less
  2. null (Ed.)
    ABSTRACT Water security is a powerful concept that is still in its early days in the field of nutrition. Given the prevalence and severity of water issues and the many interconnections between water and nutrition, we argue that water security deserves attention commensurate with its importance to human nutrition and health. To this end, we first give a brief introduction to water insecurity and discuss its conceptualization in terms of availability, access, use, and stability. We then lay out the empirical grounding for its assessment. Parallels to the food-security literature are drawn throughout, both because the concepts are analogous and food security is familiar to the nutrition community. Specifically, we review the evolution of scales to measure water and food security and compare select characteristics. We then review the burgeoning evidence for the causes and consequences of water insecurity and conclude with 4 recommendations: 1) collect more water-insecurity data (i.e., on prevalence, causes, consequences, and intervention impacts); 2) collect better data on water insecurity (i.e., measure it concurrently with food security and other nutritional indicators, measure intrahousehold variation, and establish baseline indicators of both water and nutrition before interventions are implemented); 3) consider food and water issues jointly in policy and practice (e.g., establish linkages and possibilities for joint interventions, recognize the environmental footprint of nutritional guidelines, strengthen the nutrition sensitivity of water-management practices, and use experience-based scales for improving governance and regulation across food and water systems); and 4) make findings easily available so that they can be used by the media, community organizations, and other scientists for advocacy and in governance (e.g., tracking progress towards development goals and holding implementers accountable). As recognition of the importance of water security grows, we hope that so too will the prioritization of water in nutrition research, funding, and policy. 
    more » « less
  3. Ethics and social responsibility are often viewed as key areas of concern for many engineering educators and professional engineers. Thus, it is important to consider how students and professionals understand and navigate ethical issues, explore how such perceptions and abilities change over time, and investigate if certain types of interventions and experiences (e.g., coursework, training, service activities, etc.) impact individual participants. The breadth of engineering as a profession also raises questions about how ethics and social responsibility are understood across a wide range of disciplines, subfields, and industry sectors. Recognizing a need for more empirical research to address such questions, our research team carried out a five year, longitudinal, mixed-methods study to explore students’ perceptions of ethics and social responsibility. This study relied on repeated use of quantitative measures related to ethics, along with qualitative interviews to explore how students’ perceptions of these issues change across time, between institutions, and in response to participation in certain experiences. Additionally, we are now initiating a follow-on study where we will collect survey and interview data from our previous participants now that most of them are in full-time job roles and/or pursuing graduate degrees, as well as from a new group of early career engineers to enlarge our sample. In this paper, we first give an overview of key research findings from our ongoing research that have been published or are under review. The second major part of this paper delves into some specific theoretical and methodological questions and challenges associated with our research. This paper will likely be of interest to educators and researchers who are involved with developing and/or evaluating ethical capabilities among engineering students. 
    more » « less
  4. Human brain and behavior research has traditionally—and paradoxically—taken place mostly in environments that are isolated from the public: In a typical human neuroscience study, scientists recruit university students to participate in well-controlled laboratory studies, i.e., outside of humans’ natural habitat. This model is currently under attack from multiple directions, ranging from scholars arguing that it generates biased data, to communities who express distrust toward scientists, to educators who are eager for more authentic science experiences for their students. While a growing number of researchers is turning to citizen science approaches to both educate and involve the general public in science, these initiatives are most pervasive in the ‘traditional’ sciences (e.g., ecology, astronomy), and often focus on recruiting the public to help collect data, rather than including non-scientists as partners in their scientific process. MindHive (www.mindhive.science) is an online community science platform for human brain and behavior research that engages its users in the full spectrum of scientific inquiry. Taking an open science approach, MindHive features a collaborative study design environment, comprising an experiment builder, a database of validated tasks and surveys, and a public-facing study page; a peer review center; and GDPR-compliant data collection, data management, and data visualization and interpretation functionality. We describe case studies from the COVID-19 pandemic to illustrate how MindHive envisions enabling scientists, students, educators, not-for-profit organizations, and community members globally to contribute studies, resources, and research data to the platform, as such supporting both STEM learning and scientific discovery. 
    more » « less
  5. Casual recommendations for scientists to “be human” and to “bring the public alongside the scientific process” have generated plethora of such interactions on social media. This study explores different self-presentation tactics, like sharing their research successes and failures, or contextualizing their research process, on how scientists and science issues are perceived by the public. In an online between-subjects experiment (N= 1,843), participants rated scientists who shared their failures as more benevolent and open, and as having more integrity, than those who only shared their successes. These perceptions further increased support for science and information-seeking intentions. The findings highlight how scientists’ self-presentation on social media can influence important science communication outcomes. 
    more » « less