More Like this

1. The Salt Controversy Systematic Review Reports and Primary Study Reports Network Dataset

   https://doi.org/10.13012/B2IDB-6128763_V2  

   Fu, Yuanxi ; Hsiao, Tzu-Kun ; Joshi, Manasi Ballal ( January 2022 , University of Illinois at Urbana-Champaign)

   The salt controversy is the public health debate about whether a population-level salt reduction is beneficial. This dataset covers 82 publications--14 systematic review reports (SRRs) and 68 primary study reports (PSRs)--addressing the effect of sodium intake on cerebrocardiovascular disease or mortality. These present a snapshot of the status of the salt controversy as of September 2014 according to previous work by epidemiologists: The reports and their opinion classification (for, against, and inconclusive) were from Trinquart et al. (2016) (Trinquart, L., Johns, D. M., & Galea, S. (2016). Why do we think we know what we know? A metaknowledge analysis of the salt controversy. International Journal of Epidemiology, 45(1), 251–260. https://doi.org/10.1093/ije/dyv184 ), which collected 68 PSRs, 14 SRRs, 11 clinical guideline reports, and 176 comments, letters, or narrative reviews. Note that our dataset covers only the 68 PSRs and 14 SRRs from Trinquart et al. 2016, not the other types of publications, and it adds additional information noted below. This dataset can be used to construct the inclusion network and the co-author network of the 14 SRRs and 68 PSRs. A PSR is "included" in an SRR if it is considered in the SRR's evidence synthesis. Each included PSR is cited in the SRR, but not all references cited in an SRR are included in the evidence synthesis or PSRs. Based on which PSRs are included in which SRRs, we can construct the inclusion network. The inclusion network is a bipartite network with two types of nodes: one type represents SRRs, and the other represents PSRs. In an inclusion network, if an SRR includes a PSR, there is a directed edge from the SRR to the PSR. The attribute file (report_list.csv) includes attributes of the 82 reports, and the edge list file (inclusion_net_edges.csv) contains the edge list of the inclusion network. Notably, 11 PSRs have never been included in any SRR in the dataset. They are unused PSRs. If visualized with the inclusion network, they will appear as isolated nodes. We used a custom-made workflow (Fu, Y. (2022). Scopus author info tool (1.0.1) [Python]. https://github.com/infoqualitylab/Scopus_author_info_collection ) that uses the Scopus API and manual work to extract and disambiguate authorship information for the 82 reports. The author information file (salt_cont_author.csv) is the product of this workflow and can be used to compute the co-author network of the 82 reports. We also provide several other files in this dataset. We collected inclusion criteria (the criteria that make a PSR eligible to be included in an SRR) and recorded them in the file systematic_review_inclusion_criteria.csv. We provide a file (potential_inclusion_link.csv) recording whether a given PSR had been published as of the search date of a given SRR, which makes the PSR potentially eligible for inclusion in the SRR. We also provide a bibliography of the 82 publications (supplementary_reference_list.pdf). Lastly, we discovered minor discrepancies between the inclusion relationships identified by Trinquart et al. (2016) and by us. Therefore, we prepared an additional edge list (inclusion_net_edges_trinquart.csv) to preserve the inclusion relationships identified by Trinquart et al. (2016). UPDATES IN THIS VERSION COMPARED TO V1 (Fu, Yuanxi; Hsiao, Tzu-Kun; Joshi, Manasi Ballal (2022): The Salt Controversy Systematic Review Reports and Primary Study Reports Network Dataset. University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-6128763_V1) - We added two new columns in salt_cont_author.csv, "author_id_scopus" and "author_id_mannual" to indicate which author ids were from Scopus and which were assigned by us. - We corrected a few mistakes in "last_search_year," "last_search_month," and "last_search_day" column in systematic_review_inclusion_criteria.csv. - We systematically adjusted the information related to report #12 in report_list.csv, systematic_review_inclusion_criteria.csv, supplementary_reference_list.pdf, salt_cont_author.csv, and inclusion_net_edges.csv to reflect information found in Adler 2014 (Adler, A. J., Taylor, F., Martin, N., Gottlieb, S., Taylor, R. S., & Ebrahim, S. (2014). Reduced dietary salt for the prevention of cardiovascular disease. Cochrane Database of Systematic Reviews, 12. https://doi.org/10.1002/14651858.CD009217.pub3). See our explaination in section "Explanations about report #12". - We sorted the salt_cont_author.csv file by "author_id," not by "ID" (the id of the report). 

   more » « less
2. Opening the Door to Registered Reports: Census of Journals Publishing Registered Reports (2013–2020)

   https://doi.org/10.1525/collabra.24404  

   Montoya, Amanda Kay ; Krenzer, William Leo ; Fossum, Jessica Louise ( January 2021 , Collabra: Psychology)

   Registered reports are a new publication workflow where the decision to publish is made prior to data collection and analysis and thus cannot be dependent on the outcome of the study. An increasing number of journals have adopted this new mechanism, but previous research suggests that submission rates are still relatively low. We conducted a census of journals publishing registered reports (N = 278) using independent coders to collect information from submission guidelines, with the goal of documenting journals’ early adoption of registered reports. Our results show that the majority of journals adopting registered reports are in psychology, and it typically takes about a year to publish the first registered report after adopting. Still, many journals have not published their first registered report. There is high variability in impact of journals adopting registered reports. Many journals do not include concrete information about policies that address concerns about registered reports (e.g., exploratory analysis); however, those that do typically allow these practices with some restrictions. Additionally, other open science practices are commonly encouraged or required as part of the registered report process, especially open data and materials. Overall, many journals did not include many of the fields coded by the research team, which could be a barrier to submission for some authors. Though the majority of journals allow authors to be anonymous during the review process, a sizable portion do not, which could also be a barrier to submission. We conclude with future directions and implications for authors of registered reports, journals that have already adopted registered reports, and journals that may consider adopting registered reports in the future. 

   more » « less
3. The Salt Controversy Systematic Review Reports and Primary Study Reports Network Dataset

   https://doi.org/10.13012/B2IDB-6128763_V3  

   Fu, Yuanxi ; Hsiao, Tzu-Kun ; Joshi, Manasi Ballal ; Lischwe Mueller, Natalie ( January 2023 , University of Illinois at Urbana-Champaign)

   The salt controversy is the public health debate about whether a population-level salt reduction is beneficial. This dataset covers 82 publications--14 systematic review reports (SRRs) and 68 primary study reports (PSRs)--addressing the effect of sodium intake on cerebrocardiovascular disease or mortality. These present a snapshot of the status of the salt controversy as of September 2014 according to previous work by epidemiologists: The reports and their opinion classification (for, against, and inconclusive) were from Trinquart et al. (2016) (Trinquart, L., Johns, D. M., & Galea, S. (2016). Why do we think we know what we know? A metaknowledge analysis of the salt controversy. International Journal of Epidemiology, 45(1), 251–260. https://doi.org/10.1093/ije/dyv184 ), which collected 68 PSRs, 14 SRRs, 11 clinical guideline reports, and 176 comments, letters, or narrative reviews. Note that our dataset covers only the 68 PSRs and 14 SRRs from Trinquart et al. 2016, not the other types of publications, and it adds additional information noted below. This dataset can be used to construct the inclusion network and the co-author network of the 14 SRRs and 68 PSRs. A PSR is "included" in an SRR if it is considered in the SRR's evidence synthesis. Each included PSR is cited in the SRR, but not all references cited in an SRR are included in the evidence synthesis or PSRs. Based on which PSRs are included in which SRRs, we can construct the inclusion network. The inclusion network is a bipartite network with two types of nodes: one type represents SRRs, and the other represents PSRs. In an inclusion network, if an SRR includes a PSR, there is a directed edge from the SRR to the PSR. The attribute file (report_list.csv) includes attributes of the 82 reports, and the edge list file (inclusion_net_edges.csv) contains the edge list of the inclusion network. Notably, 11 PSRs have never been included in any SRR in the dataset. They are unused PSRs. If visualized with the inclusion network, they will appear as isolated nodes. We used a custom-made workflow (Fu, Y. (2022). Scopus author info tool (1.0.1) [Python]. https://github.com/infoqualitylab/Scopus_author_info_collection ) that uses the Scopus API and manual work to extract and disambiguate authorship information for the 82 reports. The author information file (salt_cont_author.csv) is the product of this workflow and can be used to compute the co-author network of the 82 reports. We also provide several other files in this dataset. We collected inclusion criteria (the criteria that make a PSR eligible to be included in an SRR) and recorded them in the file systematic_review_inclusion_criteria.csv. We provide a file (potential_inclusion_link.csv) recording whether a given PSR had been published as of the search date of a given SRR, which makes the PSR potentially eligible for inclusion in the SRR. We also provide a bibliography of the 82 publications (supplementary_reference_list.pdf). Lastly, we discovered minor discrepancies between the inclusion relationships identified by Trinquart et al. (2016) and by us. Therefore, we prepared an additional edge list (inclusion_net_edges_trinquart.csv) to preserve the inclusion relationships identified by Trinquart et al. (2016). UPDATES IN THIS VERSION COMPARED TO V2 (Fu, Yuanxi; Hsiao, Tzu-Kun; Joshi, Manasi Ballal (2022): The Salt Controversy Systematic Review Reports and Primary Study Reports Network Dataset. University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-6128763_V2) - We added a new column "pub_date" to report_list.csv - We corrected mistakes in supplementary_reference_list.pdf for report #28 and report #80. The author of report #28 is not Salisbury D but Khaw, K.-T., & Barrett-Connor, E. Report #80 was mistakenly mixed up with report #81. 

   more » « less
4. Examining data imbalance in crowdsourced reports for improving flash flood situational awareness

   https://doi.org/10.1016/j.ijdrr.2023.103825  

   Esparza, Miguel ; Farahmand, Hamed ; Brody, Samuel ; Mostafavi, Ali ( September 2023 , International Journal of Disaster Risk Reduction)

   The use of crowdsourced data has been finding practical use for enhancing situational awareness during disasters. While recent studies have shown promising results regarding the potential of crowdsourced data (such as user-generated flood reports) for flash flood mapping and situational awareness, little attention has been paid to data imbalance issues that could introduce biases in data and assessment. To address this gap, in this study, we examine biases present in crowdsourced reports to identify data imbalance with a goal of improving disaster situational awareness. Three biases are examined: sample bias, spatial bias, and demographic bias. To examine these biases, we analyzed reported flooding from 3-1-1 reports (which is a citizen hotline allowing the community to report problems such as flooding) and Waze reports (which is a GPS navigation app that allows drivers to report flooded roads) with respect to FEMA damage data collected in the aftermaths of Tropical Storm Imelda in Harris County, Texas, in 2019 and Hurricane Ida in New York City in 2021. First, sample bias is assessed by expanding the flood-related categories in 3-1-1 reports. Integrating other flooding related topics into the Global Moran's I and Local Indicator of Spatial Association (LISA) revealed more communities that were impacted by floods. To examine spatial bias, we perform the LISA and BI-LISA tests on the data sets—FEMA damage, 3-1-1 reports, and Waze reports—at the census tract level and census block group level. By looking at two geographical aggregations, we found that the larger spatial aggregations, census tracts, show less data imbalance in the results. Through a regression analysis, we found that 3-1-1 reports and Waze reports have data imbalance limitations in areas where minority populations and single parent households reside. The findings of this study advance understanding of data imbalance and biases in crowdsourced datasets that are growingly used for disaster situational awareness. Through addressing data imbalance issues, researchers and practitioners can proactively mitigate biases in crowdsourced data and prevent biased and inequitable decisions and actions. 

   more » « less
5. Embedded student formative reporting to promote student metacognition and instructor pedagogical adjustments

   https://doi.org/10.55632/pwvas.v95i2.948  

   Richards-Babb, Michelle ( April 2023 , Proceedings of the West Virginia Academy of Science)

   Timely, formative feedback is important for instructors and students. In this project, “embedded students” provided instructors with formative feedback through anonymized reports. Students enrolled in preparatory chemistry course sections in fall 2022 self-selected to submit reports documenting their ongoing experiences in learning chemistry content. Embedded students journaled about (i) hours devoted to the course; (ii) topic-specific content mastery, comfort, and confusion; and (iii) instructor pedagogy. An average of 108 reports per week were anonymized and shared with instructors. Instructor effort was limited to agreeing to receive the weekly reports, look them over, and potentially use them to inform upcoming teaching. Embedded students benefited by thinking about their learning in the class (metacognition) and focusing their learning for the upcoming week. Overall, 300 distinct students submitted 1,513 weekly reports, with overall means of 2.81 hours per week of self-reported time devoted to attending lecture, 4.75 hours per week of self-reported time engaged in additional study, and 7.56 hours per week of self-reported total time devoted to the course. Instructors reported reading over 87% of the reports and using feedback to make an average of 4.6 instructional adjustments ranging from working additional practice problems on difficult concepts, providing answer keys to extra problems, implementing problem-solving/review days, and reviewing difficult material or content from earlier in the course. Instructors indicated that the weekly reports contributed to “instructor beneficence”. Students “felt heard” especially when instructors explicitly and constructively addressed comments from the weekly reports. This work is partially supported by the NSF-funded First2 Network. 

   more » « less