More Like this

1. Gravity Spy Volunteer Classifications of LIGO Glitches from Observing Runs O1, O2, O3a, and O3b

   https://doi.org/10.5281/zenodo.5911226  

   Zevin, Michael ; Coughlin, Scott ; Chase, Eve ; Allen, Sara ; Bahaadini, Sara ; Berry, Christopher ; Crowston, Kevin ; Harandi, Mabi ; Jackson, Corey ; Kalogera, Vicky ; et al ( January 2022 )

   Abstract

   <p>This dataset contains machine learning and volunteer classifications from the Gravity Spy project. It includes glitches from observing runs O1, O2, O3a and O3b that received at least one classification from a registered volunteer in the project. It also indicates glitches that are nominally retired from the project using our default set of retirement parameters, which are described below. See more details in the Gravity Spy Methods paper. </p> <p>When a particular subject in a citizen science project (in this case, glitches from the LIGO datastream) is deemed to be classified sufficiently it is &#34;retired&#34; from the project. For the Gravity Spy project, retirement depends on a combination of both volunteer and machine learning classifications, and a number of parameterizations affect how quickly glitches get retired. For this dataset, we use a default set of retirement parameters, the most important of which are: </p> <ol><li>A glitches must be classified by at least 2 registered volunteers</li><li>Based on both the initial machine learning classification and volunteer classifications, the glitch has more than a 90% probability of residing in a particular class</li><li>Each volunteer classification (weighted by that volunteer&#39;s confusion matrix) contains a weight equal to the initial machine learning score when determining the final probability</li></ol> <p>The choice of these and other parameterization will affect the accuracy of the retired dataset as well as the number of glitches that are retired, and will be explored in detail in an upcoming publication (Zevin et al. in prep). </p> <p>The dataset can be read in using e.g. Pandas: <br /> &#96;&#96;&#96;<br /> import pandas as pd<br /> dataset &#61; pd.read_hdf(&#39;retired_fulldata_min2_max50_ret0p9.hdf5&#39;, key&#61;&#39;image_db&#39;)<br /> &#96;&#96;&#96;<br /> Each row in the dataframe contains information about a particular glitch in the Gravity Spy dataset. </p> <p><strong>Description of series in dataframe</strong></p> <ul><li>[&#39;1080Lines&#39;, &#39;1400Ripples&#39;, &#39;Air_Compressor&#39;, &#39;Blip&#39;, &#39;Chirp&#39;, &#39;Extremely_Loud&#39;, &#39;Helix&#39;, &#39;Koi_Fish&#39;, &#39;Light_Modulation&#39;, &#39;Low_Frequency_Burst&#39;, &#39;Low_Frequency_Lines&#39;, &#39;No_Glitch&#39;, &#39;None_of_the_Above&#39;, &#39;Paired_Doves&#39;, &#39;Power_Line&#39;, &#39;Repeating_Blips&#39;, &#39;Scattered_Light&#39;, &#39;Scratchy&#39;, &#39;Tomte&#39;, &#39;Violin_Mode&#39;, &#39;Wandering_Line&#39;, &#39;Whistle&#39;] <ul><li>Machine learning scores for each glitch class in the trained model, which for a particular glitch will sum to unity</li></ul> </li><li>[&#39;ml_confidence&#39;, &#39;ml_label&#39;] <ul><li>Highest machine learning confidence score across all classes for a particular glitch, and the class associated with this score</li></ul> </li><li>[&#39;gravityspy_id&#39;, &#39;id&#39;] <ul><li>Unique identified for each glitch on the Zooniverse platform (&#39;gravityspy_id&#39;) and in the Gravity Spy project (&#39;id&#39;), which can be used to link a particular glitch to the full Gravity Spy dataset (which contains GPS times among many other descriptors)</li></ul> </li><li>[&#39;retired&#39;] <ul><li>Marks whether the glitch is retired using our default set of retirement parameters (1&#61;retired, 0&#61;not retired)</li></ul> </li><li>[&#39;Nclassifications&#39;] <ul><li>The total number of classifications performed by registered volunteers on this glitch</li></ul> </li><li>[&#39;final_score&#39;, &#39;final_label&#39;] <ul><li>The final score (weighted combination of machine learning and volunteer classifications) and the most probable type of glitch</li></ul> </li><li>[&#39;tracks&#39;] <ul><li>Array of classification weights that were added to each glitch category due to each volunteer&#39;s classification</li></ul> </li></ul> <p> </p> <p>&#96;&#96;&#96;<br /> For machine learning classifications on all glitches in O1, O2, O3a, and O3b, please see Gravity Spy Machine Learning Classifications on Zenodo</p> <p>For the most recently uploaded training set used in Gravity Spy machine learning algorithms, please see Gravity Spy Training Set on Zenodo.</p> <p>For detailed information on the training set used for the original Gravity Spy machine learning paper, please see Machine learning for Gravity Spy: Glitch classification and dataset on Zenodo. </p>
   More>>
2. Data quality up to the third observing run of Advanced LIGO: Gravity Spy glitch classifications

   https://doi.org/10.1088/1361-6382/acb633  

   Glanzer, Jane ; Banagari, Sharan ; Coughlin, Scott ; Soni, Siddharth ; Zevin, Michael ; Berry, Christopher Philip Luke ; Patane, Oli ; Bahaadini, Sara ; Rohani, Neda ; Crowston, Kevin ; et al ( January 2023 , Classical and Quantum Gravity)

   Understanding the noise in gravitational-wave detectors is central to detecting and interpreting gravitational-wave signals. Glitches are transient, non-Gaussian noise features that can have a range of environmental and instrumental origins. The Gravity Spy project uses a machine-learning algorithm to classify glitches based upon their time–frequency morphology. The resulting set of classified glitches can be used as input to detector-characterisation investigations of how to mitigate glitches, or data-analysis studies of how to ameliorate the impact of glitches. Here we present the results of the Gravity Spy analysis of data up to the end of the third observing run of Advanced LIGO. We classify 233981 glitches from LIGO Hanford and 379805 glitches from LIGO Livingston into morphological classes. We find that the distribution of glitches differs between the two LIGO sites. This highlights the potential need for studies of data quality to be individually tailored to each gravitational-wave observatory.
3. Gravity Spy: Humans, Machines and The Future of Citizen Science

   https://doi.org/10.1145/3022198.3026329  

   Crowston, Kevin ( January 2017 , Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing)

   Gravity Spy is a citizen science project that draws on the contributions of both humans and machines to achieve its scientific goals. The system supports the Laser Interferometer Gravitational Observatory (LIGO) by classifying “glitches” that interfere with observations. The system makes three advances on the current state of the art: explicit training for new volunteers, synergy between machine and human classification and support for discovery of new classes of glitch. As well, it provides a platform for human-centred computing research on motivation, learning and collaboration. The system has been launched and is currently in operation.
4. A biodiversity dataset graph: DataONE

   https://doi.org/10.5281/zenodo.1486279  

   Poelen, Jorrit H. ( January 2018 )

   Abstract

   <p>The intended use of this archive is to facilitate meta-analysis of the Data Observation Network for Earth (DataONE, [1]). </p> <p>DataONE is a distributed infrastructure that provides information about earth observation data. This dataset was derived from the DataONE network using Preston [2] between 17 October 2018 and 6 November 2018, resolving 335,213 urls at an average retrieval rate of about 5 seconds per url, or 720 files per hour, resulting in a data gzip compressed tar archive of 837.3 MB .  </p> <p>The archive associates 325,757 unique metadata urls [3] to 202,063 unique ecological metadata files [4]. Also, the DataONE search index was captured to establish provenance of how the dataset descriptors were found and acquired. During the creation of the snapshot (or crawl), 15,389 urls [5], or 4.7% of urls, did not successfully resolve. </p> <p>To facilitate discovery, the record of the Preston snapshot crawl is included in the preston-ls-* files . There files are derived from the rdf/nquad file with hash://sha256/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f . This file can also be found in the data.tar.gz at data/8c/67/e0/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f/data . For more information about concepts and format, please see [2]. </p> <p>To extract all EML files from the included Preston archive, first extract the hashes assocated with EML files using:</p> <p>cat preston-ls.tsv.gz | gunzip | grep &#34;Version&#34; | grep -v &#34;deeplinker&#34; | grep -v &#34;query/solr&#34; | cut -f1,3 | tr &#39;\t&#39; &#39;\n&#39; | grep &#34;hash://&#34; | sort | uniq &gt; eml-hashes.txt</p> <p>extract data.tar.gz using:</p> <p>~/preston-archive$ tar xzf data.tar.gz </p> <p>then use Preston to extract each hash using something like:</p> <p>~/preston-archive$ preston get hash://sha256/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa<br /> &lt;eml:eml xmlns:eml&#61;&#34;eml://ecoinformatics.org/eml-2.1.1&#34; xmlns:xs&#61;&#34;http://www.w3.org/2001/XMLSchema&#34; xmlns:xsi&#61;&#34;http://www.w3.org/2001/XMLSchema-instance&#34; xmlns:stmml&#61;&#34;http://www.xml-cml.org/schema/stmml_1.1&#34; packageId&#61;&#34;doi:10.18739/A24P9Q&#34; system&#61;&#34;https://arcticdata.io&#34; scope&#61;&#34;system&#34; xsi:schemaLocation&#61;&#34;eml://ecoinformatics.org/eml-2.1.1 ~/development/eml/eml.xsd&#34;&gt;<br />   &lt;dataset&gt;<br />     &lt;alternateIdentifier&gt;urn:x-wmo:md:org.aoncadis.www::d76bc3b5-7b19-11e4-8526-00c0f03d5b7c&lt;/alternateIdentifier&gt;<br />     &lt;alternateIdentifier&gt;d76bc3b5-7b19-11e4-8526-00c0f03d5b7c&lt;/alternateIdentifier&gt;<br />     &lt;title&gt;Airglow Image Data 2011 4 of 5&lt;/title&gt;<br /> ...</p> <p>Alternatively, without using Preston, you can extract the data using the naming convention:</p> <p>data/[x]/[y]/[z]/[hash]/data</p> <p>where x is the first 2 characters of the hash, y the second 2 characters, z the third 2 characters, and hash the full sha256 content hash of the EML file.</p> <p>For example, the hash hash://sha256/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa can be found in the file: data/00/00/2d/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa/data . For more information, see [2].</p> <p>The intended use of this archive is to facilitate meta-analysis of the DataONE dataset network. </p> <p>[1] DataONE, https://www.dataone.org<br /> [2] https://preston.guoda.bio, https://doi.org/10.5281/zenodo.1410543 . DataONE was crawled via Preston with &#34;preston update -u https://dataone.org&#34;.<br /> [3] cat preston-ls.tsv.gz | gunzip | grep &#34;Version&#34; | grep -v &#34;deeplinker&#34; | grep -v &#34;query/solr&#34; | cut -f1,3 | tr &#39;\t&#39; &#39;\n&#39; | grep -v &#34;hash://&#34; | sort | uniq | wc -l<br /> [4] cat preston-ls.tsv.gz | gunzip | grep &#34;Version&#34; | grep -v &#34;deeplinker&#34; | grep -v &#34;query/solr&#34; | cut -f1,3 | tr &#39;\t&#39; &#39;\n&#39; | grep &#34;hash://&#34; | sort | uniq | wc -l<br /> [5] cat preston-ls.tsv.gz | gunzip | grep &#34;Version&#34; | grep  &#34;deeplinker&#34; | grep -v &#34;query/solr&#34; | cut -f1,3 | tr &#39;\t&#39; &#39;\n&#39; | grep -v &#34;hash://&#34; | sort | uniq | wc -l</p> <p>This work is funded in part by grant NSF OAC 1839201 from the National Science Foundation.</p>
   More>>
5. A biodiversity dataset graph: GBIF, iDigBio, BioCASe

   https://doi.org/10.5281/zenodo.3852671  

   Poelen, Jorrit H. ( January 2020 )

   Abstract

   <p>A biodiversity dataset graph: GBIF, iDigBio, BioCASe</p> <p>The intended use of this archive is to facilitate meta-analysis of the Global Biodiversity Information Facility, Integrated Digitized Biocollections, Biological Collection Access Service (GBIF, iDigBio, BioCASe). GBIF, iDigBio and BioCASe help provide access to biological data collections.</p> <p>This dataset provides versioned provenance logs of snapshots of the GBIF, iDigBio, BioCASe network as tracked by Preston [2] between 2018-09-03 and 2020-05-02 using &#34;preston update -u https://gbif.org,https://idigbio.org,http://biocase.org&#34;.</p> <p>This publication contains two types of files: index files and provenance logs. Associated data files are hosted elsewhere for pragmatic reasons. Index files provide a way to link provenance files in time to establish a versioning mechanism. Provenance logs describe how, when, what and where the GBIF, iDigBio, BioCASe content was retrieved. For more information, please visit https://preston.guoda.bio or https://doi.org/10.5281/zenodo.1410543 .  </p> <p>To retrieve and verify the downloaded GBIF, iDigBio, BioCASe biodiversity dataset graph, use the preston[2] command-line tool to &#34;clone&#34; this dataset using:</p> <p>$ java -jar preston.jar ls --remote https://zenodo.org/record/3852671/files &gt; /dev/null</p> <p>Optionally, you can retrieve all associated data (&gt;500GB) files using:</p> <p>$ java -jar preston.jar clone --remote https://zenodo.org/record/3852671/files,https://archive.org/download/biodiversity-dataset-archives/data.zip/data/,https://deeplinker.bio</p> <p>Please note https://archive.org/download/biodiversity-dataset-archives/data.zip/data/ and https://deeplinker.bio are Preston remotes that provided access to GBIF, iDigBio, BioCASe data files at time of writing (25 May 2020). These remotes can replaced with any other Preston remote(s) if needed. This may take a while depending on network speed and hardware constraints. See also https://archive.org/details/biodiversity-dataset-archives .</p> <p>After that, verify the index of the archive by reproducing the following provenance log history:</p> <p>$ java -jar preston.jar history</p> <p>&lt;0659a54f-b713-4f86-a917-5be166a14110&gt; &lt;http://purl.org/pav/hasVersion&gt; &lt;hash://sha256/c253a5311a20c2fc082bf9bac87a1ec5eb6e4e51ff936e7be20c29c8e77dee55&gt; .<br /> &lt;hash://sha256/b83cf099449dae3f633af618b19d05013953e7a1d7d97bc5ac01afd7bd9abe5d&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/c253a5311a20c2fc082bf9bac87a1ec5eb6e4e51ff936e7be20c29c8e77dee55&gt; .<br /> &lt;hash://sha256/7efdea9263e57605d2d2d8b79ccd26a55743123d0c974140c72c8c1cfc679b93&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/b83cf099449dae3f633af618b19d05013953e7a1d7d97bc5ac01afd7bd9abe5d&gt; .<br /> &lt;hash://sha256/05a877bdb8617144fe166a13bf51828d4ad1bc11631c360b9e648a9f7df2bbcd&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/7efdea9263e57605d2d2d8b79ccd26a55743123d0c974140c72c8c1cfc679b93&gt; .<br /> &lt;hash://sha256/b5a30bbd8d51e9faf08d4ddebbc5bda9bab1b12545172f1524ac5ebdb0038bd4&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/05a877bdb8617144fe166a13bf51828d4ad1bc11631c360b9e648a9f7df2bbcd&gt; .<br /> &lt;hash://sha256/1d3817d9cb9fc7de7a3b7a4181daba8de1e52b348280154e8a163c7dd7ee1a7e&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/b5a30bbd8d51e9faf08d4ddebbc5bda9bab1b12545172f1524ac5ebdb0038bd4&gt; .<br /> &lt;hash://sha256/24b3f981c88c747f44ad3372095767cd15dcf81bd6cd2e54328a90a21409df43&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/1d3817d9cb9fc7de7a3b7a4181daba8de1e52b348280154e8a163c7dd7ee1a7e&gt; .<br /> &lt;hash://sha256/ba02b235fd445904eae45b50bc637a195f25e9ca1637bcf26b2dc7f8698aa1fe&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/24b3f981c88c747f44ad3372095767cd15dcf81bd6cd2e54328a90a21409df43&gt; .<br /> &lt;hash://sha256/102cbfb1e800ef795ba1e1c51a34bff9b463b34c9443435069ddc76970c1e9c9&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/ba02b235fd445904eae45b50bc637a195f25e9ca1637bcf26b2dc7f8698aa1fe&gt; .<br /> &lt;hash://sha256/fd27b0552c8a6800a8b3b1b822a2063a3215c1d9887badad09a62746b80846bc&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/102cbfb1e800ef795ba1e1c51a34bff9b463b34c9443435069ddc76970c1e9c9&gt; .<br /> &lt;hash://sha256/20d36a6f879ba1dd797d4288a4f2e32719d3c674156194c2765a3ec6b43f5e17&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/fd27b0552c8a6800a8b3b1b822a2063a3215c1d9887badad09a62746b80846bc&gt; .<br /> &lt;hash://sha256/7801a034fe3c7920e032d2338a690b700ca41a90a92d878fc3a67111cad16d29&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/20d36a6f879ba1dd797d4288a4f2e32719d3c674156194c2765a3ec6b43f5e17&gt; .<br /> &lt;hash://sha256/c1b50502b1ca87046eeb7fe4863d0cf9319b6645ff2142db69f21b4cc23332b6&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/7801a034fe3c7920e032d2338a690b700ca41a90a92d878fc3a67111cad16d29&gt; .<br /> &lt;hash://sha256/dc293e26154b89273791b9674d81110029f987c686b386184d0b66a5b95f9cda&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/c1b50502b1ca87046eeb7fe4863d0cf9319b6645ff2142db69f21b4cc23332b6&gt; .<br /> &lt;hash://sha256/f3ed6aa1bd15ee43d05e138b935040aaa745f6ca8c7e8f2dfbb0a3ae0df66f36&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/dc293e26154b89273791b9674d81110029f987c686b386184d0b66a5b95f9cda&gt; .<br /> &lt;hash://sha256/650a28fff3e03dadba70dc05a34c580c04203380187953fa4a2fb778353fee79&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/f3ed6aa1bd15ee43d05e138b935040aaa745f6ca8c7e8f2dfbb0a3ae0df66f36&gt; .<br /> &lt;hash://sha256/e4e5736e8bfec6c686eedde4c6dfa62845930d04e12dfa6f8a7d70abc3d087df&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/650a28fff3e03dadba70dc05a34c580c04203380187953fa4a2fb778353fee79&gt; .<br /> &lt;hash://sha256/e69d186ff3be11830c2da67d1bfeb896ec6398fc9d555fa26eaae1baa54450fb&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/e4e5736e8bfec6c686eedde4c6dfa62845930d04e12dfa6f8a7d70abc3d087df&gt; .<br /> &lt;hash://sha256/3e7f19a8a78b51437240f49c499e6e7f89b8d58d4e3ceb9480d4356721645cee&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/e69d186ff3be11830c2da67d1bfeb896ec6398fc9d555fa26eaae1baa54450fb&gt; .<br /> &lt;hash://sha256/5c469224fa0b6159bf33a59ddaa0246634e81bddd1728e7bf3540745055eccfa&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/3e7f19a8a78b51437240f49c499e6e7f89b8d58d4e3ceb9480d4356721645cee&gt; .<br /> &lt;hash://sha256/eb2c716ec85158a0785216de1b09965173fc368d12f213c1bf747bbc2e49c6a6&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/5c469224fa0b6159bf33a59ddaa0246634e81bddd1728e7bf3540745055eccfa&gt; .<br /> &lt;hash://sha256/3dd674b7ad16391629948981a9cb6f6f86937d016861c3e59cd6e6bf3589f3b7&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/eb2c716ec85158a0785216de1b09965173fc368d12f213c1bf747bbc2e49c6a6&gt; .<br /> &lt;hash://sha256/480868b59e95f3ce2324a7308dba65795e857d34cfbdcea7440a6f2620c6fbf6&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/3dd674b7ad16391629948981a9cb6f6f86937d016861c3e59cd6e6bf3589f3b7&gt; .<br /> &lt;hash://sha256/58daa9a51e5dc0911163aa1b98d68c801106734cd29eab9980814057351aeb70&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/480868b59e95f3ce2324a7308dba65795e857d34cfbdcea7440a6f2620c6fbf6&gt; .<br /> &lt;hash://sha256/a0a18b0e32f933112084b846863438038f66f63eeeb22fa9d8d734e8a25bb208&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/58daa9a51e5dc0911163aa1b98d68c801106734cd29eab9980814057351aeb70&gt; .<br /> &lt;hash://sha256/a7a5e7c6a4b21bdf67f48d6bea85f438b8133f674027b04625dfadec3ff985f6&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/a0a18b0e32f933112084b846863438038f66f63eeeb22fa9d8d734e8a25bb208&gt; .<br /> &lt;hash://sha256/0e6b49850d96b4b58ea3759ecea45d273a48f074c4edaaec5e008791d7718781&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/a7a5e7c6a4b21bdf67f48d6bea85f438b8133f674027b04625dfadec3ff985f6&gt; .<br /> &lt;hash://sha256/8c0752dc6425b9c716837c9713ce284158b4cff70a1e66be2beb0677018831f4&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/0e6b49850d96b4b58ea3759ecea45d273a48f074c4edaaec5e008791d7718781&gt; .<br /> &lt;hash://sha256/d99fa37caa268f8061980001146ed2a566e814d0740bb1974b76847512be95d3&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/8c0752dc6425b9c716837c9713ce284158b4cff70a1e66be2beb0677018831f4&gt; .<br /> &lt;hash://sha256/af0bb2c89571a30815d4488e72dede84a2ffc102bb87961f06884509fd5d1dae&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/d99fa37caa268f8061980001146ed2a566e814d0740bb1974b76847512be95d3&gt; .<br /> &lt;hash://sha256/261177a96185166f1c301beacf7350abff03d1b5710be6bfd8c4aff9caffef12&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/af0bb2c89571a30815d4488e72dede84a2ffc102bb87961f06884509fd5d1dae&gt; .<br /> &lt;hash://sha256/5a39b7bbe9d1bc46ed2eb7bd76c490b5c85a09369a7cf7dc18fa04532679e9a7&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/261177a96185166f1c301beacf7350abff03d1b5710be6bfd8c4aff9caffef12&gt; .<br /> &lt;hash://sha256/af8f9ed321d9c403617f54a96e3217adc918970fbbfe8b8715359669f4890b63&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/5a39b7bbe9d1bc46ed2eb7bd76c490b5c85a09369a7cf7dc18fa04532679e9a7&gt; .<br /> &lt;hash://sha256/9a41d2583f0b8169ffdd44fb2d3a5e057eba4a10e5d9193d0c6e9dcf07c3119e&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/af8f9ed321d9c403617f54a96e3217adc918970fbbfe8b8715359669f4890b63&gt; .<br /> &lt;hash://sha256/b9864a749112cad2fe19e62bf5d8bad580a7036d363d16d81d5c16be325fa0fd&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/9a41d2583f0b8169ffdd44fb2d3a5e057eba4a10e5d9193d0c6e9dcf07c3119e&gt; .<br /> &lt;hash://sha256/09574d9c1330c2b1bec9b7bf3a55ab9273bedbfed78affd70a058a1a25e052d2&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/b9864a749112cad2fe19e62bf5d8bad580a7036d363d16d81d5c16be325fa0fd&gt; .<br /> &lt;hash://sha256/668d5d6e9c9e7ddb410073ff75eb7f2935c60cc62944ba1fd96ca60feec4a103&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/09574d9c1330c2b1bec9b7bf3a55ab9273bedbfed78affd70a058a1a25e052d2&gt; .<br /> &lt;hash://sha256/6387c9ebed9507a0fbba2d161e83c2da73e0d6fa6dd51fb19ac4a4ca75b839c7&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/668d5d6e9c9e7ddb410073ff75eb7f2935c60cc62944ba1fd96ca60feec4a103&gt; .<br /> &lt;hash://sha256/d79fb9207329a2813b60713cf0968fda10721d576dcb7a36038faf18027eebc1&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/6387c9ebed9507a0fbba2d161e83c2da73e0d6fa6dd51fb19ac4a4ca75b839c7&gt; .<br /> &lt;hash://sha256/6fb7271a2da1543036e39bcdb4c415a46b5437569eaaf0ffdef3e907a2f4309f&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/d79fb9207329a2813b60713cf0968fda10721d576dcb7a36038faf18027eebc1&gt; .<br /> &lt;hash://sha256/ab62f4a9601f30d23353a479830f9d2dfc7898e15d2cc2d81977e898d885c908&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/6fb7271a2da1543036e39bcdb4c415a46b5437569eaaf0ffdef3e907a2f4309f&gt; .<br /> &lt;hash://sha256/ff74959ec6e5e98e7db674afcb915f50725f049b968e9a9f10de169aa0a3dcb5&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/ab62f4a9601f30d23353a479830f9d2dfc7898e15d2cc2d81977e898d885c908&gt; .<br /> &lt;hash://sha256/6c4c94cdb224d39e7c655b1a1a6afbba8daf3c9ac64c42ba72dfd346d5d3a547&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/ff74959ec6e5e98e7db674afcb915f50725f049b968e9a9f10de169aa0a3dcb5&gt; .<br /> &lt;hash://sha256/9c17ce013b33c3c9e6bc513cb49a14660fad9bd6f87a4f21568cc871b10ba39b&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/6c4c94cdb224d39e7c655b1a1a6afbba8daf3c9ac64c42ba72dfd346d5d3a547&gt; .<br /> &lt;hash://sha256/5dcf876c6cb0c5b15197acf1ea6989d41c1a1333c6a7e0437f035aa9d22a3790&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/9c17ce013b33c3c9e6bc513cb49a14660fad9bd6f87a4f21568cc871b10ba39b&gt; .<br /> &lt;hash://sha256/39f83f5805f32f765003c5e9ee8c69adb3889d9f26dd61bf4aa3a829ac744e2c&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/5dcf876c6cb0c5b15197acf1ea6989d41c1a1333c6a7e0437f035aa9d22a3790&gt; .<br /> &lt;hash://sha256/916255b2b73680595dcb22b30991a757dd223208473fb4fbe90405757bc07953&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/39f83f5805f32f765003c5e9ee8c69adb3889d9f26dd61bf4aa3a829ac744e2c&gt; .<br /> &lt;hash://sha256/3b39831bcc286c1db44787e21b736378f5847a16b7c39bdac3dd2011e9189dc1&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/916255b2b73680595dcb22b30991a757dd223208473fb4fbe90405757bc07953&gt; .<br /> &lt;hash://sha256/f13b15a20e4fe70b4a111e67ac20ef676404b8456dfc39694f2cb3a4c62a2b2d&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/3b39831bcc286c1db44787e21b736378f5847a16b7c39bdac3dd2011e9189dc1&gt; .<br /> &lt;hash://sha256/8aacce08462b87a345d271081783bdd999663ef90099212c8831db399fc0831b&gt; &lt;http://purl.org/pav/previousVersion&gt; &lt;hash://sha256/f13b15a20e4fe70b4a111e67ac20ef676404b8456dfc39694f2cb3a4c62a2b2d&gt; .</p> <p><br /> If you retrieved data files, you can check the integrity of the extracted archive by confirming that each line produce by the command &#34;preston verify&#34; produces lines as shown below, with each line including &#34;CONTENT_PRESENT_VALID_HASH&#34;. Depending on hardware capacity, this may take a while.</p> <p>$ java -jar preston.jar verify<br /> hash://sha256/3eff98d4b66368fd8d1f8fa1af6a057774d8a407a4771490beeb9e7add76f362    file:/home/preston/preston-archive/data/3e/ff/3eff98d4b66368fd8d1f8fa1af6a057774d8a407a4771490beeb9e7add76f362    OK    CONTENT_PRESENT_VALID_HASH    89931<br /> hash://sha256/184886cc6ae4490a49a70b6fd9a3e1dfafce433fc8e3d022c89e0b75ea3cda0b    file:/home/preston/preston-archive/data/18/48/184886cc6ae4490a49a70b6fd9a3e1dfafce433fc8e3d022c89e0b75ea3cda0b    OK    CONTENT_PRESENT_VALID_HASH    210344<br /> hash://sha256/1846abf2b9623697cf9b2212e019bc1f6dc4a20da51b3b5629bfb964dc808c02    file:/home/preston/preston-archive/data/18/46/1846abf2b9623697cf9b2212e019bc1f6dc4a20da51b3b5629bfb964dc808c02    OK    CONTENT_PRESENT_VALID_HASH    210344<br /> hash://sha256/554fdab07f2372bf363a1d7ef30fcf4c32e1da98b95a6342780c5eb35e0e7b38    file:/home/preston/preston-archive/data/55/4f/554fdab07f2372bf363a1d7ef30fcf4c32e1da98b95a6342780c5eb35e0e7b38    OK    CONTENT_PRESENT_VALID_HASH    202701</p> <p>Note that a copy of the java program &#34;preston&#34;, preston.jar, is included in this publication. The program runs on java 8&#43; virtual machine using &#34;java -jar preston.jar&#34;, or in short &#34;preston&#34;.</p> <p>Files in this data publication:</p> <p>--- start of file descriptions ---</p> <p>-- description of archive and its contents (this file) --<br /> README</p> <p>-- executable java jar containing preston[2] v0.1.15. --<br /> preston.jar</p> <p>-- individual provenance index files --</p> <p>049b0eb995b484c1e64184f582f51b3c608dcade70c4aefc2d53f903bae45098<br /> 073315c32d7fd19868449bef1b11b15a86981dee53a31f7f5c882f7e3be413c3<br /> 1172c6927e58113db668409d36b6a2cd84cf1a93e85b50d65d0bd008a5d8aaa4<br /> 1707cb11cd9f696f1a86fd06742c1e14fad856747be88791f79f6fc7c979d5a6<br /> 272ff1f12a573c667634d934d06b8bab0dd9cc6558795287ea99fab87620d005<br /> 2a5de79372318317a382ea9a2cef069780b852b01210ef59e06b640a3539cb5a<br /> 2bbbe11bb1932c6c8fbbc2ed16dde182f53c4cecbe0dd4f779c32f527a61bc62<br /> 37b8b636e939072d0df7246bf077ead4279f9dd33929be322e631104b0641308<br /> 3901b6af522d535fb164823704686e72f73b7798a2a64eaeb817134552c69e2c<br /> 395ed0c95a624f8853116442690965acf69151acd6b33cc4fc710f567828f784<br /> 460c14ed0129c1469c9149ed1030cdc133f110fb32048748323982cb88dd7eda<br /> 477b6c4e9ecf5c8cd1b5502e0245c8622fa4b358f6710f97db39b473ed3d8235<br /> 52b7274f5d795e4987964bb1a327dd6d6e4f65870e6a7aac172481d0ba3013d4<br /> 54786bde04751bc31bf38c9e89c010cfee7de91760e1f5f31218ff11acff8a70<br /> 6135b237a49b37b857801836494f2c36bcb1526bdacf001a9d11727fff6bf1f1<br /> 674937568c0572bc2873f502dca2fe691ba230869f0aba73f5938422654c05cc<br /> 69b4d5ca9643c14501a48a2b1eb24971a6da68da5033c304f7f00b94e16a11d9<br /> 6df3363a236d4f026154ef86b34d9672b111333d0c2be179c43db146864f6ed3<br /> 70066ea7c6a9dd6c2193cdc90b3b1ff7664af235ab245f6c03d1dd497b376570<br /> 7084702f8025c99a6608a3355ccad5ff5e644ad544121f5d524961f7fe29ceb6<br /> 7e9934a1fc580c3f591c295306ab364c2e7a589e91590ab6334514e4b5c28062<br /> 7ebb008412baaac3afcc8af68b796bf4ca98f367cfd61a815eee82cdffeab196<br /> 886edb8d22973bb04fe3b42d12106029a00b9deab3fb77d8787123327b77ae3b<br /> 8a2426eb4b38af30c6ee764463b8684e0dec400e4472a2a53e6eedf246dab178<br /> 8a6d7e2ab026ff56380235fd9696f5e538e5e426b9374f2ddf3a705e186a7788<br /> 8d44c9e36a505e5c3f125e1702ef7473280bf5bcfa624fe5d3998694b67e0887<br /> 94290680edef0f8ac81d5d4d5b8b680ba5ce821df17c4de62464429552c3360e<br /> 95f88f27ed3448534206406738dfb5c5030fe3d6883c6dda261649357600883f<br /> 9d12cae409e8ea0a546f7945cc629d622400000c3338e4710d9c6084fca9274d<br /> 9fa9ea50db419c75251026708183add8973d9e68a79062f7808b110bef21006e<br /> a24abbe089556f51fe9c2a51febdcaf893b419556312bcc63515713fc4a52922<br /> a3b0477fe46f09b0f51c0f651691665c149bc341f5c19996675d849252e86453<br /> a486474333f05884580dd10c54c95999063c7d1bc22e2cbe3bead604aca0a183<br /> a524b9af3f172793998e1f9c5c0e9c949cc935624a17ed3364d32bc0391c9382<br /> aa0e508aeb96f240b551fe92ff4224325ddcdf66f97eef95ac78aec62e53a169<br /> ab34300942ec02cca7adf2744f6fbc1ab7587060bea09ef92b65b66f89d1ddcd<br /> b05d4a17d9a02180669d7eb017102dd1a739fb4615759cba94baf944b2aee29c<br /> b37c79f95c22fc4d657cc89dedd7a870923285da690ad4f5121962492484a142<br /> bc699639e5515a5fc9da9d442357cc8a9ff310a177e54f1646e002723de49f1d<br /> be6d8cd5f1405a5e3e8aa492fb8dab41f6521608834d746e6cbc58d2f550f918<br /> c06f4413a97a5540fbdd40bdbfb194435c154533df7fe388dfdd378084e19c3d<br /> c585b8addfb7f7991ad74c0bae158aecefc6be5b11c28b020135e0f13040e187<br /> c66587e9730a6f68e961240038892df656ea99a1a25f4ff8ce556c07b09a4878<br /> ca289dce66c8b9955c223fe3e906b8f26c12cf53506cebe651b004961f7964af<br /> cea1aab236de5de8da8954797d846c225bf2ad4f8fe3cd413e60ab029f9e1b3e<br /> da05cc27a47e755ebe912fafae434df5bd31a5d92658fe1943acc0a2023fab32<br /> ee473aeda889fd12ac2c76aae06314e5f279cce5f1a736d39bfc097657a82060<br /> fcb2ee4d630a9a1440417b0c46da5bc1578a388d6aedd12189a23283b60dde7d<br /> fef548489bd7bea43ae1c2b7755d38a87f4a8b038a466bf7e7b4ac64d665fd62<br /> ff32a7cbc99eaf6b67695fd94284a9b1b47a76497ef4d10ffc4dae199cc0d7c3</p> <p>--- individual provenance logs --</p> <p>05a877bdb8617144fe166a13bf51828d4ad1bc11631c360b9e648a9f7df2bbcd<br /> 09574d9c1330c2b1bec9b7bf3a55ab9273bedbfed78affd70a058a1a25e052d2<br /> 0e6b49850d96b4b58ea3759ecea45d273a48f074c4edaaec5e008791d7718781<br /> 102cbfb1e800ef795ba1e1c51a34bff9b463b34c9443435069ddc76970c1e9c9<br /> 1d3817d9cb9fc7de7a3b7a4181daba8de1e52b348280154e8a163c7dd7ee1a7e<br /> 20d36a6f879ba1dd797d4288a4f2e32719d3c674156194c2765a3ec6b43f5e17<br /> 24b3f981c88c747f44ad3372095767cd15dcf81bd6cd2e54328a90a21409df43<br /> 261177a96185166f1c301beacf7350abff03d1b5710be6bfd8c4aff9caffef12<br /> 39f83f5805f32f765003c5e9ee8c69adb3889d9f26dd61bf4aa3a829ac744e2c<br /> 3b39831bcc286c1db44787e21b736378f5847a16b7c39bdac3dd2011e9189dc1<br /> 3dd674b7ad16391629948981a9cb6f6f86937d016861c3e59cd6e6bf3589f3b7<br /> 3e7f19a8a78b51437240f49c499e6e7f89b8d58d4e3ceb9480d4356721645cee<br /> 480868b59e95f3ce2324a7308dba65795e857d34cfbdcea7440a6f2620c6fbf6<br /> 58daa9a51e5dc0911163aa1b98d68c801106734cd29eab9980814057351aeb70<br /> 5a39b7bbe9d1bc46ed2eb7bd76c490b5c85a09369a7cf7dc18fa04532679e9a7<br /></p>
   More>>