More Like this

1. Data from: Dynamics and scaling of a small river discharging into the surf zone

   https://doi.org/10.5061/dryad.2280gb608  

   Lou, Yingzhong; Horner-Devine, Alexander; Derakhti, Morteza; Giddings, Sarah; Spydell, Matthew; Rodriguez, Angelica; Simpson, Alexandra (January 2025, Dryad)

   We use an idealized numerical model to investigate the dynamics and fate of a small river discharging into the surf zone. Our study reveals that the plume reaches a steady state, at which point the combined advective and diffusive freshwater fluxes from the surf zone to the inner shelf balance the river discharge. At a steady state, the surf zone is well-mixed vertically due to wave-enhanced vertical turbulent diffusion and has a strong cross-shore salinity gradient. The horizontal gradient drives a cross-shore buoyancy-driven circulation, directed offshore at the surface and onshore near the bottom, which opposes the wave-driven circulation. Using a scaling analysis based on momentum and freshwater budgets, we determine that the steady-state alongshore plume extent (Lp) and the fraction of river water trapped in the surf zone depend on the ratio of the near-field plume length to the surf zone width (Lnf/Lsz) across a wide range of discharge and wave conditions, and a limited set of tidal conditions. This scaling also allows us to predict the residence time and freshwater fraction (or dilution ratio) in the steady-state plume within the surf zone, which range from approximately 0.1 to 10 days and 0.1 to 0.3, respectively. Our findings establish the basic dynamics and scales of an idealized plume in the surf zone, as well as estimates of residence times and dilution rates that may provide guidance to coastal managers. # Data from: Dynamics and scaling of a small river discharging into the surf zone [https://doi.org/10.5061/dryad.2280gb608](https://doi.org/10.5061/dryad.2280gb608) The present dataset includes the [COAWST model](https://www.usgs.gov/centers/whcmsc/science/coawst-a-coupled-ocean-atmosphere-wave-sediment-transport-modeling-system) outputs used to describe the dynamics and scaling of a small river discharging into the surf zone. ## File structure The data are structured as follows: 1. plume_scale.mat - Data of plume scales of all the cases, where * Hs: significant wave height [m] * Q: river discharge [m^3 s^-1] * L_nf: near-field plume length [m] * L_p: alongshore plume extent [m] * h_sz: water depth at the surf zone edge [m] * x_sz: surf zone width [m] * S_in: inflow salinity [PSU] * g_p: reduced gravity at the river mouth [m s^-2] * g_p*_*0: reduced gravity at the river mouth calculated using the density difference between river inflow and ambient ocean water [m s^-2] * Eta_0: water surface elevation anomaly at the river mouth [m] * V_sz: total volume of freshwater trapped in the surf zone [m^3] * T: the time required for the plume to reach a steady state [day] * L_t: plume turning distance [m] * S_bar: averaged salinity in the plume [PSU] 2. DepthAveraged.mat - Depth-averaged flow fields. DepthAveraged_BaseCase.mat, DepthAveraged_Case1.mat, DepthAveraged_Case3.mat, DepthAveraged_Case4.mat, DepthAveraged_Case6.mat, DepthAveraged_Case7.mat, DepthAveraged_Case8.mat, DepthAveraged_Case9.mat, DepthAveraged_Case16.mat, DepthAveraged_Case17.mat, DepthAveraged_Case18.mat, DepthAveraged_Case19.mat includes the results of the base case, cases 1, 3, 4, 6-9, and 16-19, respectively. In these files: * Wetdry_mask: wet/dry mask on RHO-points [binary] * Wetdry_mask_u: wet/dry mask on U-points [binary] * Wetdry_mask_v: wet/dry mask on V-points [binary] * Z: free-surface [m] * S: surface salinity [PSU] * Hs: significant wave height [m] * U: vertically integrated u-momentum component [m s^-1] * U_st: vertically-integrated u-Stokes drift velocity [m s^-1] * V: vertically integrated v-momentum component [m s^-1] * V_st: vertically-integrated v-Stokes drift velocity [m s^-1] 3. FullField_BaseCase.mat - 3D flow fields for the base case, where * Z: free-surface [m] * S: salinity [PSU] * Hs: significant wave height [m] * Lw: mean wavelength [m] * U: u-momentum component [m s^-1] * U_st: u-Stokes drift velocity [m s^-1] * V: v-momentum component [m s^-1] * V_st: v-Stokes drift velocity [m s^-1] * W: w-momentum component [m s^-1] * W_st: w-Stokes drift velocity [m s^-1] * Aks: salinity vertical diffusion coefficient [m^2 s^-1] * Akv: vertical viscosity coefficient [m^2 s^-1] * Cs_r: S-coordinate stretching curves at RHO-points [-] * Cs_w: S-coordinate stretching curves at W-points [-] 4. FreshwaterTrace_BaseCase.mat - Time series of freshwater volume and fluxes for the base case, where * i_sz: XI-index of the location of the surf zone edge [-] * i_shore: XI-index of the location of the shoreline [-] * Vsz: volume of freshwater in the plume in the surf zone [m^3] * Vis: volume of freshwater in the plume in the inner shelf [m^3] * Vsz_total: total volume of freshwater in the surf zone [m^3] * Vis_total: total volume of freshwater in the inner shelf [m^3] * R2SZ_flux: freshwater flux discharging into the surf zone [m^3 s^-1] * Vchannel: volume of freshwater in the plume in the river channel [m^3] * Vchannel_total: volume of freshwater in the river channel [m^3] * SBoundary_flux_SZ: the freshwater fluxes through the southern domain boundaries of the surf zone [m^3 s^-1] * SBoundary_flux_IS: the freshwater fluxes through the southern domain boundaries of the inner shelf [m^3 s^-1] * NBoundary_flux_SZ: the freshwater fluxes through the northern domain boundaries of the surf zone [m^3 s^-1] * NBoundary_flux_IS: the freshwater fluxes through the northern domain boundaries of the inner shelf [m^3 s^-1] * WBoundary_flux: the freshwater fluxes through the westhern domain boundary [m^3 s^-1] 5. DepthAveraged_XDiagnostic.mat - Depth-averaged diagnostic output of cross-shore momentum terms. DepthAveraged_XDiagnostic_BaseCase.mat includes the results of the base case at the steady state, and DepthAveraged_XDiagnostic_0day_1mWave.mat includes those at the start of river flow. In these files: * ubar_xadv: time-averaged 2D u-momentum, horizontal XI-advection term [m s^-2] * ubar_yadv: time-averaged 2D u-momentum, horizontal ETA-advection term [m s^-2] * ubar_xvisc: time-averaged 2D u-momentum, horizontal XI-viscosity term [m s^-2] * ubar_yvisc: time-averaged 2D u-momentum, horizontal ETA-viscosity term [m s^-2] * ubar_prsgrd: time-averaged 2D u-momentum, pressure gradient term [m s^-2] * ubar_zqsp: time-averaged 2D u-momentum, quasi-static pressure [m s^-2] * ubar_zbeh: time-averaged 2D u-momentum, Bernoulli head [m s^-2] * ubar_bstr: time-averaged 2D u-momentum, bottom stress term [m s^-2] * ubar_wbrk: time-averaged 2D u-momentum, wave breaking term [m s^-2] 6. DepthAveraged_YDiagnostic_BaseCase.mat - Depth-averaged diagnostic output of alongshore momentum terms, where * vbar_xadv: time-averaged 2D v-momentum, horizontal XI-advection term [m s^-2] * vbar_yadv: time-averaged 2D v-momentum, horizontal ETA-advection term [m s^-2] * vbar_xvisc: time-averaged 2D v-momentum, horizontal XI-viscosity term [m s^-2] * vbar_yvisc: time-averaged 2D v-momentum, horizontal ETA-viscosity term [m s^-2] * vbar_prsgrd: time-averaged 2D v-momentum, pressure gradient term [m s^-2] * vbar_zqsp: time-averaged 2D v-momentum, quasi-static pressure [m s^-2] * vbar_zbeh: time-averaged 2D v-momentum, Bernoulli head [m s^-2] * vbar_bstr: time-averaged 2D v-momentum, bottom stress term [m s^-2] * vbar_wbrk: time-averaged 2D v-momentum, wave breaking term [m s^-2] 7. grid.zip - Model grid file. * This grid file is designed for use with [ROMS](https://www.myroms.org/index.php), the hydrodynamic module of the COAWST modeling system. A diagram illustrating how the variables are placed on the grid and where the boundaries lie relative to the grid is available on [WikiROMS](https://www.myroms.org/wiki/Grid_Generation). * This grid file is in NetCDF format, which can be opened and used by a wide range of application software such as MATLAB, Python, and Panoply. For more detailed information, please refer to its [official website](https://www.unidata.ucar.edu/software/netcdf/). ## Code/Software All the post-processing scripts and data are prepared by MATLAB. 

   more » « less
2. Simulation Data for Design of Peptides that Fold and Self-Assemble on Graphite

   https://doi.org/10.5281/zenodo.6426152  

   Comer, Jeffrey; Thakkar, Ravindra; Velásquez-Silva, Astrid; Miranda-Carvajal, Ingrid (January 2022, Zenodo)

   {"Abstract":["This data set for the manuscript entitled "Design of Peptides that Fold and Self-Assemble on Graphite" includes all files needed to run and analyze the simulations described in the this manuscript in the molecular dynamics software NAMD, as well as the output of the simulations. The files are organized into directories corresponding to the figures of the main text and supporting information. They include molecular model structure files (NAMD psf or Amber prmtop format), force field parameter files (in CHARMM format), initial atomic coordinates (pdb format), NAMD configuration files, Colvars configuration files, NAMD log files, and NAMD output including restart files (in binary NAMD format) and trajectories in dcd format (downsampled to 10 ns per frame). Analysis is controlled by shell scripts (Bash-compatible) that call VMD Tcl scripts or python scripts. These scripts and their output are also included.<\/p>\n\nVersion: 2.0<\/p>\n\nChanges versus version 1.0 are the addition of the free energy of folding, adsorption, and pairing calculations (Sim_Figure-7) and shifting of the figure numbers to accommodate this addition.<\/p>\n\n\nConventions Used in These Files\n===============================<\/p>\n\nStructure Files\n----------------\n- graph_*.psf or sol_*.psf (original NAMD (XPLOR?) format psf file including atom details (type, charge, mass), as well as definitions of bonds, angles, dihedrals, and impropers for each dipeptide.)<\/p>\n\n- graph_*.pdb or sol_*.pdb (initial coordinates before equilibration)\n- repart_*.psf (same as the above psf files, but the masses of non-water hydrogen atoms have been repartitioned by VMD script repartitionMass.tcl)\n- freeTop_*.pdb (same as the above pdb files, but the carbons of the lower graphene layer have been placed at a single z value and marked for restraints in NAMD)\n- amber_*.prmtop (combined topology and parameter files for Amber force field simulations)\n- repart_amber_*.prmtop (same as the above prmtop files, but the masses of non-water hydrogen atoms have been repartitioned by ParmEd)<\/p>\n\nForce Field Parameters\n----------------------\nCHARMM format parameter files:\n- par_all36m_prot.prm (CHARMM36m FF for proteins)\n- par_all36_cgenff_no_nbfix.prm (CGenFF v4.4 for graphene) The NBFIX parameters are commented out since they are only needed for aromatic halogens and we use only the CG2R61 type for graphene.\n- toppar_water_ions_prot_cgenff.str (CHARMM water and ions with NBFIX parameters needed for protein and CGenFF included and others commented out)<\/p>\n\nTemplate NAMD Configuration Files\n---------------------------------\nThese contain the most commonly used simulation parameters. They are called by the other NAMD configuration files (which are in the namd/ subdirectory):\n- template_min.namd (minimization)\n- template_eq.namd (NPT equilibration with lower graphene fixed)\n- template_abf.namd (for adaptive biasing force)<\/p>\n\nMinimization\n-------------\n- namd/min_*.0.namd<\/p>\n\nEquilibration\n-------------\n- namd/eq_*.0.namd<\/p>\n\nAdaptive biasing force calculations\n-----------------------------------\n- namd/eabfZRest7_graph_chp1404.0.namd\n- namd/eabfZRest7_graph_chp1404.1.namd (continuation of eabfZRest7_graph_chp1404.0.namd)<\/p>\n\nLog Files\n---------\nFor each NAMD configuration file given in the last two sections, there is a log file with the same prefix, which gives the text output of NAMD. For instance, the output of namd/eabfZRest7_graph_chp1404.0.namd is eabfZRest7_graph_chp1404.0.log.<\/p>\n\nSimulation Output\n-----------------\nThe simulation output files (which match the names of the NAMD configuration files) are in the output/ directory. Files with the extensions .coor, .vel, and .xsc are coordinates in NAMD binary format, velocities in NAMD binary format, and extended system information (including cell size) in text format. Files with the extension .dcd give the trajectory of the atomic coorinates over time (and also include system cell information). Due to storage limitations, large DCD files have been omitted or replaced with new DCD files having the prefix stride50_ including only every 50 frames. The time between frames in these files is 50 * 50000 steps/frame * 4 fs/step = 10 ns. The system cell trajectory is also included for the NPT runs are output/eq_*.xst.<\/p>\n\nScripts\n-------\nFiles with the .sh extension can be found throughout. These usually provide the highest level control for submission of simulations and analysis. Look to these as a guide to what is happening. If there are scripts with step1_*.sh and step2_*.sh, they are intended to be run in order, with step1_*.sh first.<\/p>\n\n\nCONTENTS\n========<\/p>\n\nThe directory contents are as follows. The directories Sim_Figure-1 and Sim_Figure-8 include README.txt files that describe the files and naming conventions used throughout this data set.<\/p>\n\nSim_Figure-1: Simulations of N-acetylated C-amidated amino acids (Ac-X-NHMe) at the graphite\u2013water interface.<\/p>\n\nSim_Figure-2: Simulations of different peptide designs (including acyclic, disulfide cyclized, and N-to-C cyclized) at the graphite\u2013water interface.<\/p>\n\nSim_Figure-3: MM-GBSA calculations of different peptide sequences for a folded conformation and 5 misfolded/unfolded conformations.<\/p>\n\nSim_Figure-4: Simulation of four peptide molecules with the sequence cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) at the graphite\u2013water interface at 370 K.<\/p>\n\nSim_Figure-5: Simulation of four peptide molecules with the sequence cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) at the graphite\u2013water interface at 295 K.<\/p>\n\nSim_Figure-5_replica: Temperature replica exchange molecular dynamics simulations for the peptide cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) with 20 replicas for temperatures from 295 to 454 K.<\/p>\n\nSim_Figure-6: Simulation of the peptide molecule cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) in free solution (no graphite).<\/p>\n\nSim_Figure-7: Free energy calculations for folding, adsorption, and pairing for the peptide CHP1404 (sequence: cyc(GTGSGTG-GPGG-GCGTGTG-SGPG)). For folding, we calculate the PMF as function of RMSD by replica-exchange umbrella sampling (in the subdirectory Folding_CHP1404_Graphene/). We make the same calculation in solution, which required 3 seperate replica-exchange umbrella sampling calculations (in the subdirectory Folding_CHP1404_Solution/). Both PMF of RMSD calculations for the scrambled peptide are in Folding_scram1404/. For adsorption, calculation of the PMF for the orientational restraints and the calculation of the PMF along z (the distance between the graphene sheet and the center of mass of the peptide) are in Adsorption_CHP1404/ and Adsorption_scram1404/. The actual calculation of the free energy is done by a shell script ("doRestraintEnergyError.sh") in the 1_free_energy/ subsubdirectory. Processing of the PMFs must be done first in the 0_pmf/ subsubdirectory. Finally, files for free energy calculations of pair formation for CHP1404 are found in the Pair/ subdirectory.<\/p>\n\nSim_Figure-8: Simulation of four peptide molecules with the sequence cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) where the peptides are far above the graphene\u2013water interface in the initial configuration.<\/p>\n\nSim_Figure-9: Two replicates of a simulation of nine peptide molecules with the sequence cyc(GTGSGTG-GPGG-GCGTGTG-SGPG) at the graphite\u2013water interface at 370 K.<\/p>\n\nSim_Figure-9_scrambled: Two replicates of a simulation of nine peptide molecules with the control sequence cyc(GGTPTTGGGGGGSGGPSGTGGC) at the graphite\u2013water interface at 370 K.<\/p>\n\nSim_Figure-10: Adaptive biasing for calculation of the free energy of the folded peptide as a function of the angle between its long axis and the zigzag directions of the underlying graphene sheet.<\/p>\n\n <\/p>"],"Other":["This material is based upon work supported by the US National Science Foundation under grant no. DMR-1945589. A majority of the computing for this project was performed on the Beocat Research Cluster at Kansas State University, which is funded in part by NSF grants CHE-1726332, CNS-1006860, EPS-1006860, and EPS-0919443. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562, through allocation BIO200030."]} 

   more » « less
3. Molecular Dynamics Simulation of a Designed Cyclic Peptide Bound to CTLA4

   https://doi.org/10.5281/zenodo.7186684  

   Thakkar, Ravindra; Comer, Jeffrey (October 2022, RSC medicinal chemistry)

   This data set for the manuscript entitled "Computational Design of a Cyclic Peptide that Inhibits the CTLA4 Immune Checkpoint Pathway" includes all files needed to run and analyze the simulations of a designed cyclic peptide (Peptide 16) bound to CTLA4 in the putative most stable binding configuration, which is detailed in Figure 6 of the paper. These files include molecular model structure files (NAMD psf), force field parameter files (in CHARMM format), initial atomic coordinates (pdb format), NAMD configuration files, NAMD output including restart files (in binary NAMD format) and trajectories in dcd format (downsampled to 10 ns per frame). Analysis is controlled by shell scripts (Bash-compatible) that call VMD Tcl scripts. These scripts and their output are also included. Version: 1.0 Conventions Used in These Files =============================== Structure Files ---------------- - ctla4_P16_wat.psf (original NAMD (XPLOR?) format psf file including atom details (type, charge, mass), as well as definitions of bonds, angles, dihedrals, and impropers for each dipeptide.) - ctla4_P16.pdb (initial coordinates before equilibration) - repart_*.psf (same as the above psf files, but the masses of non-water hydrogen atoms have been repartitioned by VMD script repartitionMass.tcl) - rest*.pdb (same as the above pdb files, but atoms have been marked for restraints in NAMD. These files are generated by doPrep.sh, with restraints applied to different atoms.) Force Field Parameters ---------------------- CHARMM format parameter files: - par_all36m_prot.prm (CHARMM36m FF for proteins) - toppar_water_ions_prot.str (CHARMM water and ions with NBFIX parameters needed for protein and others commented out) Template NAMD Configuration Files --------------------------------- These contain the most commonly used simulation parameters. They are called by the other NAMD configuration files (which are in the namd/ subdirectory): - template_min.namd (minimization) - template_rest.namd (NPT equilibration with different parts of the protein restrained) - template_prod.namd (for the long production simulations) Minimization ------------- - namd/min_*.0.namd Restraints ------------- - namd/rest_*.0.namd (both CTLA4 binding site and peptide atoms are restrained) - namd/rest_*.1.namd (CA atoms of CTLA4 and all atoms of the peptide are restrained) - namd/rest_*.2.namd (all atoms of only the peptide are restrained) - namd/rest_*.3.namd (only CA atoms of only the peptide are restrained) - namd/rest_*.4.namd (no atoms are restrained) Production ------------- - namd/pro_*.{D,E,F}.0.namd Analysis ------------- - interaction.sh (Shell script for running analysis with VMD) - calcSeparationNearestAtom.tcl (Calculate the separation between two selections, taking the shortest distance between any pair of atoms spanning the two selections. Accounts for (orthogonal) periodic boundary conditions.) - useful.tcl (VMD Tcl script with a library of useful procs, used by the script above) - sep_*.dat (Output of the above analysis containing rows with two columns: time in nanoseconds and minimum distance in Å) Scripts ------- Files with the .sh extension can be found throughout. These usually provide the highest level control for submission of simulations and analysis. Look to these as a guide to what is happening. 

   more » « less
4. Experimentation Framework for Wireless Communication Systems under Jamming Scenarios Dataset

   https://doi.org/10.5281/zenodo.4749668  

   Jacovic, Marko (January 2021, Zenodo)

   {"Abstract":["Data files were used in support of the research paper titled "\u201cExperimentation Framework for Wireless\nCommunication Systems under Jamming Scenarios" which has been submitted to the IET Cyber-Physical Systems: Theory & Applications journal. <\/p>\n\nAuthors: Marko Jacovic, Michael J. Liston, Vasil Pano, Geoffrey Mainland, Kapil R. Dandekar\nContact: krd26@drexel.edu<\/p>\n\n---------------------------------------------------------------------------------------------<\/p>\n\nTop-level directories correspond to the case studies discussed in the paper. Each includes the sub-directories: logs, parsers, rayTracingEmulation, results. <\/p>\n\n--------------------------------<\/p>\n\nlogs:    - data logs collected from devices under test\n    - 'defenseInfrastucture' contains console output from a WARP 802.11 reference design network. Filename structure follows '*x*dB_*y*.txt' in which *x* is the reactive jamming power level and *y* is the jaming duration in samples (100k samples = 1 ms). 'noJammer.txt' does not include the jammer and is a base-line case. 'outMedian.txt' contains the median statistics for log files collected prior to the inclusion of the calculation in the processing script. \n    - 'uavCommunication' contains MGEN logs at each receiver for cases using omni-directional and RALA antennas with a 10 dB constant jammer and without the jammer. Omni-directional folder contains multiple repeated experiments to provide reliable results during each calculation window. RALA directories use s*N* folders in which *N* represents each antenna state. \n    - 'vehicularTechnologies' contains MGEN logs at the car receiver for different scenarios. 'rxNj_5rep.drc' does not consider jammers present, 'rx33J_5rep.drc' introduces the periodic jammer, in 'rx33jSched_5rep.drc' the device under test uses time scheduling around the periodic jammer, in 'rx33JSchedRandom_5rep.drc' the same modified time schedule is used with a random jammer. <\/p>\n\n--------------------------------<\/p>\n\nparsers:    - scripts used to collect or process the log files used in the study\n        - 'defenseInfrastructure' contains the 'xputFiveNodes.py' script which is used to control and log the throughput of a 5-node WARP 802.11 reference design network. Log files are manually inspected to generate results (end of log file provides a summary). \n        - 'uavCommunication' contains a 'readMe.txt' file which describes the parsing of the MGEN logs using TRPR. TRPR must be installed to run the scripts and directory locations must be updated. \n        - 'vehicularTechnologies' contains the 'mgenParser.py' script and supporting 'bfb.json' configuration file which also require TRPR to be installed and directories to be updated. <\/p>\n\n--------------------------------<\/p>\n\nrayTracingEmulation:    - 'wirelessInsiteImages': images of model used in Wireless Insite\n            - 'channelSummary.pdf': summary of channel statistics from ray-tracing study\n            - 'rawScenario': scenario files resulting from code base directly from ray-tracing output based on configuration defined by '*WI.json' file \n            - 'processedScenario': pre-processed scenario file to be used by DYSE channel emulator based on configuration defined by '*DYSE.json' file, applies fixed attenuation measured externally by spectrum analyzer and additional transmit power per node if desired\n            - DYSE scenario file format: time stamp (milli seconds), receiver ID, transmitter ID, main path gain (dB), main path phase (radians), main path delay (micro seconds), Doppler shift (Hz), multipath 1 gain (dB), multipath 1 phase (radians), multipath 1 delay relative to main path delay (micro seconds), multipath 2 gain (dB), multipath 2 phase (radians), multipath 2 delay relative to main path delay (micro seconds)\n            - 'nodeMapping.txt': mapping of Wireless Insite transceivers to DYSE channel emulator physical connections required\n            - 'uavCommunication' directory additionally includes 'antennaPattern' which contains the RALA pattern data for the omni-directional mode ('omni.csv') and directional state ('90.csv')<\/p>\n\n--------------------------------<\/p>\n\nresults:    - contains performance results used in paper based on parsing of aforementioned log files\n <\/p>"]} 

   more » « less
5. A biodiversity dataset graph: DataONE

   https://doi.org/10.5281/zenodo.1486279  

   Poelen, Jorrit H. (January 2018, Zenodo)

   {"Abstract":["The intended use of this archive is to facilitate meta-analysis of the Data Observation Network for Earth (DataONE, [1]). <\/p>\n\nDataONE is a distributed infrastructure that provides information about earth observation data. This dataset was derived from the DataONE network using Preston [2] between 17 October 2018 and 6 November 2018, resolving 335,213 urls at an average retrieval rate of about 5 seconds per url, or 720 files per hour, resulting in a data gzip compressed tar archive of 837.3 MB .  <\/p>\n\nThe archive associates 325,757 unique metadata urls [3] to 202,063 unique ecological metadata files [4]. Also, the DataONE search index was captured to establish provenance of how the dataset descriptors were found and acquired. During the creation of the snapshot (or crawl), 15,389 urls [5], or 4.7% of urls, did not successfully resolve. <\/p>\n\nTo facilitate discovery, the record of the Preston snapshot crawl is included in the preston-ls-* files . There files are derived from the rdf/nquad file with hash://sha256/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f . This file can also be found in the data.tar.gz at data/8c/67/e0/8c67e0741d1c90db54740e08d2e39d91dfd73566ea69c1f2da0d9ab9780a9a9f/data . For more information about concepts and format, please see [2]. <\/p>\n\nTo extract all EML files from the included Preston archive, first extract the hashes assocated with EML files using:<\/p>\n\ncat preston-ls.tsv.gz | gunzip | grep "Version" | grep -v "deeplinker" | grep -v "query/solr" | cut -f1,3 | tr '\\t' '\\n' | grep "hash://" | sort | uniq > eml-hashes.txt<\/p>\n\nextract data.tar.gz using:<\/p>\n\n~/preston-archive$$ tar xzf data.tar.gz <\/p>\n\nthen use Preston to extract each hash using something like:<\/p>\n\n~/preston-archive$$ preston get hash://sha256/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa\n<eml:eml xmlns:eml="eml://ecoinformatics.org/eml-2.1.1" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:stmml="http://www.xml-cml.org/schema/stmml_1.1" packageId="doi:10.18739/A24P9Q" system="https://arcticdata.io" scope="system" xsi:schemaLocation="eml://ecoinformatics.org/eml-2.1.1 ~/development/eml/eml.xsd">\n  <dataset>\n    <alternateIdentifier>urn:x-wmo:md:org.aoncadis.www::d76bc3b5-7b19-11e4-8526-00c0f03d5b7c</alternateIdentifier>\n    <alternateIdentifier>d76bc3b5-7b19-11e4-8526-00c0f03d5b7c</alternateIdentifier>\n    <title>Airglow Image Data 2011 4 of 5</title>\n...<\/p>\n\nAlternatively, without using Preston, you can extract the data using the naming convention:<\/p>\n\ndata/[x]/[y]/[z]/[hash]/data<\/p>\n\nwhere x is the first 2 characters of the hash, y the second 2 characters, z the third 2 characters, and hash the full sha256 content hash of the EML file.<\/p>\n\nFor example, the hash hash://sha256/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa can be found in the file: data/00/00/2d/00002d0fc9e35a9194da7dd3d8ce25eddee40740533f5af2397d6708542b9baa/data . For more information, see [2].<\/p>\n\nThe intended use of this archive is to facilitate meta-analysis of the DataONE dataset network. <\/p>\n\n[1] DataONE, https://www.dataone.org\n[2] https://preston.guoda.bio, https://doi.org/10.5281/zenodo.1410543 . DataONE was crawled via Preston with "preston update -u https://dataone.org".\n[3] cat preston-ls.tsv.gz | gunzip | grep "Version" | grep -v "deeplinker" | grep -v "query/solr" | cut -f1,3 | tr '\\t' '\\n' | grep -v "hash://" | sort | uniq | wc -l\n[4] cat preston-ls.tsv.gz | gunzip | grep "Version" | grep -v "deeplinker" | grep -v "query/solr" | cut -f1,3 | tr '\\t' '\\n' | grep "hash://" | sort | uniq | wc -l\n[5] cat preston-ls.tsv.gz | gunzip | grep "Version" | grep  "deeplinker" | grep -v "query/solr" | cut -f1,3 | tr '\\t' '\\n' | grep -v "hash://" | sort | uniq | wc -l<\/p>\n\nThis work is funded in part by grant NSF OAC 1839201 from the National Science Foundation.<\/p>"]} 

   more » « less