More Like this

1. NSF COLDEX Ice Penetrating Radar Derived Grids of the Southern Flank of Dome A:Subtitle

   https://doi.org/10.18738/T8/M77ANK  

   Young, Duncan A; Paden, John P; Yan, Shuai; Kerr, Megan E; Singh, Shivangini; Vega_Gonzàlez, Alejandra; Kaundinya, Shravan; Greenbaum, Jamin S; Chan, Kristian; Blankenship, Donald D; et al (January 2025, Texas Data Repository)

   <p>NSF COLDEX performed two airborne campaigns from South Pole Station over the Southern Flank of Dome A and 2022-23 and 2023-24, searching for a potential site of a continuous ice core that could sample the mid-Pleistocene transition. Ice thickness data extracted from the MARFA radar system has allow for a new understanding of this region.</p> <p>Here we generate crustal scale maps of ice thickness, bed elevation, specularity content, subglacial RMS deviation and fractional basal ice thickness with 1 km sampling, and 10 km resolution. We include both masked and unmasked grids.</p> <p> The projection is in the SCAR standard ESPG:3031 polar stereographic projection with true scale at 71˚S.</p> <p>These geotiffs were generated using performed using GMT6.5 (<a href="https://doi.org/10.1029/2019GC008515">Wessel et al., 2019</a>) using the pygmt interface, by binning the raw data to 2.5 km cells, and using the <a href="https://github.com/sakov/nn-c"> nnbathy </a> program to apply natural neighbor interpolation to 1 km sampling. A 10 km Gaussian filter - representing typical lines spacings - was applied and then a mask was applied for all locations where the nearest data point was further than 8 km. </p> Ice thickness, bed elevation and RMS deviation @ 400 m length scale (<a href="http://dx.doi.org/10.1029/2000JE001429">roughness</a>) data includes the following datasets: <ul> <li> UTIG/CRESIS <a href="https://doi.org/10.18738/T8/J38CO5">NSF COLDEX Airborne MARFA data</a></li> <li> British Antarctic Survey <a href="https://doi.org/10.5285/0f6f5a45-d8af-4511-a264-b0b35ee34af6">AGAP-North</a></li> <li> LDEO <a href="https://doi.org/10.1594/IEDA/317765"> AGAP-South </a></li> <li> British Antarctic Survey <a href="https://doi.org/10.5270/esa-8ffoo3e">Polargap</a></li> <li> UTIG Support Office for Airborne Research <a href="https://doi.org/10.15784/601588">Pensacola-Pole Transect (PPT) </a></li> <li> NASA/CReSIS <a href="https://doi.org/10.5067/GDQ0CUCVTE2Q"> 2016 and 2018 Operation Ice Bridge </a> </li> <li> ICECAP/PRIC <a href="https://doi.org/10.15784/601437"> SPICECAP Titan Dome Survey </a> </ul> <p>Specularity content (<a href="https://doi.org/10.1109/LGRS.2014.2337878">Schroeder et al. 2014</a>) is compiled from <a href="https://doi.org/10.18738/T8/KHUT1U"> Young et al. 2025a </a> and <a href="https://doi.org/10.18738/T8/6T5JS6"> Young et al. 2025b</a>.</p> <p>Basal ice fractional thickness is complied from manual interpretation by Vega Gonzàlez, Yan and Singh. </p> <p>Code to generated these grids can be found at <a href="https://github.com/smudog/COLDEX_dichotomy_paper_2025"> at github.com </a></p> 

   more » « less
2. HISDAC-ES: Historical Settlement Data Compilation for Spain (1900-2020)

   https://doi.org/10.6084/m9.figshare.22009643.v2  

   Uhl, Johannes H.; Royé, Dominic; Burghardt, Keith; Aldrey Vázquez, José Antonio; Borobio Sanchiz, Manuel Sanchiz; Leyk, Stefan (January 2023, figshare)

   The historical settlement data compilation for Spain (HISDAC-ES) is a geospatial dataset consisting of over 240 gridded surfaces measuring the physical, functional, age-related, and evolutionary characteristics of the Spanish building stock. We scraped, harmonized, and aggregated cadastral building footprint data for Spain, covering over 12,000,000 building footprints including construction year attributes, to create a multi-faceted series of gridded surfaces (GeoTIFF format), describing the evolution of human settlements in Spain from 1900 to 2020, at 100m spatial and 5 years temporal resolution. Also, the dataset contains aggregated characteristics and completeness statistics at the municipality level, in CSV and GeoPackage format.!!! UPDATE 08-2023 !!!: We provide a new, improved version of HISDAC-ES. Specifically, we fixed two bugs in the production code that caused an incorrect rasterization of the multitemporal BUFA layers and of the PHYS layers (BUFA, BIA, DWEL, BUNITS sum and mean). Moreover, we added decadal raster datasets measuring residential building footprint and building indoor area (1900-2020), and provide a country-wide, harmonized building footprint centroid dataset in GeoPackage vector data format.File descriptions:Datasets are available in three spatial reference systems:HISDAC-ES_All_LAEA.zip: Raster data in Lambert Azimuthal Equal Area (LAEA) covering all Spanish territory.HISDAC-ES_IbericPeninsula_UTM30.zip: Raster data in UTM Zone 30N covering all the Iberic Peninsula + Céuta and Melilla.HISDAC-ES_CanaryIslands_REGCAN.zip: Raster data in REGCAN-95, covering the Canary Islands only.HISDAC-ES_MunicipAggregates.zip: Municipality-level aggregates and completeness statistics (CSV, GeoPackage), in LAEA projection.ES_building_centroids_merged_spatjoin.gpkg: 7,000,000+ building footprint centroids in GeoPackage format, harmonized from the different cadastral systems, representing the input data for HISDAC-ES. These data can be used for sanity checks or for the creation of further, user-defined gridded surfaces.Source data:HISDAC-ES is derived from cadastral building footprint data, available from different authorities in Spain:Araba province: https://geo.araba.eus/WFS_Katastroa?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetCapabilitiesBizkaia province: https://web.bizkaia.eus/es/inspirebizkaiaGipuzkoa province: https://b5m.gipuzkoa.eus/web5000/es/utilidades/inspire/edificios/Navarra region: https://inspire.navarra.es/services/BU/wfsOther regions: http://www.catastro.minhap.es/INSPIRE/buildings/ES.SDGC.bu.atom.xmlData source of municipality polygons: Centro Nacional de Información Geográfica (https://centrodedescargas.cnig.es/CentroDescargas/index.jsp)Technical notes:Gridded dataFile nomenclature:./region_projection_theme/hisdac_es_theme_variable_version_resolution[m][_year].tifRegions:all: complete territory of Spaincan: Canarian Islands onlyibe: Iberic peninsula + Céuta + MelillaProjections:laea: Lambert azimuthal equal area (EPSG:3035)regcan: REGCAN95 / UTM zone 28N (EPSG:4083)utm: ETRS89 / UTM zone 30N (EPSG:25830)Themes:evolution / evol: multi-temporal physical measurementslanduse: multi-temporal building counts per land use (i.e., building function) classphysical / phys: physical building characteristics in 2020temporal / temp: temporal characteristics (construction year statistics)Variables: evolutionbudens: building density (count per grid cell area)bufa: building footprint areadeva: developed area (any grid cell containing at least one building)resbufa: residential building footprint arearesbia: residential building indoor areaVariables: physicalbia: building indoor areabufa: building footprint areabunits: number of building unitsdwel: number of dwellingsVariables: temporalmincoy: minimum construction year per grid cellmaxcoy: minimum construction year per grid cellmeancoy: mean construction year per grid cellmedcoy: median construction year per grid cellmodecoy: mode (most frequent) construction year per grid cellvarcoy: variety of construction years per grid cellVariable: landuseCounts of buildings per grid cell and land use type.Municipality-level datahisdac_es_municipality_stats_multitemporal_longform_v1.csv: This CSV file contains the zonal sums of the gridded surfaces (e.g., number of buildings per year and municipality) in long form. Note that a value of 0 for the year attribute denotes the statistics for records without construction year information.hisdac_es_municipality_stats_multitemporal_wideform_v1.csv: This CSV file contains the zonal sums of the gridded surfaces (e.g., number of buildings per year and municipality) in wide form. Note that a value of 0 for the year suffix denotes the statistics for records without construction year information.hisdac_es_municipality_stats_completeness_v1.csv: This CSV file contains the missingness rates (in %) of the building attribute per municipality, ranging from 0.0 (attribute exists for all buildings) to 100.0 (attribute exists for none of the buildings) in a given municipality.Column names for the completeness statistics tables:NATCODE: National municipality identifier*num_total: number of buildings per municperc_bymiss: Percentage of buildings with missing built year (construction year)perc_lumiss: Percentage of buildings with missing landuse attributeperc_luother: Percentage of buildings with landuse type "other"perc_num_floors_miss: Percentage of buildings without valid number of floors attributeperc_num_dwel_miss: Percentage of buildings without valid number of dwellings attributeperc_num_bunits_miss: Percentage of buildings without valid number of building units attributeperc_offi_area_miss: Percentage of buildings without valid official area (building indoor area, BIA) attributeperc_num_dwel_and_num_bunits_miss: Percentage of buildings missing both number of dwellings and number of building units attributeThe same statistics are available as geopackage file including municipality polygons in Lambert azimuthal equal area (EPSG:3035).*From the NATCODE, other regional identifiers can be derived as follows:NATCODE: 34 01 04 04001Country: 34Comunidad autónoma (CA_CODE): 01Province (PROV_CODE): 04LAU code: 04001 (province + municipality code) 

   more » « less
3. Software Environments in Binder Containers

   https://doi.org/10.5281/zenodo.4891790  

   Shaffer, Tim; Chard, Kyle; Thain, Douglas (January 2021, Zenodo)

   {"Abstract":["Binder is a publicly accessible online service for executing interactive notebooks based on Git repositories. Binder dynamically builds and deploys containers following a recipe stored in the repository, then gives the user a browser-based notebook interface. The Binder group periodically releases a log of container launches from the public Binder service. Archives of launch records are available here. These records do not include identifiable information like IP addresses, but do give the source repo being launched along with some other metadata. The main content of this dataset is in the binder.sqlite<\/code> file. This SQLite database includes launch records from 2018-11-03 to 2021-06-06 in the events<\/code> table, which has the following schema.<\/p>\n\nCREATE TABLE events(\n version INTEGER,\n timestamp TEXT,\n provider TEXT,\n spec TEXT,\n origin TEXT,\n ref TEXT,\n guessed_ref TEXT\n);\nCREATE INDEX idx_timestamp ON events(timestamp);\n<\/code>\n\nversion<\/code> indicates the version of the record as assigned by Binder. The origin<\/code> field became available with version 3, and the ref<\/code> field with version 4. Older records where this information was not recorded will have the corresponding fields set to null.<\/li>timestamp<\/code> is the ISO timestamp of the launch<\/li>provider<\/code> gives the type of source repo being launched ("GitHub" is by far the most common). The rest of the explanations assume GitHub, other providers may differ.<\/li>spec<\/code> gives the particular branch/release/commit being built. It consists of <github-id>/<repo>/<branch><\/code>.<\/li>origin<\/code> indicates which backend was used. Each has its own storage, compute, etc. so this info might be important for evaluating caching and performance. Note that only recent records include this field. May be null.<\/li>ref<\/code> specifies the git commit that was actually used, rather than the named branch referenced by spec<\/code>. Note that this was not recorded from the beginning, so only the more recent entries include it. May be null.<\/li>For records where ref<\/code> is not available, we attempted to clone the named reference given by spec<\/code> rather than the specific commit (see below). The guessed_ref<\/code> field records the commit found at the time of cloning. If the branch was updated since the container was launched, this will not be the exact version that was used, and instead will refer to whatever was available at the time (early 2021). Depending on the application, this might still be useful information. Selecting only records with version 4 (or non-null ref<\/code>) will exclude these guessed commits. May be null.<\/li><\/ul>\n\nThe Binder launch dataset identifies the source repos that were used, but doesn't give any indication of their contents. We crawled GitHub to get the actual specification files in the repos which were fed into repo2docker when preparing the notebook environments, as well as filesystem metadata of the repos. Some repos were deleted/made private at some point, and were thus skipped. This is indicated by the absence of any row for the given commit (or absence of both ref<\/code> and guessed_ref<\/code> in the events<\/code> table). The schema is as follows.<\/p>\n\nCREATE TABLE spec_files (\n ref TEXT NOT NULL PRIMARY KEY,\n ls TEXT,\n runtime BLOB,\n apt BLOB,\n conda BLOB,\n pip BLOB,\n pipfile BLOB,\n julia BLOB,\n r BLOB,\n nix BLOB,\n docker BLOB,\n setup BLOB,\n postbuild BLOB,\n start BLOB\n);<\/code>\n\nHere ref<\/code> corresponds to ref<\/code> and/or guessed_ref<\/code> from the events<\/code> table. For each repo, we collected spec files into the following fields (see the repo2docker docs for details on what these are). The records in the database are simply the verbatim file contents, with no parsing or further processing performed.<\/p>\n\nruntime<\/code>: runtime.txt<\/code><\/li>apt<\/code>: apt.txt<\/code><\/li>conda<\/code>: environment.yml<\/code><\/li>pip<\/code>: requirements.txt<\/code><\/li>pipfile<\/code>: Pipfile.lock<\/code> or Pipfile<\/code><\/li>julia<\/code>: Project.toml<\/code> or REQUIRE<\/code><\/li>r<\/code>: install.R<\/code><\/li>nix<\/code>: default.nix<\/code><\/li>docker<\/code>: Dockerfile<\/code><\/li>setup<\/code>: setup.py<\/code><\/li>postbuild<\/code>: postBuild<\/code><\/li>start<\/code>: start<\/code><\/li><\/ul>\n\nThe ls<\/code> field gives a metadata listing of the repo contents (excluding the .git<\/code> directory). This field is JSON encoded with the following structure based on JSON types:<\/p>\n\nObject: filesystem directory. Keys are file names within it. Values are the contents, which can be regular files, symlinks, or subdirectories.<\/li>String: symlink. The string value gives the link target.<\/li>Number: regular file. The number value gives the file size in bytes.<\/li><\/ul>\n\nCREATE TABLE clean_specs (\n ref TEXT NOT NULL PRIMARY KEY,\n conda_channels TEXT,\n conda_packages TEXT,\n pip_packages TEXT,\n apt_packages TEXT\n);<\/code>\n\nThe clean_specs<\/code> table provides parsed and validated specifications for some of the specification files (currently Pip, Conda, and APT packages). Each column gives either a JSON encoded list of package requirements, or null. APT packages have been validated using a regex adapted from the repo2docker source. Pip packages have been parsed and normalized using the Requirement class from the pkg_resources package of setuptools. Conda packages have been parsed and normalized using the conda.models.match_spec.MatchSpec<\/code> class included with the library form of Conda (distinct from the command line tool). Users might want to use these parsers when working with the package data, as the specifications can become fairly complex.<\/p>\n\nThe missing<\/code> table gives the repos that were not accessible, and event_logs<\/code> records which log files have already been added. These tables are used for updating the dataset and should not be of interest to users.<\/p>"]} 

   more » « less
4. NSF COLDEX/Open Polar Radar Example Delay Doppler Classification of Englacial Reflectors

   https://doi.org/10.18738/T8/DM10IG  

   Young, Duncan; Ng, Gregory; Kempf, Scott D; Young, Duncan (January 2025, Texas Data Repository)

   <p>This is an example line of NSF COLDEX MARFA ice penetrating radar data (CLX/MKB2o/R66a) that has been processed to provide azimuthal information about radar echos from below, and to the front and back of the aircraft. The input was 1 meter slow time resampled coherent range record with phase intact. The data were pulse compressed and an azimuth fast Fourier transform was used to convert to azimuth angles in 1 km chunks, then slices at -19°, +19˚ and nadir were selected for these numpy arrays. These can be displayed as an RGB image with Blue = nadir, red = forward and green = rear</p> <p>The nadir slice should dominate specular echos, as seen with englacial reflecting horizons; where this trades to more balanced returns across all three channels, scattering dominates, as with rough bed rock or volume scattering. A gmt text file contains information about where this transition occurs in the ice column.</p> <p>Details in delay Doppler processing can be found in <a href="http://pds-geosciences.wustl.edu/mro/mro-m-sharad-5-radargram- v1/mrosh_2001/document/rgram_processing.pdf">Campbell et al., 2014</a>; the idea for using this approach for looking at englacial structure was discussed by <a href="https://doi.org/10.5194/egusphere-egu23-2856">Arenas-Pingarrón, Á. et al., 2023</a>. Details of HiCARs/MARFA focused processing can be found in <a href="http://dx.doi.org/10.1109/TGRS.2007.897416">Peters et al., 2007</a>.</p> 

   more » « less
5. Mitigating RF Jamming Attacks at the Physical Layer with Machine Learning Dataset

   https://doi.org/10.5281/zenodo.6304194  

   Jacovic, Marko; Rivas, Xaime Rey (January 2022, Zenodo)

   {"Abstract":["Data files were used in support of the research paper titled \u201cMitigating RF Jamming Attacks at the Physical Layer with Machine Learning<\/em>" which has been submitted to the IET Communications journal.<\/p>\n\n---------------------------------------------------------------------------------------------<\/p>\n\nAll data was collected using the SDR implementation shown here: https://github.com/mainland/dragonradio/tree/iet-paper. Particularly for antenna state selection, the files developed for this paper are located in 'dragonradio/scripts/:'<\/p>\n\n'ModeSelect.py': class used to defined the antenna state selection algorithm<\/li>'standalone-radio.py': SDR implementation for normal radio operation with reconfigurable antenna<\/li>'standalone-radio-tuning.py': SDR implementation for hyperparameter tunning<\/li>'standalone-radio-onmi.py': SDR implementation for omnidirectional mode only<\/li><\/ul>\n\n---------------------------------------------------------------------------------------------<\/p>\n\nAuthors: Marko Jacovic, Xaime Rivas Rey, Geoffrey Mainland, Kapil R. Dandekar\nContact: krd26@drexel.edu<\/p>\n\n---------------------------------------------------------------------------------------------<\/p>\n\nTop-level directories and content will be described below. Detailed descriptions of experiments performed are provided in the paper.<\/p>\n\n---------------------------------------------------------------------------------------------<\/p>\n\nclassifier_training: files used for training classifiers that are integrated into SDR platform<\/p>\n\n'logs-8-18' directory contains OTA SDR collected log files for each jammer type and under normal operation (including congested and weaklink states)<\/li>'classTrain.py' is the main parser for training the classifiers<\/li>'trainedClassifiers' contains the output classifiers generated by 'classTrain.py'<\/li><\/ul>\n\npost_processing_classifier: contains logs of online classifier outputs and processing script<\/p>\n\n'class' directory contains .csv logs of each RTE and OTA experiment for each jamming and operation scenario<\/li>'classProcess.py' parses the log files and provides classification report and confusion matrix for each multi-class and binary classifiers for each observed scenario - found in 'results->classifier_performance'<\/li><\/ul>\n\npost_processing_mgen: contains MGEN receiver logs and parser<\/p>\n\n'configs' contains JSON files to be used with parser for each experiment<\/li>'mgenLogs' contains MGEN receiver logs for each OTA and RTE experiment described. Within each experiment logs are separated by 'mit' for mitigation used, 'nj' for no jammer, and 'noMit' for no mitigation technique used. File names take the form *_cj_* for constant jammer, *_pj_* for periodic jammer, *_rj_* for reactive jammer, and *_nj_* for no jammer. Performance figures are found in 'results->mitigation_performance'<\/li><\/ul>\n\nray_tracing_emulation: contains files related to Drexel area, Art Museum, and UAV Drexel area validation RTE studies.<\/p>\n\nDirectory contains detailed 'readme.txt' for understanding.<\/li>Please note: the processing files and data logs present in 'validation' folder were developed by Wolfe et al. and should be cited as such, unless explicitly stated differently. \n\tS. Wolfe, S. Begashaw, Y. Liu and K. R. Dandekar, "Adaptive Link Optimization for 802.11 UAV Uplink Using a Reconfigurable Antenna," MILCOM 2018 - 2018 IEEE Military Communications Conference (MILCOM), 2018, pp. 1-6, doi: 10.1109/MILCOM.2018.8599696.<\/li><\/ul>\n\t<\/li><\/ul>\n\nresults: contains results obtained from study<\/p>\n\n'classifier_performance' contains .txt files summarizing binary and multi-class performance of online SDR system. Files obtained using 'post_processing_classifier.'<\/li>'mitigation_performance' contains figures generated by 'post_processing_mgen.'<\/li>'validation' contains RTE and OTA performance comparison obtained by 'ray_tracing_emulation->validation->matlab->outdoor_hover_plots.m'<\/li><\/ul>\n\ntuning_parameter_study: contains the OTA log files for antenna state selection hyperparameter study<\/p>\n\n'dataCollect' contains a folder for each jammer considered in the study, and inside each folder there is a CSV file corresponding to a different configuration of the learning parameters of the reconfigurable antenna. The configuration selected was the one that performed the best across all these experiments and is described in the paper.<\/li>'data_summary.txt'this file contains the summaries from all the CSV files for convenience.<\/li><\/ul>"]} 

   more » « less