A ubiquitous problem in aggregating data across different experimental and observational data sources is a lack of software infrastructure that enables flexible and extensible standardization of data and metadata. To address this challenge, we developed HDMF, a hierarchical data modeling framework for modern science data standards. With HDMF, we separate the process of data standardization into three main components: (1) data modeling and specification, (2) data I/O and storage, and (3) data interaction and data APIs. To enable standards to support the complex requirements and varying use cases throughout the data life cycle, HDMF provides object mapping infrastructure to insulate and integrate these various components. This approach supports the flexible development of data standards and extensions, optimized storage backends, and data APIs, while allowing the other components of the data standards ecosystem to remain stable. To meet the demands of modern, large-scale science data, HDMF provides advanced data I/O functionality for iterative data write, lazy data load, and parallel I/O. It also supports optimization of data storage via support for chunking, compression, linking, and modular data storage. We demonstrate the application of HDMF in practice to design NWB 2.0, a modern data standard for collaborative science across the neurophysiology community.
more »
« less
Nonlinear thermoremanence acquisition and implications for paleointensity data (Dataset)
Paleomagnetic, rock magnetic, or geomagnetic data found in the MagIC data repository from a paper titled: Nonlinear thermoremanence acquisition and implications for paleointensity data
more »
« less
- Award ID(s):
- 2126298
- PAR ID:
- 10558648
- Publisher / Repository:
- Magnetics Information Consortium (MagIC)
- Date Published:
- Subject(s) / Keyword(s):
- Igneous Intrusive Pluton Anorthosite 2700000000 2700000000 Years BP
- Format(s):
- Medium: X
- Location:
- (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139); (Latitude:45.45; Longitude:-102.139)
- Right(s):
- Creative Commons Attribution 4.0 International
- Institution:
- Paleomagnetic Lab Scripps Institution Of Oceanography, UCSD, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Pooling and sharing data increases and distributes its value. But since data cannot be revoked once shared, scenarios that require controlled release of data for regulatory, privacy, and legal reasons default to not sharing. Because selectively controlling what data to release is difficult, the few data-sharing consortia that exist are often built around data-sharing agreements resulting from long and tedious one-off negotiations. We introduce Data Station, a data escrow designed to enable the formation of data-sharing consortia. Data owners share data with the escrow knowing it will not be released without their consent. Data users delegate their computation to the escrow. The data escrow relies on delegated computation to execute queries without releasing the data first. Data Station leverages hardware enclaves to generate trust among participants, and exploits the centralization of data and computation to generate an audit log. We evaluate Data Station on machine learning and data-sharing applications while running on an untrusted intermediary. In addition to important qualitative advantages, we show that Data Station: i) outperforms federated learning baselines in accuracy and runtime for the machine learning application; ii) is orders of magnitude faster than alternative secure data-sharing frameworks; and iii) introduces small overhead on the critical path.more » « less
-
Recent studies have shown that several government and business organizations experience huge data breaches. Data breaches increase in a daily basis. The main target for attackers is organization sensitive data which includes personal identifiable information (PII) such as social security number (SSN), date of birth (DOB) and credit card /debit card (CCDC). The other target is encryption/decryption keys or passwords to get access to the sensitive data. The cloud computing is emerging as a solution to store, transfer and process the data in a distributed location over the Internet. Big data and internet of things (IoT) increased the possibility of sensitive data exposure. Most methods used for the attack are hacking, unauthorized access, insider theft and false data injection on the move. Most of the attacks happen during three different states of data life cycle such as data-at-rest, data-in-use, and data-in-transit. Hence, protecting sensitive data at all states particularly when data is moving to cloud computing environment needs special attention. The main purpose of this research is to analyze risks caused by data breaches, personal and organizational weaknesses to protect sensitive data and privacy. The paper discusses methods such as data classification and data encryption at different states to protect personal and organizational sensitive data. The paper also presents mathematical analysis by leveraging the concept of birthday paradox to demonstrate the encryption key attack. The analysis result shows that the use of same keys to encrypt sensitive data at different data states make the sensitive data less secure than using different keys. Our results show that to improve the security of sensitive data and to reduce the data breaches, different keys should be used in different states of the data life cycle.more » « less
-
The rate at which humanity is producing data has increased sig- nificantly over the last decade. As organizations generate unprece- dented amounts of data, storing, cleaning, integrating, and ana- lyzing this data consumes significant (human and computational) resources. At the same time organizations extract significant value from their data. In this work, we present our vision for develop- ing an objective metric for the value of data based on the recently introduced concept of data relevance, outline proposals for how to efficiently compute and maintain such metrics, and how to utilize data value to improve data management including storage organi- zation, query performance, intelligent allocation of data collection and curation efforts, improving data catalogs, and for making pric- ing decisions in data markets. While we mostly focus on tabular data, the concepts we introduce can also be applied to other data models such as semi-structure data (e.g., JSON) or property graphs. Furthermore, we discuss strategies for dealing with data and work- loads that evolve and discuss how to deal with data that is currently not relevant, but has potential value (we refer to this as dark data). Furthermore, we sketch ideas for measuring the value that a query / workload has for an organization and reason about the interaction between query and data value.more » « less
-
null (Ed.)Biomedical research results in the collection and storage of increasingly large and complex data sets. Preserving those data so that they are discoverable, accessible, and interpretable accelerates scientific discovery and improves health outcomes, but requires that researchers, data curators, and data archivists consider the long-term disposition of data and the costs of preserving, archiving, and promoting access to them. Life Cycle Decisions for Biomedical Data examines and assesses approaches and considerations for forecasting costs for preserving, archiving, and promoting access to biomedical research data. This report provides a comprehensive conceptual framework for cost-effective decision making that encourages data accessibility and reuse for researchers, data managers, data archivists, data scientists, and institutions that support platforms that enable biomedical research data preservation, discoverability, and use.more » « less
An official website of the United States government
