Why DataLad?

When publishing data, merely providing file access is insufficient for a simple reason: data are not static. Released data often (and should!) continue to evolve; file formats can change, bugs will be fixed, new data are added, and derived data needs to be integrated.

While version control systems are a de-facto standard for open source software development, a similar level of tooling and culture is not present in the open data community.

DataLad builds on top of git-annex and extends it with an intuitive command-line interface. It enables users to operate on data using familiar concepts, such as files and directories, while transparently managing data access and authorization with underlying hosting providers.

A powerful and complete Python API is also provided to enable authors of data-centric applications to bring versioning and the fearless acquisition of data into continuous integration workflows.

Contributors

So far, more than 30 individuals have contributed to the development of DataLad. See an up-to-date list of them on GitHub. In addition, there are a number of DataLad extensions packages. Some of them, and their contributors, can be found on GitHub too. Additionally, the DataLad Handbook, a versatile educational resource on (research) data management, has also received contribution by over 35 individuals.

Acknowledgments

DataLad development is funded as a US-German project on collaborative research in computational neuroscience (CRCNS):

  • DataLad - a decentralized system for integrated discovery, management, and publication of digital objects of science (Halchenko / Hanke), co-funded by the US National Science Foundation (NSF 1912266) and the German Federal Ministry of Education and Research (BMBF 01GQ1905).
  • DataGit: converging catalogues, warehouses, and deployment logistics into a federated "data distribution" (Halchenko / Hanke), co-funded by the US National Science Foundation (NSF 1429999) and the German Federal Ministry of Education and Research (BMBF 01GQ1411)

and additional support is provided by

NSF logo BMBF logo NIH NIBIB logo HBP logo Virtual Brain Cloud logo Dartmouth logo Jülich Research Center Logo Heinrich Heine University Logo OvGU logo

DataLad is from the Makers of

NeuroDebian logo PyMVPA logo Studyforrest logo