Update 04_create_dataset_card.m
Browse files
sections/04_create_dataset_card.md
CHANGED
|
@@ -2,85 +2,85 @@
|
|
| 2 |
|
| 3 |
The `README.md` is a markdown file that is displayed when goes to the front page for the dataset. It should give appropriate context for the dataset and guidance on how to use it. As a template, consider having these sections--where the parts in brackets should be filled in. See the [MIP](https://huggingface.co/datasets/RosettaCommons/MIP/blob/main/README.md) dataset as an example.
|
| 4 |
|
| 5 |
-
|
| 6 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
|
| 9 |
-
\#\# Quickstart Usage
|
| 10 |
-
|
| 11 |
-
\#\#\# Install HuggingFace Datasets package
|
| 12 |
-
|
| 13 |
-
Each subset can be loaded into python using the HuggingFace \[datasets\](https://huggingface.co/docs/datasets/index) library.
|
| 14 |
-
First, from the command line install the \`datasets\` library
|
| 15 |
-
|
| 16 |
-
$ pip install datasets
|
| 17 |
-
|
| 18 |
-
Optionally set the cache directory, e.g.
|
| 19 |
-
|
| 20 |
-
$ HF\_HOME=${HOME}/.cache/huggingface/
|
| 21 |
-
$ export HF\_HOME
|
| 22 |
-
|
| 23 |
-
then, from within python load the datasets library
|
| 24 |
-
|
| 25 |
-
\>\>\> import datasets
|
| 26 |
-
|
| 27 |
-
\#\#\# Load model datasets
|
| 28 |
-
|
| 29 |
-
To load one of the \<DATASET ID\> model datasets, use \`datasets.load\_dataset(...)\`:
|
| 30 |
-
|
| 31 |
-
\>\>\> dataset\_tag \= "\<DATASET TAG\>"
|
| 32 |
-
\>\>\> dataset \= datasets.load\_dataset(
|
| 33 |
-
path \= "\<HF PATH TO DATASET\>",
|
| 34 |
-
name \= f"{dataset\_tag}",
|
| 35 |
-
data\_dir \= f"{dataset\_tag}")\['train'\]
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
and the dataset is loaded as a \`datasets.arrow\_dataset.Dataset\`
|
| 39 |
-
|
| 40 |
-
\>\>\> dataset
|
| 41 |
-
\<RESULT OF LOADING DATASET MODEL\>
|
| 42 |
-
|
| 43 |
-
which is a column oriented format that can be accessed directly, converted in to a \`pandas.DataFrame\`, or \`parquet\` format, e.g.
|
| 44 |
-
|
| 45 |
-
\>\>\> dataset.data.column('\<COLUMN NAME IN DATASET\>')
|
| 46 |
-
\>\>\> dataset.to\_pandas()
|
| 47 |
-
\>\>\> dataset.to\_parquet("dataset.parquet")
|
| 48 |
-
|
| 49 |
-
\#\#\# \<BREIF EXAMPLE OF HOW TO USE DIFFERENT PARTS OF THE DATASET\>
|
| 50 |
-
|
| 51 |
-
\#\# Dataset Details
|
| 52 |
-
|
| 53 |
-
\#\#\# Dataset Description
|
| 54 |
-
\<DETAILED DESCRIPTION OF DATASET\>
|
| 55 |
-
|
| 56 |
-
\- \*\*Acknowledgements:\*\*
|
| 57 |
-
\<ACKNOWLEDGEMENTS\>
|
| 58 |
-
|
| 59 |
-
\- \*\*License:\*\* \<LICENSE\>
|
| 60 |
-
|
| 61 |
-
\#\#\# Dataset Sources
|
| 62 |
-
\- \*\*Repository:\*\* \<URL FOR SOURCE OF DATA\>
|
| 63 |
-
\- \*\*Paper:\*\* \<APA CITATION REFERENCE FOR SOURCE DATA\>
|
| 64 |
-
\- \*\*Zenodo Repository:\*\* \<ZENODO LINK IF RELEVANT\>
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
\#\# Uses
|
| 68 |
-
\<DESCRIPTION OF INTENDED USE OF DATASET\>
|
| 69 |
-
|
| 70 |
-
\#\#\# Out-of-Scope Use
|
| 71 |
-
\<DESCRIPTION OF OUT OF SCOPE USES OF DATASET\>
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
\#\#\# Source Data
|
| 75 |
-
\<DESCRIPTION OF SOURCE DATA\>
|
| 76 |
-
|
| 77 |
-
\#\# Citation
|
| 78 |
-
\<BIBTEX REFERENCE FOR DATASET\>
|
| 79 |
-
|
| 80 |
-
\#\# Dataset Card Authors
|
| 81 |
-
\<NAME/INFO OF DATASET AUTHORS\> |
|
| 82 |
-
| :---- |
|
| 83 |
-
|
| 84 |
## **5 Add Metadata to the Dataset Card**
|
| 85 |
|
| 86 |
### **Overview**
|
|
|
|
| 2 |
|
| 3 |
The `README.md` is a markdown file that is displayed when goes to the front page for the dataset. It should give appropriate context for the dataset and guidance on how to use it. As a template, consider having these sections--where the parts in brackets should be filled in. See the [MIP](https://huggingface.co/datasets/RosettaCommons/MIP/blob/main/README.md) dataset as an example.
|
| 4 |
|
| 5 |
+
# <DATASET TITLE>
|
| 6 |
+
<short descriptive abstract the dataset>
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
## Quickstart Usage
|
| 10 |
+
|
| 11 |
+
### Install HuggingFace Datasets package
|
| 12 |
+
|
| 13 |
+
Each subset can be loaded into python using the HuggingFace \[datasets\](https://huggingface.co/docs/datasets/index) library.
|
| 14 |
+
First, from the command line install the \`datasets\` library
|
| 15 |
+
|
| 16 |
+
$ pip install datasets
|
| 17 |
+
|
| 18 |
+
Optionally set the cache directory, e.g.
|
| 19 |
+
|
| 20 |
+
$ HF_HOME=${HOME}/.cache/huggingface/
|
| 21 |
+
$ export HF_HOME
|
| 22 |
+
|
| 23 |
+
then, from within python load the datasets library
|
| 24 |
+
|
| 25 |
+
>>> import datasets
|
| 26 |
+
|
| 27 |
+
### Load model datasets
|
| 28 |
+
|
| 29 |
+
To load one of the <DATASET ID> model datasets, use `datasets.load\_dataset(...)`:
|
| 30 |
+
|
| 31 |
+
>>> dataset_tag = "<DATASET TAG\>"
|
| 32 |
+
>>> dataset = datasets.load\_dataset(
|
| 33 |
+
path = "<HF PATH TO DATASET>",
|
| 34 |
+
name = f"{dataset_tag}",
|
| 35 |
+
data_dir = f"{dataset_tag}")['train']
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
and the dataset is loaded as a `datasets.arrow_dataset.Dataset`
|
| 39 |
+
|
| 40 |
+
>>> dataset
|
| 41 |
+
<RESULT OF LOADING DATASET MODEL>
|
| 42 |
+
|
| 43 |
+
which is a column oriented format that can be accessed directly, converted in to a `pandas.DataFrame`, or `parquet` format, e.g.
|
| 44 |
+
|
| 45 |
+
>>> dataset.data.column('<COLUMN NAME IN DATASET>')
|
| 46 |
+
>>> dataset.to_pandas()
|
| 47 |
+
>>> dataset.to_parquet("dataset.parquet")
|
| 48 |
+
|
| 49 |
+
### <BREIF EXAMPLE OF HOW TO USE DIFFERENT PARTS OF THE DATASET>
|
| 50 |
+
|
| 51 |
+
## Dataset Details
|
| 52 |
+
|
| 53 |
+
### Dataset Description
|
| 54 |
+
<DETAILED DESCRIPTION OF DATASET>
|
| 55 |
+
|
| 56 |
+
- **Acknowledgements:**
|
| 57 |
+
<ACKNOWLEDGEMENTS>
|
| 58 |
+
|
| 59 |
+
- **License:** <LICENSE>
|
| 60 |
+
|
| 61 |
+
### Dataset Sources
|
| 62 |
+
- **Repository:** <URL FOR SOURCE OF DATA>
|
| 63 |
+
- **Paper:** <APA CITATION REFERENCE FOR SOURCE DATA>
|
| 64 |
+
- **Zenodo Repository:** <ZENODO LINK IF RELEVANT>
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
## Uses
|
| 68 |
+
<DESCRIPTION OF INTENDED USE OF DATASET>
|
| 69 |
+
|
| 70 |
+
### Out-of-Scope Use
|
| 71 |
+
<DESCRIPTION OF OUT OF SCOPE USES OF DATASET>
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
### Source Data
|
| 75 |
+
<DESCRIPTION OF SOURCE DATA>
|
| 76 |
+
|
| 77 |
+
## Citation
|
| 78 |
+
<BIBTEX REFERENCE FOR DATASET>
|
| 79 |
+
|
| 80 |
+
## Dataset Card Authors
|
| 81 |
+
<NAME/INFO OF DATASET AUTHORS>
|
| 82 |
|
| 83 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
## **5 Add Metadata to the Dataset Card**
|
| 85 |
|
| 86 |
### **Overview**
|