# 🤗Datasets

**URL:** https://discuss.huggingface.co/c/datasets/10.md

[Latest](https://discuss.huggingface.co/latest.md) · [Categories](https://discuss.huggingface.co/categories.md)

---

## [Best way to copy a ~1.3 PB Storage Bucket to Azure Blob? (rate limits, egress, tooling)](https://discuss.huggingface.co/t/best-way-to-copy-a-1-3-pb-storage-bucket-to-azure-blob-rate-limits-egress-tooling/192107)

<div class="topic-metadata">

**Author:** [@adas-ash](https://discuss.huggingface.co/u/adas-ash)\
**Replies:** 0\
**Last updated:** [November 18, 2026, 8:19pm UTC](https://discuss.huggingface.co/t/best-way-to-copy-a-1-3-pb-storage-bucket-to-azure-blob-rate-limits-egress-tooling/192107 "2026-11-18T20:19:01Z")

</div>

We need to copy the Storage Bucket nvidia/human-1m-100k-oss (~1.32 PB, ~2.56M files) into Azure Blob storage: one full copy, then a catch-up pass. First test: an Azure VM running hf sync into a Blob container mounted wi…

---

## [Request to register DedeProGames/phone-tool-bench as an official benchmark on the Hub](https://discuss.huggingface.co/t/request-to-register-dedeprogames-phone-tool-bench-as-an-official-benchmark-on-the-hub/192104)

<div class="topic-metadata">

**Author:** [@DedeProGames](https://discuss.huggingface.co/u/DedeProGames)\
**Replies:** 0\
**Last updated:** [November 18, 2026, 5:21pm UTC](https://discuss.huggingface.co/t/request-to-register-dedeprogames-phone-tool-bench-as-an-official-benchmark-on-the-hub/192104 "2026-11-18T17:21:49Z")

</div>

Hello HF team, I would like to request the registration of my dataset as a benchmark on the Hugging Face Hub: Dataset: DedeProGames/phone-tool-bench ← (link 1: DedeProGames/phone-tool-bench · Datasets at Hugging Face) …

---

## [\[SEEKING\] Indic Document Dataset (India) — Invoices, Receipts, Utility Bills, Payment Advices, Packing Lists, Commercial Invoices, Credit Notes](https://discuss.huggingface.co/t/seeking-indic-document-dataset-india-invoices-receipts-utility-bills-payment-advices-packing-lists-commercial-invoices-credit-notes/177055)

<div class="topic-metadata">

**Author:** [@quantvectors](https://discuss.huggingface.co/u/quantvectors)\
**Replies:** 7\
**Last updated:** [November 17, 2026, 6:00am UTC](https://discuss.huggingface.co/t/seeking-indic-document-dataset-india-invoices-receipts-utility-bills-payment-advices-packing-lists-commercial-invoices-credit-notes/177055 "2026-11-17T06:00:08Z")

</div>

Hi, We’re looking for annotated document datasets in Indic languages for AI/OCR model training. We are specifically looking for documents originating from India. We’re open to both open-source and commercial datasets. L…

---

## [Tarot card data for all 78 cards (CSV/JSON, CC BY 4.0)](https://discuss.huggingface.co/t/tarot-card-data-for-all-78-cards-csv-json-cc-by-4-0/191945)

<div class="topic-metadata">

**Author:** [@tarostarot](https://discuss.huggingface.co/u/tarostarot)\
**Replies:** 0\
**Last updated:** [November 16, 2026, 1:37pm UTC](https://discuss.huggingface.co/t/tarot-card-data-for-all-78-cards-csv-json-cc-by-4-0/191945 "2026-11-16T13:37:08Z")

</div>

I’ve published a small structured data set for the 78 tarot cards: tarostarot/tarot-card-data · Datasets at Hugging Face Each card has its name, number, arcana, suit, rank, element, eight keywords, a one-sentence uprigh…

---

## [Accessibility Datasets](https://discuss.huggingface.co/t/accessibility-datasets/191938)

<div class="topic-metadata">

**Author:** [@GarethFW](https://discuss.huggingface.co/u/GarethFW)\
**Replies:** 0\
**Last updated:** [November 16, 2026, 11:10am UTC](https://discuss.huggingface.co/t/accessibility-datasets/191938 "2026-11-16T11:10:37Z")

</div>

I am building up a library of datasets relating to all aspects of accessibility and inclusive design for websites, mobile applications, games design, customer experience design, regulatory frameworks and programme manage…

---

## [Request to register GKB-1000 as an official benchmark](https://discuss.huggingface.co/t/request-to-register-gkb-1000-as-an-official-benchmark/191424)

<div class="topic-metadata">

**Author:** [@Lelonthecodeur](https://discuss.huggingface.co/u/Lelonthecodeur)\
**Replies:** 4\
**Last updated:** [November 13, 2026, 7:15pm UTC](https://discuss.huggingface.co/t/request-to-register-gkb-1000-as-an-official-benchmark/191424 "2026-11-13T19:15:54Z")

</div>

Subject: Request to register GKB-1000 as a Hugging Face Benchmark Hello Hugging Face team, I would like to request the registration of my dataset, General Knowledge Benchmark (GKB-1000), as an official benchmark on the…

---

## [Looking for one independent annotator: our LLM raters agree with each other at chance level and we can't tell why](https://discuss.huggingface.co/t/looking-for-one-independent-annotator-our-llm-raters-agree-with-each-other-at-chance-level-and-we-cant-tell-why/180645)

<div class="topic-metadata">

**Author:** [@leventbulut](https://discuss.huggingface.co/u/leventbulut)\
**Replies:** 9\
**Last updated:** [November 13, 2026, 3:15pm UTC](https://discuss.huggingface.co/t/looking-for-one-independent-annotator-our-llm-raters-agree-with-each-other-at-chance-level-and-we-cant-tell-why/180645 "2026-11-13T15:15:15Z")

</div>

Short version of the ask: I have 100 scenes, a locked set of human reference labels, and five machine raters that disagree with the human and with each other at roughly chance level. I cannot tell whether that is because…

---

## [Random vs scaffold splits: single-seed baselines on 6 chemistry datasets](https://discuss.huggingface.co/t/random-vs-scaffold-splits-single-seed-baselines-on-6-chemistry-datasets/190952)

<div class="topic-metadata">

**Author:** [@woongworks](https://discuss.huggingface.co/u/woongworks)\
**Replies:** 5\
**Last updated:** [November 12, 2026, 7:34am UTC](https://discuss.huggingface.co/t/random-vs-scaffold-splits-single-seed-baselines-on-6-chemistry-datasets/190952 "2026-11-12T07:34:47Z")

</div>

Random splits can place close analogues in both training and test sets. A Bemis–Murcko split keeps molecules with the same scaffold together, so it tests one form of structural generalization, although Murcko scaffolds a…

---

## [How to get articles on a topic from Tim-Pinecone / LOC-Chronicling-America dataset](https://discuss.huggingface.co/t/how-to-get-articles-on-a-topic-from-tim-pinecone-loc-chronicling-america-dataset/191549)

<div class="topic-metadata">

**Author:** [@rxshah](https://discuss.huggingface.co/u/rxshah)\
**Replies:** 3\
**Last updated:** [November 8, 2026, 3:02pm UTC](https://discuss.huggingface.co/t/how-to-get-articles-on-a-topic-from-tim-pinecone-loc-chronicling-america-dataset/191549 "2026-11-08T15:02:52Z")

</div>

I have gone through the examples on how to extract data from Tim-Pinecone/LOC-Chronicling-America dataset : README.md · Tim-Pinecone/LOC-Chronicling-America at main Are we getting the individual articles or the entire …

---

## [UncertaintyGym: A Benchmark for LLM Epistemic Calibration and Uncertainty Expression](https://discuss.huggingface.co/t/uncertaintygym-a-benchmark-for-llm-epistemic-calibration-and-uncertainty-expression/178647)

<div class="topic-metadata">

**Author:** [@Ill-Ness](https://discuss.huggingface.co/u/Ill-Ness)\
**Replies:** 6\
**Last updated:** [November 8, 2026, 10:42am UTC](https://discuss.huggingface.co/t/uncertaintygym-a-benchmark-for-llm-epistemic-calibration-and-uncertainty-expression/178647 "2026-11-08T10:42:03Z")

</div>

Hi Hugging Face team, We would like to submit UncertaintyGym for official benchmark indexing and Evaluation Hub integration. Overview: UncertaintyGym evaluates an LLM’s meta-cognitive calibration—measuring whether mod…

---

## [Reliability check on my own dataset's annotation layer: five machine raters, one definition, answers from 0 to 78](https://discuss.huggingface.co/t/reliability-check-on-my-own-datasets-annotation-layer-five-machine-raters-one-definition-answers-from-0-to-78/178375)

<div class="topic-metadata">

**Author:** [@leventbulut](https://discuss.huggingface.co/u/leventbulut)\
**Replies:** 4\
**Last updated:** [November 5, 2026, 6:30pm UTC](https://discuss.huggingface.co/t/reliability-check-on-my-own-datasets-annotation-layer-five-machine-raters-one-definition-answers-from-0-to-78/178375 "2026-11-05T18:30:37Z")

</div>

I publish a small Turkish narrative corpus that ships a machine-generated annotation layer — six binary craft features per scene. Those flags were never validated against human judgement, so I ran three studies to check …

---

## [Why is there almost no manipulation data for agriculture?](https://discuss.huggingface.co/t/why-is-there-almost-no-manipulation-data-for-agriculture/178257)

<div class="topic-metadata">

**Author:** [@viharshah85](https://discuss.huggingface.co/u/viharshah85)\
**Replies:** 3\
**Last updated:** [November 5, 2026, 10:28am UTC](https://discuss.huggingface.co/t/why-is-there-almost-no-manipulation-data-for-agriculture/178257 "2026-11-05T10:28:08Z")

</div>

I farm, and I’ve been going through the robotics datasets on the Hub for a few months trying to find anything useful. Almost everything is indoor tabletop. Open X-Embodiment, DROID, most of what’s under LeRobot — pick up…

---

## [Ifparse 2.0 Release](https://discuss.huggingface.co/t/ifparse-2-0-release/191309)

<div class="topic-metadata">

**Author:** [@Ill-Ness](https://discuss.huggingface.co/u/Ill-Ness)\
**Replies:** 2\
**Last updated:** [November 4, 2026, 10:14am UTC](https://discuss.huggingface.co/t/ifparse-2-0-release/191309 "2026-11-04T10:14:52Z")

</div>

IFParse v2.0: models still can’t turn a log line into exact JSON We released IFParse v2.0, a benchmark where a model gets a raw machine record (nginx line, syslog, CloudTrail, k8s audit event, Zeek conn row, CEF, Sysmon…

---

## [Trying to figure out the best parameters!](https://discuss.huggingface.co/t/trying-to-figure-out-the-best-parameters/191390)

<div class="topic-metadata">

**Author:** [@GeorgeZervas](https://discuss.huggingface.co/u/GeorgeZervas)\
**Replies:** 1\
**Last updated:** [November 4, 2026, 7:55am UTC](https://discuss.huggingface.co/t/trying-to-figure-out-the-best-parameters/191390 "2026-11-04T07:55:56Z")

</div>

Hi,good morning. Anybody here to help? I am trying to successfully put 2 different objects(balls) into their matching basket. I have record almost 150 episodes but the results are not very good.Is it better to start with…

---

## [What are the biggest challenges you face when sourcing high-quality, domain-specific datasets for training and evaluating AI models?](https://discuss.huggingface.co/t/what-are-the-biggest-challenges-you-face-when-sourcing-high-quality-domain-specific-datasets-for-training-and-evaluating-ai-models/178544)

<div class="topic-metadata">

**Author:** [@viharshah85](https://discuss.huggingface.co/u/viharshah85)\
**Replies:** 3\
**Last updated:** [November 2, 2026, 6:02am UTC](https://discuss.huggingface.co/t/what-are-the-biggest-challenges-you-face-when-sourcing-high-quality-domain-specific-datasets-for-training-and-evaluating-ai-models/178544 "2026-11-02T06:02:49Z")

</div>

I’d be particularly interested in experiences with multimodal data (text, audio, image, video), multilingual datasets, and human-annotated data—and what criteria you use to determine whether a dataset is reliable enough …

---

## [RSB-1000: 1,000-record synthetic benchmark for phishing detection (CC-BY-4.0)](https://discuss.huggingface.co/t/rsb-1000-1-000-record-synthetic-benchmark-for-phishing-detection-cc-by-4-0/190933)

<div class="topic-metadata">

**Author:** [@RevealWire](https://discuss.huggingface.co/u/RevealWire)\
**Replies:** 1\
**Last updated:** [October 26, 2026, 2:20pm UTC](https://discuss.huggingface.co/t/rsb-1000-1-000-record-synthetic-benchmark-for-phishing-detection-cc-by-4-0/190933 "2026-10-26T14:20:55Z")

</div>

Hi — I run Reveal Wire (youtube.com/@RevealWire), a channel that tests AI claims with real experiments. For an episode on phishing detection I looked for published detection scores for open-weight models and found zero —…

---

## [Can someone with a Chinese Baidu NetDisk account help me download this dataset?](https://discuss.huggingface.co/t/can-someone-with-a-chinese-baidu-netdisk-account-help-me-download-this-dataset/160296)

<div class="topic-metadata">

**Author:** [@tferhan](https://discuss.huggingface.co/u/tferhan)\
**Replies:** 4\
**Last updated:** [October 23, 2026, 1:38am UTC](https://discuss.huggingface.co/t/can-someone-with-a-chinese-baidu-netdisk-account-help-me-download-this-dataset/160296 "2026-10-23T01:38:42Z")

</div>

I’m working on a real-world satellite imagery segmentation project. The dataset is UBCv2 (vec), hosted on Baidu NetDisk, and I’m blocked from downloading it due to the Chinese phone number restriction. Link: https://pan…

---

## [Revenue prediction datasets](https://discuss.huggingface.co/t/revenue-prediction-datasets/74897)

<div class="topic-metadata">

**Author:** [@jonaspoelmans](https://discuss.huggingface.co/u/jonaspoelmans)\
**Replies:** 1\
**Last updated:** [October 17, 2026, 8:46am UTC](https://discuss.huggingface.co/t/revenue-prediction-datasets/74897 "2026-10-17T08:46:48Z")

</div>

We are working on a novel revenue prediction algorithm benchmark study. The dataset should contain either daily, weekly or monthly revenues for at least 50 companies for a period of 1 year minimum. The companies can be o…

---

## [Datasets \>= 5 ruins sharding when shuffle() is called](https://discuss.huggingface.co/t/datasets-5-ruins-sharding-when-shuffle-is-called/179493)

<div class="topic-metadata">

**Author:** [@joell1](https://discuss.huggingface.co/u/joell1)\
**Replies:** 4\
**Last updated:** [October 8, 2026, 8:28pm UTC](https://discuss.huggingface.co/t/datasets-5-ruins-sharding-when-shuffle-is-called/179493 "2026-10-08T20:28:19Z")

</div>

Hi HF community I am using a streaming dataset with a training dataset that is sharded into 8 different parquet files. Before datasets 5.0, when I would call shuffle() with the dataset, there would be 8 shards in the re…

---

## [Download a fraction of data from HuggingFace Datasets](https://discuss.huggingface.co/t/download-a-fraction-of-data-from-huggingface-datasets/126236)

<div class="topic-metadata">

**Author:** [@srinjoyMukherjee](https://discuss.huggingface.co/u/srinjoyMukherjee)\
**Replies:** 5\
**Last updated:** [October 3, 2026, 10:33pm UTC](https://discuss.huggingface.co/t/download-a-fraction-of-data-from-huggingface-datasets/126236 "2026-10-03T22:33:01Z")

</div>

hey all, I want to download about 15GBs of data from each subsection/language of the stack dataset. Is there any way to do this? I can download the entire subsection using following code snippet, but i want a fraction o…

---

## [Confirmation: Can I use HuggingFace datasets for academic ML training?](https://discuss.huggingface.co/t/confirmation-can-i-use-huggingface-datasets-for-academic-ml-training/182807)

<div class="topic-metadata">

**Author:** [@Fitzz007](https://discuss.huggingface.co/u/Fitzz007)\
**Replies:** 1\
**Last updated:** [October 3, 2026, 1:31am UTC](https://discuss.huggingface.co/t/confirmation-can-i-use-huggingface-datasets-for-academic-ml-training/182807 "2026-10-03T01:31:01Z")

</div>

Hello HuggingFace community, My name is Mohammed Talbani, a Computer Science student at ESTS in Safi, Morocco. I am completing an AI internship where I need to build an image dataset of traditional Arabic footwear (babo…

---

## [Dataset & LoRA Training Pipeline for Room Emptying / Object Removal (Image-to-Image)](https://discuss.huggingface.co/t/dataset-lora-training-pipeline-for-room-emptying-object-removal-image-to-image/180659)

<div class="topic-metadata">

**Author:** [@suruchiarora](https://discuss.huggingface.co/u/suruchiarora)\
**Replies:** 2\
**Last updated:** [October 1, 2026, 9:33am UTC](https://discuss.huggingface.co/t/dataset-lora-training-pipeline-for-room-emptying-object-removal-image-to-image/180659 "2026-10-01T09:33:59Z")

</div>

Hi Community, I am working on a project aimed at emptying indoor spaces (removing all furniture/decor) as well as removing specific individual objects from an image using provided object masks. I plan to fine-tune a mo…

---

## [Is selling datasets way harder than building them? Or is it just me?](https://discuss.huggingface.co/t/is-selling-datasets-way-harder-than-building-them-or-is-it-just-me/177440)

<div class="topic-metadata">

**Author:** [@Soloise](https://discuss.huggingface.co/u/Soloise)\
**Replies:** 3\
**Last updated:** [September 28, 2026, 6:08pm UTC](https://discuss.huggingface.co/t/is-selling-datasets-way-harder-than-building-them-or-is-it-just-me/177440 "2026-09-28T18:08:06Z")

</div>

Is selling datasets way harder than building them? Or is it just me? Maybe I’m thinking wrong, but I keep running into this. Let’s say you spent months building a really good dataset. Now you want to sell it. The buy…

---

## [Hausa Healthcare LLM Preference & Safety Evaluation Dataset](https://discuss.huggingface.co/t/hausa-healthcare-llm-preference-safety-evaluation-dataset/180613)

<div class="topic-metadata">

**Author:** [@abdulfatah740](https://discuss.huggingface.co/u/abdulfatah740)\
**Replies:** 3\
**Last updated:** [September 19, 2026, 6:04pm UTC](https://discuss.huggingface.co/t/hausa-healthcare-llm-preference-safety-evaluation-dataset/180613 "2026-09-19T18:04:00Z")

</div>

Hello Huggingface Community, I am a computational linguist and Hausa NLP/AI data specialist working on a Hausa LLM Healthcare & Safety Preference Evaluation Dataset. The dataset is developed collaboratively by computat…

---

## [FlyEye escape-neuron-v1: 3,376-neuron / 78,797-edge compact MaleCNS-derived graph](https://discuss.huggingface.co/t/flyeye-escape-neuron-v1-3-376-neuron-78-797-edge-compact-malecns-derived-graph/180617)

<div class="topic-metadata">

**Author:** [@wihaha](https://discuss.huggingface.co/u/wihaha)\
**Replies:** 3\
**Last updated:** [September 18, 2026, 6:40pm UTC](https://discuss.huggingface.co/t/flyeye-escape-neuron-v1-3-376-neuron-78-797-edge-compact-malecns-derived-graph/180617 "2026-09-18T18:40:43Z")

</div>

We have published the first FlyEye compact neuron-level dataset together with the browser/runtime developer kit. Dataset: Tubban/flyeye-escape-neuron-v1 · Datasets at Hugging Face It contains a 3,376-neuron / 78,797 si…

---

## [Access to the gated repository, StanfordAIMI/CheXpert-v1.0-512](https://discuss.huggingface.co/t/access-to-the-gated-repository-stanfordaimi-chexpert-v1-0-512/180567)

<div class="topic-metadata">

**Author:** [@Ephy3897](https://discuss.huggingface.co/u/Ephy3897)\
**Replies:** 3\
**Last updated:** [September 18, 2026, 4:18pm UTC](https://discuss.huggingface.co/t/access-to-the-gated-repository-stanfordaimi-chexpert-v1-0-512/180567 "2026-09-18T16:18:09Z")

</div>

I kindly request access to the gated repository, StanfordAIMI/CheXpert-v1.0-512 on Hugging Face. I intend to use the dataset for my ongoing research on artificial intelligence in medical imaging. I have made a request,…

---

## [Open call: test your agent memory layer on an adversarial coding benchmark](https://discuss.huggingface.co/t/open-call-test-your-agent-memory-layer-on-an-adversarial-coding-benchmark/179762)

<div class="topic-metadata">

**Author:** [@Gde05](https://discuss.huggingface.co/u/Gde05)\
**Replies:** 5\
**Last updated:** [September 17, 2026, 9:36am UTC](https://discuss.huggingface.co/t/open-call-test-your-agent-memory-layer-on-an-adversarial-coding-benchmark/179762 "2026-09-17T09:36:42Z")

</div>

I built AMB, an open and preregistered benchmark for memory layers in coding agents. Most memory benchmarks test whether a system retrieves a relevant chunk. AMB tests whether that memory actually helps an agent complet…

---

## [Finding a working mirror /copy of the PxSLU/PxCorpus dataset beacause Zenodo keeps returning a 504 Gateway Time-out](https://discuss.huggingface.co/t/finding-a-working-mirror-copy-of-the-pxslu-pxcorpus-dataset-beacause-zenodo-keeps-returning-a-504-gateway-time-out/180350)

<div class="topic-metadata">

**Author:** [@OGB2000](https://discuss.huggingface.co/u/OGB2000)\
**Replies:** 2\
**Last updated:** [September 13, 2026, 2:27pm UTC](https://discuss.huggingface.co/t/finding-a-working-mirror-copy-of-the-pxslu-pxcorpus-dataset-beacause-zenodo-keeps-returning-a-504-gateway-time-out/180350 "2026-09-13T14:27:45Z")

</div>

Looking for someone who has successfully downloaded PxSLU / PxCorpus (Zenodo 6524162). Zenodo currently returns 504 Gateway Time-out when I try to download pxslu.zip (375.5 MB). Has anyone already downloaded this dataset…

---

## [Open-source tool for detecting issues in robot-learning datasets](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602)

<div class="topic-metadata">

**Author:** [@omert27](https://discuss.huggingface.co/u/omert27)\
**Replies:** 5\
**Last updated:** [September 12, 2026, 4:50pm UTC](https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602 "2026-09-12T16:50:33Z")

</div>

Open-source tool for robot dataset issues I’ve been working on Calibra, an open-source toolkit for analyzing robot-learning datasets. While working with demo data, I noticed a common issue: we often waste compute train…

---

## [Trajlens: a validator for LeRobotDataset, audited 100 Hub datasets](https://discuss.huggingface.co/t/trajlens-a-validator-for-lerobotdataset-audited-100-hub-datasets/177276)

<div class="topic-metadata">

**Author:** [@Kunal-Somani](https://discuss.huggingface.co/u/Kunal-Somani)\
**Replies:** 1\
**Last updated:** [September 9, 2026, 4:06pm UTC](https://discuss.huggingface.co/t/trajlens-a-validator-for-lerobotdataset-audited-100-hub-datasets/177276 "2026-09-09T16:06:42Z")

</div>

Posting here because this is directly about Hub data quality. LeRobotDataset is now the largest dataset category on the Hub. I built trajlens, an open-source linter for it, and ran it against 100 random public lerobot-t…

[Next page](https://discuss.huggingface.co/c/datasets/10.md?page=1)
