A free, monthly-refreshed compliance matrix of all 4,893 Indic-language datasets on the Hub, showing that 65% declare no license — so teams can avoid licensing traps (and missing-tag repos) before they train on them

🤗Datasets
1

Dataset: 65% of 4,893 Indic-language datasets declare no license — full compliance matrix

I scanned every Indic-language dataset on the Hub (hindi, tamil, bengali, telugu, marathi, malayalam, kannada, urdu, +9 more) and published the full results.

Headline numbers (scan date: 2026-08-01)

  • 3,185 of 4,893 datasets (65.1%) declare no license tag
  • 46.1% of all downloads hit no-license repos — including ai4bharat/IndicCorpV2 (2,888 downloads) and cfilt/iitb-english-hindi (1,003 downloads)
  • 154 datasets are CC-BY-NC — a silent trap for commercial training (e.g. ai4bharat/IndicParaphrase, 4,611 downloads)
  • 139 datasets carry ambiguous tags (other, cc, unknown)

The data

Full matrix with risk buckets + guidance for every repo (CSV + Parquet, CC0):

:backhand_index_pointing_right: hardik90/indic-dataset-license-matrix

Also in that repo: a sample audit report with a deep-dive on 3 flagship corpora, plus an FAQ covering commercial use, CC-BY-NC, and how to verify any dataset’s license.

Why it matters

Under the EU AI Act GPAI documentation duties and standard enterprise procurement policies, a missing license tag effectively disqualifies a dataset from commercial pipelines — regardless of the author’s original intent.

If you train or ship on Indic data, filter the matrix by risk_bucket and build your watchlist before your next fine-tune.

Maintainers: I’d love to see your license tags + provenance notes — the matrix is refreshed monthly, and I’m happy to help draft the card text.

The matrix is factual, API-derived metadata released under CC0. Not legal advice — verify anything you rely on.

2

Useful matrix. When you review these datasets, is the larger problem missing license metadata or not knowing whether the publisher still supports the listed version? Would an expiring, publisher-signed version record materially improve the matrix?