Croissant

Import MLCommons Croissant dataset metadata as one queryable markdown page.

MLCommons Croissant describes a dataset, not its rows. KiwiFS imports one Croissant document as one page: dataset metadata goes into frontmatter, and the file list and column schema go into kiwi-data blocks you can query with DQL. Use it to build a searchable catalog of the datasets your team uses.

Import

URL

Terminal

kiwifs import --from croissant \
  --url https://www.kaggle.com/datasets/OWNER/NAME/croissant/download \
  --root ./knowledge

File

Terminal

kiwifs import --from croissant --file croissant.json --root ./knowledge

REST

Terminal

curl -X POST http://localhost:3333/api/kiwi/import \
  -H "Content-Type: application/json" \
  -d '{"from": "croissant", "url": "https://huggingface.co/api/datasets/OWNER/NAME/croissant"}'

The page is written to <dataset-slug>/data.md, where the slug comes from the dataset name. Pass --prefix to choose another folder. Fetched documents are limited to 32 MB and 60 seconds.

What the page contains

Part	Content
Frontmatter	title, kind: dataset, description, url, license, version, keywords, creator, cite-as, date-published, record-sets, fields, croissant-origin
## Files	A kiwi-data block of kind dataset-file, one record per file or file set
## Record set: <name>	A kiwi-data block of kind dataset-schema, one record per column

Each dataset-schema record has these fields:

Field	Description
record-set	The record set the column belongs to
name	Column name
dtype	Short type such as integer, float, text, boolean, date, url, or image. Null when the type isn’t recognized.
description	Column description
column, source-file	Where the value comes from in the source files
references	The field this column references, if any
field-id	The Croissant field ID

Query the schema

Use FROM RECORDS to query the column records across every imported dataset. The page’s frontmatter, such as title, is available alongside each record’s fields.

DQL

TABLE title, name, dtype, column, source-file
FROM RECORDS "dataset-schema"
WHERE dtype = "float"
SORT title ASC

Croissant is JSON-LD. KiwiFS expands the document to absolute IRIs before reading it, so emitters that use different property names for the same thing still import correctly. Contexts resolve offline: an inline @context, which Kaggle and Hugging Face emit, and the well-known MLCommons context URL both work without network access. KiwiFS refuses any other remote context rather than fetching it.

Related

Widgets

How kiwi-data blocks render and store records.

DQL

Query syntax, including FROM RECORDS.