unslothai--unsloth/studio/backend/plugins/data-designer-github-repo-seed

data-designer-github-repo-seed

A Data Designer seed-reader plugin for Unsloth Studio that scrapes real GitHub data (issues, pull requests, commits) from one or more repositories and hands it to the recipe pipeline as a seed dataset.

Designed to ship with Studio as a default seed source so any user with a GitHub token can build training datasets straight from live repos.

What it does

Given a list of owner/name repos, a GitHub token, and a per-resource limit, the plugin uses GitHub's GraphQL API to fetch issues, pull requests, and/or commits, with labels, state, authors, and the first N comments of each item, and materialises a single JSONL with uniform columns so the rest of the recipe (LLM text / LLM structured / processors) can treat it like any other seed table.

Column	Description
item_type	issue / pull / commit
repo	owner/name
number	Issue/PR number, or commit SHA
title	Title (or commit message headline)
body	Issue/PR body (or full commit message)
state	OPEN / CLOSED / MERGED (empty for commit)
author	GitHub login of the author
created_at	ISO8601
closed_at	ISO8601 (empty for commits)
url	Permalink
labels	List of label names
comments	First N comments concatenated

Usage in a recipe

{
  "seed_config": {
    "source": {
      "seed_type": "github_repo",
      "repos": ["unslothai/unsloth", "unslothai/unsloth-zoo"],
      "token": "",
      "item_types": ["issues", "pulls"],
      "limit": 100,
      "include_comments": true,
      "max_comments_per_item": 30
    },
    "sampling_strategy": "shuffle",
    "selection_strategy": null
  }
}

Leave token empty to fall back to the server's GH_TOKEN / GITHUB_TOKEN environment variable, useful when the recipe is published and shouldn't carry a secret.

Auth

A GitHub personal access token with public_repo scope is enough for public repositories; repo scope is required for private ones. GraphQL requests are rate-limit aware: the client inspects x-ratelimit-* headers and sleeps until reset when the budget drops below a safety threshold.

Install

Shipped as a default Studio plugin. For development:

pip install -e .

Registered automatically via the data_designer.plugins entry point.