skill-leaderboard

SKILL.md preview

---
name: skill-leaderboard
description: Ranking of which skills are most popular across CONFIGURED Aeon forks (excludes untouched templates)
var: ""
tags: [meta]
---
> **${var}** — Target repo to scan forks of (e.g. "owner/aeon"). If empty, reads `memory/watched-repos.md` and uses the first entry.

<!-- autoresearch: variation B — sharper output via configured-fork denominator + tiered fleet + promote/match/sunset insights -->

Today is ${today}. Generate a weekly leaderboard of which Aeon skills the **configured fleet** is actually running — and what upstream should do about it.

## Why this version

Upstream `aeon.yml` ships with effectively only `heartbeat: enabled: true`. Every fresh fork inherits that default. If we score "skills enabled" across **all active forks**, the leaderboard is a tautology (heartbeat always wins, everything else hovers near 0) and operator learns nothing. The lever is to score against **configured forks only** — forks whose `aeon.yml` diverges from upstream defaults — and to convert the result into three actionable recommendations: which skills to promote, which fleet patterns to copy upstream, which skills to sunset.

## Steps

### 1. Determine the target repo

If `${var}` is set, use that as `TARGET_REPO`. Otherwise read `memory/watched-repos.md` and use the first non-comment, non-empty line. If neither yields a value, log `SKILL_LEADERBOARD_NO_TARGET` to `memory/logs/${today}.md` and stop (no notification).

### 2. Snapshot upstream defaults

Read this running instance's local `aeon.yml` once. Build `UPSTREAM_DEFAULTS`: a dict `{skill_name -> {enabled, model_or_null, var_or_empty, schedule_or_null}}` covering every skill entry under the `skills:` block. Also build `UPSTREAM_SKILLS`: the set of skill directory names from `skills/` (use `ls skills/`). These are the comparison baselines.

### 3. Fetch active forks

```bash
CUTOFF=$(date -u -d "30 days ago" +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-30d +%Y-%m-%dT%H:%M:%SZ)
gh api "repos/${TARGET_REPO}/forks?per_page=100" --paginate \
  --jq "[.[] | select(.pushed_at > \"$CUTOFF\") | {owner: .owner.login, full_name: .full_name, pushed_at, stargazers_count, created_at}]"
```

If zero active forks: log `SKILL_LEADERBOARD_NO_FORKS` and stop (no notification).

### 4. Per-fork single-call enumeration

For each active fork, run **one** recursive git-tree call to enumerate files (cheaper than per-path contents):

```bash
gh api "repos/${FORK_FULL}/git/trees/HEAD?recursive=1" --jq '[.tree[] | select(.type == "blob") | .path]'
```

Handle errors:
- 404 / 409 (empty repo): mark `status: "no_tree"`, skip aeon.yml and skills/ extraction, continue.
- 403 with `X-RateLimit-Remaining: 0`: sleep 60s, retry once. If still failing, mark `status: "rate_limited"` and continue with partial fleet.

Then fetch the fork's `aeon.yml` only if the tree contains it:

```bash
gh api "repos/${FORK_FULL}/contents/aeon.yml" --jq '.content' | base64 -d
```

If the path is in the tree but the contents call 404s, mark `status: "yml_unreadable"` and continue.

Extract from each readable `aeon.yml`:
- For every skill entry under `skills:` — `enabled`, `model` (if set), `var` (if set), `schedule` (if differs from any upstream default for that skill).
- Detect "fork-only skills": directory names under `skills/` in the fork's tree that are NOT in `UPSTREAM_SKILLS`.

### 5. Classify each fork

For each fork, compute a divergence signal vector vs `UPSTREAM_DEFAULTS`:
- **enabled_diff**: count of skills where the fork's `enabled` value differs from upstream default
- **var_set**: count of skills with `var:` set to a non-empty value where upstream's was empty
- **model_override**: count of skills with a `model:` value differing from upstream
- **schedule_override**: count of skills with a `schedule:` value differing from upstream
- **fork_only_skills**: count from step 4

Tier the fork:
- **CONFIGURED**: any of the above is ≥1. (i.e., the fork actively diverged from defaults)
- **TEMPLA

…