signal-verdict

SKILL.md preview

---
name: signal-verdict
description: Accountability check on a configured set of tracker skills. Verifies each tracker is producing citable signals in articles/newsletters. Surfaces uncited trackers so the operator can demote or kill them.
schedule: "0 8 * * 1"
commits: true
permissions:
  - contents:write
tags: [meta, ops]
---

Today is ${today}. Read `memory/MEMORY.md` before starting.

## Voice

If `soul/SOUL.md` and `soul/STYLE.md` are populated, read both and match the operator's voice for the notification. If they are empty templates or absent, write in a clear, direct, neutral tone — position first, no hedging, no corporate softeners.

## Why this skill exists

When you ship many tracker skills in a short window, it's easy to lose track of which trackers are actually feeding written output and which are just running. `skill-health` answers "did the skill run?" — `signal-verdict` answers a different question: **did the output inform anything that got published outward?** Trackers that fail this check repeatedly should be demoted (less-frequent schedule) or retired.

## Config

This skill is parameterized via `memory/topics/tracked-skills.md`. If the file doesn't exist, create the seed below and exit on this run (no notification — operator needs to fill it in):

```markdown
# Tracked Skills (signal-verdict)

## Verdict Bar
- **Deadline:** <YYYY-MM-DD>   # by this date, each tracker must have ≥1 citation
- **Rule:** Each tracker must cite a signal in a daily/newsletter article between its first run and the deadline, or get demoted.

## Trackers Under Review

| Tracker | First Run | Verdict Bar | Status | Citation Weeks | Total Weeks |
|---------|-----------|-------------|--------|----------------|-------------|
| <skill-name> | YYYY-MM-DD | YYYY-MM-DD | pending | 0 | 0 |

## Keyword Map

```yaml
<skill-name>:
  - "<keyword 1>"
  - "<keyword 2>"
<skill-name-2>:
  - "<keyword 1>"
```

## Weekly Log
(populated on each run)
```

Each `Tracker` row + the matching `Keyword Map` entry defines one tracker under review. To add or remove a tracker, edit this file — no skill code changes.

## Steps

### 1. Load context

Read `memory/MEMORY.md` and the tracker list + keyword map from `memory/topics/tracked-skills.md`. Extract:
- Per-tracker current status and citation counts
- Date of last run
- Verdict deadline
- Keyword map (used in step 3)

If the keyword map is empty for a tracker, log a warning and skip that tracker for this run.

### 2. Find articles from the past 14 days

```bash
ls articles/ | grep -E "^[0-9]{4}-[0-9]{2}-[0-9]{2}\.md$" | sort | tail -14
ls articles/ | grep -E "newsletter|weekly" | sort | tail -10
```

Read each file. **Exclude** ops articles (these don't count as "outward content"):
- `articles/self-review-*.md`
- `articles/skill-evals-*.md`
- `articles/vuln-scan-*.md`
- `articles/batch-health-*.md`
- `articles/repo-pulse-*.md`

What counts as "outward content":
- Daily articles (`articles/YYYY-MM-DD.md`)
- Weekly newsletters (`articles/newsletter-*.md`, `articles/weekly-*.md`)
- Any article whose main thesis or a substantive section discusses the tracker's topic area

### 3. Score each tracker for this week

For each tracker in the keyword map, search the article texts (from step 2) for any keyword match. A tracker is:

- **CITED** if at least one of its keywords appears in a substantive paragraph (not just a passing reference) of an outward-content article from the last 7 days
- **UNCITED** if no qualifying mention exists

**Citation threshold:** Minimum 1 substantive mention. A log entry in `memory/logs/` does NOT count — this check is about content published outward. A passing mention in a notification preamble does NOT count.

### 4. Compute verdict status

For each tracker:
- **CITED** this week: `citation_weeks += 1`, `total_weeks += 1`
- **UNCITED** this week: `total_weeks += 1` only

**Verdict determination** (only if today ≥ deadline):
- `citation_weeks ≥ 1` → **PASS** (keep on current schedule)
- `

…