Proof of concept · 29 Sep 2026
Triaging GitHub issues with Jev
A single Python script sends every unlabeled issue to Jev (TypeSafe AI's System One classifier) and labels it by type, priority, and whether an AI coding agent can pick it up. Those are the same labels Looper already reads, so Looper needs no changes.
15 / 15issues classified correctly as bug, feature, or question
3 / 3vague or incomplete issues flagged
needs-more-info0risky issues (credentials, big design work) handed to the agent
~790tokens per issue (~700 in, ~90 out)
How it works
1. Fetch
open issues with no labels
gh issue listopen issues with no labels
→
2. Ask Jevtitle + body, 4 questions
one API call per issue
one API call per issue
→
3. Decidescores → labels
thresholds in code
thresholds in code
→
4. Label
only with
gh issue editonly with
--apply→
5. Looperpicks up
opens PRs automatically
agent issuesopens PRs automatically
Jev doesn't generate text. It answers typed questions with a label and a probability, which makes it fast, cheap, and easy to put thresholds on.
The four questions Jev answers
choice
Type
bug, feature, or questionscore 0–2
Priority
p2 low · p1 normal · p0 critical (crash, data loss, security)yes / no
Well described?
Could someone new act on it: what happened, what was expected, how to reproduce?yes / no
Agent-ready?
Clearly scoped, no human decision, no external access, and can be verified by testsLabeling rules
- needs-more-info when "well described" is 0.5 or below. A wrong call here only asks the reporter for more detail, so the bar is low.
- agent only when the issue is well described, agent-ready is above 0.85, and it isn't a question. A wrong yes costs a paid agent run, so the bar is high.
- needs-human-triage when Jev's confidence on type or priority is below 0.7. The unsure label is skipped and a person decides.
- Dry run by default: it prints what it would do and changes nothing until you pass
--apply.
Results on 15 test issues
Test issues cover clear bugs, a security hole, data loss, typos, vague one-liners, questions, small and large features, and a task that needs production access. Green bars cleared their threshold.
| # | Issue | Type | Priority (conf) | Well described | Agent-ready | Labels |
|---|
What we learned
Works well
- Type was right on every issue, almost always with 0.95+ confidence.
- Vague issues ("app is broken", "login doesn't work sometimes") scored below 0.1 on "well described".
- Agent-readiness splits cleanly: everything scored either 0.80 or higher, or 0.62 or lower.
- Jev kept the agent away from credential rotation (0.03) and multi-workspace design (0.02).
Needs tuning
- Priority is the weakest answer. Every
needs-human-triagecame from low priority confidence, which happens when an issue sits between two levels. - "app is broken" got p0 with 0.98 confidence, probably because the reporter wrote "asap".
- "Search should match tags" scored 0.80 on agent-ready, just under the 0.85 bar.
Next steps
- Lower the agent bar from 0.85 to 0.75. This only adds #13, with no false positives.
- Skip the priority label when an issue gets
needs-more-info, since you can't prioritize what isn't clear yet. - Run
--applyon a real repo and have the team review the labels for a week. - If it holds up, run it on a schedule or as a GitHub Action so new issues are triaged automatically before Looper picks them up.