ejstembler.com / posts / pi-classifier-router

Model Routing with pi-classifier-router

Sep 22, 2026 Software EngineeringMachine Learning 31 views
Model Routing with pi-classifier-router

Update Sep 23 2026

The day after this post, I had the repo reviewed and fixed what turned up. Vibe-coded, remember? It found a real security issue and a real bug, so I'd recommend upgrading:

The post below is updated to match.

I recently vibe-coded a small extension for pi and Oh My Pi (omp) called pi-classifier-router. Before each turn it reads the prompt, asks a small System-One model how demanding the request is, and switches the session to a model that fits. "Rename this variable" goes to a fast, cheap model. "Refactor this module" goes to the big one.

Here's why I built it, how it works, and the config I run with omp.

Why bother routing

Every prompt was going to the same model. A one-line answer and a cross-cutting refactor cost the same, and both waited on the same big model. I wanted the easy 90% on something small and fast, and the hard 10% on the heavyweight, without typing /model every few minutes or guessing which is which.

Also, I can't always tell how hard a request is until I've made it. "Why is this test flaky?" reads like a one-liner, and can turn into an hour of digging. So I wanted the call made before the turn reaches the provider, by something that's good at judging that.

How it works

The extension hooks the before_agent_start event:

The rest is safety features around those three steps. A per-model circuit breaker stops a failing model from being retried forever, fallback chains move to another model when a circuit opens, and every decision is recorded on the session as a class-router.decision entry, so I can see what it chose and why.

Most importantly, it never breaks a turn. If classification fails, a model doesn't resolve, or there's no auth for it, the session just keeps its current model and I get a notification. Worst case, routing does nothing.

Two backends

The classifier is pluggable, with two options.

Jev (TypeSafe) is the one I use. It's an HTTP call to TypeSafe's System-One endpoint, with an API key from an environment variable. It's the quickest to set up (no weights, no Python), and it costs tokens on every prompt.

Keep in mind that with Jev, the text of every prompt it classifies is sent to TypeSafe. Slash commands are never sent, and prompts over 8,000 characters are shortened first (keeping the start and end, where the actual request usually is). That limit is the routing.maxPromptChars setting. If your prompts can't leave your machine, use Laya.

Laya is the local option, and the more interesting one to me. It runs in a local Python sidecar, or on a host you control over HTTP, and in the sidecar case nothing leaves the machine. The catch: by default it serves Laya's base checkpoints, which are close to chance on typed decisions used as-is. To get good routing you need a checkpoint fine-tuned for your questions, loaded with router: false. Also budget for the first run, which downloads multi-gigabyte weights from Hugging Face.

One extension, two hosts

I use omp, but the extension targets the upstream pi extension API, so one entry point loads on both. On omp it automatically uses what omp adds: ctx.models for resolving models, managed timers, and the auto_retry_start / auto_retry_end events for spotting failures. Only types are imported from the pi package, so it adds no runtime dependency to either host.

For me, the practical difference is model names. omp resolves role aliases the same way its --model flag does, so I can write @smol, @default, and @slow in the config and let my omp roles decide what those mean.

pi has no role aliases, so on pi the defaults do nothing until you map each category to a real model, like fireworks/accounts/fireworks/routers/deepseek-pro-latest. The Quick start in the README has an example.

Installing it

It's on npm:

omp plugin install @ejstembler/pi-classifier-router

# or on pi
pi install npm:@ejstembler/pi-classifier-router

Then start a new session. Extensions load at session start, so a session that was already running won't have it, and it'll look like it's doing nothing.

My config

The extension has its own config file. It's not omp's ~/.omp/agent/config.yml or pi's settings file. Mine lives at ~/.omp/class-router.yml:

backend: jev

jev:
  endpoint: https://api.typesafe.ai/v1/systemone
  model: jev-latest
  apiKeyEnvVar: TYPESAFE_API_KEY
  timeoutMs: 3000

routing:
  primaryQuestion: task_complexity
  modelMapping:
    trivial: "@smol"
    standard: "@default"
    hard: "@slow"
  fallbackChains:
    "@smol":
      - "@smol"
      - "@default"
    "@default":
      - "@default"
      - "@slow"
    "@slow":
      - "@slow"
      - "@default"
  confidenceThreshold: 0.5
  defaultCategory: standard

circuitBreaker:
  failureThreshold: 3
  cooldownMs: 120000
  halfOpenMaxTrials: 1

notify: true
applyTo: all

A few notes:

Watching it work

There's one command, /class-router:

Everything comes back as a notification prefixed [class-router], and never includes the prompt text or a token. For history, each decision is saved in the session file with the category, confidence, chosen model, the model it resolved to, whether the model actually changed, the reason, the backend, how long it took, and an error code when classification failed.

Before you turn it on

Where to get it

It's MIT licensed:

Start with the README's Quick start. It has everything above, plus a table of every setting and what each decision reason means. If you're curious how it works inside, that's in docs/ARCHITECTURE.md.

It's a small thing, but I'm no longer thinking about which model to use for each prompt. Give it a try!