AMD DEVELOPER HACKATHON · ACT II · TRACK 1

A COST-AWARE
LLM ROUTING AGENT

ANSWER ALMOST EVERYTHING WITH A SMALL LOCAL GEMMA MODEL (LLAMA.CPP ON CPU, FREE), AND ESCALATE TO FIREWORKS ONLY WHEN LOCAL CONFIDENCE IS LOW. LOCAL INFERENCE COUNTS FULLY TOWARD ACCURACY AND COSTS ZERO TOKENS.

account_tree GITHUB deployed_code DOCKER HUB play_circle DEMO VIDEO slideshow SLIDE DECK
route

ROUTE A QUERY

TYPE A PROMPT — THE REAL CLASSIFIER + GATE LOGIC RUNS LIVE IN YOUR BROWSER
bar_chart

EVALUATION CHARTS

ACCURACY BY CATEGORY (ALL 19 CORRECT)
ROUTING DISTRIBUTION
LATENCY DISTRIBUTION (SECONDS)
tune

CONFIDENCE THRESHOLD (τ) SIMULATOR

DRAG τ — TASKS ESCALATE WHEN LOCAL CONFIDENCE < τ
τ = 0.02
LOCAL ROUTES
17
REMOTE ROUTES
2
MEASURED REMOTE TOKENS
345

table_rows

PER-TASK RESULTS

ROUTE
CATEGORY
SORT BY
TASK ID CATEGORY ROUTE CONFIDENCE LATENCY RMT TOKENS ANSWER