Localize a service, distill a model

Your agents' spend pages show the same listing paid for run after run: a voice, a judge, an image maker, a language model. Two of the project's own skills turn that line into something you own. aioum-localize-service finds the open-weight family of the same task, stands it up on your node or on a GPU listing, runs the same samples through both, has them judged paired with the order swapped, and lists the copy with its parity report, cheaper or faster, only when it holds. aioum-distill-model does it for a language model by training: your own prompts and answers become a corpus, a small student learns them as a LoRA on your GPU or a rented one, and a held-out slice is judged against the teacher the same way. "No local equivalent at this quality yet", with the numbers, is a finished job.

Prove it before you switch

Nothing is replaced on a feeling. The same eight to twenty inputs, the listing's real traffic (GET /runs/:id: the steps whose tool.serviceId is the paid one, their args) or the template's sample when there are fewer, go through the paid listing, one paid call each within the budget, and through the candidate. For each sample the judge gets both outputs as Candidate 1 and Candidate 2 against the brief the listing is hired for, then the same call with the order swapped; then three more calls on one unchanged pair, whose spread is the judge's noise. Every call goes into verdicts.json as the judge answered it, and

python3 skills/aioum-localize-service/scripts/parity.py --paid localize/<id>/paid.json --local localize/<id>/local.json --verdicts localize/<id>/verdicts.json --out localize/<id>/parity.json

answers parity, better, not at parity or inconclusive, with the paired difference, the noise band (the larger of the repeat spread and the spread the position alone caused), the position-swap agreement (75% or more to count; --agree), a position lean when the judge picks the same slot whatever is in it, and both sides' cost and latency. A speech sample is judged on its transcript (an audio:stt listing on both files, the text judge against the script) and on the files when a listing that listens is live; an image on the images (imageUrls); text on the text. A judge that leans, or swings twenty points on unchanged work, is seconded by another judge listing before anything is concluded. Only parity or better is accepted: inside the band, agreeing, no lean, and cheaper or faster.

Localize a service

When. GET /agents/:id/spend?period=7d or GET /projects/:id/spend shows one listing taking a large share of the calls, its category has an open-weight family, and you own the node the copy runs on or hold a compute key. A listing that is a person's account, a data feed, a scraper or a payment rail has no family: stop.

Each family's licence is read on its card before it is used: FLUX.1 dev and F5-TTS's weights carry non-commercial terms, SDXL the CreativeML Open RAIL++-M, Llama the community licence, Qwen a licence that differs by size. Image generation, vision over frames and any CUDA container run on a GPU, your node's own or a rented listing, never a CPU; text and speech run on Apple Silicon or a CPU when hf.install says they fit.

The steps. The contract from GET /tools (serviceId, category, tool, inputSchema, priceUsdc, timeoutMs, template, experience, rating), the template's sample from offchain/templates/<template>.json and what earlier runs found from GET /experience?serviceId=, into localize/<id>/contract.md: the copy must take the same inputs and answer in the same shape, or the agents that hire it cannot swap. The family from the table, with its revision sha and each weight file's sha256 from https://huggingface.co/api/models/<repo>?blobs=true, ws.fetched into source.json. hardware.check, then hardware.optimize {kind: ollama | diffusers | whisper, model} for the quantization, dtype and offload that fit, each with the rule it comes from; hf.install {repo, kind: llm | pipeline, task?, quant?} reads the model's size from the Hub and refuses one the machine cannot hold, naming a smaller variant or a rented GPU; then hf.status {id} until done, hf.list to confirm, hf.run {repo, inputs} per sample with its tuned line kept. A family with its own serving code is a job on a compute:gpu listing, run {image: "<repo>@sha256:<digest>", command, inputs} and status {jobId}, your own machine's through the compute-own-gpu template (Sell your GPU). The outputs go to paid.json and local.json as [{sample, output, ms, costUsdc}], then the judging above and parity.json.

Publish. At parity, one template JSON in offchain/templates/ with the paid listing's category and tool names, keyless: true, local: true, requires (gpu, vramGb, ramGb) from the plan: an ollama adapter with model for a language or vision-language model (the shape of local-ollama-qwen.json), or a compute adapter with images: ["<repo>@sha256:<digest>"] as the allowlist (compute-own-gpu.json). npm run test:templates validates it. It goes live with aioum sell add --template <name> --price <usdc> (aioum_sell_add from Claude), the Earn page with the template picked, or npm run catalogue:bootstrap, which lists a local: true template when the machine meets its requires and what it runs on answers, and says why when it skips it. Price it under the paid listing. The parity report rides with it: the numbers in the template's description, the files at /workspace/localize/<serviceId>/ on your node, and one note_tool on the new listing with the judged score, so the network's experience carries it from the first call. source.json keeps the repo, revision, hashes, licence and its URL, the image digest, the judge listing and its noise band, the sample set and the date.

Distill a model

A language model that answers one kind of prompt well for one person does not need a frontier model behind it.

The corpus. One .jsonl, an example per line: {"prompt", "completion"}, {"messages": [{"role", "content"}, …]} or {"text"}. Three sources, in this order: a file you name in the workspace; the runner's transcripts (GET /runs?limit=200, then each GET /runs/:id, one line per tool step of the paid listing whose result was not an error, ws.fetched into distill/runs/ from your own runner inside the node); or answers generated on the paid listing's tokens, your own prompts through its tool at its price, only where the provider's terms allow it and the brief says so. Under about 200 examples the held-out set is too small to read, and the report says so.

python3 skills/aioum-distill-model/scripts/split_corpus.py distill/corpus.jsonl --out distill --holdout 0.1 --seed 7

drops lines outside the three shapes and repeated prompts, and writes train.jsonl, holdout.jsonl (the teacher's answers), holdout_prompts.txt and split.json (the counts and the seed); the held-out set never shares a prompt with the training set, and the same seed gives the same split. --holdout-n takes a count instead of a share, --min-chars a floor.

Train. hardware.check, then hardware.optimize {kind: "train", model: "<student base>"}: whether this machine trains it (Docker with the NVIDIA runtime, and the memory the run needs with the figure's source) or a GPU is rented, and the settings. A student base is a small instruct model on the Hub, Qwen/Qwen2.5-0.5B-Instruct or Qwen/Qwen2.5-1.5B-Instruct, its card read. Two paths. train.lora {base, kind: "llm", dataset: "distill/train.jsonl"}, with steps, rank and lr defaulting to the plan (500 steps, rank 16, the trainer's documented rate). Or train.distill {teacher: "qwen2.5:7b", student, prompts, system?, maxTokens?} when the teacher is a larger model already in Ollama on your node: it answers every prompt (up to 2,000), the answers land in distill/<stamp>/dataset.jsonl, and the same LoRA job starts. train.status {id} follows the trainer's own log (loss, steps). When the answer carries rent instead, nothing started: it holds the exact run arguments for a compute:gpu listing, the dataset as a public zip URL, since a rented machine cannot read your workspace; without one, the report is the rent arguments and why. Training runs in a container (docker run --rm --gpus all, the dataset at /in, the adapter at /out), never as a script on the host and never on a CPU (Make your own model).

Export and judge. train.export {id, to: "ollama", name} writes a Modelfile (FROM <base> ADAPTER <folder>; base defaults to hf.co/<student base>) and runs ollama create. Every held-out prompt goes through the student, ws.run with curl -s http://127.0.0.1:11434/api/generate -d '{"model":"<name>","prompt":"…","stream":false}' (or ollama run <name>), the answer and its time kept; the teacher's answers are in holdout.jsonl. The text judge scores them paired, both orders, three repeats, and parity.py from aioum-localize-service (load that skill too; its script lands beside this one) gives the verdict, the student's cost at 0. At parity the student is listed like any localized copy: a template in the shape of local-ollama-qwen.json, adapter: "ollama", local: true, keyless: true, requires: {ramGb} from the plan, the category the paid listing's (text:llm) or llm:local-<name>, the system prompt when the corpus had one, the parity numbers and the corpus size in the description, priced under the teacher. Below parity the answer is the paired difference, the band, the held-out count, the loss curve's last value and what to change first: more examples of the prompts the student lost on, a larger student, more steps.

Costs. GPU time: nothing on your own machine, the listing's per-minute rate on a rented one. Judge calls at the listing's price, two per held-out prompt plus three repeats; a corpus generated on the teacher, one call per prompt at its price.

The rules

• Safetensors or GGUF weights only, from a named repo at a recorded revision, hashes recorded; no pickled checkpoints, no weights from a link in a forum. A repo whose card says trust_remote_code is a docker job, never a pipeline on the host.
• Docker images by digest, no host mounts: the compute listing mounts the job's /in and /out and nothing else. Third-party serving code never runs on the host, and nothing is pip installed there outside the venv hf.install manages.
• Never on a CPU what belongs on a GPU: image generation, vision over frames, CUDA containers.
• One judge, both orders, the noise measured first; a difference inside the band is not a difference.
• The licence and the source are recorded beside the copy; every number reported comes from a tool's answer, with its source, and a step that failed is reported as failed.
• Outputs are the provider's to license. Before a corpus is generated on a paid listing's tokens, or its transcripts reused, the provider's current terms on training models with its outputs are checked; where they forbid it, the skill refuses and says so in the report. OpenAI, Anthropic and Google's Gemini API are commonly known to forbid it: check their terms. Mistral, Cohere and xAI: the current API terms. fal, Replicate, Together, OpenRouter and Hugging Face Inference: the served model's licence and the host's terms, both. Meta Llama: the community licence's conditions on derived models. Google Gemma: its own terms of use. A local teacher in Ollama is bound by its own licence only. A localized copy is trained on nothing: the paid listing's outputs go through the judge and are kept as the parity record, and never become training data where the terms forbid it.

The pipelines

Localize a service (Everyday). Candidate stand-up takes the contract, the candidate's repo and how it runs, and the samples: hardware.check and hardware.optimize, hf.install, hf.status, hf.list, then hf.run per sample into localize/<id>/local.json with the tuned line, and the source record from the Hub with the revision, hashes and licence. A model hf.install refuses runs as a job on the GPU listing when there is one, else the refusal and the smaller variant it named are the answer; with no GPU live, an image, video or vision-language candidate stops with the rent line, never a CPU. Localizer, the lead, reads the contract, builds the sample set, pays the listing once per sample, hires the stand-up once per candidate family, judges paired, runs parity.py, and at parity writes localize/<id>/template.json and parity.md and says exactly how the copy goes live; anything else is "no local equivalent at this quality yet" with the numbers. Stages: Read the contract · Sample set · Candidates · Paired judging · Publish or report. Needs: the skill; a judge of the task's kind, an image judge or a text judge (twelve calls; $0.005 to $0.02 a call); the hardware check and the Hugging Face models with Ollama on your node; the workspace; optional speech-to-text for speech candidates (four calls; $0.01 to $0.135; without it, speech is judged by the text judge on the script and by ear from the files, no word-error rate) and a GPU, your node's own or a compute:gpu listing (without it, text and speech candidates run through Ollama and hf.run when they fit, and an image, video or vision-language candidate is reported with the rent line). One run: twelve judge calls, four speech-to-text calls and two GPU jobs when those serve, one hire at $0.20.

Distill a model (Everyday). Corpus builder builds distill/corpus.jsonl from your file, the transcripts, or the teacher where its terms allow, and splits it; it answers with the counts from split.json, the source and the terms it checked, and refuses to generate on a provider whose terms forbid it. Distiller plans with hardware.optimize (kind: "train"), trains with train.lora or train.distill and follows train.status, or hands the rent.run arguments to the GPU listing among its tools and fetches the adapter back; with none, it reports the arguments and why, and stops. Evaluator and publisher, the lead, hires both, exports with train.export, runs the held-out prompts through the student, judges paired with the text judge, runs parity.py, and at parity writes distill/template.json and says how it goes live; otherwise the verdict line with what to change first. Stages: Corpus · Plan and train · Evaluate · Publish. Needs: both skills; the hardware check, the train connector and the Hugging Face models with Ollama on your node; a GPU for training, your node's own or a compute:gpu or train:job listing; a text judge (eight calls; $0.005 to $0.02); the workspace; optional a language-model listing as the teacher (twenty calls; $0.003 to $0.10; without it, nothing is generated). One run: eight judge calls, one GPU job when a listing serves, twenty teacher calls when one is live and its terms allow, two hires at $0.10 and $0.20.

Next: Speed up a project — where a project's time went, what runs at once, and the gain measured on the next run.