shikumi-optimize
Safe HaskellNone
LanguageGHC2024

Shikumi.Optimize.Instruction

Description

M3 — instruction search (a MIPRO/COPRO-style optimizer). For each node, the grounded proposer (EP-19, proposeInstructions) suggests several candidate instruction strings — fed a dataset summary, the program's pseudo-code and description, the node's real input/output field names, the instruction history, and a stylistic tip — and each candidate is scored; the best is kept. Optimization is greedy coordinate ascent — one node at a time, holding the others fixed — so the candidate count is linear in nodes × proposals.

The proposer and its signal-gatherers are themselves ordinary shikumi Programs, so they are typed, cached, traced, and testable with the same stub-LM machinery as everything else — the optimizer is written in the framework it optimizes. The current effective instruction (override, or signature base when no override is present) is always retained as a candidate (the proposer guarantees it), and keeping that candidate writes no redundant override, so a node can never end up worse than where it started.

Budget. The grounded proposer reserves 4 + proposalsPerNode predicted LM completions per node (dataset summary, program describe, module describe, and one generation per proposal); scoring one candidate reserves one completion per dataset example per predict node. The search stops — returning the best found so far — before either bound in the Budget would be exceeded. When the remaining budget cannot cover a node's full proposal, that node keeps its current instruction (no proposer call) rather than partially proposing.

This module re-points V1's blind proposer at EP-19's grounded surface; the old ProposeInProposeOutproposeInstruction predictor is removed (the grounded GenerateInstructionIn/GenerateInstructionOut replaces it, still emitting a proposedInstruction output field).

Synopsis

Documentation

instructionSearch :: (ToJSON i, ToJSON o) => Int -> Budget -> Optimizer i o Source #

Search for a better instruction at every node by greedy coordinate ascent under an explicit LM-call budget, using the grounded proposer to generate candidates.