Finally figured out that SIA (Self Improving AI) framework from Hexo Labs after scrolling through a ton of posts. Gonna jot this down. Back in the day, self-improvement stuff was split into two camps: one tweaks the harness and scaffolding, like Darwin Gödel Machine; the other does test-time training, directly messing with weights. SIA’s approach is to shove both paths into the same iterative loop, where a Feedback-Agent reads the full execution trace and reward, then decides whether to modify the scaffolding, run a LoRA weight update, or do both.
What’s even more convenient is you don’t even have to pick an RL algorithm yourself. If the reward is dense step-level signals, it goes with PPO; if rollouts are cheap and you only verify at the end, it uses GRPO; for right-skewed distributions where correct solutions are rare, it picks EAW. This auto-algorithm-selection feature is honestly the most practical part for me. The whole harness part runs on CPU, just needs an Anthropic key to get going, and only the weight updates require an H100. So if you’re trying to save cash, you can just run the harness version first, figure out the upper limits, then decide if you want to throw in a GPU. Code’s under MIT license, up on GitHub.