Imagine going to sleep after writing a single Markdown specification and waking up to find that an AI agent ran dozens of LLM fine-tuning experiments overnight on your behalf - discovering optimal LoRA ranks, refining learning rate schedules, tuning batch sizes and committing each verified improvement to Git.
This is no longer a fantasy. Earlier this year, the autoresearch project showcased how autonomous LLM agents can iteratively explore pre-training in a self-contained loop. Taking inspiration from this paradigm, we created autofinetune: applying autonomous research loops to LLM post-training (Supervised Fine-Tuning and Reinforcement Learning via GRPO), using Google’s full AI stack—Tunix, Gemma, and Cloud TPUs orchestrated with Antigravity CLI and Gemini Flash 3.7.
In this post, we’ll explore how the autonomous research loop works for post-training and walk through a couple of real-world LLM finetuning case studies.
Traditional post-training involves a repetitive, manual cycle:
attn_vec_einsum to the LoRA target modules improve accuracy?" or "What happens if we change rollout temperature during GRPO?").As demonstrated in autoresearch, we can now automate this whole process with the power of AI agents:
program.md): human defines the loop, boundary conditions, evaluation criteria, and constraints.run.py): A single, clean, self-contained finetuning script.program.md: it modifies run.py, runs the training job, measures the target metric, retains winning commits or reverts regressions, and logs results in results.tsv.
In the first experiment in autofinetune, we took the same SFT setup in our previous blog and extended it by creating the autoresearch loop to optimize google/functiongemma-270m-it on the google/mobile-actions dataset.
The agent was given boundaries in program.md:
Here is a sample trajectory from sample_runs/SFT_results.tsv demonstrating how the agent hill climbed.
As you can see, the agent is able to automatically adjust LoRA rank/alpha, optimizer, learning rate, etc. to keep improving the model’s accuracy in terms of generating correct function calls.
Supervised fine-tuning is only a simple test. For our second case study, we took the official GRPO example from the Tunix repository (which trains Gemma 3 1B for math reasoning using GSM8K; the trained model has better numerical accuracy and format accuracy in its answers) and set it up for autonomous RL finetuning. Reinforcement learning is subject to hyperparameter sensitivity, instability, and longer execution times - making this task more challenging and time-consuming.
Post_RL_metric, which is simply numerical_accuracy + format_accuracy (you can of course use other metrics, i.e., using different weights).Below is a sample trajectory logged in sample_runs/RL_results.tsv, showing the agent’s progress. The agent was able to identify better LoRA configurations, rollout temperature, KL penalty, system prompt, etc. to improve the total reward by ~10%.
We hope this project shows you the power of AI agents in the domain of LLM post-training and inspires you to think about how to leverage them to automate your LLM finetuning workflows using Tunix on TPUs. Please check out the code, sample runs, and program.md templates in the autofinetune GitHub repository, explore the Tunix library, and start building your own autonomous post-training lab today!
