HumanInTheLoop

Generate, evaluate, iterate: synthetic data to supercharge AI judges

Teach your AI judge—faster—with synthetic test cases Evaluating AI with "LLM-as-a-judge" works—until you run out of good examples. This study introduces a tool that generates rich, customizable synthetic test cases to refine your evaluation criteria, all within a human-in-the-loop workflow. * Configure domains, personas, lengths, desired

Teaching AI to Finish Long, Complex Tasks with Timely Human Help

AI agents are getting good at coding and research, but they still struggle with long, domain-specific projects. Training them is hard: full-time human annotation is too costly, while pure trial-and-error rarely finds successful paths. Apollo is a new sampling framework that blends lightweight, asynchronous human guidance with strict action-level filtering.