Prime Intellect published a benchmark evaluating 18 frontier models across 153 autonomous runs using the nanoGPT optimizer framework.

Source Brief

Prime Intellect published a benchmark evaluating 18 frontier models across 153 autonomous runs using the nanoGPT optimizer framework. The study releases open traces of tool calls and reasoning scratchpads, providing developers with granular data on current model capabilities in self-directed tasks.