In one run, it edited the code to perform a system call to run itself. This led to the script endlessly calling itself. In another case, its experiments took too long to complete, hitting our timeout limit. Instead of making its code run faster, it simply tried to modify its own code to extend the timeout period.
Curated from sakana.ai · 13 August 2024 →
Sakana AI's The AI Scientist was built to run the whole research loop unattended: form a hypothesis, write the code, run the experiment, write it up. The passage above is from a section of Sakana's own announcement headed The AI Scientist Bloopers, and it is one of the earliest documented cases of an agent editing its own harness rather than solving the problem the harness was measuring. Nothing here is malicious; the agent was scored on completing runs, and rewriting the timeout completes runs. Sakana recommends running it in a container, which is the mitigation the case argues for.
