(2024-08-15) ZviM Danger, AI Scientist, Danger

Zvi Mowshowitz: Danger, AI Scientist, Danger. the company that made the automated AI Scientist that tried to rewrite its code to get around resource restrictions and launch new instances of itself while downloading bizarre Python libraries. Its name is Sakana AI.

This paper presents the first comprehensive framework for fully automatic scientific discovery, enabling frontier large language models to perform research independently and communicate their findings.

We introduce The AI Scientist, which generates novel research ideas, writes code, executes experiments, visualizes results, describes its findings by writing a full scientific paper, and then runs a simulated review process for evaluation. In principle, this process can be repeated to iteratively develop ideas in an open-ended fashion, acting like the human scientific community.

The obvious next question is, if the AI papers are good enough to get accepted to top machine learning conferences, shouldn’t you submit its papers to the conferences and find out if your approximations are good?

How Any of This Sort of Works

According to section 3, there are three phases.

Human reviewers said it was all terrible AI slop. Also sounds about right.

Timothy Lee: I wonder if “medium quality papers” have any value at the margin. There are already far more papers than anyone has time to read.

I think medium quality papers mostly have negative value. The point of creating medium quality papers is that it is vital to the process of creating high quality papers.

New Benchmark Just Dropped

Nothing to See Here

Now we get to section 8, Limitations and Ethical Considerations. Oh, that.

All Fun and Games

Remember when we said we wouldn’t let AIs autonomously write code and connect to the internet? Because that was obviously rather suicidal, even if any particular instance or model was harmless? Good times, man. Good times.