Map / Core AI / Benchmarks & Leaderboards / CORE-Bench Hard

CORE-Bench Hard

Agent installs deps, runs published scientific code, and answers questions

CuratedSoftware Engineering Benchmarks

I use this adds CORE-Bench Hard to your stack and finds people with similar stacks. Every mark feeds the counts on Pulse. Sign in to mark it.

No ratings yet

Who uses it

0 use this

0use it now

0tried it, dropped it

0want to try

Nobody has marked it yet. Press I use this, Tried it or Want to try to be the first.

Ratings

Only from people who use it or tried it

No ratings yet

The first rating comes from someone who uses CORE-Bench Hard or tried it.

Use CORE-Bench Hard?

What it does

CORE-Bench Hard is the hardest level of CORE-Bench, a benchmark from Princeton researchers that tests whether AI agents can reproduce the results of published scientific papers. Given only a paper's code repository, the agent must install dependencies, run the code and answer questions from the outputs and figures. Results appear on Princeton's Holistic Agent Leaderboard (HAL).

Alternatives on the map

All 26 alternatives

Benchmarks & Leaderboards, starting with the other Software Engineering Benchmarks.

Compare with

People who use it also use

Not enough marks yet to tell. When people who use CORE-Bench Hard mark the rest of their stack, it shows up here.

Open website