Benchmarks & Evaluation

Standardised tests and leaderboards used to measure and compare model performance on software engineering tasks

• Adopt
  • SWE-bench Verified
• Trial
  • LiveCodeBench
  • BigCodeBench
• Hold
  • HumanEval