Good LLM benchmark has no public answers or is live updated, has deterministic eval and has result far from 0/100% for current LLMs. The first is why I'm not optimistic regarding @DanHendrycks humanity's last exam long-term, and the last is why I use @lmarena_ai…

We resolve the moving sofa problem by showing that Gerver's construction with 18 curve sections attains the maximum area $2.2195\cdots$.

The NLTS (No Low-Energy Trivial State) conjecture of Freedman and Hastings [2014] posits that there exist families of Hamiltonians with all low energy states of non-trivial complexity (with...

The emergence of conversational natural language processing models presents a significant challenge for Higher Education. In this work, we use the entirety of a UK physics undergraduate (BSc with...

Our database of benchmark results, featuring the performance of leading AI models on challenging tasks. It includes results from benchmarks evaluated internally by Epoch AI as well as data collected from external sources. The dashboard tracks AI progress…

We show that electron crystals compete closely with non-Abelian fractional Chern insulators in the half-full second moiré band of twisted bilayer MoTe$_2$. Depending on the twist angle and...

Following the successful 2023 edition, we organised the Second Perception Test challenge as a half-day workshop alongside the IEEE/CVF European Conference on Computer Vision (ECCV) 2024, with the...

We introduce Best-of-N (BoN) Jailbreaking, a simple black-box algorithm that jailbreaks frontier AI systems across modalities. BoN Jailbreaking works by repeatedly sampling variations of a prompt...

Our new quantum chip demonstrates error correction and performance that paves the way to a useful, large-scale quantum computer.

Optimizing neural networks with loss that contain high-dimensional and high-order differential operators
is expensive to evaluate with back-propagation due to $\mathcal{O}(d^{k})$ scaling of the...

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for...

$1M for the AI that can close 90% of new GitHub issues

The modern paradigm in machine learning involves pre-training on diverse data, followed by task-specific fine-tuning. In reinforcement learning (RL), this translates to learning via offline RL on...

Elm: making sense of your wealth. Take the guesswork out of wealth management with our sensible-yet-sophisticated approach to investing.

@iandanforth @doomie

Recent advances in multimodal large language models (MLLMs) have expanded research in video understanding, primarily focusing on high-level tasks such as video captioning and question-answering....