Safeguarded AI: Theory

Tender description

ARIA is an R&D funding agency built to unlock scientific and technological breakthroughs that benefit everyone. We empower scientists and engineers to pursue research at the edge of what is technologically or scientifically possible. We reach across disciplines, sectors and institutions to shape, fund and manage projects across the R&D ecosystem, from startups to universities, to break down silos and discover new pathways. We're looking for proposals for Safeguarded AI: Theory, for info see https://www.aria.org.uk/programme-safeguarded-ai/ Lot 1: As AI becomes more capable, it has the potential to power scientific breakthroughs, enhance global prosperity, and safeguard us from disasters. But only if it's deployed wisely. Current techniques working to mitigate the risk of advanced AI systems have serious limitations, and can't be relied upon empirically to ensure safety. To date, very little R&D effort has gone into approaches that provide quantitative safety guarantees for AI systems, because they're considered impossible or impractical. By combining scientific world models and mathematical proofs we will aim to construct a 'gatekeeper', an AI system tasked with understanding and reducing the risks of other AI agents. In doing so we'll develop quantitative safety guarantees for AI in the way we have come to expect for nuclear power and passenger aviation. Our goal: to usher in a new era for AI safety, allowing us to unlock the full economic and social benefits of advanced AI systems while minimising risks. For more info see https://www.aria.org.uk/programme-safeguarded-ai/