A tale of two Regulatory Markets: the role of institutional incentives in supporting sustainable Regulatory Markets for future AI systems

In the near and long term, the deployment of powerful AI
capabilities raises concerns of accidents, misuse, and systemic risk (Brundage et al., 2018; Shevlane and Dafoe, 2019;
Zwetsloot and Dafoe, 2019; Hernandez-Orallo et al. ´ , 2019).
These capabilities also require new techniques to audit and
certify (Cihon et al., 2021a; Gursoy and Kakadiaris, 2022).
Regulatory Markets could help AI governance to be more
adaptive (Clark and Hadfield, 2019). Governments set targets and mandate that companies employ the services of private regulators to demonstrate compliance with those targets. Private regulators must compete with each other to
regulate AI companies. This competition may lead to innovations in methods to detect unsafe behaviour and better
understand what safe development practises look like.
While the size of these benefits is uncertain, regulators
must be incentivised to invest in better methods in the first
place. One can ask what role governments can play in providing incentives for higher quality regulators to join the regulatory market. To this end, this extended abstract highlights
findings from a recent evolutionary game analysis. The paper explores how different institutional incentives influence
the evolutionary dynamics of interactions between AI companies and regulators (Bova et al., 2023). Namely, the paper
considers two types of incentives governments might consider, showing that only one of these types, dubbed ”Vigilant
Incentives”, can support regulators in evaluating cuttingedge AI systems.