The U.K.’s AI Safety Institute has released a free, open-source testing platform that evaluates the safety of new AI models. Dubbed Inspect, the toolset should provide a “consistent approach” towards the creation of secure AI applications around the world.
Inspect is the first AI safety testing platform created by a state-backed body to be made freely available to the public. Ultimately, it will accelerate improvements in the development of secure AI models and safety testing efficacy.
How to use the Inspect software library
The Inspect software library, launched on May 10, can be used to assess the safety of an AI model’s features, including its core knowledge, ability to reason and autonomous capabilities, in a standardised way. Inspect provides a score based on its findings, revealing both how safe the model is and how effective the evaluation was.
As the Inspect source code is openly accessible, the global AI testing community — including businesses, research facilities and governments — can integrate it with their models and get essential safety information more quickly and easily.
SEE: Top 5 AI Trends to Watch in 2024
AI Safety Institute Chair Ian Hogarth said the Inspect team was inspired by leading open-source AI developers to create a building block towards a “shared, accessible approach to evaluations.”
He said in the press release, “We hope to see the global AI community using Inspect to not only carry out their own model safety tests, but to help adapt and build upon the open source platform so we can produce high-quality evaluations across the board.”
Secretary of State for Science, Innovation and Technology Michelle Donelan added that safe AI will improve various sectors in the U.K. from “our NHS to our transport network.”
The AISI, along with the expert group Incubator for Artificial Intelligence and Prime Minister Rishi Sunak’s office, is also recruiting AI talent to test and develop new open-source AI safety tools.
Inspect: What developers need to know
A guide on how to use the Inspect toolkit in its base form can be found on the U.K. government’s GitHub. However, the software comes with an MIT License that allows it to be copied, modified, merge published, distributed, sold and sub-licensed; this means anyone can amend or add new testing methods to the script via third-party Python packages to improve its capabilities.
Developers looking to use Inspect first need to install it and ensure they have access to an AI model. They can then build an evaluation script using the Inspect framework and run it on their model of choice.
Inspect evaluates the safety of AI models using three main components:
- Datasets of sample test scenarios for evaluation, including prompts and target outputs.
- Solvers that execute the test scenarios using the prompt.
- Scorers that analyse the output of the solvers and generate a score.
The source code can be accessed through the U.K. government’s GitHub repository.
What the experts are saying about Inspect
The overall response to the U.K.’s announcement of Inspect has been positive. The CEO of community AI platform Hugging Face Clément Delangue posted on X that he is interested in creating a “public leaderboard with results of the evals” of different models. Such a leaderboard could both showcase the safest AIs and encourage developers to make use of Inspect so their models can rank.
Linux Foundation Europe also posted that the open-sourcing of Inspect “aligns perfectly with our call for more open source innovation by the public sector.” Deborah Raji, a research fellow at Mozilla and AI ethicist, called it a “testament to the power of public investment in open source tooling for AI accountability” on X.
The U.K.’s moves towards safer AI
The U.K.’s AISI was launched at the AI Safety Summit in November 2023 with the three primary goals of evaluating existing AI systems, performing foundational AI safety research and sharing information with other national and international actors. Shortly after the summit, the U.K.’s National Cyber Security Centre published guidelines on the security of AI systems along with 17 other international agencies.
With the explosion in AI technologies over the past two years, there is a dire need to establish and enforce robust AI safety standards to prevent issues including bias, hallucinations, privacy infringements, IP violations and intentional misuse, which could have wider social and economic consequences.
SEE: Generative AI Defined: How it Works, Benefits and Dangers
In October 2023, the G7 countries, including the U.K., released the ‘Hiroshima’ AI code of conduct, which is a risk-based approach that intends “to promote safe, secure and trustworthy AI worldwide and will provide voluntary guidance for actions by organizations developing the most advanced AI systems.”
This March, G7 nations signed an agreement committing to explore how artificial intelligence can improve public services and boost economic growth. It also covered the joint development of an AI toolkit to inform policy-making and ensure AI used by public sector services are safe and trustworthy.
The following month, the U.K. government formally agreed to work with the U.S. on developing tests for advanced artificial intelligence models. Both countries agreed to “align their scientific approaches” and work together to “accelerate and rapidly iterate robust suites of evaluations for AI models, systems, and agents.”
This action was taken to uphold the commitments established at the first global AI Safety Summit, where governments from around the world accepted their role in safety testing the next generation of AI models.
