This morning, President Biden issued a highly anticipated executive order that establishes new actions on the safe, secure, and trustworthy use of artificial intelligence. The White House previously disclosed that it was drafting this action in July when it announced that 15 companies had agreed to voluntary commitments on the safe, secure, and transparent development of AI. This executive order marks the most significant regulatory action taken by the federal government to date, and while it targets federal agencies, the order has significant implications for the private sector as well.
The executive order is a positive step forward to help organizations effectively manage the unique security, ethical, and operational risks that come with AI adoption. Advances in AI have far outpaced companies’ abilities to secure these systems. As an engineering discipline, AI development is nascent compared to traditional software development. We largely don’t have equivalent testing and security habits - in part because the dynamics of AI models makes this a challenging problem.
The executive order puts this into motion by requiring certain elements that were previously voluntary. While there are many parts to the order, AI red teaming is a central component. This is a new motion for most companies, but even those that have the specialized skillset know that manual red teaming doesn’t scale. Companies need to automate the parts that can be automated, which also has the added benefits of standardizing the assessment process and operationalizing risk management frameworks. Ultimately, AI red teaming is about assurance. At Robust Intelligence, we believe that organizations will require novel approaches to meet this new paradigm in AI security and comply with the executive order.
Scope of the US Executive Order
Biden’s sweeping executive order primarily targets federal agencies' deployment of AI, which includes the development of new resources, processes, and entities. This lays the groundwork for adherence by both the private and public sector. Many enterprises have already started to implement AI standards and policies, spurred by proposed legislation and voluntary frameworks - namely the EU AI Act and the NIST AI Risk Management Framework (AI RMF). The Biden Administration has consulted with over 20 countries on AI governance, which signals the global implications of any individual nation’s actions.
The executive order addresses specific focus areas of AI risk, including:
- Develop new standards for safety and security, including AI red team tests to be set by NIST
- Protect privacy, including guidelines to evaluate the effectiveness of privacy-preserving techniques
- Advance civil rights, including enabling the Department of Justice and Federal civil rights offices to investigate and prosecute civil rights violations
The order also creates a new White House AI Council established to oversee the implementation of this executive order and sets timelines for federal agencies to meet the specified requirements. For instance, NIST has 270 days to develop a new companion resource to the AI RMF focused on the use and development of generative AI and the Secretary of Labor has 365 days to publish federal guidance on the use of AI-assisted hiring tools regarding nondiscrimination.
Third-Party Red Teaming
The inclusion of AI red teaming in the executive order was deliberate. It builds on the “external red teaming” language used in the voluntary AI commitments declaration and independent “testing” used in the NIST AI RMF and EU AI Act. This is an acknowledgment that all models carry some level of risk. By thoroughly assessing every model prior to use, companies can determine if the level of risk exceeds their standards or thresholds. Without such assessments, companies may unknowingly open themselves up to unnecessary security, ethical, and operational risk.
NIST has been tasked with setting the criterial for AI red teaming. As a preeminent physical science laboratory attached to the US Department of Commerce, National Institute of Standards and Technology (NIST) standards are designed to enhance economic security and improve our quality of life. Their AI RMF is the most widely adopted framework to date.
We’ll have to wait to see how NIST interprets these evaluations because there’s no single accepted definition of what AI red teaming entails, however the executive order defines AI red teaming as a practice used to "adopt adversarial methods to identify flaws and vulnerabilities, such as harmful or discriminatory outputs from an AI system, unforeseen or undesirable system behaviors, limitations, or potential risks associated with the misuse of the system." AI red teaming as a cybersecurity practice has a history of contentious definitions. Its military roots stem from war gaming, USAF red flag exercises, TSA screening, DARPA challenges, and more. Practically speaking, the primary purpose of AI red teaming is assurance, and it should be conducted at three points in the AI lifecycle:
- the model level
- the application level before release
- continuously after release because an application’s behavior depends on new data that is always changing
It’s important to note that red teaming is not limited to security and privacy vulnerabilities. This practice should be used to identify ethical and operational weaknesses as well, such as bias, toxicity, hallucinations, factual inconsistency, and more.
Where to Start
Starting an AI red team may seem daunting, but there are certain practices and tools that can help you get started an increase your effectiveness. Automated solutions, such as the one offered by Robust Intelligence, should be used for the elements that can be automated. For example, automation is best used for focused testing at the system component level on fine-tuned models and the like, whereas manual, open-ended testing is best done at the system level to find novel failure modes between components.
It’s also helpful to aspire to a 90/10 rule, meaning you should spend less than 10% of your time getting 90% of testing completed through tooling and automation. The rest of your time budget (90% or more) is to get the last 10% of corner cases through manual trial and error.
An automated approach to AI red teaming has the added benefit for providing an organizational-level standard. When considering solutions, you should consider the following capabilities:
- Identify risks early in development - use hundreds of specialized tests, as well as algorithmically-generated red teaming attacks, to automatically identify model vulnerabilities and unintended behavior
- Spot new risks in production models - continue testing models periodically and analyzes the outputs over time
- Validate models quickly - translate statistical test results into clearly defined outputs that meet major AI risk frameworks and regulatory requirements
- Enforce custom AI risk standards - tune parameters and add custom tests to meet your needs
In Summary
The AI executive order is a significant milestone, and not just for federal agencies or those doing business with the government. The new resources, processes, and entities will lay the groundwork for enterprises to accelerate adoption of AI red teaming, which in turn will give companies the confidence to deploy more AI-powered applications and realize the full potential of AI.
To learn more about using automated red teaming to satisfy AI risk standards and regulatory compliance, visit our website or request a demo.
