Experts at WMG at the University of Warwick and Deepen AI have launched credit-based public access to the largest public store of scenarios for testing automated vehicles.
Known as ‘The Safety Pool ™ Scenario Database’, it features over 250,000 different scenarios to create standards that will make the operation of automated vehicles much safer for users and pedestrians everywhere.
Over 450 organisations have enrolled in the database since the launch of the project, and with increased safety around autonomous technology, the experts hope that the use of Artificial Intelligence (AI) in road vehicles will increase.
Dr Maria J Alonso, Lead, Automotive in the Software-Driven Era Initiative, World Economic Forum, explained: “Collaboration among stakeholders is key to ensure that automated vehicles offer the highest possible levels of safety. The Safety Pool ™ Scenario Database, with its over 250,000 scenarios and its collaborative approach, provides a platform to contribute to safer mobility on our roads.”
Do autonomous vehicles have a high safety risk?
Safety is the biggest challenge that remains in increasing the use of AI in road vehicles. On average, there are 9.1 self-driving car accidents per million miles driven, compared to 4.1 accidents per million miles for regular vehicles.
Current autonomous vehicles on the market still require a certain level of driver concentration, but vehicles advertised as self-driving lead drivers to become more reliant on the technology. This means that the driver’s reaction time will be slower in the event of an accident.
Moreover, the software systems of automated vehicles can also unexpectedly malfunction, which increases the likelihood of collisions with other road users. Until AI becomes more widely used in the transport industry, experts will have limited information on how safe self-driving vehicles really are.
The Safety Pool ™ Scenario Database
The new database provides a large range of scenarios, provided by organisations, in different operational design domains (ODDs) that can be leveraged by governments, industry and academia to test and benchmark Automated Driving Systems.
The scenarios are generated by novel scenario generation methods, such as knowledge-based and data-based. They focus on uncovering failures in automated vehicles to capture more scenarios.
As access to the store is credit-based, credits are awarded to contributing organisations based on the uniqueness of the scenario and its validity. The credits can then be used to gain access to more scenarios. This system encourages users to contribute to the growing database and make more scenarios available to the autonomous vehicle community.
Dr Siddartha Khastgir, Head of Verification & Validation, at WMG University of Warwick, said: “With the public launch of the Safety Pool™ Scenario Database, we are democratising critical data that was not easily available in the ecosystem. By aligning with international standards, and working closely with regulators and developers in the UK and internationally, we are driven by the mission of making the safety of automated vehicles pre-competitive.”
The scenarios also cover a diverse set of attributes and manoeuvre types, such as urban environments, highways and different weather conditions, where automated vehicles will have to perform different manoeuvre types such as cut-ins and overtaking.
Jamie McFaden, Head of Automated Vehicle Technologies Group at the Vehicle Certification Agency said: “The Vehicle Certification Agency, the UK’s Type Approval Authority, recognises the importance of ODD-based testing, and as a result, is evaluating the use of Safety Pool ™ Scenario Database as part of the Type Approval process for Automated Driving Systems.
“Coherent with our mission to align with international standards, today, each scenario has been launched in ASAM OpenScenario 1.1 and ASAM OpenDRIVE 1.6 formats. This provides further interoperability between stakeholders using ASAM OpenX standards for their simulation-based testing of automated vehicles.”
Use cases cover the Automated Lane Keeping System, low-speed shuttle and urban level four. Users can efficiently search the scenarios through the design domain and behaviour tags in line with ASAM OpenLabel standards.
Mohammad Musa, CEO of Deepen AI, concluded: “By contributing and working within the Safety Pool™ framework, commercial stakeholders across AV fleet operators, OEMs, Tier 1 suppliers and all other Tier 2+ suppliers can massively accelerate verification & validation as well as share insights & edge cases that are very hard for a single stakeholder to collect by themselves.”
