This is the third installment in a series of articles addressing engineering challenges and opportunities associated with the verification and validation of autonomous and semi-autonomous vehicles. Click here to read part 1 and part 2.

Testing a digital twin of a system begins at the test bench, which is an abstraction that provides the ability to execute a system under test (SUT) and then access data within it. The digital twin of an electrical/electronic (E/E) system includes simulations of all its ECUs, networks, mechatronic hardware, and other parts. Each type of data or access requires a different interface, and is configured and managed by different specialized tools. For E/E systems, the primary data include:

• Signals and parameters in embedded software and simulation models
• Data from the diagnostic functionalities of ECUs
• Measurable variables and calibration data within automotive software
• Data controlling fault- and error-injection within the system
• Signals encoded into messages communicated over an automotive network

Test automation software supports the testing sequence (or test case) and utilizes the interfaces exposed by the test bench to access data and exercise the digital twin’s functionality. Overall, test cases must have enough coverage to verify and validate all aspects of the E/E system requirements.

If test automation software is directly coupled to the test bench using a rigid interface, test reuse is not possible across the development process, and large disconnects will remain between verification and validation (V&V) at the model-level, and V&V at the implementation-level. This is a significant issue because every test case will not be converted or back-adapted to each level of abstraction – and tool-dependent tests will be developed multiple times by different people with varying skillsets.

Factors that make test reuse difficult include:

• Different levels of abstraction of the data and signals within the test benches
• Multiple modeling and programming languages
• Proprietary means to sequence and control the various specialized tools or otherwise stimulate and trace the digital twin

Without standardization, test case reuse is unattainable because otherwise there is no consistent way to communicate between test cases and test benches;describe, configure, and initialize test benches, or map and access data within test benches.

ASAM XIL is an API standard for the communication between test automation tools and test benches. It addresses the above challenges, while promoting interoperability between solutions from different vendors.

Key ASAM XIL benefits include:

• Supports test case reuse by decoupling test automation software from test hardware
• Establishes interoperability between different vendors relative to test automation software and test benches
• Provides a means for controlling test benches based on simulation tools
• Significantly reduces testing effort
• Supports long-term protection of testing investments
• Establishes a common approach for test bench setup and initialization
• Collects time-aligned trace data and specifies its data format
• Supports injection of faults and errors into the SUT
• Supports test events and triggering
• Reduces training costs

ASAM XIL is built into two major parts. The first is a test bench for the simulation models, ECU data (e.g. parameters, variables, and diagnostics), electrical error simulation, and automotive networks. This test bench provides interfaces to different types of tools via specific test bench ports. These ports offer standardized access to ECUs, including interfaces for calibration, measurement, model access, diagnostics, network signals, and electric error simulation.

The second major part of ASAM XIL is a framework for mapping units, data types or variable identifiers, and for configuring actions, measurement, logging, triggering, initialization, ordering, and sequencing.

The ASAM XIL mapping framework is key to addressing the particularly challenging task of decoupling test automation software from test benches. It does this by allowing data within a test bench to differ in value and type using mappings. By providing a new mapping for each test bench, the test case remains unchanged and can be reused against any XIL abstraction level within the digital twin and across engineering phases. Port independence to test cases is achieved via object-oriented access to variables on the test bench, and an abstraction of ports within the framework layer. Based on these variable objects, the framework provides objects for signal recording, signal generation, and event watching and triggering.

ASAM XIL is also beneficial from a business perspective. It reduces training costs and supports testing early in the development process when issues are least expensive to rectify. Test cases can be reused across the entire development process. OEMs and suppliers can exchange test cases and test benches very efficiently. And finally, verification engineers can switch between the best test automation software and the best test benches.

The fourth installment in this series will address the role of generative model-driven development in automotive V&V.