Research

Software Testing with Reinforcement Learning

Software verification and validation is a critical component of modern development processes. With the increasing functionality of modern software systems, the demand for efficient, scalable, and automated testing methods is growing. Reinforcement Learning (RL), with its approaches based on agent-environment interaction, offers a promising framework for automating software testing. RL is particularly suited for identifying complex failure scenarios triggered by specific and non-trivial sequences of states and actions. By iteratively adapting test strategies based on thier reward function, RL agents can dynamically generate test scenarios that uncover previously undetected bugs and anomalies. At ITIV&FZI, research is conducted on the development of novel RL methods that aim at optimizing the software testing process in terms of both efficiency and coverage of potential defect states, as well as facilitating the discovery of hard-to-detect anomalies.

Intelligent and Adaptive Traffic Simulation

The validation of automated driving functions poses a central challenge due to the discrepancy between simulations and real-world traffic situations. Previous approaches that rely on pre-recorded trajectories of traffic participants are limited, as these agents are unable to adequately respond to the behavior of automated vehicles. This leads to unrealistic simulation results that are insufficient for reliable evaluation of driving functions. Learning-based methods, however, offer new potential for developing realistic and reactive traffic agents that dynamically react to the behavior of automated vehicles and specifically generate critical traffic scenarios. This allows a targeted investigation of the behavior of automated vehicles in challenging situations and contributes to the improvement of their robustness. At ITIV&FZI, novel methods are investigated to efficiently implement these approaches and to significantly improve the validation of automated vehicles through realistic and adaptive simulation environments.

Student Works
Title	Type	Date

Publications

2024

Generation of Adversarial Trajectories using Reinforcement Learning to Test Motion Planning Algorithms
Ransiek, J.; Schütt, B.; Hof, A.; Sax, E.
2024. 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Institute of Electrical and Electronics Engineers (IEEE), Bilbao, 24th - 28th September 2023, 2819–2826, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/ITSC57777.2023.10422130

2023

Statistical Consideration of the Representativeness of Open Road Tests for the Validation of Automated Driving Systems
Langner, J.; Pohl, R.; Ransiek, J.; Elspas, P.; Sax, E.
2023. 2023 IEEE International Automated Vehicle Validation Conference (IAVVC), Institute of Electrical and Electronics Engineers (IEEE)Austin, 16th-18th October 2023, 1–8, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/IAVVC57316.2023.10328090

1001 ways of scenario generation for testing of self-driving cars: A survey
Schütt, B.; Ransiek, J.; Braun, T.; Sax, E.
2023. IEEE Intelligent Vehicles Symposium (IV), Institute of Electrical and Electronics Engineers (IEEE), Anchorage, 4th-7th June 2023, 1–8, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/IV55152.2023.10186735

2022

Robust Parameter Estimation and Tracking through Lyapunov-based Reinforcement Learning
Rudolf, T.; Ransiek, J.; Schwab, S.; Hohmann, S.
2022. IECON 2022 – 48th Annual Conference of the IEEE Industrial Electronics Society, 1–6, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/IECON49645.2022.9968893