Scenario Simulation for Automated Driving with MATLAB, Simulink, and RoadRunner - MATLAB & Simulink
Video Player is loading.
Current Time 0:00
Duration 16:05
Loaded: 1.03%
Stream Type LIVE
Remaining Time 16:05
 
1x
  • Chapters
  • descriptions off, selected
  • en (Main), selected
    Video length is 16:05

    Scenario Simulation for Automated Driving with MATLAB, Simulink, and RoadRunner

    Don Bradfield, MathWorks
    Linghui Zhang, MathWorks

    Hear an overview of workflows used to develop scenes and scenarios for automated driving applications, including designing scenes and scenarios interoperable with common driving simulations tools, building scenarios from recorded sensor data, and simulating driving applications for early design and test. See how these workflows are applied to a case study featuring a Euro NCAP test suite for an autonomous emergency braking application.

    Published: 28 May 2024

    Hello, and thank you for participating in this session, designing scenarios for automated driving applications. My name is Linghui Zhang. Today, I will share some of our recent investments and high-level workflows for driving applications. And my colleague, Don, will introduce the details about our tools and workflows.

    There are several trends that are driving our recent investment in virtual scenarios. As you may have guessed, the primary trend is the need to leverage simulation for early design and test. This is especially true in the context of closed-loop systems that need to take into account recoil dynamics. Engineers wanted to reduce the amount of time physically testing the vehicle. And the replaying recorded vehicle data is often insufficient.

    Another trend we see is that there are many different driving simulation tools on the market today, both from commercial and open source. Different teams may use different tools. Or the same team may use different tools during the design and test the process. The ASAM industry body has been working to make it easier to interoperate between driving simulation tools with standards like OpenDRIVE and OpenSCENARIO.

    The third trend is that to maximize simulation value. The scenarios under test must be representative of real-world scenarios. We see a lot of customer interest in taking logs of their recorded data and building equivalent virtual scenarios for simulations.

    So in today's session, we will introduce our tools that can help you address each of these trends. First, you will learn about RoadRunner tools and how the tools can help you authoring scenes and scenarios that are interoperable with common driving simulation tools through ASAM standards like OpenDRIVE and OpenSCENARIO. You will then learn how you can use these scenes and scenarios with Simulink and Automated Driving Toolbox in order to simulate and test driving applications. Lastly, you learn how you can use MATLAB and Automated Driving Toolbox to help you build virtual scenarios from your recorded vehicle data. And then you can either use that with Simulink or other driving simulation tools. Now, my colleague, Don, is going to walk us through each of these different three workflows.

    Thanks, Linghui. My name is Don Bradfield. And I'm an application engineer here at MathWorks. And to start, I'm going to show you how you can design scenes and scenarios for common driving simulation tools.

    For scenes, I'd like to talk about our RoadRunner platform, which is used to design 3D scenes for automated driving applications. The core of this platform is RoadRunner itself, which is an intuitive tool for designing scenes. You can design scenes from scratch using the graphical editor and the RoadRunner asset library, which contains 3D models and assets you often find in scenes. RoadRunner also lets you import data from a variety of formats, including OpenDRIVE, Zenrin, and more.

    Some of you might be working with various HD map formats like here, TomTom, or even your own custom format. And you can import these HD maps into RoadRunner using the RoadRunner Scene Builder product. The scenes you create in RoadRunner can be exported to file formats like OpenDRIVE, Filmbox, or directly exported to a variety of third-party simulators.

    So let's see it in action. RoadRunner has been used very frequently to author scenes, including roads, roadside objects such as trees, traffic signs and buildings. And those static scenes are the foundations for scenario simulation.

    Customers like the ease of designing roads with RoadRunner and naturally wanted to add vehicles, trajectories, and actions. To meet this request, we provide RoadRunner Scenario. Roadrunner Scenario is an interactive editor that lets you design scenarios for simulating and testing automated driving systems.

    You can import trajectories from CSV files and from OpenSCENARIO and edit them in RoadRunner Scenario. If you have a scenario built in RoadRunner Scenario, you can export them to either OpenSCENARIO 1 or OpenSCENARIO 2 formats. You can simulate actor behaviors from other tools like Simulink and CARLA. And you can also perform automation using MATLAB, Python, or any other gRCP supported language.

    With RoadRunner Scenario, users can add different kinds of vehicles and pedestrians to a scenario. Vehicles can have various types of behaviors, such as follow road and lanes, change speed, change lane, or offset with respect to lane centers. Almost all such variables can be parameterized to generate scenario variations programmatically. The users can then simulate using the In Editor simulation.

    Once you create a scenario within a scene, you often want to test the scenario logic on other scenes. For this, you can easily relocate the scenarios with the Remap Anchor API. You can relocate the scenario within the same scene or even relocate the scenario to different scenes.

    After you author a scenario in RoadRunner Scenario, you can export to ASAM OpenSCENARIO 1, newly named XML. And you can visualize this in esmini, which is a popular open source tool for OpenSCENARIO. You can also export to OpenSCENARIO 2, newly named DSL. We engage with ASAM, especially in the areas of OpenDRIVE and OpenSCENARIO. And these standards allow engineers to develop and reuse scenarios across different teams and simulation platforms, creating a better framework for development.

    So let's jump into some new features to help you design scenes and scenarios. First, here's an example on how to specify routes to import HERE HD Live Map data and build scenes. This is especially helpful to import only the parts of a map that you're interested in.

    Next, the Actor Orientation tool enables you to modify the orientation data of a vehicle at waypoints along a path. And this allows you to simulate scenarios, such as drifting and rollovers. Last, the OpenSCENARIO export enhancements make exporting to third-party simulators, such as CARLA, easier by specifying export options for non-instantaneous actions.

    Next, let's learn about how you can simulate driving applications for early design and test. The Automated Driving Toolbox includes patterns of applications for various driving simulations. These application examples demonstrate workflows across products.

    You can use RoadRunner and RoadRunner Scenario for scene and scenario creation, add vehicle dynamics blocks for high fidelity dynamics. For algorithm development, using toolboxes for lidar, computer vision, sensor fusion and tracking, navigation, and a model predictive controls can help you in your focus areas. And some of these examples also showcase code generation and test automation capabilities using our verification and validation products.

    An essential part of developing such applications is simulating sensors. In the Automated Driving Toolbox, vehicles and other actors can be represented as simple box shapes as polygon meshes. This supports sensors, such as radar, lidar, ultrasonic vision, and lane detections. The Automated Driving Toolbox also provides sensors that can be used with the Unreal Engine, such as the monocular camera and fisheye camera. There are also sensors that detect position, such as Wheel Encoder, GPS, IMU, and INS.

    There are multiple application examples representing each of these categories. Each example implements features like adaptive cruise control, emergency braking, or high-level functions, such as lane change and parking. You can use these examples as a basis for designing and testing automated driving applications.

    So let's dive into the emergency braking and talk about some new offerings and workflows to help you and your team. One area that we're seeing a lot of interest is the Euro NCAP testing for autonomous emergency braking. So we released the test suite for Euro NCAP protocols. Let's look at the Euro NCAP offerings with RoadRunner Scenario and Simulink.

    We ship out pre-built Euro NCAP scenarios that have been generated for the ego asset as RoadRunner Scenario files. The test suite is primarily focused on Euro NCAP safety assist AEB car-to-car scenarios. The ego vehicle is controlled using an AEB Simulink test bench that co-simulates with RoadRunner Scenario. The autonomous emergency braking results for all tests are consolidated to provide a detailed Euro NCAP report.

    Let's take a look at a single Euro NCAP scenario, the car-to-car rear moving and the AEB test bench in action. You'll see here that we ship a pre-built AEB controller. But you can easily implement your own controller.

    We have a system that represents a vehicle and its dynamics. And last, we have a scenario and an environment subsystem. This contains RoadRunner Scenario co-simulation interfaces and sensors.

    Let's take a look at the steps to simulate an AEB Euro NCAP test scenario. The first step in the process is generating the scenario. There are 11 different vehicle under test speeds and five impact overlap percentages leading to 55 variations of this AEB car-to-car rear moving scenario.

    Next, let's make sure a collision occurs to ensure this is a valid test. So we'll disable AEB and verify the collision. It looks like a collision and valid test to me. And we can confirm this with our impact plot.

    Last, we'll enable AEB and verify avoidance. It looks like AEB was triggered right around 16 seconds. And the AEB algorithm worked as intended.

    Now that I've showed you how to simulate a single AEB Euro NCAP test scenario, let's take a look at how we can automate testing with Simulink test with more of the 7 Euro NCAP scenarios. The AEB results for the tests are consolidated to provide a detailed Euro NCAP report. Simulink tests facilitates management of all test iterations with RoadRunner Scenario and also facilitates logging of the AEB results.

    The first step in automate testing for AEB Euro NCAP scenarios is to configure the Euro NCAP scenario. Once you've selected and configured your scenarios, you can press Run in Simulink Test. Once you press Run and Simulink Test, the first iteration will start running, then iterate through all your configured test cases and variations.

    Next, we can review and explore the generated test report. So let's open up a report. Details about the variation in scoring are specific to the scenario that was tested. Here, you can see the speeds and collision status for turn across path and now the overlap percentages and impact speeds for removing.

    We want to make sure visualizing the results are just as easy as running the test. When navigating through the test report, you can click on any iteration variation. Here, the iteration name will update, and you can quickly replay the simulation. I'll click on another iteration variation to view the results. The test suite for Euro NCAP protocols is a powerful tool to help you with your Euro NCAP testing.

    Last, I'll show you how to build scenarios from maps and recorded sensor data. The Scenario Builder Support Package enables you to mimic real-world driving scenarios and improve your test coverage. Many of you have access to map data and recorded data from sensors like camera, lidar, and IMUs. The Scenario Builder uses this to generate vehicle trajectories for creating driving scenarios. In doing so, it makes use of perception, navigation, and sensor fusion workflows.

    So let's quickly go through an example. Here, we'll use camera images, lidar point cloud data, and label data for this reconstruction. You can follow this workflow to generate a RoadRunner Scenario from your sensor data.

    Let's look at some recently added examples of that. This example generates a road scene containing traffic signs extracted from labeled, camera, and lidar sensor data. The traffic signs are labeled in camera frames. And the lidar data provides accurate depth and dimensional measurements. In this example, we extract trees, buildings, and road elevation information from lidar data and reconstruct a road network with OpenStreetMap data. And last, this example shows the extraction of key events, such as left turns, right turns, acceleration and deceleration, lane change, and cut-ins from the ego and target actor trajectories.

    I demonstrated how recent investments from MathWorks can help you design scenarios for use with common driving simulation tools, use those scenarios for simulating driving applications, and how you can generate scenarios from your recorded data. And everything that I showed you is based off of shipping examples that you can reproduce today.

    We help customers get started with these tools and workflows. Actually, we also work closely with customers through technical engagements and consulting service to extend and customize solutions. For example, we have worked with customers to extend scenario simulation workflow to model traffic signals. You will learn more about this in Seo-Wook's upcoming presentation.

    We also help customers deploy their simulation to the cloud and integrate with software-defined vehicle workflows. You can see an example of integration with Qualcomm at the booth session. More recently, we have seen growing interest in constraint solving to reduce scenario hyperspaces. This allows you to identify invalid scenarios early on before introducing more expensive, higher fidelity simulations.

    These are just some of the areas where we engage with customers. Please reach out if you have projects you would like to engage with us. Along with today's conference, we also offer a post webinar event on June 6. The topic is "Design and Simulate Scenes and Scenarios for Automated Driving Applications." Thank you for your attention.