Automate Testing for Highway Lane Change
This example shows how to assess the functionality of a lane change application by defining scenarios based on requirements, and automating testing of components and the generated code for those components. The components include a lane change planner and controller. This example builds on the Highway Lane Change example.
The highway lane change system enables the ego vehicle to automatically move from one lane to another lane on a highway. The system models longitudinal and lateral control dynamics for automated lane change. System-level simulation enables you to assess the functionality of the system-level test bench model. You can configure simulations to test scenarios based on system requirements. Automatically running these simulations enables regression testing to verify system-level functionality.
The Highway Lane Change example shows how to design and simulate a system-level model for lane change using ground truth information. This example shows how to automate the testing of that model against multiple scenarios by using Simulink® Test™. The scenarios are based on system-level requirements. In this example, you:
Review requirements — The requirements describe system-level test conditions. Create simulation test scenarios to represent these conditions.
Review the test bench model — Review the system-level lane-change test bench model that contains metric assessments. These metric assessments integrate the test bench model with Simulink Test for automated testing.
Disable runtime visualizations — Disabling runtime visualizations reduces the execution time for automated testing.
Automate testing — Configure a test manager to simulate each test scenario, assess success criteria, and report results. Explore the results dynamically in the test manager and export them to a PDF for external reviewers.
Automate testing with generated code — Configure the lane change planner and controller components to generate C++ code. Run automated testing on the generated code and verify the behavior.
Automate testing in parallel — Increase overall execution speed for running tests by using parallel computing on a multicore computer.
To explore the test bench model, load the highway lane change project.
Requirements Toolbox™ enables you to author, analyze, and manage requirements within Simulink. This example contains 15 test scenarios, with high-level testing requirements defined for each scenario. Open the requirement set.
Alternatively, you can open the file from the Requirements tab of the Requirements Manager app in Simulink.
Each row in this file specifies the testing requirements of a lane-change system test scenario in textual and graphical formats. These scenarios enable you to test the lane change planner and controller components.
scenario_LC_01_SlowMoving— Three-lane straight road scene with a slow-moving lead vehicle in the ego lane.
scenario_LC_02_SlowMovingWithPassingCar— Three-lane straight road scene with a slow-moving lead vehicle in the ego lane and a passing car in the left lane.
scenario_LC_03_DisabledCar— Three-lane straight road scenario with a disabled vehicle in the ego lane.
scenario_LC_04_CutInWithBrake— Three-lane straight road scene with a lead car cutting into the ego lane with a brake.
scenario_LC_05_SingleLaneChange— Four-lane straight road scene with multiple vehicles in the scenario and a slow-moving lead vehicle.
scenario_LC_06_DoubleLaneChange— Four-lane straight road scene with multiple vehicles in the scenario, including a slow-moving lead vehicle and a fast-moving rear vehicle.
scenario_LC_07_RightLaneChange— Three-lane straight road scene with a lead car in the ego lane and a passing vehicle in the left lane that causes a right-lane change trigger for the ego vehicle.
scenario_LC_08_SlowmovingCar_Curved— Curved road scene with a slow-moving lead car in the ego lane.
scenario_LC_09_CutInWithBrake_Curved— Curved road scene with a lead car cutting into the ego lane.
scenario_LC_10_SingleLaneChange_Curved— Curved road scene with multiple vehicles in the scenario and a slow-moving lead vehicle.
scenario_LC_11_MergingCar_HighwayEntry— Highway entry scene with three vehicles in the scenario. One of the vehicles is merging into the ego lane from the highway-entry road segment.
scenario_LC_12_CutInCar_HighwayEntry— Highway entry scene with four vehicles in the scenario. One of the vehicle cuts into the ego lane on the curved segment of the road network.
scenario_LC_13_DisabledCar_Ushape— U-shaped scene with multiple vehicles in the scenario, including a disabled car in the adjacent lane of the ego vehicle and a slow-moving vehicle in the ego lane.
scenario_LC_14_DoubleLaneChange_Ushape— U-shaped scene with multiple vehicles in the scenario, including a disabled car in the adjacent lane of the ego vehicle and a slow-moving vehicle in the ego lane.
scenario_LC_15_StopnGo_Curved— Curved road scene that contains six vehicles in the scenario. The lead vehicle slows down while other vehicles travel in adjacent lanes.
Review Test Bench Model
This example reuses the
HighwayLaneChangeTestBench model from the Highway Lane Change example. Open the test bench model.
The test bench model contains these subsystems:
Scenario and Environment— Specifies the scene, vehicles, and map data used for simulation.
Planner Configuration Parameters— Specifies the configuration parameters required for the planner algorithm.
Highway Lane Change Planner— Implements the lane change planner algorithm for highways.
Lane Change Controller— Algorithm model that specifies the controller.
Vehicle Dynamics— Specifies the dynamic model for the ego vehicle.
Metrics Assessment— Assesses system-level behavior.
Configure this test bench model by using the
helperSLHighwayLaneChangeSetup script. This setup script takes
scenarioName as input, where
scenarioName can be any one of the previously described test scenarios. To run the setup script, use this code:
scenarioName = "scenario_LC_15_StopnGo_Curved";
You can now simulate the model and visualize the results. For more details on the design of individual components in the test bench model, see the Highway Lane Change example.
Next, automate the simulation runs for this test bench model using Simulink Test for the different test scenarios. The
Metrics Assessment subsystem enables integration of system-level metric evaluations with Simulink Test. This subsystem uses Check Static Range (Simulink) and Check Static Lower Bound (Simulink) blocks for this integration. Open the
Metrics Assessment subsystem.
Metric Assessment subsystem outputs these metrics:
Verify Time gap— Verifies that the time gap between the ego vehicle and the lead vehicle is greater than 0.8 seconds. The time gap between the two vehicles is the ratio of the calculated headway distance to the ego vehicle velocity.
Verify No Collision— Verifies that the ego vehicle does not collide with any vehicle in the scenario at any point during the simulation.
Verify Longitudinal Jerk— Verifies that the ego vehicle has a longitudinal jerk value between –5 m/s^3 and 5 m/s^3.
Verify Lateral Jerk— Verifies that the ego vehicle has a lateral jerk value between –5 m/s^3 and 5 m/s^3.
Verify Safety— Verifies the safety of the ego vehicle with respect to the vehicles present in the current lane, left lane, and right lane. This ensures that the ego vehicle performs a safe maneuver while following the trajectory.
Disable Runtime Visualizations
The system-level test bench model opens a visualization window showing the ego vehicle, the sampled trajectories, and a capsule list in both the chase view and top view of the scenario. You can disable the visualization window to make the simulation run faster.
Configure the Visualization block to disable the visualization window.
blk = "HighwayLaneChangeTestBench/Visualization"; set_param(blk,EnableChaseView="off") set_param(blk,EnableTopView="off")
Configure the Test Manager to automate the testing of the lane change application. Open the
HighwayLaneChangeMetricAssessments.mldatx test file in the Test Manager.
testFile = sltest.testmanager.load("HighwayLaneChangeMetricAssessments.mldatx");
Observe the populated test cases previously authored in this file. These tests are configured to run the model.
Each test case uses the POST-LOAD callback to run the setup script with appropriate inputs. After simulating the test case, the Test Manager uses the
helperLaneChangeReport function from the CLEAN-UP callback to generate the plots for curvature, lateral deviation, heading angle, velocity, steering angle, acceleration, and jerk.
Run and Explore Results for Single Test Scenario
Test the system-level model on the
testSuite = getTestSuiteByName(testFile,"Test Scenarios"); testCase = getTestCaseByName(testSuite,"scenario_LC_15_StopnGo_Curved"); resultObj = run(testCase);
Generate test report after the simulation.
sltest.testmanager.report(resultObj,"Report.pdf", ... Title="Highway Lane Change", ... IncludeMATLABFigures=true, ... IncludeErrorMessages=true, ... IncludeTestResults=false, ... LaunchReport=true)
Report.pdf file. Observe that the
Test environment section shows the platform on which the test is run and the MATLAB version used for testing. The
Summary section shows the outcome of the test and duration of the simulation in seconds. The
Results section shows pass or fail results based on the assessment criteria. This section also shows the plots logged from the
Run and Explore Results for All Test Scenarios
Simulate the system for all the tests by using the
run(testFile) command. Alternatively, you can simulate the system by clicking Play in the Test Manager app.
When the test simulations are complete, view the results for all of the tests in the Results and Artifacts tab of the Test Manager. For each test case, the Test Manager uses the Check Static Range (Simulink) and Check Static Lower Bound (Simulink) blocks in the model to visualize overall pass or fail results.
You can find the generated report in the current working directory. This report contains a detailed summary of pass or fail statuses and plots for each test case.
Verify Test Status in Requirements Editor
Open the Requirements Editor and select Display. Then, select Verification Status to see a verification status summary for each requirement. Green and red bars indicate the pass or failure, respectively, of simulation results for each test.
Automate Testing with Generated Code
HighwayLaneChangeTestBench model enables integrated testing of the
Highway Lane Change Planner and
Highway Lane Change Controller components. Generating code for these components requires a Simulink Coder™ license. This workflow enables you to verify that the generated code produces expected results that match the system-level requirements throughout simulation.
Highway Lane Change Planner to run in software-in-the-loop (SIL) mode.
model = "HighwayLaneChangeTestBench/Highway Lane Change Planner"; set_param(model,SimulationMode="Software-in-the-loop")
Lane Change Controller to run in SIL mode.
model = "HighwayLaneChangeTestBench/Lane Change Controller"; set_param(model,SimulationMode="Software-in-the-loop")
Simulate the system for all the test scenarios by using the
run(testFile) command. After the tests are complete, review the plots and results in the generated report. If you have a license for Simulink Coverage™, you can also get the code coverage analysis for the generated code in the generated report. This license also enables you to visualize the coverage results for individual test cases and aggregated coverage results.
Automate Testing in Parallel
If you have a Parallel Computing Toolbox™ license, you can configure the Test Manager to execute tests in parallel using a parallel pool. To run tests in parallel, save the models after disabling the runtime visualizations by using the
save_system("HighwayLaneChangeTestBench") command. The Test Manager uses the default Parallel Computing Toolbox cluster and executes tests only on the local machine. Running tests in parallel speeds up execution and decreases the amount of time it takes to get test results. For more information on how to configure tests in parallel from the Test Manager, see Run Tests Using Parallel Execution (Simulink Test).