padv.AssessmentResult

Create and access results from assessment

Description

This object requires CI/CD Automation for Simulink Check.

A padv.AssessmentResult object represents the results from an assessment. When you run a task, the build system automatically runs assessments associated with the task. The assessment results also contain a table of assessed artifacts and their compliance status. When the assessment runs, the assessment action function creates the assessment results. When you define an assessment, you can categorize task artifacts by using the assessment result object functions addCompliantArtifacts, addNonCompliantArtifacts, and addWarningArtifacts.

Starting in R2024a, when you point to the task status, you can see a breakdown of the individual assessments for a task, the assessment results, and the impact of those assessment results on the overall task status. The assessment information provides the specific objectives associated with the failures, warnings, and passing results that you see in the Details column.

Creation

Syntax

padv.AssessmentResult(assessmentId)

padv.AssessmentResult(___,Name=Value)

Description

padv.AssessmentResult(assessmentId) creates an assessment result object that represents the result from an assessment with the ID assessmentId.

padv.AssessmentResult(___,Name=Value) sets properties using one or more name-value arguments. For example, padv.AssessmentResult("A1",Status="Compliant") creates an assessment result for a compliant test assessment.

Input Arguments

assessmentId — Unique identifier for assessment that produced assessment result

string

Unique identifier for assessment that produced the assessment result, specified as a string.

Example: "A1"

Data Types: string

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: padv.AssessmentResult("A1",Status="NonCompliant",Summary="No test results available.")

Status — Outcome of assessment

padv.ComplianceStatus.Compliant (default) | padv.ComplianceStatus.Warning | padv.ComplianceStatus.NonCompliant

Outcome of assessment, specified as either:

• padv.ComplianceStatus.Compliant — Compliant with the assessment objective

• padv.ComplianceStatus.Warning — Review for potential issues with compliance to the assessment objective

• padv.ComplianceStatus.NonCompliant — Not compliant with the assessment objective

These values belong to the enumeration class padv.ComplianceStatus.

The Status property of the assessment result reflects an overall status for the entire assessment result. To see the statuses of individual artifacts that the assessment assessed, see the ArtifactStatus property. Depending on the value of the Status property, the assessment enforces constraints on the ArtifactStatus property to make sure that the overall assessment result status aligns with the collective artifact statuses.

This property becomes read-only after object creation.

Example: padv.ComplianceStatus.NonCompliant

Summary — Summary of assessed artifacts

string

Summary of assessed artifacts, specified as a string.

The Summary can provide an overview of the current status of the artifacts. For detailed artifact statuses, see the ArtifactStatus property.

Example: "No test results available."

Data Types: string

Properties

AssessmentId — Unique identifier for assessment that produced assessment result

string

Unique identifier for assessment that produced the assessment result, specified as a string.

Example: "A1"

Data Types: string

Summary — Summary of assessed artifacts

string

Summary of assessed artifacts, specified as a string.

The Summary can provide an overview of the current status of the artifacts. For detailed artifact statuses, see the ArtifactStatus property.

Example: "No test results available."

Data Types: string

Status — Outcome of assessment

padv.ComplianceStatus.Compliant (default) | padv.ComplianceStatus.Warning | padv.ComplianceStatus.NonCompliant

Outcome of assessment, specified as either:

• padv.ComplianceStatus.Compliant — Compliant with the assessment objective

• padv.ComplianceStatus.Warning — Review for potential issues with compliance to the assessment objective

• padv.ComplianceStatus.NonCompliant — Not compliant with the assessment objective

These values belong to the enumeration class padv.ComplianceStatus.

The Status property of the assessment result reflects an overall status for the entire assessment result. To see the statuses of individual artifacts that the assessment assessed, see the ArtifactStatus property. Depending on the value of the Status property, the assessment result enforces constraints on the ArtifactStatus property to make sure that the overall assessment result status aligns with the collective artifact statuses.

This property becomes read-only after object creation.

Example: padv.ComplianceStatus.NonCompliant

ArtifactStatus — Assessed artifacts and their statuses

table

Assessed artifacts and their statuses, specified as a table.

You can use the table to identify which artifacts the assessment considers as compliant, warning, or noncompliant against the assessment objectives. By default, the property creates a table with columns for Artifact and Status. The rows in the table contain each artifact that the assessment assessed and the status of that artifact.

The Status property of the assessment result reflects an overall status for the entire assessment result. Depending on the value of the Status property, the assessment result enforces constraints on the ArtifactStatus property to make sure that the overall assessment result status aligns with the collective artifact statuses. The statuses for the assessed artifacts must follow these rules.

• If the Status of the assessment result is "Compliant":

  • All artifacts in ArtifactStatus must have compliant statuses.

  • You can add compliant artifacts to the assessment result by using the addCompliantArtifacts function.

• If the Status of the assessment result is "Warning":

  • Artifacts in ArtifactStatus must have either compliant or warning artifact statuses.

  • You can add compliant and warning artifacts to the assessment result by using the addCompliantArtifacts and addWarningArtifacts functions respectively.

• If the Status of the assessment result is "NonCompliant":

  • Artifacts in ArtifactStatus must have either compliant or noncompliant statuses.

  • You can add compliant and noncompliant artifacts to the assessment result by using the addCompliantArtifacts and addNonCompliantArtifacts functions respectively.

Example: table('Size',[0,2],'VariableNames',["Artifact","Status"],'VariableTypes',["padv.Artifact","padv.ComplianceStatus"])

Data Types: table

Object Functions

addCompliantArtifacts	Add compliant artifacts to assessment result
addNonCompliantArtifacts	Add non-compliant artifacts to assessment result
addWarningArtifacts	Add artifacts with warnings to assessment result

Examples

Assess Quality of Task Results

You can define and manage formal assessments of your task inputs and outputs by using a padv.Assessment object. The padv.Assessment object specifies the assessment objectives, the assessment action that the assessment performs to evaluate the task and determine the task status, and recommended actions. You can associate an assessment with a specific task by using the addAssessments method or Assessments task property and the build system automatically performs your assessments when you run the task. You create and access assessment results by using a padv.AssessmentResult object.

Open the Process Advisor example project.

processAdvisorExampleStart

The model AHRS_Voter opens with the Process Advisor pane to the left of the Simulink^® canvas.

In the Process Advisor pane, click the Edit process model button to open the processmodel.m file for the project.

Replace the contents of the processmodel.m file with the following example code. The code defines:

• T — A custom task that uses the action function myTaskAction to create a task output file output.txt.

• A1 — An assessment that uses the action function assessOutputIsTxt to check if that output file is a text file.

• A2 — An assessment that uses the action function assessOutputFileContent to check if the output file is empty.

The assessment action functions summarize the results of the assessment by using padv.AssessmentResult objects.

function processmodel(pm)
    % Define process model for project

    arguments
        pm padv.ProcessModel
    end

    % --- Assessments ---
    A1 = padv.Assessment("A1",Objective="Output file must be a TXT file.",Action=@assessOutputIsTxt);
    A2 = padv.Assessment("A2",Objective="Output file should not be empty.",Action=@assessOutputFileContent);

    % --- Task ---
    T = padv.Task("MyTask",Action=@myTaskAction,OutputDirectory=fullfile("$PROJECTROOT$"),...
        Assessments=[A1,A2]);

    % Add the task to the process model
    pm.addTask(T);

end

function taskResult = myTaskAction(~)
    % Create a file named "output.txt"
    filename = "output.txt";
    fileID = fopen(filename,'w');
    fclose(fileID);
    % Create task result and specify file as task output
    taskResult = padv.TaskResult;
    taskResult.Status = padv.TaskStatus.Pass;
    taskResult.OutputPaths = fullfile(pwd,filename);
end

function res = assessOutputIsTxt(assessment, inputs, taskResult)
    try
        % Get first output artifact and its file address
        outputFile = taskResult.OutputArtifacts(1);
        fileAddress = outputFile.ArtifactAddress.getFileAddress();
        
        % Check if file has .txt extension
        [~, ~, ext] = fileparts(fileAddress);
        if strcmp(ext, ".txt")
            res = padv.AssessmentResult(assessment.Id, Status="Compliant");
            res.Summary = "Output file is TXT.";
            res.addCompliantArtifacts(outputFile);
        else
            res = padv.AssessmentResult(assessment.Id, Status="Warning");
            res.Summary = "Output file is not TXT.";
            res.addWarningArtifacts(outputFile);
        end
    catch
        res = padv.AssessmentResult(assessment.Id, Status="NonCompliant");
        res.Summary = "Output file is inaccessible or missing.";
    end
end

function res = assessOutputFileContent(assessment, inputs, taskResult)
    try
        % Get first output artifact and its file info
        outputFile = taskResult.OutputArtifacts(1);
        fileAddress = outputFile.ArtifactAddress.getFileAddress();
        fileID = fopen(fileAddress, 'r');
        fileContent = fread(fileID);
        fclose(fileID);
        % Check if file is empty
        if isempty(fileContent)
            res = padv.AssessmentResult(assessment.Id, Status="Warning");
            res.Summary = "Output file is empty.";
            res.addWarningArtifacts(outputFile);
        else
            res = padv.AssessmentResult(assessment.Id, Status="Compliant");
            res.Summary = "Output file is not empty.";
        end
    catch
        res = padv.AssessmentResult(assessment.Id, Status="NonCompliant");
        res.Summary = "Output file is inaccessible or missing.";
    end
end

In Process Advisor, run the task by clicking the Run All button.

In the Tasks column, the task status shows that the task passed. But the Details column shows one warning and one passing result.

Point to the task status to the left of the task name. Starting in R2024a, the Assessments section shows that the passing result comes from the A1 assessment because the task successfully generated a text file. The warning result comes from the A2 assessment because the task generates an empty text file.

View a breakdown of the assessments, assessment results, and the compliant, warning, and non-compliant artifacts by clicking Assessments in the task status pop-up. If you add recommended actions to your assessments by using the addRecommendedActions function, you can view those recommended actions in the Assessments dialog.

By default, non-compliant assessments cause the task to fail. But suppose that you want the warning assessment result to cause the task to fail. You can change the task definition in the process model to make the task fail on any warning results from the assessments by using the task property FailTaskOn.

    % --- Task ---
    T = padv.Task("MyTask",Action=@myTaskAction,OutputDirectory=fullfile("$PROJECTROOT$"),...
        Assessments=[A1,A2],FailTaskOn="AnyWarning");

Run the task in Process Advisor by clicking Refresh Tasks and clicking the run button for the task.

In the Tasks column, the task status now shows that the task failed. If you point to the task status, the Assessments section now indicates that warnings and non-compliant assessments cause the task to fail.