Verification and Validation Workflows for Medical Device Development - MATLAB & Simulink
Video length is 49:43

Verification and Validation Workflows for Medical Device Development

Overview

Whether you are an embedded software developer or an engineer designing a medical device, software quality is critical. 

Model-Based Design incorporates verification and validation into the software development workflow. As a result, the software is comprehensively tested and verified before you integrate it into a medical device. In addition, parts of the documentation required by IEC 62304 are automatically generated for regulatory compliance.   

But how does Model-Based Design work in practice? And how do you get started? Join this webinar to learn the best practices for improving software quality while reducing development time and costs. We will go through the best practices for designing software for medical devices, from requirements management through design, implementation, and integration to verification and testing. You will learn how to produce higher-quality software while reducing design errors and development time.

Highlights

  • Requirements management and system architecture design
  • Automated unit and system level testing with test report generation
  • IEC 62304 supported workflow for FDA regulatory compliance 

About the Presenter

Akhilesh Mishra is a senior application engineer for the medical devices and healthcare industry at MathWorks. He specializes in signal/data processing, artificial intelligence, and GPU computing workflows. He has been with MathWorks for 4+ years. Akhilesh holds a master’s from the University of Kansas where he was the signal processing lead in a group working on radar systems for sounding the ice sheets of Greenland and Antarctica to study global sea-level rise.

Recorded: 8 Dec 2021

Welcome to today's talk on IEC 62304 certification, and verification and validation workflows for medical device development with MATLAB and Simulink. My name is Akhilesh Mishra, and I'm a Senior Application Engineer at MathWorks. I am based off at the Plano, Texas office in USA. And part of my job is to support customers like yourself using MATLAB and Simulink for being successful in their medical device development.

Before I begin, I would like to remind you that throughout the session I'm available in the Q&A panel for answering any questions you have. So as we go along, feel free to post your questions in the Q&A panel, and we would be taking it during the session itself. All right, so the key takeaways.

So I do want to highlight two things at least in the talk today. First of all, you can develop your medical device software end to end while complying with the regulatory guidelines, by using MATLAB and Simulink. And MATLAB and Simulink has this comprehensive framework which is tied in with the IEC 62304 guidelines for development and even maintaining the software as a medical device, or what we'll refer to as SMD throughout the session.

Little introduction about The MathWorks. We are a private company based off-- our headquarters is in the Natick, Massachusetts location. We have almost 4 million users worldwide using our software for developing a wide variety of different applications. So the IEC 62304 standards give us the guidelines for software as a medical device development. And what we'll focus on today is this nice diagram which was given on page number 7 of the document, which describes the entire pipeline of gathering the customer needs, coming up with the software requirements, coming up with the architectural design of your software, and then the detailed design, and then also implementing the unit verification and validation, plus the system integration and validation and testing before the software is actually released.

And what we'll see in today's session is how MathWorks has these comprehensive set of tools which allows you to implement the 62304 guidelines in your device development throughout the lifecycle of your software. And not only just to develop it, but then these tools help you to maintain your software once the product is actually deployed and it is in the field as well.

So a little bit about why this compliance to V&V and why all these certification standards, why is it so important. So from the FDA standpoint, like for any software for that matter, not only medical device actually. If I speak about other industries, it's kind of the same thing, which happens everywhere, is that the most errors are introduced during-- we are in the development phase, where we gather the requirements and come up with the specs, come up with the code. And then finally, once the code is deployed on the hardware, and if the errors are found in the field, then the cost to fix those errors is a lot.

And then there's a lot of repercussions which come along the way. And the most important repercussion being the device recalls in the medical industry. This is some stats I was able to pull together from the Sedgwick Recall Index. It seems like from Q1 to Q2 in 2021, the recalls did go down a bit, almost 18%.

However, one thing important to note is that still the software is the main issue for all these recalls. And apparently, 20th time in the last 21 quarters, it has been the software issues which were responsible for the recalls. And maybe a lot of the people on the session today can comprehend.

But the last thing what any medical device developer wants is this software based device recall. So we definitely want to avoid that. And that's the reason why we have this V&V framework and this IEC 62304 guidelines to prevent such kind of a situation to happen. Pick out the bugs, errors, issues, early in the phase. And then have your clean and reliable code deployed onto the target which would be running together with all the medical devices.

Now we have the Simulink framework for developing your models, a simulation based model for whatever device you're building, whatever software controls, signal processing, AI, you can put it all together at a system level and simulate it, test it. But then also we have this V&V framework, which ties in back to this modeling where you can link your requirements to the entities in the model. And then based on the requirements, you can also come up with an architectural design, which we'll see today.

Have some static analysis done before you actually take your model and then generate a code from it, which would get deployed. And once you're in the process of generating code, you can also do an equivalence testing, and then also an equivalence checking. So all the test cases which you've put together for your specific model which you had built, they would equally apply to the generated code from the model. And you can have a testing on that, and then also an equivalence checking where you have a software actually which is deployed onto the target itself, like an embedded processor or an FPGA. And then you can have your test cases run through that code once in what we call the processor in the loop mode.

So we'll review a lot of these tools throughout the guidelines. And then we also have-- for the whole picture, we have the IEC 62304 Certification Kit which we'll also explore, for getting additional certificates and then also the test cases for The MathWorks product itself. All right, so now let's come to Simulink and see some of these tools in action.

What I have over here in Simulink is actually a simple rendition model of an infusion pump. And for those of you who do not know the infusion pump, it is a device to deliver medicine or bolus to the patient. Now it is a simplified model of the infusion pump. If you would like to see the full complicated model, we do have this video series on-- if you just Google search infusion pump and Simulink. So it will bring it to you, this video series link where we have a full detailed design of an infusion pump.

But over here what we have is a supervisory logic. There's a PID controller. There's a motor which actually delivers a bolus. And then there's a delivery line dynamics subsystem, which actually corresponds how the patient reacts to the bolus and then what's the output pressure-- the actual pressure of the delivery line.

We have a HMI panel, where the doctor or the nurse can select what should be the set delivery line pressure. And then there are some lamps for indication of if it's alarming more, or if the device is in trip mode. So let me just operate this device real quick to show you what it is doing.

So there is definitely a complex set of requirements over here. So one of them is like, this is a normal mode operation where the bolus is getting delivered in every two seconds, and the pump goes off for every two seconds. A normal mode.

But let's re-simulate it again and try to increase the pressure and see what this does. So when the pressure became greater than 80, the alarm turned on and the delivery is only for one second now. Let's say the pressure comes back to normal. Then you have to press the reset alarm. Let's re-simulate it-- actually, you know what, I'll just simulate it for infinite time, so that we can see the response.

So right now again in the normal mode. Once it comes to the alarming mode. Whenever it becomes greater than 80. Although the set point is 70, but due to the dynamics, it is actually going above 80. So it will operate only for one second, and let's say what we do is-- I'll just minimize this over here to make it more clearer.

Now if the pressure for whatever reason comes back to normal, you have to press the reset alarm button in order to reset the alarm. And now the device goes in the normal delivery mode again. And let's say for whatever reason, if the pressure becomes greater than 100, which is a total unsafe region, then the device would trip, shut down. And then somebody needs to check the device to actually forcefully reset it.

So some requirements basically put together. And then this is how the device is operating. So this is the model, which we have. But starting from the requirements, before even coming up with the model-- like IEC 62304 requires us to come up with the requirements of the detailed design, and then come up with an architectural diagram. So for that, we will jump to this app.

And like I said before, MATLAB has this comprehensive set of tools. And the first tool I want to open over here is Simulink requirements. So you see there are three options for the requirements over here in the section of Model Verification, Validation, and Test. So what I'll open here is, I'll open the Requirements Editor. And then in parallel, I'll open that. Also I'll open this Requirements Manager tool. So let's quickly open both of them.

And what I have over here for you is, in the Requirements Editor, you can actually import. Let's say if you have a requirement put together in a Word document, Excel sheet or if you have a different tool which you are working with, then you can bring the requirements from that tool directly in Simulink in the ReqIF format. Or if you're starting from scratch, like what I did, so you can create a new requirement set, and then you can add your subsequent requirements, all the detailed design, inside it.

So for instance, a normal running mode, I have these requirements where if the delivery line pressure is less than 80, then the motor should turn on in every two seconds and turn off in every two seconds of time. Similarly for the HMI requirements, I have-- if the alarm is in alarming mode, then turn it to red, otherwise turn it to green in the normal mode, and so forth. So you can come up with the requirements, add child requirements to it. And then last but not the least, once you have these requirements put together, You can actually open the same set of requirements in your Requirement Manager tool, which is over here.

And you have this nice little bar of showing the status, if the requirement is implemented or verified. The verified part, I'll come to it a little bit later. But for the implementation part, all you need to do is just pretty much drag and-- let's say my supervisory logic has this unit which is a state chart, which is doing my normal running mode. I just have to literally drag and drop the requirement. And now we have established a link to this requirement.

And I would like to emphasize that this link is not only to this model, but all the way from this model once we generate the code, like the C code, C++ code, even the FPGA code like a Verilog or VHDL code. The requirement would be tied to even each and every single line of the generated code. And all of this tying back, once you're implementing the tests, the requirements would also be tied to your test cases. And you can put together all those in artifacts so that you can put in your 510(k) filing.

All right, so we saw how this implementation was done. And ideally, we like all our requirements to be implemented. And this bar should be entirely blue, and then all the requirements should also be verified. So that being said, I do want to also introduce to you another tool, which is the Architectural Management tool.

So right now, I'm linking the requirements directly to the Simulink model. But I do want to emphasize here that we do have another tool, which is called System Composer. And even before coming up with the detailed design of this software implementation, you can actually start off with putting together these components or the UML diagrams of your design. And then you can link them together and then come up with the entire architectural design of what you're doing.

Like in this case, maybe this is my delivery line. This is a component. One of the components is my logic, let's say. And similarly, you can start detailing it in more and more in detail. You can go in the subunit and see what are the implementations? Or one of the good things is that, whatever you've done in Simulink, you can actually link it to the Simulink-- you can link this component to a Simulink behavior, and then have a Simulink implementation embedded and tied back to the architectural design.

And this architectural design is also linked with requirements. You can have the same requirements which I had over there, tied back to this architectural model. And then from the architectural model, actually start building the detailed design in your Simulink environment.

So that would be the big workflow. But in the interest of time, I'm not going in the details of this System Composer. Just want to put it out there that the architecture can be linked with requirements to this Composer tool. So just wanted to throw it out there that in the interest of time, I would not be able to cover it in complete. But you know, before even coming to this model, you link your requirements to the UML diagram, your architectural design. And then through that architectural design, you start coming up with this Simulink design.

All right so let's go back to the slide 1 again. So this was the first tool which we covered, which is the requirements. One last thing I did want to mention is-- let's say during the development phase, you do change certain aspects of your design. Then whatever the links we've established were the live links. And if there is any change detected, in the links overview you would see those changes. So let me actually show that to you, I guess.

So let's say we have this Requirements Editor. You can see all the links. And you will see that these components are all linked together. But whenever there's an issue in the link, you would actually see that, OK, this has got some issue.

And you'll see that in this case, there was an unresolved source item, or I linked it with something which does not exist anymore. Or these links are for instance-- you know, it's not able to find where these links live. And if I change any link, you can also track them.

One last but not the least-- another thing I should mention here, is that you can also generate reports right from the requirements, once you've linked to your different components. You can generate a nice report which could be used for your filing, or it could be used for like artifacts, just for archiving. And then this would give you the detailed design. This report would contain the live links to your model.

Or you can also export out the model in an HTML format, and have live links living over there. But you can see that it's got this nice particular layout with all the links. And whatever is the status with your verified, implemented, you would see the details over here in this full report. Awesome.

So now that being said, let's go to the second piece-- the verification piece. So to verify, right now what we saw was the Simulink requirements and then a Simulink System Composer. But for the verification piece, let's go back to the slides.

We have two different aspects. So the first aspect I would like to emphasize over here is, you can have some static analysis done, where-- because the end goal is to generate the code. So you can have your IEC 62304 checks implemented in your model. Check for readiness, for code generation.

Make sure that it is 62304 compliant design. There are no things like division by zero, or there's no situation in the model which can cause an issue down in the downstream. Like one of the things which comes up a lot, is data type mismatches where you have one data type like a double, and then the other one is single. In the Simulink it's working fine, but on your hardware itself, I think that could create issues. So it goes through all those checks to ensure that it is compliant with what we are trying to do, and it is also compliant with the hardware in which we are trying to put the code.

So that is what we call a static analysis and it could be done using what we call as a Simulink Check, so let me show that to you first. So you have this tool again, going back to this app. You open this app, which is called Model Advisor. And in this Model Advisor, let's say in this case, I am more concerned with my supervisory logic, for instance.

So let's say we want to have the supervisory logic-- is what I will generate the code for in this design. So we can start the Model Advisor. And in this Model Advisor, that you can actually select these checks. Like over here, I have instead of doing by task, what I'll do is I'll directly jump to the 62304 modeling standards.

And if you see this dropdown menu, you'll see that there are a lot of different checks for the different elements which are implemented in the design. For instance, these high integrity systems is what I was talking about. So if you-- let's say to see what these checks are, so you can see that. You know, check usage of log and log ten operations. Because we don't want a situation where there's a negative number and then we take the logarithmic of that.

Or see like another one-- another good one is actually requirements check, where we want to check that all the model elements are linked to the requirements. And it should highlight which model elements are not linked to the requirements. So you can select these checks, run them.

I'll just run one check because it does take a time to run all these checks. But once you run these checks, whatever the results you get-- like in this case, actually, I do not have it all implemented. So it will give a warning or it will give an error. But then you can also, even for these checks, have a nice report spit out.

So let me run another check right now, the one which actually passes. So in this case, like identify absolute block, which have unreachable code or produce overflows. And if you run this check, you can actually generate a nice report for all these checks which you have run. And then it will give you these reports on what checks passed, what failed, and then give you a nice way to go back and forth between the model.

And these reports, actually I'm emphasizing, these reports are super important once it comes like-- if you're working with a big team also, and they're seeing the model for the first time. But they can quickly go through the reports, and the reports have the live link to the different model components. So it allows you to easily navigate between the report, and look back at the model element. And also we'll see down the line that these reports also have live links to even the generated code. So the actual C code which we'll generate from the design would have these live links.

All right, so that was a Simulink check tool which I just showed you right now for implementing those checks. But now moving on, let's see about the more important tool, which is the Simulink Test. So Simulink Test allows you to implement this functional testing. And what did-- we have capability like whatever your big model is, your system is, you can create-- isolate-- a certain segment, like a unit, and create what we call as a test harness of it, which is a total independent model.

Test harness can have a wide variety of custom inputs. Like you can take a baseline input, let's say, if you have from an actual instrument. Or you can create a test sequence, Excel file, or you can create a StateFlow model to give your input complex set of signals. And then for your assessments, we have the different options where you can measure with the baseline, like in actual data, which you might have. Or you can have additional logical and temporal assessments built in.

You also have the capability to simulate like a software in the loop, processor in the loop, and hardware in the loop workflows. And additionally, the Simulink Test scales up very nicely on parallel computing, if you have multiple cores. So let's go ahead and see this Simulink Test capability right now.

So we have this model. We have this implementation status. And what I'll do is, again we'll go to this Our Favorites app section over here and open this app, which is called Simulink Test. And once you open the Simulink Test Manager, what I have done over here is-- I already have created a test harness for my supervisory logic, which is over here, for which I intend to generate code.

And you can open this test harness. I already have it over here, so let me show you what it looks like. So this is my test harness. I isolated that supervisory logic which is oscillating the normal mode and alarming mode.

And what I've given is some test signals over here using a Test Sequence Editor block. Where I say that, hey, for the first five seconds, keep the pressure this, keep the reset alarm signal this. Next stage do this. Next after five seconds do this, and so forth. I simulate what I feel could be good enough conditions to analyze the entire requirements, and analyze the entire running stages or the running modes of my device.

So this model, you can just simulate it. You can log your signals. You know, all you need to do is log it. You can simulate it for infinite time or you can simulate it for your-- so you can simulate it for a desired amount of time.

You'll see that for my particular simulation, I have these signals coming in, whatever I've logged. Like, the latest one being the trip signal. And you can see that in this case, the trip signal goes high, from minus 1 to 1, at somewhere around the 25 second mark.

So all right. So this test harness is linked back to your original model. I mean if you want to make changes-- let's say you found out that there were errors and you fixed it-- then you can push the changes to the original model. But before doing that, I do want to show you how we model the test cases.

So in this case, just like Simulink Editor, you have this capability where you can have this test manager-- let me open the test cases. I have already created test cases in the past for this one. So it allows you to alter different, different test cases. And these test cases are linked back to the requirements that you are verifying.

So let me show you this. Like, let's say I have this first test case altered, where I say that my reset alarm-- and it's an operational test. So this is my harness. It's linked to the current harness, which is open.

But over here is where I have linked it to the requirement. And if you go back to the Requirement Manager, this is how we get this yellow bar over here. Like, if you would really like to see the links, let me show it to you this. So if I go back to my requirements, I have this mode where it says that transit alarming to normal mode when reset alarm button is pressed, and the delivery line pressure is less than 80.

So you can see that I've got two links over here. One is the implementation link of this logic, and then I also have this verification link where it actually brings me to my test case on how I'm testing the requirement. And right now if you see, all this is yellow because I haven't really verified these-- I've implemented it. I've got the test case, but it doesn't really know if the test is passed or failed.

So let me go ahead and run all these test cases at once. And see the status changing. So what you'll see over here-- I've got these tests for alarming mode, motor working test, normal operation mode, and then a few safety tests like check the delivery line pressure, if it is actually tripping or not, what's the maximum RPM of the motor, what's the maximum voltage of the device, and whatnot. So like I said before, you can even run this in parallel, but right now I've just run it in serial mode.

So I'll just go ahead and actually run the entire test suite right now. So you can see that tests are running. A lot of the tests have passed, some are failed, and some are still in the running mode. But if I go back to this design now, so you'll see that this verified status as the tests are passing, the ones which are linked to the requirement, they're turning green. And the ones which are not run are still yellow. And the test which will actually fail would actually turn into red.

So just give it a few more seconds to run real quick. And I think we have the results. So you see that five test cases passed and four failed in this case, which is all right. Not a problem. And you can see that this green and red kind of showing up like that.

So let's go ahead and see how this tool helps us to actually debug what might be happening. So I'll just pick up one of these test cases just to show you what it looks like. So first, let's see our test case which passed.

So in this case, the reset alarm light passed. And let's go ahead and see what this test is-- is what are the options we have. So over here when I designed this test, we added the requirements, we had the system on the test. We have a simulation mode, which I'll come to later. Right now it's in a normal mode.

But like I said before, you can also run these tests on the generated code, in software in the loop mode, or actually have the generated code on the hardware and run these tests on the hardware. So this is where the PIL mode comes in. But having said that, you can add a baseline criteria for measuring your output, and then comparing it with the actual output and seeing what's the tolerance error, and stuff of that sort.

Or what I have over here in this case, because it's a logic test-- So I have this logical temporal and temporal assessment created. And the way to alter this is actually very neat in Simulink. What you need to do is, you can almost read it like a normal English language. So in this case, I have these signals like pressure, alarm, and reset alarm. And I say that at any point of time, if pressure is less than or equal to 80, and reset alarm is pushed on, then without any delay the alarm must reset.

So you can add multiple assessments. Over here, you can have things like customize, trigger response, bound checks. Or let's show you for the sake of it-- let's say at any point of time, what is always less than-- let's say the pressure is always less than an upper bound and 80.

Then once you do that, so the assessment here is like, at any point of time I have this upper bound of the signal. Like, my pressure should be less than 80, you know. So you can kind of add your own assessments in this fashion, but this is actually not true for this medical device.

But once you have that all scripted in the results section, what you'll see is, if you go to this reset alarm-- we have all these signals logged out. You can see that my alarm-- and it is plotted against-- let's see. So I'll just-- see my alarm.

I'll uncheck my trip signal. I'll just plot my alarm signal. And then with the alarm signal, I do want to see like, when do my-- reset alarm pressure was-- what's happening over here. So alarm goes on and then it goes off and then goes on again.

And I can see the logical assessment over here, what happened. So there are instances at 20 seconds, at 0 seconds, where at any assessment, it became less than 80 and the alarm button was pressed. And then the signal-- alarm became 0. So this is like a passed assessment right over here.

Similarly, for the safety one, let's take a look at this one a little bit of detail. So in the trip deliver pressure test, where it should have tripped, the logic was-- let's take a look at the logic again. So this is logic was, that at any point of time, if the pressure becomes greater than 100, then without any delay the trip signal should be on, and the output pressure, the set point pressure, should become equal to 0.

And what we see over here is that it actually does fail at this 25 second mark somewhere. And the expected behavior was that whenever it became true, the response should have become true. But whenever this became true, actually it stayed false. And apparently this happens at 24.99999783 seconds, when this condition was supposed to be true and it is not true.

So let's take a look back at the test harness. Now we know that there's some problem over here. So for debugging purposes, we can go back to our test harness. And what I've done here is, I've locked these signals, this pressure signal and the trip signal. I'll open this Data Inspector.

And in my debugging mode, what I'll say is like, I let this model stop the simulation at 24.9 seconds. Let's go ahead and simulate it real quick, and then see the plots on the Data Inspector. So what I'll do is, I'll just put them side by side so that we can have a good comparison. Let's plot these signals out, our trip, and then my pressure signal which is coming out of here.

So this is my pressure and that is my logic. You can see that my trip signal is down over here, it's minus 1. And this is how the pressure is, and you see that the pressure has never gone above 100 until now. But in this debugging, let's go ahead and see the state logic now. And we'll see what is happening around 24.9 seconds.

So just-- now we can step the model one step at a time. So even now the pressure is still below 100, and the pressure is like 55. And as we are at 24.990, as we reach 25 right there, exact 25 second.

What happened was, the signal went above 100. This is like 100 point-- just a little bit above 100. You can zoom in to actually see this, 100.1 maybe. And the trip signal is still false.

And the reason is, when it became greater than 100, it went through this logic where it-- I checked if the pressure was greater than 80. It came to the alarming mode first. And then in the next time step after the alarming mode, that's where it checked this greater than 100.

So obviously, this is a design flaw. There is a delay because my logic from the normal running mode never goes directly to the trip mode, but it goes through the alarming mode. And that's why there's one time step delay before actually the trip initiates over here. So Test Manager, like I said before, it gives you these insights. And now to rectify it, I can do something like, I can create a-- oops.

So I'll have to stop this simulation first. And then I can make an edit where I directly have this link over here, where I say that if my pressure is ever greater than 100, then it should directly go to off. And now let's go back. We can save this over here. You know, let's do a Save All.

So now let's go to the Test Manager. We can try to rerun this test and see what the results are. So trip delivery pressure test-- hopefully the test passes this time. And yes, it does. So now if we see the temporal assessment-- so at instance when it became greater than 100, the situation was true.

And we can further insure it through our plots over here. So if we see this plot-- let's take a look at this plot. So the moment the pressure went above 100, exactly at 25, now the signal became true, or the trip was initiated. All right.

So that being said, now for all the tests which we ran in the artifact folder-- like, let's go back to our first test where we ran all the tests. You can actually, even with test, you can create reports. When you ran all the tests, you can create like, what are the tests which passed, which failed, and then more artifacts in a document or a PDF format. Let's go ahead and quickly create it.

And now we see that it's creating the report. Let's take a look at it. So the report which was generated looks something like this. And you'll see that you have the options to customize it.

But in this case, I'm logging all the signals, I'm putting all the plots. And all my test cases are again linked back to the requirements. And I have the status, if it has passed or it's failed.

And then if it failed, then we also have the details about the logical assessment, what really happened and what was expected and so forth. It's a nice looking report for us, another artifact which could be used for a filing. And it can be customized.

Now that being said, the last piece I do want to emphasize over here before I even end this, is the coverage results. So we do have yet another tool in Simulink which is called Simulink Coverage. And this is to ensure that whatever we are implementing in our logic is 100% covered and tested.

So the different types of coverage which we support are, the MCDC design decision, condition, and whatnot. So let's take a look at what this looks like. By default for all my test suites, I have enabled my coverage.

And let's say for my supervisory logic, you can see that my decision coverage was 92%, condition 83%, MCDC 67%, and relation boundary 25%, and so forth. So if you want to see the detailed results of the coverage, you can launch the report. Let me just rerun it again.

So I've run the simulation again. And for the entire test suite, like in this case, we have five passed test cases and a few failed test cases. That's all right. But you can see that this is a different coverage. And we can get a detailed report, again, for the coverage. And in this case, the supervisory logic for a verification status.

Like we do not have 100% coverage as we can see. You can see that the different areas where-- what where those different states where we tested our model against, and what were the states which were not even reachable or we did not even get there. So this is still-- we're talking at the model level. But everything I'm going to show you right now it could also be implemented at the code level or you can have the same coverage report implemented for the generated C code of your model.

So for instance, as an example, the state MotorOn, it was not hit, like none of the times when I was testing the state alarming of this state. And the state MotorOn was never even-- in my test cases, I was not able to cover that. So this is the coverage report over here.

We can go get more detailed coverage report. These reports are again tied back to the model, and then all the way back to the requirements. You have this additional Coverage tab here, where you can create custom settings of your coverage. And in this case, because I'm working with the test harness-- so I'll reopen the test harness again. I think I closed it.

So even in this test harness, we have this Coverage Analyzer. And this is a part of our tool, which is called Simulink Coverage. And you can have certain customized settings for your coverage, what to cover, or what not to kind of analyze, and then have it enabled. So enable the Coverage Analysis, and you can see the certain metrics and whatnot.

So that being said, now I do want to bring out another tool-- yet another tool is, which makes this whole Simulink thing very, very powerful is, how do we ensure that we get 100% coverage? Now we're here-- like, you saw that I was manually creating all the test cases and to the best of my knowledge, I did come up with signals which would include and cover entire requirements. But as you can see, I'm not getting 100% coverage right now.

So the other tool which I want to show you today is Simulink Design Verifier. And let's go back to the main model. And what this Design Verifier does is-- again going back to our Favorite app sections. And in this one, you have this Design Verifier.

And you have the different options-- like it has three different things which it does. One is the Design Detection Error, if there are some errors in the design. One is Property Proofing and then the third one is Test Generation.

So what you can do using this Design Verifier is, you can auto generate test cases for your model or for your component, especially this one, which is going to go in the code. And what I can do is, I can select what my settings are for my code, for my coverage. And in this case, my desired outcome is that I would like to have 100% coverage over here.

So in the Test Generation setting, you can set your objective. Like let's say for my decision, I want 100% coverage. Or you can say that for my condition and decision, I want 100% coverage. And I'll go ahead and generate test. It does take a while to generate, so just give it a minute or two.

So it kind of goes in, analyzes the model, and then auto generates those test cases which could be opened in the Simulink Test Harness and the Simulink Test Manager. So in this case, I think it came up with like 39 different objectives to have the 100% condition decision coverage obtained. And once this coverage is-- this Design Verifier, the auto test generation is completed, we can select one of these options.

In this case, I'll just export it to Test Manager over here. So you know, let's create a new harness for this logic. That's totally fine. But what it does is, with the new harness, which it created-- let me show you the harness as well. It created this new harness of our model which is fine. It's another harness.

But over here, like this was our original test suite which we had created. And it came up with this new test suite with these new test cases linked with this new test harness, which is fine. And let's go ahead and run this and see the coverage results on these test cases. So it came up with a baseline test with four different-- so what it's doing behind the scenes, is automatically creating new signals for me.

And if I were to see the coverage results, now you see that my decision is 100%. And my condition even now is still 83%, which is fine. I can set another objective and get this condition also to 100% if I want.

But you see that my decision is now 100%-- the object is achieved. So a very, very, very powerful tool. Because auto generating those test cases for achieving 100% coverage is something which is really important for a medical device, to ensure the regulators that our code auto generated or the model has been thoroughly tested for each and every use case, and there is no dead logic anywhere which is not being used and whatnot.

So this is the Design Verifier. Let me just pull this up again. And you can see that this is the new report which I got over here, which has 100% decision. And that was my objective which I had selected.

So that being said, now after the Design Verifier, I do want to bring this about-- although the next step after doing this coverage is getting the code generation in place. And for generating the code, we do have this tool called Embedded Coder, which allows you to generate a customized code for your respective hardware. So the way to do that is, you can go in the apps again and you can go to this Embedded Coder tab, and it would guide you through the particular settings, what you need for your particular device at hand, model at hand, and then create the customized code with the customized libraries.

Like for instance, I'll give an example, like for an Arm Cortex processor, you can actually generate the code which uses CMSIS libraries for your implementation and whatnot. But one thing which I do want to emphasize-- and in the interest of time, I won't be able to show it to you today. But do let us know if you're interested in seeing more.

What we have is that you can create something what we call as a software in the loop and the hardware in the loop implementation of your design. And what that entails is-- let me just show it to you real quick. So in your core generation settings, you can actually in your Verification tab, you can specify that you want to do a SIL or a PIL testing. PIL is like processing in the loop.

If you're working with the hardware which is actually connected to Simulink, you can generate the code and have it deployed on the hardware, and then run it in the loop with your test cases. Or you can do a software in the loop implementation. And once you do that, you can quickly generate the code for your system.

Like in this case, maybe let's say I just want to generate quickly-- I'll do it for you-- generate the code for my supervisory logic over here. And what it does is, it will convert all the logic to the C code. And everything which I was doing, the test harness, the test cases which were running on the model, would run now on the generated C code.

And also my test harness, what I've created over here is-- it creates a block, Simulink block, which is like the embedded C code inside it. And that block, I can use against my test harness and then rerun all the test cases which I had for my original test harness, but replace it with this SIL block, which is actually a C code implementation of my design. And use it for my test harness and then get my detailed coverage reports, not on the model this time, but then on the actual C code.

So if you see on the right, we have this entire C code implementation of that block. And once we start doing those coverage analysis, now we're doing it on the C code. And even for your design verification, or the Design Verifier where I auto generated test cases, now you'll auto generate the test cases to have 100% coverage on this generated C code. So you can take it a step beyond.

All right, so that was SIL and the PIL. I have the slide over here just again to recap. For the PIL, is where your actual block is running the algorithm on the hardware itself. And then you're doing all the same verification and validation all together.

That being said, there's another tool-- and this is the last one which I want to cover today-- is the IEC 62304 Certification Kit. So we actually do have the Certification Kit which has these reports, the reports for all the different tools which we saw today, like the Design Verifier, Embedded Code, or Simulink Requirements, Simulink Test. It is at your disposal.

And then additionally, if you go to MATLAB cases which you can run to validate the tools itself, it opens this nice little browser. And over here, we do support the different IEC certifications. But what we are interested is in the 62304.

And for any of these tools which you might be using, let's say you use Simulink Check, you have these test cases over here where we have created all those files to run those test cases. And then there's instructions over here on how to run those test cases and publish them as a report. So we have provided the test cases, and then you can execute the test by clicking this command over here. And it would create those reports, which you can include in your filing.

All right, so that being said now, let's see the last piece over here, is the scaling. Now everything what we did is the testing, the static analysis, equivalence testing, checking, we can also automate this by connecting it with the continuous integration servers, and scale it up so that now it's not one person or one developer doing all of this. But then entire team could be working together and doing a continuous testing, and then improvement in the design, before you actually come to your final, final production code which gets deployed, and bring their device to the market.

And this continuous integration servers, like one which we support out of the box is Jenkins. But then if you're working with a different environment, do let us know. We can help.

All right, so that was all I had today. One hour is not sufficient to showcase the entire V&V capabilities, but I hope you got enough idea to help you understand the value of the proposition of our tools, and how this comprehensive framework can side by side, while you're developing the design, comply to IEC 62304 guidelines for your SaMD device development.

Test, validate, verify so that your production code is pretty high in quality, and then you don't have to go through towards things like software recall, device recall, or have any of the bugs propagated down streams. So MATLAB and Simulink is this platform which allows you to build those high safety standard medical devices end to end.

Tools which I covered today-- I've listed them over here so it would be available to you for your reference for future. And if you're interested in talking to us more about a particular tool, or the set of tools which we saw today, then do reach out to us.

We do have a very comprehensive support model to help you guys out. We have technical support, in case you're using the tool and stuck with a technical question. We do have what we call as our Application Engineering and Pilot Engineering Group which helps you understand-- where we bring our engineers on board and then they look at what your requirements are, and help you adapt the solution and make sure that you're getting the right tools for the right kind of problems which you're trying to solve in your day to day work life.

And that being said, we also offer training and consulting services. We have a comprehensive training on all of these tools to ensure that you are able to get the full potential out of these verification and validation capabilities which we offer. And consulting is where one of our engineers can tie up work together with you guys to actually accomplish a project or implement a certain task at hand.

So do reach out to us. Do reach out to your local representatives for any additional questions or follow-ups which you might be interested in. And that's all I had today.

So again, I'm Akhilesh Mishra. Feel free to add me on LinkedIn. My username is akhilesh-mishra-mathworks at LinkedIn, or you can also send me an email for your questions directly. And I have the email on the screen.

So at this time, we will take the remaining questions for the remainder of the time which we have. But thank you, everybody, for taking out the time today and attending this session on the verification validation workflows for medical device development. Really appreciate your presence today. Thank you.

View more related videos