Performing Hardware-Based Verification and Testing on FPGAs with MATLAB and Simulink - MATLAB & Simulink
Video Player is loading.
Current Time 0:00
Duration 28:44
Loaded: 0.58%
Stream Type LIVE
Remaining Time 28:44
 
1x
  • descriptions off, selected
  • en (Main), selected
    Video length is 28:44

    Performing Hardware-Based Verification and Testing on FPGAs with MATLAB and Simulink

    Overview

    Learn how you can save time by using MATLAB and Simulink hardware-based verification and testing of algorithms on FPGA development boards. In this webinar, MathWorks engineers will show you how you can reduce the amount of time needed to set up dedicated hardware testing on FPGA boards by using hardware-specific capabilities in MATLAB and Simulink.  

    In a series of demonstrations, you will see how you can: 

    • Perform hardware-based verification by using MATLAB and Simulink test benches running on a desktop computer to test a design-under-test (DUT) programmed into an FPGA development board
    • Insert probes into HDL implementations and retrieve signals for analysis and visualization in MATLAB
    • Control and test IP cores running on Xilinx Zynq SoC development boards without having to write throwaway embedded code for Arm processors to test new IP cores

    Please allow approximately 45 minutes to attend the presentation and Q&A session. We will be recording this webinar, so if you can't make it for the live broadcast, register and we will send you a link to watch it on-demand.

    About the Presenter

    Eric Cigan is a principal product marketing manager at MathWorks for ASIC and FPGA verification. Prior to joining MathWorks, he held technical marketing roles at Mentor Graphics, MathStar, and AccelChip. Eric earned BS and MS degrees in mechanical engineering from the Massachusetts Institute of Technology.

    In his spare time, Eric curates a wide-ranging, ever-growing collection of FPGA development boards from manufacturers around the world.

    Recorded: 14 Sep 2021

    Hello, my name is Eric Cigan. Today I'll be presenting to you about some of the methods we support in the verification and testing of algorithms running on FPGA boards. This is an area of ongoing investment for MathWorks, and you can expect to see more innovations in upcoming releases.

    What you'll see today, though, through several demonstrations, is a variety of ways that you can work with FPGA hardware in the process of developing, verifying, and testing algorithms created in MATLAB and Simulink.

    First, a little bit on my background. I joined MathWorks in 2007. I work here in the product marketing team for our HDL products.

    Prior to joining MathWorks, I was really involved in the electronic design automation and FPGA industries in product marketing, technical marketing, industry marketing, and business development roles.

    I have experience here in the last 20 years in high-level design tools for FPGAs and ASICs, although my background coming out of grad school was really more in the realm of controls for space vehicles. One of the earliest projects I worked on was control systems for what's now the International Space Station.

    So to nowadays, what I do is technical marketing really in two areas. One is SoC hardware/software codesign and also with FPGA, ASIC and SoC verification, which is more relevant to today's presentation.

    Just a quick outline here. We're already into the introduction. So we're just going into drivers for verification. And we'll spend the bulk of the time here covering methods for hardware-based verification and testing.

    After that, we'll do a little bit of a summary on supported FPGA and SoC hardware, and we'll get into some of the next steps you can do to learn more beyond this presentation.

    Now, one thing I'd like to refer to here in the next couple of slides is this study that's been done over the last decade plus by Wilson Research Group that were commissioned by Mentor Graphics-- which of course now is Siemens EDA. And they really got into various aspects of functional verification in adjacent areas.

    Here we're seeing a couple of graphs that give you part of the picture. A topic that's covered over and over again in these surveys has been, what sort of flaws in FPGA designs have contributed to issues when end products go into production.

    Since 2012, their survey's consistently shown that the number one type of error was logic or functional flaws, which impact half of all FPGA design projects.

    Now when they drilled into this-- what sort of logical or functional flaws-- they found there were basic design errors in three out of four of these design projects. And then in over half the design, the root cause was either changes in specification, or specifications that were incorrect or incomplete from the start.

    This is happening despite the fact that FPGA designers themselves are spending half their time performing verification tasks.

    Now, one of the other big takeaways here is that 2/3 of FPGA design projects are behind schedule, with a substantial number being at least 50% behind schedule. And along with projects being completed behind schedule, we still see that 83% of FPGA projects have substantial bugs when they go into production. According to the 2020 report, 12% of projects have six or more such bugs.

    Given these concerning statistics and trends, how can you make sure that the algorithms you target to FPGAs don't get released into the world with bugs?

    Well, let's consider some of the techniques supported by MathWorks to help you in this area. In this workflow, I start off developing algorithms in MATLAB or Simulink.

    I refine those models to the point that I have a well-developed test bench and a hardware-friendly model of the design under test, or DUT, in MATLAB or Simulink. That hardware-friendly model becomes my golden reference from here on out.

    Next, I need to come up with the RTL to implement the DUT, either by hand coding the Verilog or VHDL or by generating automatically using HDL Coder.

    From there on, I'd use tools from Xilinx, Intel, and Microsemi to synthesize the RTL for my target hardware, then run place and route, and ultimately generate a bitstream to program my board.

    In this sort of top-down workflow, I have several opportunities to verify and test the DUT. First, I can verify my RTL against my reference model. If that's successful, the next opportunity is to program the FPGA development board and then test how the implementation compares with the reference model.

    The methods we support for these are HDL cosimulation and FPGA-in-the-Loop testing.

    From there on, I may want to do more open-ended testing in hardware-- testing the implementation at speed with live inputs. And for that, we have two more features-- FPGA Data Capture to bring results into your MATLAB session, and another feature that you can use to control the DUT from your MATLAB session.

    The starting point, then, is with HDL cosimulation. Here we are reusing the test bench and conducting a back-to-back test of the reference model in MATLAB or Simulink with the RTL code in HDL simulators. We support ModelSim and Questa simulators from Siemens, and the Xcelium simulator from Cadence.

    FPGA vendor-locked versions of-- FPGA vendor-locked versions of ModelSim aren't supported.

    You can use this method whether you hand code your RTL or generate it from Simulink using HDL Coder.

    You can run the HDL simulator in the background or you can use its full user interface, including its debugging features. You can also evaluate HDL code coverage with the HDL simulator.

    Here's the example we'll use for the first couple demos. It's a current controller for a motor velocity control system, and it uses field-oriented control.

    Let's start with the system test bench. We have a PMSM motor with drive electronics that constitute the plant. Over here, we've got the current controller. This model is set up for HDL code generation using HDL Coder. It will become our design under test, or DUT, when we go into verification through HDL cosimulation and FPGA-in-the-Loop testing.

    You can see the internals of the field-oriented control algorithm with the Clarke and Park transforms that convert the three phase currents into two independent variables we refer to as D and Q. The D and Q channels each have their own PI controller. So if you look, you can see the proportional gain in this part, on this block, and then the integral gain down here.

    The system inputs will put the motor through a series of step changes in its rotational velocity.

    This just takes a brief amount of time to simulate, and then we can see the command current and delivered current here, and then the phase voltage is in the bottom graph. Again, this becomes our test bench and our golden reference as we proceed downstream.

    The next step is to get our RTL code for synthesizing the controller. We launch HDL Coder to generate the RTL, and we're going to be targeting this controller to a Microsemi PolarFire development board.

    When we do that, we specify that we want to generate a cosimulation model for use with ModelSim or Questa from Siemens EDA. We can also check off that we want to look at HDL code coverage within Questa. The last thing we need to do is to specify that we want to generate a test bench.

    Now we see that our HDL is generated, and we're set up to do a back-to-back test. Our HDL Verifier product has generated a model that has a reference model of the DUT we looked at earlier on top, and then on the bottom here, we can see a cosimulation block that will connect to a session with Questa's simulator that will simulate the generated RTL.

    At this point, ready to run a cosimulation. So we click on the play button to run with the same stimulus as we had before.

    We can watch the phase voltages on the scope. And you can see the outputs from the reference model on top, and the outputs from Questa in the middle.

    In this back-to-back test, the bottom chart shows the differential between the reference model and the RTL running in Questa, so we can see the difference is effectively 0.

    Remember that we could run with the HDL simulator in the background, or we could run with it interactively with access to the simulator's GUI. We can use all the views within the simulator and even set up break points in the source views.

    Since we set up this run for code coverage, we can also get reports on code coverage for this particular set of stimulus to the model. This will give you a good first check on the coverage of your test bench.

    Now, if test bench coverage is of interest to you, MathWorks offers model verification and validation tools that would help you build up test benches to strive for and attain 100% coverage. We can't cover all that material here, though, but we'd be happy to talk to you more about that as a followup.

    FPGA-in-the-Loop behaves a lot like HDL cosimulation. It lets you reuse your MATLAB or Simulink test bench to perform back-to-back testing with the reference model and the implementation running on the FPGA or SoC development board.

    We offer support for a very large number of COTS boards-- commercial off-the-shelf boards-- with devices from Xilinx, Intel, and Microsemi. We also offer the ability to extend this support to custom boards if those boards can match our specifications for custom board support.

    Now let's take a look at a demo. We'll use that same design as we showed before, but this time we'll be targeting a board-- in this case, again, the Microsemi PolarFire development board.

    Here's our FOC current controller again. We're picking up where we left off with HDL cosimulation, so we bring up this HDL Workflow Advisor tool that comes along with a HDL Coder, and set up the options for FPGA-in-the-Loop.

    And we're going to use Ethernet for connectivity, and we specify the IP address and MAC address for the board as part of this process.

    Then we launch the process to build the FPGA-in-the-Loop block. This runs a series of scripts, in this case with the Microsemi's Libero SoC tool to synthesize the RTL, run place and route, and generate the programming file.

    Obviously all that takes a while, so let's skip ahead.

    The DOS console shows us that Libero has completed its tasks, and HDL Verifier generates a model similar to what we saw with HDL cosimulation. The model, the Simulink reference is on top. We can browse down into that to view the design under test.

    The block on the bottom connects the Simulink session to the PolarFire FPGA board. We load the programming file into the PolarFire board. For this board, we actually need to cycle the power off and on, so we'll do that.

    Now these blocks connect the original test bench by reference, setting up another back-to-back test.

    Then we can start the FPGA-in-the-Loop run. We can see the results come out as they did with HDL cosimulation, with the bottom graph confirming that we have negligible error between the golden reference and the FPGA implementation.

    Now remember that this stimulus just covered half a second of time. Clearly this ran slower, right? Probably 10 times slower than that. So that's one thing to make note of here.

    Also, we didn't spin any motor here. This was just testing the current controller algorithm in hardware. So for a design like this, you're probably going to want to be able to run it at speed controlling a real PMSM or brushless DC motor.

    This brings us to the next feature, FPGA Data Capture. So let's get a brief overview of that.

    With FPGA Data Capture, you can get visibility into internal signals within the FPGA without having to rework the design to make them explicit outputs.

    Once you specify the internal signals, HDL Verifier or generates the IP to capture those signals. You can set up triggers to capture the signals or just snag them immediately upon demand.

    A big advantage of FPGA Data Capture compared to FPGA-in-the-Loop is you can capture signals at the FPGA fundamental clock rate, when you get much higher frequency signals than you could possibly obtain with FPGA-in-the-Loop without a lot of extra work.

    If you're using HDL Coder, FPGA Data Capture is integrated into the HDL Workflow Advisor tool. That makes it pretty easy to set up. So let's look at a demonstration.

    In this demo, we are essentially using a more elaborate version of the field-oriented control motor example. For the model, you'll see that we've added an inertial load to the motor. Makes it somewhat more complicated.

    The controller, though, is substantially different. In this case, we're targeting a Zynq-7000 board that has an ARM Cortex-A9 dual-core processor on it.

    The subsystem here shows how the controller has been partitioned. The block on the left would be targeted to the ARM core. It contains a velocity controller that would run at just 1 kilohertz. That's essentially an outer loop.

    This velocity controller has the logic to start the motor from a dead stop, and then to calibrate its encoder peripheral, finally then to go into closed loop regulation of the motor shaft velocity.

    The block on the right represents the partition for Zynq's programmable logic. The current controller is running at a 25 kilohertz rate. So if we drill down inside the current controller, you'll see that it's fundamentally the same as the model we just ran a moment ago with HDL cosimulation and FPGA-in-the-Loop.

    But now you can see that we've also got some peripherals here-- a model for the A/D converter, that measures phase currents, an encoder peripheral that detects shaft position and computes the shaft velocity, and a pulse-width modulator, or PWM output, drives the phase voltages to the power MOSFETs and ultimately drives the motor drive electronics and the motor itself.

    Now the clock rate of PWMs is 50 megahertz. In other words, it's 2,000 times faster than the current controller's processing rate.

    So in this demo, we want to test the PWM to make sure it's operating correctly. As a result, we've got a PWM test bench here that delivers commands to the PWM signals and displays the outputs.

    We can simulate this so you can see the three output phases of the PWM with different duty cycles-- 25%, 50%, 75%. We would never run them like that, but this at least gives you an indication, visibility into how that would look.

    We're going to want to see how the PWMs operate when we're actively controlling the motor. So we set probes on the PWM outputs, and that'll let us monitor them during operation.

    We can specify the number of samples that will be retrieved into MATLAB. And we set this buffer size to 4,096, 4K. We choose that number because the current controller, again, is running at 25 kilohertz. PWM clock rate is 2,000 times faster at 50 megahertz. So with a buffer of 4K samples, we should be able to capture two complete cycles of the PWMs, and that'll let us check and observe how they're running.

    Then we can generate the HDL code for this PWM peripheral, again using HDL Coder, but with the Data Capture IP included that will save the buffer to memory.

    The PWM peripheral is set up with external ports for its input and output. We can see that. And we're using HDL Coder's IP Core Generation Workflow, and it generates this report for the core so we can confirm all the details.

    Then we'll build a bitstream for inclusion in the reference design for this Zynq FOC design. That means that the PWM outputs would normally be internal signals within that reference design, and we wouldn't normally have visibility on those without FPGA Data Capture.

    As before-- we've skipped ahead of it-- we've already generated the bitstream for the PWM and incorporated it into the reference design. So as a start, we're going to run a test for the same 8-second velocity step profile that we ran in simulation.

    To run this test, here we have a Simulink model that's set up to run in external mode. It has a number of different input setups, but we, to start, have configured it to go through that velocity step test.

    We'll use a video panel here so you can see the motor shafts spinning during testing. Then we can run through the 8-second test. So we can see the motor shaft spinning and then stopping.

    We then run the resulting shaft velocity and phase current from the run. We then display the resulting shaft velocity and phase current from the run. You can see this is pretty close to the simulation of the reference model itself.

    But the whole point here was taking visibility on those PWM signals, right. So to do that, we'll change the stimulus to the controller so that we can interactively control the motor velocity.

    We start the external mode model running, and we set up our screen with the Data Capture tool on the upper right and then the Logic Analyzer screen below it. And we can toggle the switch to start driving the motor.

    We set up the capture to occur when we click on the Capture Data button. As we vary the motor velocity, you'll see the duty cycle of the three phases changing. And we can measure this directly using cursors in the Logic Analyzer tool.

    Let's look at the two traces for PWM C, that's the one on the bottom. We place cursors to measure two full PWM cycles, then place others to show the on/off transitions. Using this method, we can measure that the on time is 1,330 clocks, and the off time is around 670. That ends up giving us a duty cycle of 66.5%.

    Now, what that does here is that saves us doing things like taking an oscilloscope and probing the actual board itself. So this is really a more scalable method and more precise method as well.

    I put together this slide here to summarize the comparison between FPGA-and-the-Loop and FPGA Data Capture.

    As I mentioned going into that demo, the execution speed is a big difference. With FPGA-in-the-Loop, you're generally running at rates below 100 megahertz, and the attainable speed will depend on the type of design, the transport mechanism with JTAG being the slowest, PCIe being the fastest, and Ethernet somewhere in the middle.

    With FPGA Data Capture, you're running the design at speed, not at some fraction of the speed.

    That relates to another difference. With FPGA-in-the-Loop, all the inputs come to the board from MATLAB or Simulink sessions. This is largely a practical matter since the board's running at a fraction of the frequency of the ultimate implementation.

    As we saw in the demo with FPGA Data Capture, you're running at speed and be operating on signals coming in, such as in this case, the closed-loop field area controller with its encoder and voltage signals coming into the model.

    You also saw in the demo that with FPGA-in-the-Loop, data was returned to MATLAB or Simulink just as it was with HDL cosimulation. So you can watch the outputs develop over time using scopes or other syncs.

    With FPGA Data Capture, you're snagging signals based on triggers or direct commands. And you're able to get outputs or input signals, as we saw in the case of the PWM signals of the demo.

    That brings us to the last of the hardware test methods supported with HDL Verifier. Its purpose is to let you control tests for MATLAB, either for direct interactive testing or with scripts.

    This feature is referred to as MATLAB AXI Master. We are in the process of renaming this feature to use more inclusive terminology so you will soon be seeing us refer to this feature as AXI Manager.

    There are two primary ways that you can use this feature. First, you can use it to write to DDR memories on development boards. That can be quite handy if you want to efficiently load large communications, waveforms, or large video frames into the board's memory.

    A second use is to write to or read from AXI-accessible registers, such as the ones you would have for IP cores, for FPGAs, and SoCs.

    We saw that customers using SoCs were having to write C code just for the purposes of testing the IP cores that we generated out of HDL Coder. That C code was essentially throwaway code, though, because it was really only for testing. With this feature, engineers could avoid having to write that throwaway code and just write simple, interactive MATLAB scripts. Much less work.

    This would even be a bigger factor for customers who are developing cores on conventional FPGAs where you don't have a processor core, since they would then have to generate a soft processor like a Xilinx MicroBlaze or Intel NIOS II, and then write their embedded C code to run on those soft processors.

    Let's look at a simple demo. A really simple one that shows you how you could use this feature to test IP cores, in this case for a Xilinx Kintex-7 FPGA.

    I'm using a design that was essentially our "Hello, world" example for Zynq SoCs some years ago. It's a design with an IP core with a counter that blinks the eight LEDs on this KC705 board.

    We can control the speed and direction of the counter. To start or run this in simulation, we can interactively toggle the direction of the counter. And we can also control the speed of the counter and view its value with the scope.

    OK, so our next step, then, is to build this core. So we'll go ahead and we're going to launch the HDL Workflow Advisor again to do this with HDL Coder. So we have to specify the IP Core Generation workflow, and we're targeting the KC705 board.

    But then for the reference design that HDL Coder uses, we specify that we're going to use this MATLAB AXI Master feature.

    Then we go ahead and proceed on, and we can set up the ports so that the inputs are really coming through the AXI Interface that we're going to control for MATLAB, and the LED output pins go the LEDs, and so forth.

    So then we can just go ahead and see, look at the documentation for the IP core that HDL Coder generates. And what we want to scroll down to is look at the register mapping, and we'll especially look at these different registers and their offsets. We're going to use that when we actually run this feature from MATLAB.

    All right, so now we've gone through the whole process, skipped some steps, and we are loading this-- We're loading this programming file into the board, and you see right away that the LEDs start flickering. That's because it wasn't really reset to 0. The counter's already started.

    So we've got that. Now we've gone to our MATLAB session, and we're going to run the AXI Master features. So we define the system object here. And then first we write to the enable a 0. That's going to disable the counter. And we'll see it lock up.

    And then we run our next MATLAB command. We're just going to set that enable back to 1, and we'll see the counter starting up again.

    Great. OK, now another thing we could do is just, instead of doing all this from MATLAB commands, we could connect it through our App Designer to a GUI and control all the features of this very simple counter.

    So you can see here, we've got a toggle switch for this Enable/Disable that we've been doing. We've got another toggle to set the direction up or down of the counter, and a slider bar to control the frequency of the counters. And so we'll go ahead and try running that.

    So we can see first, we'll go ahead and enable the counter. We can change its direction up and down. We can control the frequency, so you can see it go faster and then slower. And finally, we can reset it, and that just brings it to a stop.

    Admittedly, that was an elementary example, right. However, one of our customers delivered a couple IEEE conference papers showing their use of MATLAB AXI Master to control IP cores on development boards.

    In the first paper, the authors reported that using MATLAB AXI Master, they were able to reduce their firmware and embedded software development efforts. In the second paper, they actually showed an application of that, highlighting the role that the feature played in the development and testing of synthetic beamforming applications on a high-end Zynq UltraScale+ RFSoC development board.

    So their work is really kind of a better indicator of the true potential of MATLAB AXI Master in commercial applications.

    Now that we've covered these features of HDL Verifier, we should review support for boards. We currently support well over 50 COTS boards, from Xilinx, Intel, Microsemi, Avnet, and Arrow. Support does vary by feature-- FPGA-in-the-Loop. FPGA Data Capture, and then read-write access to AXI registers and memories.

    It also varies in terms of connectivity through JTAG, Ethernet, and PCIe.

    This support matrix here is included in our documentation. You can view this list at any time by browsing to the HDL Verifier Online Documentation, then browsing to FPGA Verification Requirements, and specifically Supported FPGA Boards for FPGA Verification.

    Just to give you an idea of what this support involves, here's one of our testing labs where we have a few dozen of the supported boards at any given time. This lets our development and quality engineering teams run tests on boards for each of our releases.

    So to summarize, we covered how HDL cosimulation is a good first check when you compare handwritten or automatically generated Verilog or VHDL with a reference model using your original test bench. We showed how FPGA-in-the-Loop provides a next level of verification by testing on actual hardware.

    Then we saw, with FPGA Data Capture, how you could get greater visibility into designs through at-speed testing. And finally we've got to look at HDL Verifier and its ability to access AXI4 registers and DDR memories. And we saw how you could test IP cores on FPGA boards in example usage.

    Now if what you've seen here sounds applicable to your projects, here are some resources you can go to as a next step. You can visit the HDL Verifier product page, which has links to videos, worked examples, product demonstrations, and more.

    You can visit our FPGA, ASIC, and SoC Development solution pages. In particular, the Getting Started page that's showing here gives you access to a wide range of videos and other resources to help you learn more.

    Finally, you can request a trial license or just contact MathWorks Sales if you'd like to learn more. I should note that our training services team also has training materials that they can deliver on some of these topics to help you ramp up on them.

    Thanks very much for attending today.