Introduction to AI with MATLAB

Hi, everyone. Thank you so much for joining us today for our introduction to AI webinar with MATLAB. I'm Heather. And I'm joined by Johanna. Let's do some introductions.

I'm going to start out. I am Heather Gorr. I have a PhD in physics. And through that, I did machine learning and a lot of model assessment and things like that with MATLAB. And I've been here for about eight years. And I'm joined by Johanna.

Hi, I'm Johanna. And my background is in image processing, and more recently AI. I write the AI blog for MATLAB. And I'm here today to show you a great resources and ask Heather questions when she's going through the demos.

Awesome.

All right, so let's get started. First, we want to talk a little bit about you and what we hope you are going to get from this. So first, we're assuming no knowledge of AI prior to this. However, we're hoping that you want to incorporate AI into your applications and into your work. We're hoping that you want some good AI and MATLAB resources.

And like I said, no AI experience is required. So all of the demos that you're going to see are going to be in-product. And you're going to be able to access those whenever you want to get started in AI today.

That's the idea.

That's the idea. OK, so let's start with the question about what is AI? Good question to kind of level set. So this is a broad answer, which is basically any technique that allows a human-- or I'm sorry, a computer, to be intelligent.

But like you see on the screen, that's a very broad image. It's always a brain that we see from this. So let's swap that out with maybe something a little bit more realistic because AI is about building real systems.

So when we think of AI we typically think of a model to start. And engineers and scientists start by building AI models. But we're here today to show you that a lot is involved in creating a model besides just the model itself. You've got the preprocessing--

AI is so much more than that.

Absolutely, absolutely. So you've got the preprocessing, all of the information that you need to understand the data and bring it into the model. And then you also have to test the model after you have it made. So that's basically what we're going to be starting to talk about today.

And then finally deployment as well because you want to make sure that it doesn't just live on your computer, but it actually goes out into the world to create exciting AI results.

You can actually use these things IRL.

Exactly. All right, so here's the agenda today. We're going to start with a demo to get started. Then we're going to move on to AI models and talk a little bit about MATLAB and TensorFlow together. Then we're going to talk about real AI applications. We're going to switch over to technologies such as deep learning and machine learning that are the core components of AI.

And then finally, we're going to say what's next, because this is an introduction to AI, so there's a lot more that you can do after this, and we want to give you a heads up on that too.

So let's start with our most frequently asked question, which is, how do I get started with AI?

Well, you just do it. You just try it. Actually, that's literally what I'm saying. There's so many examples out there for all kinds of different applications. The best place to check to begin with is the MATLAB documentation and GitHub. There are tons of examples that'll go through the entire flow, the process that you need.

Like data prep-- what do you need to think about before you use a model? Also using pre-trained models, which is awesome. There are so many people out there working on this full time and sharing all of their results. Like why start over from scratch?

Right, that's kind of what I think too is the accuracy of a pre-trained model, at least starting out, is going to be much higher than something you do yourself.

Exactly. And then you can tweak, once you test it, make sure it's working for you. You can go through and adjust as you need to. So let's go ahead and go for it.

And again, I've been doing this a long time. The very first place I start is the MATLAB documentation. It's fantastic for this. And you can see very nice-- there's a category for all of this, lots of great examples. Again, just find something that works for you or that kind of sounds similar to what you want to do.

So I can just open up any of those examples and get started?

Yeah. In fact, you don't even need to have MATLAB installed. You don't need anything. You need internet connection in this case. But you could even just open it in MATLAB online. So if you have a MATLAB account or a MathWorks account, it'll open in full MATLAB online with all the bells and whistles. And otherwise, it'll just open right in browser. And you can still use it and just try it.

That's great.

It's a great way to get started. So if you take a look here, there are a number of pre-trained networks. And so there are lots of people again working on all these things. They have lots of images. This one's like LeNet from Google. Google has lots of images. They trained a model with them. So let's use that.

Again many, many from the community-- we'll go ahead and bring that in. And then you always actually need to prepare the data or think about how your data need to be organized for the model. And of course, in this case, we need to make sure about the images are going to be the right size that the model will accept and do the classification.

OK, so the model wants a certain size input. And you have to make sure that your image fits that size.

Exactly, and that information is in the layers. So not a big deal. We can just run it here like you see. And then also we could take a look at the class names. What were these things trained on? What are all the options for the outputs-- so another example of how we can just take a look at what we're working with-- first step that we need to do whenever we're using a pre-trained model.

So then again, we need to do our preprocessing of our data-- a little simpler with images because often you just need to read or resize them or read them, do a little cleaning up. But we'll go ahead and resize them to the same size that we need.

And then we classify it. We don't have to really do anything. Just go ahead and classify. And sure enough, bell pepper.

Well, I mean, I guess it's a bell pepper, right? But there's other things in there, too. So I don't-- how did it decide that?

Good point. So there are other kinds of jalapenos and garlic and things like that. So if you take a look at the classify function and predict functions have a score. And so you can actually get the top labels and the scores.

And so in this case, if you take a look that's about 95% sure. I mean, that's pretty sure. It's pretty sure.

I mean there are peppers in the image. So I would imagine that that's fairly accurate.

But you can also see what the model thinks are the next likely predictions. And so it's really helpful, again, in this kind of case, maybe you want to go back and actually train this to be jalapenos instead of bell, or both.

So again, this great example goes right through everything you need to think about, everything you need to do to apply AI to your work.

And one of the other things I wanted to just point out before we move on-- I mean, we went through this, it's wonderful, but because we made this mysterious network and didn't really talk about it. I wanted to just show quickly what this thing is, or what this thing looks like.

Again, MATLAB users tend to be very smart and mathematical. And they don't like to just hand wave and say yeah, OK, sure, I'll just run your doc example.

Yeah, I've noticed it's all about the details.

Exactly, and again applying those details to your problem-- and that's really where again MATLAB really comes in handy. Because you can use a lot of these apps, you can use the well, I don't know, 40 years of expertise in the community and these topics, and signal processing, and the data processing, all of that.

So one of the nice things-- I think probably one of my favorite things that came out in the last few years is the Deep Network Designer, Experiment Manager, and the machine learning apps, of course, have gotten so much love because, again, we want to try out all these different models.

We have no idea what we-- have some idea what we're doing. But we have no idea what model we want to use in the end. So we really want to take advantage of trying those out.

And so as you can see, there is a nice view where you can look at the overview of the model. Sometimes you want to use a simpler one versus a more complicated one. You can get some information. You can download it and even zoom in and kind of understand what those layers are. I know of course, this is an intro. We don't want to get into too much.

But if you look here, it's just convolution and aggregation. These are kind of sensible things for many of us. So it makes a lot of sense. And again, you can really take advantage of the apps to really understand and tweak those if you need to.

Right, and I think what's interesting too is it repeats itself over and over again. So once you understand the basics of the different layers that you have available, then it repeats itself over and over. And you expect that these networks are going to be complicated because of the really amazing results that they produce too.

Exactly. And we'll talk about some of that later, too, whenever we're talking about different models, what's good for different applications. But even if you take a look here, all the layers, you have all the information here. You can really be enabled to feel more confident about what you're doing using the apps. So we have a lot of people that really enjoy that. You can train. You can do code generation.

So I guess we could step back and see what we learned so far. So far, we talked about, very minimally, but we talked a little bit about data preparation. In this case, we just had to resize the images. But we also wanted to think about those-- is that label right? Should that have been garlic? Is bell pepper correct? If not, we can go back and label again.

Then again, we'll talk more about this when we train a model, but we used a lot of nice research that's been done already, take advantage of that, and do our AI modeling with a pre-trained network.

And then we want to test it based on our results. And there's also a lot of cases where, especially in predictive maintenance I think, where you don't want to break equipment to train a nice model. You want to-- if you need bad data, lot of times you can simulate that.

So we see a lot of cases where people are using Simulink and cogeneration to sort of supplement the data to train a better model. And of course we're not just doing this for academic purposes. We want to actually use these models. We want to put them in a vehicle or an airplane or whatever.

So taking that next step and thinking about how the model is going to behave on that device or in the cloud, we'll talk a little bit about that and point you in the right direction.

Right, so I think the next question that we're going to really zoom in on to the modeling part of this. So you showed GoogLeNet, but what model should I use? What model's best.

Good question. I think we could probably spend hours and hours. And we could look at Twitter and all sorts of communities and see what people think. But I think that's why I wanted to show the app earlier because you can really take advantage of it. Try all of them.

Sometimes you don't know what model is going to be best for your application. So one thing-- we showed the documentation. Of course, there's amazing references, amazing information, examples, professional writers who have backgrounds and these things are making the examples.

But there's also a slew of information on our GitHub. So for example, you can see the model Hub that David Willingham has put together. And it's really exciting because, again, people are doing these for different things. Maybe you want to do computer vision. Or you're doing NLP or text or audio, LiDAR.

So certain things are going to be better, certain models are going to be better for certain applications. And so you can-- it's organized like this, you can see all of the different models that are kind of popular in the community. Some of the metrics, how well they do.

So this is a great place to start, especially after that initial example, and seeing if it works at all. We can kind of fine tune from there.

Right, so it looks like what you're showing, too, is that all of these are kind of grouped by applications. So you really want to get into-- so example, you're an audio engineer, you want to go into the audio examples.

Exactly. And there's some sense of what you're trying to do. You're trying to classify sound, or classify words that someone's saying. So you want to take a look at these, for example.

So go to the model Hub for the most up to date models.

Exactly. And then of course, the documentation is always wonderful. But the latest and greatest is going to be found on GitHub.

Excellent. So speaking of applications, I think now is a good time to take a step back. And we've been showing kind of broad, general examples. But the demos that you're seeing today are supposed to be broad because we want them to be easy to understand. We're trying to cover the basics. And we also don't want to go into an application that may not be relevant for you.

However, there's ways that MATLAB is really being used in the world that is quite exciting, and really showcases the applications that are being used for AI.

I have three customer examples that I find exciting. The first example I'd like to show comes from a company called Drass. They were looking to use AI for ship operators, using real time object detection. They were able to use label data to train a suitable model.

However, one of the biggest challenges was the team had to integrate it into the main object detection algorithm used in ships, which was written in C++. So MATLAB coders were able to automatically generate code and integrate the model successfully, which is now running on their ships.

The next story comes from Shell who wanted to develop an application that predicted reservoir volumes for prospective locations. The geologists wanted to apply their domain expertise to develop an algorithm that can predict new features in a new region.

They ended up developing a regression model as part of the solution to be accurate to within 12%, which across the industry it's not unusual to find predictions off by an order of magnitude, especially in new areas.

And finally, Mitsui Chemicals deployed an AI system that incorporates TensorFlow and MATLAB. The engineers developed an open source model for visual inspection on the production line. But they wanted to use MATLAB to create the applications.

The TensorFlow model was imported into MATLAB. And they created an application using MATLAB Compiler. Multiple engineers are now using the app reducing visual inspection time by up to 80%.

So those are three examples in which customers are using MATLAB for really diverse real world applications, which perhaps can inspire those watching to think of how AI could be implemented in their own work. And for more details, we have these and other customer examples on our website.

All right, so let's move on to our next question, which is, can I use MATLAB with TensorFlow.

Absolutely. So we were talking about pre-trained networks. Why not include TensorFlow, PyTorch, some of the really popular ones that are being implemented in Python?

Let's take a look again back at good old handy GitHub because there are quite a few really nice examples in the MATLAB deep learning area that shows the multiple ways that you can call TensorFlow or PyTorch from MATLAB.

So for example, again we'll share all of these links so you can go through line by line. But essentially it's just like having the same example that we did, but using that TensorFlow code instead of the MATLAB code. It's really great.

And one of the nice things is that you could use it as a Python model. Or you could convert it to a MATLAB model. And a lot of the reason that people want to do that is because they need to deploy.

Just like we were talking about, you want to put this in a car or you want to incorporate your AI into a plane, or do something very clever in your engineering application, you need to deploy that. That's much easier from MATLAB with a couple of clicks.

And then of course, the data processing that we were talking about, there's apps that you'll see even more. You can try all these models. You can try out all the different kinds of data prep, and then ultimately generate the code with a couple of clicks.

Right, so what it sounds to me like is that the model could come from anywhere, really, like TensorFlow, PyTorch, maybe even ONNX or something like that. And then you use MATLAB for the functions and the examples and the apps.

Exactly, so you kind of do everything else. And then of course, if you wanted to develop a little bit more with your TensorFlow model, your PyTorch model, all those latest and greatest, update your layers, of course you can co-execute and just call Python from MATLAB or vice versa. So we also see some people doing that.

But rest assured, whatever model that you've heard of out there in the world, we can totally work with it. And you can use all these great apps, and we can help you out.

That's great. So that was a breeze through TensorFlow. The answer is yes, you can use TensorFlow. However, there's so much more detail that we would love to show you today. However, we don't have the time.

So what we're going to do is we're going to put some resources available for you and it's going to talk about all about MATLAB and open source, working with Python, and also a couple of I think seminars that you were starring in that goes through all of the details of MATLAB and Python together.

Absolutely.

So moving on-- now let's talk about machine learning versus deep learning. And I know some of you out there right now are screaming that machine learning is deep learning. And deep learning is machine learning. So we're going to get into the--

--we'll get back to you--

Absolutely. So we're going to get into the details in just a second. But what I'd like to talk about is machine learning being the definition of the key technology that encompasses everything that we've been talking about today.

So within that--

I think you talked about, AI is kind of that whole thing. Like the AI is like the robot or the system. And then machine learning kind of encompasses that model that does the learning. But there's just so much to it.

I like that. So within machine learning, you've got the classical machine learning algorithms that you may be familiar with, KNN, SVM, all of these other algorithms as well that you have access to in MATLAB.

But then you have the other models that Heather was talking about a little bit earlier. So this is the deep learning networks, things like neural networks, things like transformers. And then on top of that, you also have reinforcement learning. But we're not getting into that today. So we're just going to kind of gray that out for now.

But really, the idea is that idea-- yeah, exactly. So all of these algorithms are available to you. And they all are within machine learning. But when people are talking about machine learning versus deep learning, they typically mean classical machine learning versus deep learning. And so we want to have a quick discussion on that.

So here are some questions that you can ask yourself when you're getting ready to maybe decide between machine learning or deep learning. So first is resources. So let's talk about the fact that you've got machine learning, you've got deep learning. But it's really going to be a spectrum.

Right, and deep learning being deeper, generally there's more data involved, there's more calculations, more going on. So you tend to need more resources.

Right, so I'll ask Heather, do you have a GPU?

I do, that's why my laptop is a little better than yours. Extra fancy--

Absolutely.

I think so. But because I do these things all the time, I need to train models. It's kind of weak because it's a laptop GPU. But I also have one back at the office just to make these things faster.

You don't absolutely need one. I think that's one of the nice things, really. You can also use the cloud. It's super easy to just like rent some from cloud provider for a little while and use whatever GPUs you can. No big deal.

But that will make it faster. But it's not absolutely crucial. And of course with machine learning, I mean, you could do it with pen and pencil like Cleve did in 1976. So anyway--

Right, so the idea is that it's on a spectrum. If you have a GPU, you're more likely to be successful with deep learning. However, it's not-- it's just one of many factors.

Another factor is how large of a data set do you have? If you're training deep learning, you're going to want a larger data set. And if you don't have as much data, maybe machine learning might be right.

And we'll talk about this shortly, but a lot of times whenever you have machine learning, you want to really think about how you're representing the data because you don't need all of it. But with deep learning, you just throw it all in there, for the most part. And that requires a little bit more to train the model.

Right. Now how explainable do you need the model to be? So-- exactly, I definitely put that in quotes. So if you need a very explainable model, you're going to want lean towards machine learning. And if not, then you can lean a little bit more towards deep learning.

F equals ma I feel like is extremely explainable. And even then, sometimes there's questions about whether that should be implemented into a car or an airplane. There are a lot of certification bodies that need to make sure models are working the way that they do for very good reasons.

Right, absolutely. And what you're seeing on the screen too is that there's a spectrum where neural networks may be higher predictive value. However, the interpretability of that model or the explainability of that model may be a little bit less.

Most people are going to know about y equals mx plus b. So if you can get away with that, that's ideal.

Right, absolutely. And then finally, what type of input data do you have? So the next slide here is going to be talking about the different types of data. So if you have tabular data or anything kind of in a spreadsheet, that's one type of data.

And on the other spectrum, you've got the image data. And somewhere in between is kind of the signal data and text data as well. So once again, this is on a spectrum, where if you have the tabular data, you may be more successful with machine learning. However, there are deep learning options available for you as well.

And a lot of times, I think it goes back to what does the model expect when it's doing the calculations? What is the expectation of how that data are organized?

So tabular data are already in the way that machine learning algorithms expect the data to be organized. Otherwise we need to think a little bit about it. And we might want to just kind of pass it in to the deep learning algorithms with minimal kind of processing.

Right. So I think that's the next demo that you're going to go through.

Yes, and so this is the exciting one. I know it seemed so easy earlier. Now things are not going to be easy anymore. But this is a great one because signals are a great example, again, of where you could try all of the models and see what really makes sense for your application.

And of course, this human activity recognition is one of my favorite examples. I absolutely love it because there are so many nice examples in the documentation that actually go through many different steps that will really help you.

So for the most part, the expectation for most algorithms are numeric data or categorical data, and somewhat rectangular, or at least consistent throughout. So the images are a great example because it's just kind of easy. You just need to resize it or maybe do a minimal of preprocessing.

But then when it comes to signals, you have squiggly kind of data. How do you actually represent that? You don't necessarily want to just feed it right in.

So if you think about how we want to prepare it, again, images, we can just do minimal preprocessing, feed it into the deep learning models, clean it up a bit. With time series and text, we generally go through that phase where we try to figure out how to represent it in a way that makes it rectangular, also gives the most sensible information to make that model very smart.

And so again, same, very true for tabular data as well. Sometimes you can just feed it right in. But other times you want to actually think about how you want to represent it.

And so we call that feature extraction. Or you may have heard of that before, feature representation, feature selection. And that's the most common thing for machine learning. Again this is kind of more classical where you may not have as much data, you may not have as many resources. You want to take that data and represent it in as few data points as possible.

So think about the signal. It has all kinds of noise. It has all kinds of stuff going on. But ultimately, we want this thing to say, when you're sitting, the mean of the x signal is this. The standard deviation is this. The mean of the y is this.

And so that's kind of how we want to feed it in. And again, with deep learning, you can pretty much feed in the raw signals. Again, obviously you want to do some of your domain-specific-- if you know that you want to normalize or do some smoothing, you want to do that ahead of time. But you can just kind of feed it in as is. You don't need to make it organize in that nice, rectangular fashion.

So starting with machine learning, for example, we have our signal, which is, kind of that regular kind of signal. And ultimately, we want those measurements to be represented in our variables and then labeled.

How do we do that? Well, there are many people that have spent their lives studying this. We will not. We'll just talk about it for about 30 seconds. But there are some wonderful resources out there, the statisticians and our teams, they've really collected all of this together, give you some guidance about different data and how you might want to represent it, what might make sense based on your application.

And in this case, it's a signal. MATLAB is pretty good at signals. I don't know if they knew, but it's kind of a thing. And so there's a wonderful signal analyzer app. And so a lot of times, you want to represent the signal with the spectrogram or like the spectral frequency information, which most of us live in the time domain, not the frequency domain. So that's uncomfortable.

So we can use an app and visualize, and then ultimately even get those statistics out that we want to use in our model.

So you're saying that all of those features that you were talking about, that app is going to actually figure that out for you.

Yeah, absolutely. And there's even more if you want. There's other apps that you could use that are kind of more, depending on what kind of data you have, the image labeling, processing. But it's a really great place to start because you can really represent it.

And then you can also just generate the code generate the script get the model out, and you're good to go. And now we can perform those calculations, and we have the data organized in the way that we want.

Great.

Let's go back into MATLAB. Actually, let's go to our handy dandy doc and take a look. And if you just type whatever you want, usually it can be found. Human activity recognition, done. I happen to know which examples I wanted. But it's a great place to start just searching.

And then I could use MATLAB Online, but I'm going to go ahead and use my MATLAB but I just installed, a new release, very exciting. And so I can open that example and go through the live script, and just go step by step just like we talked about.

So again, I'll load in the data. And then the next part is actually doing those feature representations that we did in the app. And so we can just replace that with what we did in the app. So very exciting.

You don't need to write all this code. It looks kind of scary. But in reality, you actually just need to hit generate code and call it. Very cool.

So now we actually have the data in a way that makes sense. Like just what we're talking about, the variables with that label. And Johanna, what's my favorite thing in MATLAB?

I would think it's apps.

It's apps, especially the machine learning apps. You can take a look. The app, obviously very, very easy to use. We could try, I think there's about 30-something models. There's--

Can you do them all?

Yeah.

Let's do them all.

Don't dare me, I will. I'll do it. So train all with parallel and my GPU.

Excellent.

Yes, very exciting. So again, just to point out a couple of the nice things in the app-- if you want to look at a couple specific ones, there's actually a nice mouse over where you could look at just a general idea of what that model does for you, just kind of give you a sense of-- again, people don't want to just accept mathematics as is. You want to make sure it's being applied properly, at least MATLAB users, if I know them.

And you do.

And I do. You can see I have four cores. So it's training four at a time, pretty quick. This one looks pretty good. And we can tell by our visualizations. And so, again, this makes it so easy. You don't have to write all kinds of code to test all these things. You can try them all, poke around, see what's up.

So let's pick that model, now what? Now what?

We just go ahead and export it.

Excellent.

So what do you want to do with it? We could take the plots. We could generate a function and generate the model. And then we can just use it as we go. If we wanted to implement it in the cloud, implement it somewhere else-- in fact, even MATLAB tells me what to do with it.

Great.

Pretty cool. So if we go back to the example, kind of just wrap it up here, make sure and-- sure enough, we got about 97% accuracy. Looks pretty similar to what we saw in our app. So our test set did well. And cool, profit.

That's it. OK, Heather, so that was machine learning. But what about deep learning?

Excellent, so we tried all those models. There's yet another subset of models that will work really well. And we can just go right to the doc and take a look.

And of course, the documentation has different applications. But of course, we can search for what we did before and open it up, and of course, open it up right in MATLAB.

So that's nice, in the documentation, there's the machine learning example and the deep learning example. And they go through the same data?

And even in a deployment example with Simulink.

Oh, nice.

So yeah, it's pretty exciting. So it really helps going through that entire process and having examples guide you along the way. So in this case, we're actually-- it's really similar to what we saw before with the image classification. But the only difference is the data, the way that's organized.

And just like we saw, each kind of sequence or signal is sort of one chunk of the data. That's really the only difference. And then we create our model. We talked before about using a pre-trained model.

LSTMs are very straightforward. There's really just a couple of layers. So we can just use the documentation, try it out, take advantage of the plots, the training visualizations. Looks like it's doing great. Obviously, if you saw that it was dropping, you could immediately just stop it and start over again.

But you could stop it, like if you were excited about the accuracy right now, you could stop that as well.

I'm very excited about it. And so yeah, that's exactly what we can do.

Excellent.

It'll actually carry through. So whatever had been trained up to that point, now we can test it. And if you take a look, just the same thing like we did before, classification, about 99%. And you can see, once again, some of those walking, running were some of the missed classifications there.

OK.

So these are the kinds of things that you want to test and try, of course, using those examples, using the apps, using your own human brain, really will help you assess what models might be best for you. That was extremely long answer to FAQ number 4.

But I think the idea too is that you could use machine learning and deep learning within MATLAB, and try out what works best for your application.

So we have a couple of considerations on the screen right now for you just to say, do I want machine learning or deep learning? But the point is that you don't really have to choose. You're going to use what's best for your application and your data.

OK, so here's our final frequently asked question. What's next? You went through an introduction today. But that's by no means the end of the story.

Right. We have a model, so what?

All right, so we have some key takeaways for you for today. Number one is focus on the workflow. So we focused a little bit on the workflow in the individual steps, but there's so much more you can do there.

You can always go back. We had great results. Of course they're documentation examples. They've been well tested. But you can always go back to the data prep, make sure things are represented well. Go back to the model training. Do the tuning. There are apps to help you try out different models.

Absolutely. And I'm pretty sure you can do all of the training in an app now.

Absolutely, even the tuning, the deployment part, everything that you see here, you can do in an app. So you don't have to start from scratch.

Right. And we just scratched the surface today. So we have free online training for you if you want to go deeper into machine learning, deep learning, or reinforcement learning.

Yeah, many other topics too, very exciting.

And then next, focus on your specific application. So peppers is only going to get you so far in life. Now you want to focus on predictive maintenance or radar or wireless communications.

So you really want to focus on your application, and chances are we have an application and an example for you.

Right. And it's just taking that the same steps that we went through, and just applying it to your thing.

Right, and the nuances.

Right, exactly. And there are so many nice examples now that really take you through that.

Right. And if you don't see your application on this list, then we want to hear from you as well.

Yeah, let us know.

All right, so that is wrapping up for today for this webinar. On behalf of Heather and myself, we'd like to say thank you for attending and have a great day.

Thank you so much.