Deploy MATLAB Programs to the Cloud, Containers, and PI Systems

Welcome to Running MATLAB in the Cloud, Containers, and OSIsoft PI. I'd like to tell you about one of our customers, Sentio, who have developed a mental health therapy program using MATLAB in the cloud. It all starts with a wristband that looks a whole lot like a Fitbit.

This wristband comes with four different sensors. There is a heart rate sensor, a galvanic skin response sensor, an infrared sensor for taking body temperature, and an inertial movement sensor. The data from these sensors is sent via Bluetooth to an app running on the user's smartphone.

Now, this app sends the data to MATLAB Production Server running on the cloud, which runs a prediction algorithm and classifies the user's emotional state as a variety of different types, for example, happiness, sadness, anger, distress, et cetera. And the user at that time is given an opportunity to augment those labels with additional information for both journaling as well as to provide feedback to the model. The model running on MATLAB Production Server also provides recommendations for the users, such as to breathe, to reduce stress, or any other recommendations that have already been provided by therapists, which the user meets on a weekly basis.

So how did these predictive models get to MATLAB Production Server on the cloud? Well, they were developed by Sentio using MATLAB. They use signal processing to clean up and normalize a lot of the time series data that came from the wrist devices. They couple that with a machine learning to be able to help classify the signals into the labels that are sent back to the user. Now, these models were then deployed to MATLAB Production Server running in AWS.

So as we talk through the workflow, we're going to be going through several phases here. And we're going to be talking about how our products, working in the cloud, can help with the different stages, like you saw were used with Sentio, for example, in the data preparation stage where they had to clean and prepare the data, the modeling and the training stage where they developed and train the model, as well as when it came time to deploy it to the cloud. So let's go through each of these stages one by one.

So in the data preparation stage, what you have here is usually a whole lot of data that has been captured through either devices or either as historical data and large files, such as CSVs in our case. This data is often stored in cloud data repositories, like AWS S3 or Azure Blob. And if you were to use MATLAB on your desktop to work with this data, you would find that it would take a long time to download this data to your local desktop. And you would have to pay data transfer fees, outbound transfer fees, to pull it out of AWS or Azure.

Ideally, what you would want is you would want your MATLAB to be co-located in close proximity to this data. And that's what we provide with the MATLAB VM that you can run in either AWS or Azure. So this MATLAB virtual machine allows you to co-locate the data exploration in close proximity to the data, often within the same data center.

In addition to being very closely located, this virtual machine that you spin up can make use of heavy duty cloud resources. So, for example, your local PC might be an 8 core machine. When you run this virtual machine in the cloud, you can specify up to a 96 core machine with maybe half a terabyte of memory, something you would never be able to get on your local PC or even to be able to cut a PO for to purchase. But you can rent this for, say, a couple of hours in the cloud.

So we have made the MATLAB Virtual Machine available as a variety of different options. You'll see them available as either reference architectures or bring your own license. We will use these terms a lot over the next couple of slides. Don't worry. At the end of this presentation, I will give you tips and guidance on when to use either reference architectures or BYOLs and talk to you about the different pros and cons of each option.

All right, so that is the data preparation stage. And what I want to do next is show you how this all works through this video. So we'll go through the steps to provision MATLAB, the MATLAB Virtual Machine on Microsoft Azure and show you how to access it, using a remote desktop client.

To get started, simply search for MATLAB in the Azure Marketplace. Click the Get It Now button. A dialog box will appear asking for your consent to share your contact information with MathWorks. Click Continue to proceed.

Since this is a bring your own license solution, if you need a MATLAB license, click here to obtain a trial license. On the next page, along with your email, provide any information that will help a MathWorks representative better understand your needs and follow up with you on licensing options.

Back to the marketplace listing, click Create to begin the deployment. First, select your subscription. Next, select an existing resource group or create a new one. It's recommended you create a new resource group for each project. It makes it easier for you to organize your projects and delete older deployments. Then, select a location for your deployment. Most locations are supported, but if you're not sure, you can leave the default and move on to the next stage.

In the Virtual Machine settings, the VM size is the size of the machine running MATLAB. A 4 BCP machine with 14 GB of memory is a reasonable default. Just make sure your VM size is sufficient to run your workload.

Next, create a username for your virtual machine. This creates a new user account. And note that this is not your Azure or MathWorks user name. It is, however, the username used to log into the virtual machine when you connect using remote desktop.

Enter a password. In the IP Addresses field specify the client IP or the IP of your PC that will be used to connect to your MATLAB deployment using remote desktop. Specifying this IP helps you to secure your deployment by limiting access. You can determine your own client IP address by googling, what is my IP? Copy and paste the client IP address followed by slash 32. The slash 32 is a mask that limits the address range to a single IP address. If you're using a network license manager, enter the optional port at host name to point to the correct license manager and move on to the next stage.

In the networking settings, the virtual network has been reconfigured for you with reasonable default. These settings can be modified to meet your specific ID needs. Please contact your network administrator before modifying the settings. If you need additional guidance on the virtual network settings, you can find a link to the MATLAB documentation here.

Next, click the Review and Create button. On the final page, Azure will validate your inputs. Make sure you read the Azure terms and conditions and privacy policy before you click Create. You will be billed by Azure for the resources consumed. Azure will now begin deploying the resources, and the process will take about 2 to 5 minutes. You'll receive a notification when the deployment completes. From that notification, click the Go To Resource Group to see the resources it created.

We'll need the DNS name to connect using remote desktop. That can be found either in the public IP or the VM. Click the IP address. Copy the DNS name. Open remote desktop on your local PC. Paste the DNS name and click Connect.

On the Login screen, enter the username and password you specified earlier. This will connect to the virtual machine desktop and may take a couple of minutes for the machine to finish starting up. Once you see the MATLAB icon on the desktop, double click it to launch MATLAB. This may again take a few minutes the first time you launch MATLAB.

MATLAB will eventually prompt you for your MathWorks account login information. Once you see the MATLAB prompt, it is ready to use. Now, go ahead and enter your favorite MATLAB command. If you no longer need to run MATLAB on Azure, simply delete the entire resource group, which will remove all the components from your deployment.

In this video, I've shown you how to bring your own license and use MATLAB on Microsoft Azure.

Once you have set up MATLAB in the cloud, we provide a wide variety of connectors to your data. In addition to connecting to AWS S3 and Azure Blob, we also provide interfaces with relational databases, NoSQL databases, and graph databases. What you see here is not a comprehensive list. To see everything we provide, , go to the GitHub link at the bottom of this page. The interfaces found on our GitHub page are provided free of charge.

All right, let's move on to the modeling and training phase. You now have access to all your data. And it's time for you to develop your algorithm or model. Often, if you try to do that on your local PC, you'll discover that it takes a long time to run your simulations, parameter sweeps, or optimization runs. So we have provided a way for you to offload a lot of that processing to the cloud with MATLAB Parallel Server.

This solution spins up a head node, as well as a cluster of compute worker nodes in the cloud. You can customize your solution by specifying the number of working nodes as well as the CPU or RAM available to each of those working nodes. Let me show you an example of this.

Here, we have a van der Pol equation running in parfor, 40,000 iterations. We have taken this same code, and we have three different environments here. We have on the top left your local PC with 20 workers. In fact, this is a fairly powerful local PC. Most times you might not have that many. You have a data center cluster with 200 workers running maybe in your local data center. And then we have spun up a cloud instance here with 1,000 workers.

So we're going to hit Play here and run this execution in-- this is not real time. This is accelerated time-- and show you the vast differences it makes to running your modeling. So over here, you'll see that the cloud instance or the cloud environment with 1,000 workers has finished execution in 2.8 minutes. Your desktop is barely even 2% completed. So you can see how by provisioning a large number of resources for a short amount of time-- this may cost you maybe a few dollars or tens of dollars-- it can really improve your productivity.

All right, so let's say, however, you have a task that requires speed up using GPUs. And by that, specifically I mean deep learning tasks. What do we provide there? Well, we provide something we call the MATLAB Deep Learning Container for NVIDIA. This is a pre-configured docker container. It has MATLAB and nine different toolboxes provided that are deep learning specific, as well as pretrained neural nets.

So how this works is first you go to AWS or Azure. You provision virtual machines that have GPUs in them, for example, an AWS P3 instance or an Azure NC-series. Then you go to the NVIDIA GPU Cloud. Now this is a bit of a misnomer. This is not a cloud. This is actually a container registry. You go there and locate the MATLAB Deep Learning Container. And you do a Docker pull into the virtual machine that you have obtained through AWS or Azure.

Now that you have it in your virtual machine, you can then run it using NVIDIA Docker run. And at that point you VNC or your remote desktop into the virtual machine, like in the same way you saw with the MATLAB example. And you can perform your deep learning tasks and specifically your deep learning training on that machine, and it should accelerate the training significantly, depending on how many GPU resources you have allocated to that VM.

All right, so that is modeling and training. Now, you have your model. You have your algorithm. And you want to deploy it. What do you do?

Well, this is where you would use the MATLAB Production Server in the cloud. The specific use cases that you would use this for are to deploy machine learning or deep learning models, like we saw with the Sentio example, or if you want to deploy your algorithm, something you've written, an optimization algorithm, for example, as a microservice API that can be called, or to perform streaming IoT analytics on data that's being streamed in near real time.

And this is available in various different options again, just like the other products you saw earlier. The one difference here is that in addition to the reference architecture and the bring your own license options, we also provide a pay as you go offering. That's new. The difference here is that you do not have to have previously obtained a MATLAB Production Server license to use this. This is something you pay for on a consumption basis. You pay for it hourly. So it's a great way to give it a try without committing to the purchase. This is something you can use. It is a full featured version.

So here's an example of an application that is calling a model. This is a load forecasting application. It calls a predictive model to provide a 24-hour load forecast on an electrical grid, as you can see over here in the top left.

Now, it's calling the model to provide this prediction that's running in Production Server in the cloud. And the front end here is being rendered on a TIBCO Spotfire GUI. So the GUI, the front end, is something you can customize to whatever needs you want and to render the output that comes out from the server.

Now, I'm going to show you here a quick demo of how to use this. This is a short video of how the API is exposed and how to call it with your front end application. We're going to skip over this whole front section where, like you saw earlier, to fill out the form to provision it. We assume that's already done.

Once the deployment has successfully completed, obtain the Cloud Console IP address from the server machine public IP resource, and enter that IP address into your browser.

You may receive a warning message from your browser about visiting an unsecure site. This is due to the use of a self-signed certificate. You can change the self-signed certificate to a CA signed certificate to eliminate this error. Details on how to do so are found in the GitHub repository. Log in using the username and password credentials you specified earlier.

The Cloud Console shows you the current status of your Production Server deployment. We specified two instances earlier. Each one of these instances running four MATLAB workers, giving us a total of eight workers. A typical Production Server license contains 24 workers. So you can scale up to six VM instances.

The URL endpoint to call your functions from your client application is displayed prominently on the Cloud Console. The first thing we need to do is to get the host IP or MAC address of the licensed server. We will need this to generate our license. At the Cloud Console, go to the Administration tab and choose Manage Licenses.

Provide the licensed server host ID to your sales rep if you are obtaining a trial license. Otherwise, provide the host license administrator who has access to the MathWorks license center to generate a license file. Once you receive the license file, usually with a dot lic extension, select the file from the File Open dialog and then upload the file.

You're now ready to upload a Production Server application. We'll go to the Applications tab and click on Upload Application. Production Server applications are packaged using MATLAB compiler SDK in a CTF file. Let's upload a Bond tools CTF from the for distribution folder of the project.

Once the application has been uploaded, you can call its functions from your front end client. In this example, we have a web-based front end client app that is calling the bond tools function we just uploaded to get the price of the bond. The web front end plan app captures the input data in the fields, packages it into JSON payload, and makes a REST call to the Production Server endpoint.

So let's recap. When it came time to data prep, we provide the MATLAB Virtual Machine in the cloud that allows you to run MATLAB in close proximity to that cloud data that you have to reduce the latency and to reduce cost as well. When it comes time to modeling and training, we provide two different options. There's Parallel Server in the cloud to help with parallelizing and speeding up your modeling. And the MATLAB Deep Learning Container is to help with speeding up the training for deep learning models. Once you have those models in place, you can go ahead and deploy them into MATLAB Production Server in the cloud.

This table provides an overview of all the cloud products available at MathWorks across Software as a Service, Platform as a Service, and Infrastructure as a Service. The products we described today are highlighted in the red box. Now, you might be wondering with all these choices, which one is the best for you? So we've provided this table as guidance.

If you want the maximum amount of flexibility and the ability to customize your deployment, we recommend going with the Cloud Reference Architectures. But most customers who wish to take their existing license and run them in the cloud, we think the bring your own license option is the best for you. And if you have the need to run long-term trials or proof of concepts, or you have very bursty workloads, then the pay as you go option is the most optimal.

So far, we have talked about running in the cloud using virtual machines. But what if you want to run using containers? Well, there are two possible scenarios here. Scenario one is you want to run MathWorks products in a container. For example, you want to run MATLAB in a container. If so, we described earlier the deep learning container for NVIDIA, which is packaged with a variety of deep learning specific toolboxes.

Now, if you wish to run generic MATLAB, then we provide the MATLAB Docker file on GitHub that you can use to build your own Docker container. If you want to run MATLAB Parallel Server or MATLAB Production Server in a container, please contact us, and we can help you containerize our server products. We have done this for several of our customers.

Scenario two is you want to run applications that you have created in MATLAB as a container. As of R2020B, we support this workflow. Using MATLAB Compiler, you package your standalone application as a Docker container image. This Docker container is a self-contained package that consists of supporting operating system libraries, a size optimized MATLAB Runtime that contains only the toolboxes you need, and your application. This Docker container can be distributed royalty free. And users can run your application with a simple Docker run command.

I have one more thing for you. And this might not be cloud related, but many of you have with OSIsoft PI systems and are wondering if you can connect MATLAB with OSIsoft PI. And the answer is yes. We provide two options.

The first is called the MATLAB Interface for OSIsoft PI System. This allows you to import your historical PI data into your MATLAB desktop for either data exploration or algorithm development. This installs into MATLAB as a tool box add-on. And once installed, we provide you with an easy-to-use Getting Started guide as a live script. From there, you can use the functions to connect to your PI server and access your data. This can be requested from the URL that you see on this page. And there's no charge for it.

The second option is the MATLAB Production Server interface for OSI PI System. This allows you to perform periodic analytics on your near real time data being streamed into your PI system. Let's see how this works.

Your sensors send data into PI Asset Framework. You install the interface we provide as a Windows service in your server. Our interface is configured using an XML file. This XML can be manually edited by you, or you can make use of the GUI that we provide to configure which PI attributes you want to retrieve data from as well as write the results back into.

Once you have configured the interface, it runs periodically to pull data from PI AF. This data is then executed against MATLAB functions running in MATLAB Production Server. The results of those functions are then returned into PI AF using the interface. Like the first option, this is no charge, and you can request it from the URL you see here.

This takes us to the end of the presentation. And we'll now open it up for questions.