Engaging First-Year Engineering Students with Deep Learning and IoT

[AUDIO LOGO]

This talk is about engaging first-year engineering students with deep learning and IoT. I'm Dr. Chao Wang, and I'm from Arizona State University. This is the outline of my talk. In 2020, American Society of Engineering Education Corporate Member Council conducted a survey of recent engineering graduate. They found that 81% of responses expressed that they were not prepared in the area of artificial intelligence. 70% of responses said that they were not prepared in the area of internet of things.

As educator, we want to prepare for their future career. That is why we want incorporate new technologies into our curriculum. There is already a lot of existing work in this area. However, most of them focuses on computer science students or upper division undergrad or graduate students, because for new technologies like this, most course materials need extensive programming background. And in the case of IoT, often we need a additional microcontroller hardware. And of course, instructor need to have the time and effort to develop new course materials.

I teach a course called Introduction to Engineering. It is a freshman-level course taken by all engineering students. They often take the course in their first semester in college. In this class, we focus on engineering design process, problem solving, and student work on a big hands-on project implementing their own design idea using sensors and actuators using Arduino.

I have long been able to incorporate new materials in this class. One of them is I always have the desire to incorporate internet of things and artificial intelligence because I think it's important for freshmen students to get exposed to these new technologies. So in case in the future they need to know more about the topics, they can take additional courses.

However, because they are freshman students, most of them don't have programming background. Actually in one year, I did a survey. One-third of my students have no programming background. And half of the female students, they've never programmed before. So it will be hard for me to introduce a new programming language in the course. And that is why I didn't get to do those things, introduce IoT and artificial intelligence in my course, until in the year of 2020, I attended a MathWorks workshop.

And during the American Society of Engineering Education Annual Conference, I got connected to a MathWorks engineer. And she introduced me to a workshop designed by MathWorks engineers on deep learning and IoT which I thought was perfect for my class. In my Introduction to Engineering class, I've already have three lecture periods dedicated to teaching MATLAB.

I use MATLAB as a data analysis and visualization tool. And I teach students how to use MATLAB to visualize data and do a basic engineering problem solving. In order to make room for this new material on deep learning and IoT, I have to move some of the existing course materials outside of the classroom. So I decided that student can take this Onramp tutorial which is designed by MathWorks. I like about this tutorial is because it is very interactive. Student can practice all those MATLAB commands, and they can get instant feedback.

So I assign this as a pre-lecture homework. Then during the first lecture, I review the concept, the commands they've learned in the Onramp tutorial in class with students with additional practice, for example, how to define, access scalar, vector, and matrix variables; how to import data from a text file and do 2D plots. And during the second lecture period, we did quite a few engineering problem solving.

For example, I led them to import a file about a solar module. And they can plot the current and versus voltage and visualize it under two different conditions, for example, like when it's-- sun is up, bright, and also under a tree, when it's shady. Then student can visualize the power output of the solar module is bigger when it is under sunny condition. And I also let student practice curve fitting, simple curve fitting, interpolation, extrapolation, because I think these are the skills that are important for all the engineering disciplines.

So during the third lecture, I have this new machine learning, deep learning in IoT that I introduce student to these new technologies. This module is designed by MathWorks. It uses an image classification application to introduce deep learning and IoT. I like about this workshop is because the student only need very basic knowledge of math. So from that online tutorial and two additional classes of practice, students have enough knowledge to do this MATLAB module with no problem. And one more nice thing about this module is it doesn't require additional hardware. It only needs a laptop with a webcam installed. On the part of me, I can get module up and running under two hours because everything is ready and prepared by MathWorks engineers.

To prepare for this module, I ask student to bring a laptop with webcam and objects to classify to class. They also need to create a MathWorks account to use MATLAB Online, which student already did when they did their Onramp tutorial. Then at the beginning of the lecture, I let student copy a code folder to their online MATLAB drive. So in this code folder, it has all the three exercises they need to do this workshop.

So once all the preparations are finished, I give a brief introduction of artificial intelligence, machine learning, deep learning, and IoT at the beginning of the lecture. This is the first exercise. Student run the code of the exercise. And then what this code does, it takes a snapshot using their webcam and then use a pre-trained deep learning model called AlexNet to assign a classification label to the image. This is the code of the exercise.

As you can see, it's pretty simple. There are only 10 lines of code. I go through them one by one, even though I think all of them student haven't seen before, but student already know the syntax of MATLAB. So they can quickly pick up with all those labels, those comments. Student can quickly understand what each line mean. So even though student have never-- they don't have their toolbox installed, deep learning toolbox installed, because we're using the online version, they don't have any trouble connecting to the webcam. They don't have any trouble using the deep learning-- pre-trained deep learning network.

So the second exercise is to send the classification label to the-- a IoT cloud service by MathWorks. It is called ThingSpeak. So this is all done in real time. So a student taking image, as you can see, taking the image with their webcam. They classify the image and the label. They get a label. And at the very bottom of the screen, you can see this label gets sent to a ThingSpeak channel with a particular channel ID and using a right key.

Then the next-- during the next exercise, student were able to collect back all those data, all those labels, and then display as a histogram. So again, I want to emphasize, this is all done in real time. So basically, each student, all student, all taking pictures using webcam. And then they have all those labels. And those labels get sent to the cloud. And then at the same time, those label get collected back and display on the student's laptop.

So, very interesting. Some of the objects get classified correctly. For example, water bottles, they get classified correctly, and also backpack. But a lot of other objects get classified wrong. One particular interesting instance is a lot of students, they-- because they are sitting in front of a webcam, they took a selfie of themselves. Their images were classified as lab coats. Soon, the student found this was really hilarious.

And this is the code for the third exercise. As you can see, we have a channel ID for the channel, public channel, we use. Then now we just need to click all the data back and visualize it using a histogram. You can see here, the x label, y label, title. Those are the things student practice in their previous exercise when they do their 2D plots. So here, they look at this code, and they all look very familiar.

I did a brief survey at the end of this lecture period, at the end of this workshop, that is called-- the survey's called Situational Motivation Scale Survey. It is a survey used to measure motivation. The metric used to measure student motivation is called Self-Determination Index. The higher the index, the more motivated student were. So I've been doing this survey for quite a few years on different activities I tried in my class. And I can tell you, the scores for this particular exercise has the highest score so far. And even though the sample size is kind of small, but one thing that you can notice is the female student seem to be much higher motivated compared to their male counterparts.

I also did another survey at the end of the semester. I used three questions on this deep learning and IoT module. And I asked them if they find a lecture to be useful, if the lecture provides them with a good introduction, and if they want to learn more about the topics in the future. So there are seven scores, numerical scores, starting from score 1, which corresponds not at all, and score number 4 is-- corresponds moderately. And score number 7 is corresponds exactly.

So as you can see, most of the student think this lecture on deep learning and IoT, they are useful, have a good introduction for them. And then some of them, they did want to learn more about the topic in the future. And even though there's-- you don't see much difference. But in this case, like with a small sample size, but we still can see female student have a slightly higher score compared to their male counterparts.

So I also have a free-response question in both of the situation survey and end of semester survey to ask students' experience in their own words about this workshop. And many of them share that I think this workshop is very cool and interesting. I have a few teams in my class. They wanted to incorporate this image classification example in their own project.

So they have to do a 6-week project in this class to implement a idea of their choice using Arduino. So they want to incorporate this image classification into their project. But because of time constraint, they wasn't able to finish. But I told them they can definitely try again when they do their senior design. They can try to incorporate internet of things and artificial intelligence in their senior design capstone project in the future.

So I hope this new workshop, this new module on deep learning and IoT, gives student a good introduction and get them exposed to this new technology, because we know now this AI and internet of things is everywhere in our daily life, from Alexa-- now we have this ChatGPT. And I think students should know more about it in the future because they will probably need this in their workplace. So that is why I think this is very important for students to know it in their earlier career.

What I like about this module from MathWorks it doesn't require any additional hardware and very little MATLAB programming background and minimum instructor preparation. So I think if someone wants to have a lightweight introduction to deep learning and IoT in their course, this is definitely a good choice, because they only need one class period, assuming student already have some basic understanding of MATLAB. And that is why I highly recommend you try this out.

A special thanks to Mrs. Gaby, who was the MathWorks engineer who helped me with all the workshop implementation. I also want to thanks to the authors of the workshop. If you want to know more about this project, there is a MathWorks newsletter article that you can access online. I also have a ASEE conference paper that you can read and know more about this project. If you have any questions, please feel free to contact me at chao.wang.6@asu.edu. Thank you for your time.

[AUDIO LOGO]