This is a guest post by Diamond Blackwell, ACM-W President at the University of Louisiana at Lafayette.
On Friday, October 26, 2018, the University of Louisiana at Lafayette opened its doors to over 900 students participating in Science Day. This is a campus-wide event where high school students throughout Louisiana come to visit university’s various science departments. Our department, the College of Computing and Informatics, put on four demos for students to partake in.
The Association of Computing Machinery – Women (ACM-W) club, an organization dedicated to increasing the number of women in tech, decided we should do something different than the previous year for our demo. Our organization’s advisor, Dr. Sonya Hsu, gave us the idea to try a Deep Learning and IoT, Internet of Things demo presented at Grace Hopper Celebration of Women in Computing this past September by the team from MathWorks. With the help from the MathWorks GHC team and colleagues, we were able to put on this spectacular demo at the Science Day at the University of Louisiana. You can read about the GHC 18 Deep Learning and IoT workshop here on this blog.
We gathered all of the required tips from Anoush Najarian of MathWorks, configured all of our laptops, and put together an informal and interactive presentation. And we had our students very entertained! They really enjoyed taking photos of the fruit (and sometimes themselves) in order to see how and which objects were classified. Students kept taking photos until their fruit was correctly labeled, seeing how holding the object in a certain position or light would affect the MATLAB program’s answer.
We did notice that with smaller groups, it was easier to help everyone and keep control of the room. This made the demonstrations and feedback sessions a lot more educational: in larger groups, most people could not get most of their questions answered in the allotted time. But, the interactive segment of the demo did make up for it.
Meanwhile, the MathWorks team in Boston could keep up with what we’re doing by looking at the data collected on ThingSpeak and analyzing it with MATLAB – that’s IoT in action!
This was an amazing outreach opportunity for our organization that could not have been executed without the help of the MathWorks team! The positive feedback and help really made this an enjoyable experience for my team, and we hope to partner with MathWorks in the future.
You too are welcome to use our GHC 18 Deep Learning and IoT workshop materials, and share your thoughts in the comments! Check out the work of awesome women in engineering and science we have been highlighting with the #shelovesmatlab hashtag!
Dear friends, we: Louvere Walker-Hannon, an application engineer who assists customers doing deep learning and data analytics, Shruti Karulkar, a quality engineering lead for test and measurement, and Anoush Najarian, a performance engineer, put together this blog post on behalf of the MathWorks GHC team.
Our team had an awesome time at GHC 18, Grace Hopper Celebration of Women in Computing. Going to the conference helped our team members get to know each other, and brought out superpowers we didn’t know existed!
This is our first year as a sponsor of the conference.Grace Hopper Celebration is the world’s largest gathering of women technologists. Besides recruiting and attending key technology talks, our team delivered a hands-on MATLAB workshop on Deep Learning and IoT.
We were thrilled to have a hands-on workshop proposal accepted at GHC, an honor and a responsibility. It turned out that we were going to be running two large sessions, full in preregistration.
We asked everyone to bring a laptop with a webcam, or share. Participants used their browser to run deep learning code in MATLAB Online, a cool framework built on top of cloud instances, and aggregated inference data to ThingSpeak, an open IoT (Internet of Things) platform, and analyzed that data.
Everyone captured images of real-world objects using webcams and used a Deep Learning network to classify them. To make things more fun, we used fruit for inference: Granny Smith apples, oranges, lemons and bananas. Our team went on a “fruitcase” expedition: we visited a local grocery store with a suitcase, bought a bunch of fruit for the workshop, and at the end of the day, gave it away to the many amazing GHC workers.
Our workshop had three exercises, and two take-home problems.
In our first exercise, we used a webcam to capture an image, and passed it along to the AlexNet Deep Learning network for inference, generating a classification label and a confidence score.
% This is code for Exercise 1 as part of the Hands on with Deep Learning % and IoT workshop presented at the Grace Hopper Celebration 2018-09-27 %% Connecting to the camera camera = webcam(1); % Connect to the camera %% Loading the neural net named: Alexnet nnet = alexnet; % Load the neural net %% Capturing and classifying image data picture = snapshot(camera); % Take a picture picture = imresize(picture, [227, 227]); % Resize the picture [label, scores] = classify(nnet, picture); % Classify the picture and % obtain confidence score [sorted_scores, ~]=sort(scores, 'descend'); % Sorting scores in % descending order image(picture); % Show the picture title(['Alexnet classification: ', char(label), ' score:', ... num2str(sorted_scores(1))]); % Show the label clear camera drawnow;
In our second exercise, we repeat what we did in Exercise 1, and post inference data to an IoT channel. Note that we use the same channel to aggregate everyone’s data.
% This is code for Exercise 2 as part of the Hands on with Deep Learning % and IoT workshop presented at the Grace Hopper Celebration 2018-09-27 %% Connecting to the camera camera = webcam(1); % Connect to the camera %% Loading the neural net named: Alexnet nnet = alexnet; % Load the neural net %% Capturing and classifying image data picture = snapshot(camera); % Take a picture picture = imresize(picture, [227, 227]); % Resize the picture [label, scores] = classify(nnet, picture); % Classify the picture and % obtain confidence score [sorted_scores, ~]=sort(scores, 'descend'); % Sorting scores in % descending order image(picture); % Show the picture title(['Alexnet classification: ', char(label), ' score:', ... num2str(sorted_scores(1))]); % Show the label clear camera drawnow; %% Aggregating label data to open IoT platform try thingSpeakWrite(123456789, string(label), 'WriteKey', 'XXXYYYZZZ') catch pause(randi(5)) end
In our third exercise, we grabbed the aggregated inference data from the IoT channel and visualized it. It was fun and a bit surprising to see what everyone’s objects ended up getting classified as.
% This is code for Exercise 3 as part of the Hands on with Deep Learning % and IoT workshop presented at the Grace Hopper Celebration 2018-09-27 %% Reading aggregated label data for the last 2 hours from ThingSpeak readChannelID = 570969; LabelFieldID = 1; readAPIKey = ''; dataForLastHours = thingSpeakRead(readChannelID, ... 'Fields', LabelFieldID, 'NumMinutes', 5, ... 'ReadKey', readAPIKey, 'OutputFormat', 'table'); %% Visualizing data using a histogram if (not(isempty(dataForLastHours))) labelsForLastHours = categorical(dataForLastHours.Label); numbins = min(numel(unique(labelsForLastHours)), 20); histogram(labelsForLastHours, 'DisplayOrder', 'descend', ... 'NumDisplayBins', numbins); xlabel('Objects Detected'); ylabel('Number of times detected'); title('Histogram: Objects Detected by Deep Learning Network'); set(gca, 'FontSize', 10) end drawnow
When our participants ran this code, we saw a histogram aggregating everyone’s inference data, with all the objects detected during the workshop. This is the power of IoT! Check out the data from all our workshop sessions aggregated together on the ThingSpeak channel.
As take-home exercises, we challenged participants to use GoogLeNet instead of AlexNet, and to create their own IoT channel and use it to post and analyze data.
It’s an honor to have a speaking proposal accepted at GHC, and delivering large hands-on sessions is a big responsibility.
We loved hearing from our participants on social media:
We heard from professors and AI and Deep Learning enthusiasts who are interested in using our materials on campuses and at maker events: below are the first two, and a few others are in the works! If you’d like to give our Deep Learning and IoT demo a shot, let us know in the comments.
Hope Rubin of our GHC team led STEM Ambassadors who brought this Deep Learning and IoT demo to the Boston Mini-Maker Faire.
Under the leadership of brave Professor Sonya Hsu and her ACM-W partners, a team ran the workshop during the Science Day at the University of Louisiana at Lafayette. Look for posts on these events on this blog!
We couldn’t have done this without our team members’ extensive experience with teaching and tech, the awesome guidance by our senior leaders, and the help from hundreds of MathWorkers and Boston SWE friends.
Boston SWE, Society of Women Engineers, one of the oldest and largest sections in the country, has been our rock! We ran our workshop to a SWE event at MathWorks the week before GHC, getting our code and materials in front of many inquisitive, engaged participants who gave us their time, their words of encouragement, and who asked us tough questions!
You too are welcome to use our GHC 18 Deep Learning and IoT workshop materials!
Visit our GHC page to meet our team and learn about working at MathWorks. Take a look at the photos our team took, or was given by session chairs, and our Twitter Moment. While you’re at it, check out the work of awesome women we have been highlighting with #shelovesmatlab hashtag! Share your thoughts in the comments.
Libelium makes the Meshlium IoT Gateway that supports commercial IoT systems and sensor applications such as waste management, forest fire detection, potable water monitoring, supply chain control, and fleet tracking. Libelium now supports ThingSpeak, so that you can take IoT applications to the next level by using ThingSpeak’s integrated MATLAB enabled IoT analytics and visualizations.
ThingSpeak is a MATLAB enabled IoT analytics platform from MathWorks, the leading developer of technical computing software for engineers and scientists. With ThingSpeak, users can view instant visualizations of live data from any Internet-connected web browser and schedule MATLAB code to run live analyses and visualizations as new data arrives. To accelerate the development of IoT analytics, MATLAB offers a full set of statistics and machine learning functionality, plus advanced methods such as nonlinear optimization, system identification, and thousands of prebuilt algorithms for signal and image processing.
Libelium sensors are used in a variety of vertical IoT applications like air quality monitoring and smart agriculture that are also common applications for users of the ThingSpeak platform. “The new integration between the Meshlium IoT Gateway and ThingSpeak will allow our customers with mutual interests to quickly analyze their data in the cloud with MATLAB,” said Eric Wetjen, Senior Product Marketing Manager for ThingSpeak.
The ThingSpeak integration allows you to easily connect your Libelium devices to the ThingSpeak IoT analytics platform by using the ThingSpeak cloud connector, which is built into the Libelium Meshlium IoT gateway. The ThingSpeak cloud connector inside of the Meshlium Manager System creates the ThingSpeak channels needed for your devices and synchronizes the data automatically without writing any custom code.
Check out the MathWorks Hardware Catalog for more information about the Libelium support for ThingSpeak and MATLAB.
We have published an example in the ThingSpeak documentation that shows you how to train a feedforward neural network to predict temperature. The feedforward neural network is one of the simplest types of artificial networks but has broad applications in IoT. Feedforward networks consist of a series of layers. The first layer has a connection from the network input. Each other layer has a connection from the previous layer. The final layer produces the network’s output. In our IoT application, the output will be the predicted temperature.
We are collecting data in a ThingSpeak channel and will use the integrated MATLAB analytics. To predict the temperature, this example makes use of the Neural Network Toolbox in MATLAB along with the data collected in a ThingSpeak channel. We will be using data collected by a weather station located at MathWorks offices in Natick, Massachusetts.
The process for creating, training, and using a feedforward network to predict the temperature is as follows:
- Gather data from the weather station
- Create a two-layer feedforward network
- Train the feedforward network
- Use the trained model to predict data
Read Data from the Weather Station ThingSpeak Channel
ThingSpeak channel 12397 contains data from the MathWorks weather station, located in Natick, Massachusetts. The data is collected once every minute. Fields 2, 3, 4, and 6 contain wind speed (mph), relative humidity, temperature (F), and atmospheric pressure (hg) data respectively. To read the data from the weather station within MATLAB, use the thingSpeakRead function.
data = thingSpeakRead(12397,'Fields',[2 3 4 6],'DateRange',[datetime('Jul 30, 2018'),datetime('Jul 31, 2018')],... 'outputFormat','table');
Create Two-Layer Feedforward Network
Use the feedforwardnet function to create a two-layer feedforward network. The network has one hidden layer with 10 neurons and an output layer.
net = feedforwardnet(10);
Train the Feedforward Network
Use the train function to train the feed-forward network.
[net,tr] = train(net,inputs,targets);
Use the Trained Model to Predict Data
After the network is trained and validated, you can use the network object to calculate the network response to any input.
output = net(inputs(:,5))
output = 74.9756
This example can be adapted to other IoT applications. Check out the ThingSpeak documentation for the code and explanation.
The ThingSpeak IoT has been building a new framework to support widgets on channel views. Widgets can be added to the public or private view of a ThingSpeak channel and even be embedded in 3rd-party systems and dashboards. The first widget that we are releasing is the gauge!
At the recent Boston TechJam, MathWorks had a ThingSpeak People Counter that used face detection to count people that came over to our booth and learned about our demo. The people counter uses MATLAB to identify faces in a live video stream from a webcam, count the number of faces, and send the results to ThingSpeak. The code and instructions for the ThingSpeak People Counter project are on File Exchange.
Now that we have the infrastructure for widgets on ThingSpeak, we can more widget types. What other widgets would you like to see on ThingSpeak?