Sunday, 24 July 2016

IoT Adventure: 4 - Sending sensor data to IoT Hub

Case
I want to send sensor data from my .NET application to Azure. How do I do that?

Solution
To send sensor data from our .NET application we will use JSON messages and send them to the Azure IoT Hub. After that, other Azure parts like Stream Analytics can 'subscribe' to the IoT Hub to actually do something with the sensor data. But that will be described in a next blogpost.
The first step is to setup an IoT Hub to receive messages. The second step is to adjust our .NET application to send those messages.

1) Create Resource Group
To create an IoT Hub we first need a Resource Group to store the data from the IoT Hub. You can skip this step by creating a Resource Group within the IoT Hub form or if you already have one in your subscription.
Go to New, Management and then Resource Group. The most important option is the Resource group location. You can't combine Azure items from different resource groups. Because we live in the Netherlands, West Europe is the most obvious location to choose for us. That way the data stays in the Netherlands.
Azure Portal - Adding Resource Group

















2) IoT Hub
Now the actual IoT Hub. Go to New, Internet of Things and then IoT Hub. For this example we are using the F1 scale tier which is free of charge, but has a limit of 8000 messages a day. So sending each second isn't possible, but once a minute is no problem. The size of the messages is limited as well to 0,5kb for F1 and 4kb for the other editions.
Azure Portal - Adding IoT Hub
















3) Consumer groups
We need to create consumer groups. We will create one for the hot path called 'PowerBi' and one for the cold path called 'AzureDB'. Stream Analytics will use these consumer groups to run the queries. Using multiple consumer groups makes it possible for several consumer applications to read data from this IoT Hub independently. If you would use only one consumer group then one consumer application will retain the lease and the others will loose the connection.
Go to the newly created IoT Hub. Click on 'All Settings', 'Messaging' and then scroll down to the consumer groups. Add two groups: PowerBi and AzureDB and click on save.
Azure Portal - Adding Consumer Groups














4) IoT Hub Connection String
Next step is to create the Connection String which we need in our code to send JSON messages to this IoT Hub. First click on the Key icon on the IoT Hub dashboard. Then in the Shared access policies window click on 'iothubowner'. Now copy the 'Connection string - primary key' for the next step.
Azure Portal - Connection string - primary key














5) Device Explorer - Connection String
With the Device Explorer tool we are going to convert the connection string from the previous step to something we can use in our code.
  • Start up Device Explorer (see prerequisites) and paste the connection string on the Configuration tab under IoT Hub Connection String and click on Update.
  • Next go to the Management tab and click on Create. Enter the unique name (or id) of your device and click on the Create button. A new line will be added in the Device Grid.
  • Right click it and choose 'Copy connection string for selected device'. This is the string we need in our code.

















6) Adjust .NET code
Go to your .NET project and use this connection string to send messages to the IoT Hub. If you are using our project then here is what you need to change:
Our Sensory project















Conclusion
Not a very exciting and visible step, but very necessary for your IoT project. Moneywise the chosen Tier is very important, but the Consumer Groups are also very essential if you have multiple consuming applications. Also read Azure Event Hub vs IoT Hub to check the differences between those two hubs.

Sunday, 17 July 2016

IoT Adventure: 3 - Create Visual Studio Project for sensors

Case
I have installed all the prerequisites, now I want to start a new Visual Studio Project. Which project do I choose and which assemblies do I need to reference for a IoT project?

Solution
Below you find the basic code you need to send sensor data to the Azure IoT Hub. The specific sensor code differs per sensor and will be posted in separate posts. Make sure to first install all prerequisites.

1) Blank App (Universal Windows)
Open Visual Studio 2015 and create a new project. We will be using C# as language. Under C#, Windows you will find the 'Blank App (Universal Windows)' project. Supply a ProjectName and Solution Name (also notice the checkbox for Source Control). Our project (and solution) is called Sensory.
Blank App (Universal Windows)

















When you create such project it will ask which version of Windows you want to use. Click on the help link if you're not sure.
Windows version










2) Source Control
The next window will be about source control because of the checkbox in the previous step. We will be using Team Foundation Server (TFS) for source control. This is recommended especially when you work in a team. You can either use your local TFS machine or use TFS online at visualstudio.com. It's also possible to use third party source control software.
TFS online
























3) Reference to Windows IoT Extensions for the UWP
A new blank project has been loaded. First we need a reference for IoT Extensions. Go to the Solution Explorer and right click on references and choose "Add Reference...". Then under Universal Windows / Extensions, locate "Windows IoT Extensions for the UWP". Sometimes you will find multiple versions. Make sure to choose the right one. We need version 10.0.10586.0.
Windows IoT Extensions for the UWP















4) Add NuGet Packages for Azure
Because we want to connect to Azure, we need to add a NuGet for Microsoft.Azure.Devices.Client.PCL. Right click the project (not the solution) and choose "Manage NuGet Packages". Then go to Browse and search for "Microsoft.Azure.Devices.Client.PCL" and click on the Install button. When you're ready, a new reference appears in the Solution Explorer.
Add NuGet for Azure















5) SensorMessage Class
For this project we have created a Sensor Message class to store the sensor values from one meassuring moment. Each x seconds/minutes we meassure for example the temperature and the illumination in a room. We store this in a SensorMessage object and then we are able to create a JSON message with these values and send this message to the Azure IoT Hub.

Right click the project and choose 'Add', 'New Item...' and in the new window choose Class. Give it the name SensorMessage. Copy and past the code below to the new class. If you used a different project name then change the namespace from Sensory to your namespace name.
Add new class file













//C# Code
using System;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;

namespace Sensory
{
    [DataContract]
    public class SensorMessage
    {
        #region Properties with get and/or set
        [DataMember]
        private string sensorName;
        public string SensorName
        {
            get { return sensorName; }
        }

        [DataMember]
        private DateTime measurementTime;
        public DateTime MeasurementTime
        {
            get { return measurementTime; } 
        }

        [DataMember]
        private decimal temperature;
        public decimal Temperature
        {
            get { return temperature; } 
            set { temperature = value; }
        }

        [DataMember]
        private decimal humidity;
        public decimal Humidity
        {
            get { return humidity; }
            set { humidity = value; }
        }

        [DataMember]
        private decimal pressure;
        public decimal Pressure
        {
            get { return pressure; }
            set { pressure = value; }
        }

        [DataMember]
        private decimal altitude;
        public decimal Altitude
        {
            get { return altitude; }
            set { altitude = value; }
        }

        [DataMember]
        private decimal decibel;
        public decimal Decibel
        {
            get { return decibel; }
            set { decibel = value; }
        }

        [DataMember]
        private int doorOpen;
        public int DoorOpen
        {
            get { return doorOpen; }
            set { doorOpen = value; }
        }

        [DataMember]
        private int motion;
        public int Motion
        {
            get { return motion; }
            set { motion = value; }
        }

        [DataMember]
        private int vibration;
        public int Vibration
        {
            get { return vibration; }
            set { vibration = value; }
        }

        [DataMember]
        private decimal illumination;
        public decimal Illumination
        {
            get { return illumination; }
            set { illumination = value; }
        }
        #endregion

        #region Constructor method
        /// 
        /// Creates SensorMessage object with default values
        /// Code should look something like:
        /// SensorMessage sensorMessage = new SensorMessage("mySensor");
        /// 
        /// The name or unique code of your sensor
        public SensorMessage(string SensorName)
        {
            sensorName = SensorName;
            // measurementTime = DateTime.Now;
            // Fix for timezone / offset troubles
            // after serializing to JSON. Without
            // this it shows UTC time with offset
            measurementTime = DateTime.UtcNow.AddHours(TimeZone.CurrentTimeZone.GetUtcOffset(DateTime.Now).Hours);
            temperature = 0;
            humidity = 0;
            pressure = 0;
            altitude = 0;
            decibel = 0;
            doorOpen = 0;
            motion = 0;
            vibration = 0;
            illumination = 0;
        }
        #endregion

        #region ToJson method
        /// 
        /// Extension method to convert object to JSON format
        /// It uses all properties with [DataMember] above it
        /// 
        /// 
        public string ToJson()
        {
            DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(SensorMessage));
            MemoryStream ms = new MemoryStream();
            ser.WriteObject(ms, this);
            string json = Encoding.UTF8.GetString(ms.ToArray(), 0, (int)ms.Length);

            return json;
        }
        #endregion
    }
}


5b) ToJson
The ToJson method in this class automatically returns a JSON message with all class properties that have [DataMember] above it. Also make sure to add [DataContract] above the class. It will generate the following message that can be read by the event Hub:
{"altitude":2.94,"decibel":7.43,"doorOpen":0,"humidity":93.97,"illumination":85.00, "measurementTime":"\/Date(1468490582115)\/","motion":0,"pressure":98.46, "sensorName":"Joost","temperature":89.49,"vibration":1}

Tip: try to keep the message as small as possible by using small columnnames or codes instead of large text values.

6) Main code
Locate the MainPage.xaml(.cs) file in the Solution Explorer and open the code behind it (the C# code). We have added some usings and methods which you need to add to your file. After that you can customize the code to your sensors/situation. In the following posts we will show you the sensor specific code for each sensor we will use in our IoT project.
MainPage.xaml.cs















//C# Code
using System;
using Microsoft.Azure.Devices.Client;   // Added
using Windows.Devices.Gpio;             // Added
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;

namespace Sensory
{
    /// 
    /// Main page
    /// 
    public sealed partial class MainPage : Page
    {
        #region Constants for config
        // GPIO PIN CONFIG
        const int PIN_DOORSENSOR = 4;

        // Azure IoT Hub connection string
        // Will be explained in next blogpost
        const string azureConnectionString = "HostName=xxxxxx.azure-devices.net;DeviceId=XXXXXXXX;SharedAccessKey=y1ANJwjTueyzCEBEAliy7MkOQHW5dOWiu6w79HcfvVc=";
        #endregion

        #region MainPage
        /// 
        /// Method that execute on startup
        /// 
        public MainPage()
        {
            this.InitializeComponent();

            // Initialize sensors
            InititializeDoorSensor();

            // Add timer event to collect and send
            // data each x seconds or minutes
            DispatcherTimer sensorTimer;
            sensorTimer = new DispatcherTimer();
            sensorTimer.Interval = TimeSpan.FromSeconds(60);
            sensorTimer.Tick += GetSensorData_Tick;
            sensorTimer.Start();
        }
        #endregion

        #region Collect sensor data
        // Method that is execute each x seconds
        // and collects the data from the sensors.
        // For this example it generates random data
        private void GetSensorData_Tick(object sender, object e)
        {
            // Create sensorMessage object and fill all
            // meassi=urements (with random data for now)
            SensorMessage sensorMessage = new SensorMessage("mySensor");
            sensorMessage.Illumination = randomDecimal();
            sensorMessage.Temperature = randomDecimal();
            sensorMessage.Humidity = randomDecimal();
            sensorMessage.Pressure = randomDecimal();
            sensorMessage.Altitude = randomDecimal();
            sensorMessage.Decibel = randomDecimal();
            sensorMessage.DoorOpen = randomInt(0.5);
            sensorMessage.Motion = randomInt(0.3);
            sensorMessage.Vibration = randomInt(0.1);

            // Send the data to the Azure IoT Hub
            SendMessage(sensorMessage.ToJson());
        }
        #endregion

        #region Initialize sensor
        // GPIO objects
        private GpioPin doorSensorPin;
        // Setup doorsensor
        private void InititializeDoorSensor()
        {
            // add sensor code
        }
        #endregion

        #region Send data to Azure IoT Hub
        // Object for sending data to azure
        private DeviceClient deviceClient = DeviceClient.CreateFromConnectionString(azureConnectionString);
        /// 
        /// Sends JSON message to Azure IoT Hub
        /// 
        /// The JSON message
        public async void SendMessage(string message)
        {
            // Send message to an IoT Hub using IoT Hub SDK 
            try
            {
                var content = new Microsoft.Azure.Devices.Client.Message(System.Text.Encoding.UTF8.GetBytes(message));
                await deviceClient.SendEventAsync(content);

#if DEBUG   // DEBUG INFO
                System.Diagnostics.Debug.WriteLine("Message Sent: {0}", message, null);
#endif
            }
            catch (Exception e)
            {
#if DEBUG   // DEBUG INFO
                System.Diagnostics.Debug.WriteLine("Exception when sending message:" + e.Message);
#endif
            }
        }
        #endregion

        #region Random Methods for test
        private int randomInt(double chance)
        {
            // Make sure each call returns a random number
            Thread.Sleep(1);

            Random test = new Random(DateTime.Now.Millisecond);

            if (test.NextDouble() >= chance)
            {
                return 1;
            }
            else
            {
                return 0;
            }
        }

        private decimal randomDecimal()
        {
            // Make sure each call returns a random number
            Thread.Sleep(1);

            Random test = new Random(DateTime.Now.Millisecond);
            return Math.Round(Convert.ToDecimal(test.NextDouble() * 100), 2);
        }
        #endregion
    }
}

6b) Random
The two random methods are temporary methods that return random data to test the application without the sensor specific code. This allows us to continue in parallel with the Azure IoT Hub, Stream Analytics and PowerBI.

7) Active Solution Platform
For building this project we need to switch to ARM (Acorn RISC Machine) instead of x86/x64. This is the processor architecture used on machines like the Raspberry Pi.
Switching Solution Platform to ARM











8) Deploying to Raspberry Pi
When your project is ready you can deploy it to your Raspberry Pi for debugging and testing. First you have to get the IP address of the Raspberry Pi device. The Windows 10 IoT Core Dashboard application is a good way to find your device(s). You will get the best result using a wired connection. When using WiFi the list below will sometimes stay empty.
Windows 10 IoT Core Dashboard

















You can also use Visual Studio to find the device. Go to the properties of your project. Then go to the Debug page and click on the Find button. On this page you can select the name or IP address from your device for deploying your code.
Find your IoT device
















Now you can use the green start button to deploy and debug your code on the remote device, but the first time you first need to select 'Remote Machine'. The first deployment could take a couple of minutes!
Start debugging









Now watch the output window in Visual Studio to see if your program is working. It's good to add a lot of Debug.WriteLine rows in de beginning to see whether everything works.
Debug output





















Conclusion
A lot of work to start with, but if the basic works you can start customizing the code for your sensors. You will need some patience because finding your Raspberry Pi sometimes need a couple of attemps (wired is more ofted successful) and the first deployment takes a couple of minutes. Don't worry if it takes 3 minutes.
In the next  few blogposts we will show the specific sensor code and show you how to create the Azure IoT Hub connection string.

Saturday, 9 July 2016

IoT Adventure: 2 - Preparing development machine

Case
We have connected multiple sensors (Door/window, Noise, Motion, Temperature, Humidity, Pressure, Altitude, Light and Vibration) to our Raspberry Pi and now we want to read those sensor values. But which software do we need and are there any prerequisites before we can start?

Solution
There are multiple programming languages and tools you can use to read your sensors, like .NET and Python. We will use C# and Visual Studio 2015 in our project, but before we can start we have to prepare our development environment.

1) Windows 10 Developer Mode
We are using a Windows 10 installation in Developer Mode. You can activate that mode by going to the settings windows of your computer. Go to 'Update & security' and then to 'For developers'. There you can choose 'Developer mode'.
Turning on Developer Mode in Windows 10















2) Visual Studio 2015
Install Visual Studio 2015 (Community, Professional or Enterprise) and make sure that you atleast use Visual Studio Update 3. Also make sure to install 'Universal Windows App Development Tools' and 'SQL Server Data Tools (SSDT)'. You can skip SSDT if you have already installed that. After installation, check your Visual Studio Version number in the Help menu. It should be at least 14.0.25208.00
Universal Windows App Development Tools & SSDT
Universal Windows App Development Tools & SSDT


















3) Windows IoT Core Project Templates
Next item to install are the project templates for Windows IoT Core Applications. The download is a .vsix file. Just double click it to execute it. You can also find it in Visual Studio in the tools menu under Extensions and Updates.
Windows IoT Core Project Templates
Windows IoT Core Project Templates



















4) Windows Azure SDK
Install Windows Azure SDK version 2.9. We need this SDK to connect to the Azure IoT Hub to send the sensor data to Azure.
Azure SDK
Azure SDK

















5) Windows IoT Core Dashboard
Install the Windows IoT Core Dashboard. This is a useful tool to find and connect your IoT devices.
Windows 10 IoT Core Dashboard
















6) Azure IoT Hub Device Explorer
Install Azure Iot Hub Device Explorer [SetupDeviceExplorer.msi]. We need this tool to make a connection to the IoT hub.
Device Explorer




















Conclusion
There are a lot of tools and addons to install before you can start programming. Downloading and installing all items could easily take two hours. But a good preparation saves you time later on. In the next blogpost we will explain how to create a Visual Studio project that can read the sensor data and send it to the Azure IoT hub.


Friday, 8 July 2016

IoT Adventure: 1 - Setting up Raspberry Pi with sensors

Case

For our Internet of Things experience we use the Raspberry pi 2. With the raspberry it’s easy to connect different sensors and program them to work. There are many connections on the Raspberry, from HDMI to USB and GPIO. 
To connect the sensors to the Raspberry you can use the GPIO (General Purpose Input/Output) pins on the Raspberry board. In figure 1 you will see a picture of the Raspberry pi with the GPIO headers marked.


Figure 1 : Raspberry Pi














You can find more information about the GPIO on the following wiki page : GPIO header.

Solution
Here you will find a brief description about the layout of the GPIO header, including some explanation of the function of the different pins. Also a simple example how to connect the pins with a breadboard.
Once you understand the basics behind the GPIO you will notice that it is pretty easy to work with.
You can use every GPIO PIN on the board as an Input or Output connector. For example you can use one of the pins to receive temperature information from a sensor as input to the Raspberry Pi or you can use a pin as output to turn on or off a LED.

In order for you to make use of these pins as input or output channels you have to write code in programming languages such as Python, C#., VB etc and upload it to the Raspberry Pi. You will find more about the programming in other blogposts.


1) Layout 
There are different Raspberry pi models, we are using the Raspberry pi B/B+ model. For more information about the different models see: Raspberry models
In figure 2 you will find the pin connections (GPIO Header) of the Raspberry Pi 2 B/B+
tip: Print this layout on paper and keep it near your Pi

Figure 1: Pin layout
Figure 2: Pin layout

There are 17 GPIO pins available, all the green pins in figure 1 are standard GPIO pins. But there are more pins.




































Below a brief summary of the remaining pins:
  • PIN 2 and 4 have an output of 5 V
  • PIN 1 and 17 have an output of 3,3V
  • PIN 3 and 5 are used for the I2C protocol (see wiki: I2C Protocol ) (SDA stands for Serial Data and SCL stands for Serial Clock).
  • PIN 19, 21, 23, 24, 26 are used for the Serial Peripheral Interface (SPI) BUS (see wiki: SPI Bus)
  • PIN 8 and 10 are used for Universal Asynchronous Receiver/Transmitter (UART) (see wiki: UART). It is possible to send data from multiple sensors using 1 wire (serial) instead of using 1 wire per sensor.
In the beginning all the different functions of the pins can be a bit daunting, but in our experience it suffices to use the common GPIO pins for the most sensors.


2) Wire Colours
In our posts we will be using the following wire colour standards;
- 3,3V pins: orange wire
- 5V pins: red wire
- GND pins: black wire
- Common GPIO pins: green wire
- I2C pins: blue wire
- UART pins: white wire
- Overall pins: yellow wire
- between the different components: Grey

It’s not necessary for you to use all the different colors I just described, you can use any color (especially if you don’t have all the colors).

3) Breadboard?
When you bought a Raspberry Pi Starters Kit, most likely they included a breadboard. A breadboard is an easy way to plug and unplug LED’s, Resitors, Sensors and more without soldering. They layout of a breadboard is pretty straight forward (if you know what you are doing). 
 
Figure 3 shows the breadboard as it is included in the Starters kit.
Figure 3: Breadboard









In the picture the horizontal line (row) is marked as number 1. The horizontal line is used to connect the 5 or 3,3 Volt pins to the + row and the GND (Ground) pin to the - row. Connecting the pins to these horizontal lines enables you to power the entire board with 5 or 3,3 Volt.


The vertical lines (depicted as the number 2 marker in Figure 2) are to be used to connect the sensors to either the horizontal lines to power the sensor or directly to the GPIO pins on the Raspberry Pi. With the horizontal lines it is usefull to know that it doesnt matter how high or low you connect the wires. The entire vertical line is used to connect to the horizontal line or the GPIO pins on the Raspberry Pi.

The breadboard is split in two, so you can use e.g. one side for the 3,3V+ and the other side for the 5V


4) Connection a sensor
Now that I have described the GPIO header layout and the breadboard we can try something simple like connecting a LED to our Raspberry Pi. In this post we will only explain how to properly connect the sensor and not how to write the correct code to activate/deactivate the sensor. The code will be explained in other posts per sensor using C#, which is our choice of code.

Figure 3 depicts how to use a 3,3V GPIO pin and a GND pin and connect it to the horizontal lines.
- GPIO pin 1 (3,3V) is connected to the breadboard pin using a orange wire
- GPIO pin 6 (GND) is connected to the breadboard using a black wire
- A 560 Ohm Resistor is connected via pins 10C and 14C to protect the power supply to the LED
- A Red led is connected via pins 14E and 15E

When you turn on the Raspberry Pi the LED will light up.

Figure 4: Breadboard


Figure 5 is a power schema of how you can connect the components to the Raspberry. In my opinion the best thing to do is first drawing a power schema. And then connect the components to the breadboard. By drawing a power schema you can see where you have to put the components and calculate e.g. the resistors.
Figure 5: Power schema



Conclusion
In this case I described the GPIO Header layout and how to connect a simple component like the LED. 
Figure 6 is the end resulate of this case.This is just the basic, from here you can build your empire of sensors ;)  
In other blogposts we will describe how to program the sensors. 


Figure 6: the end




















Note
For the schematics we are using Fritzing.

Monday, 4 July 2016

IoT Adventure

Case
What is Internet of Things (IoT)?

Solution
To discover the possibilities of IoT we started our own IoT project a few weeks ago. In this blog post series we will explain what we are doing and how we are doing it. The goal of the project is to create a couple of sensor stations, hang them in our office and create reports with the sensor data in PowerBI. No business case, just testing and learning.
IoT project overview










Sensor stations
We started by buying a couple of Microsoft IoT Packs for Raspberry Pi 2. This is a great starter kit to learn more about IoT with Microsoft technology. It has a couple of sensors in it like a Photo Cell and a Temperature, Pressure & Humidity sensor [details].
Microsoft Internet of Things Pack for Raspberry Pi 2




















But to make it even more challenging/interesting we ordered a couple of extra sensors:
- Magnetic contact switches (door sensors) [details]
- Fast Vibration sensors (easy to trigger) [details]
- Motion sensors [details]
- Electret Microphone Amplifiers (to detect sounds) [details]
In total we spent around € 150,- per sensor station (including the Raspberry Pi).
Extra sensors






















Within the following weeks we will blog about the following subjects of our IoT project:
- Setting up the Raspberry Pi with the sensors
- Reading the sensors with .NET
- Sending the data to the Azure IoT Hub
- Setting up Stream Analytics
- Creat reports in PowerBI

Joost, Mark, Menno and Ricardo


IoT Adventure: 1 - Setting up Raspberry Pi with sensors
IoT Adventure: 2 - Preparing development machine
IoT Adventure: 3 - Create Visual Studio Project for sensors
IoT Adventure: 4 - Sending sensor data to Azure IoT hub

Related Posts Plugin for WordPress, Blogger...