Overview

They say "innovation starts at home" (well, ok I changed charity with home. But then!)

Every DiY enthusiast at least try to create some solutions or product or application that he can use in his home. Therefore, home automation has been one of the popular areas of Embedded developers and DiY guys.

When the early IoT adaptation started, the companies reached out to these DiY guys, commonly referred as the makers community. So, the makers community was the one who got their hands into early IoT devices. Needless to say that home automation was again at the core of innovation, development and early solution. Being a DiY guy and my wife Moumita being a circuit makergirl, it was not a hard choice for us to decide which project we would chose if we had to chose one in DiY category. It had to be home automation.

People with a liking for history may know that in the last IoT tutorial held by CodeProject, I worked mainly with Arduino, creating my own services and working with predominantly HTTP protocol for connectivity. This time, we have teamed up to create a bigger and better solution for more critical problems.

Moumita has always been developing all the circuits I ever worked with. But this year, she has started writing and coding. That has given me an immense amount of strength to take up more complicated projects.

So, when we decided to work on Home automation, we had to brainstorm about a good product which we would like to work on to. We first detailed out the aspects of home automation.They are:

Home Monitoring - deploying sensors around the home for observing various parameters like light, temperature
Home Control - controlling AC, Fan, Light remotely
Home Automation - automating devices like switching on the bulb when it is dark
Home Security - protecting the home from thieves and intruders
Home Safety - making the home safer by preventing it from fire

Instead of working with a single concept, we decided to build a solution that demonstrated each of these aspects. While writing a beginners' tutorial, it is always a good idea to work into something that covers almost complete areas of a domain.

Therefore, in this tutorial, we are going to share with you a powerful framework of Home Automation, monitoring, Safety and Security from a single app.

In the past, GSM based home control system has been USED for remote monitoring and control. Generating a SMS for every control is not feasible as messages are costly. Also, GSM doesn't provide any visualization support which means that the variation in data cannot be observed in graphical format.

Internet of things has been of the recent growing trends in hardware technology. IoT enables the devices to connected to the internet. Therefore controlling and monitoring can be performed over a large range of devices like mobiles, tablets, PC and so on. Also, as this tutorial was planned for CodeProject's IoT tutorial contest, IoT had to be the core part of it.

Therefore, we extend our framework with a complete cloud integration with Google and ThingSpeak, a messaging gateway integration with Mqtt and call this framework HASSI.

Detailed Functionality

The project provides authentication using Face Recognition based on Eigen based PCA face recognition. Lock and Unlock commands of home are connected to face authentication. If an unauthenticated user attempts to open the door without face authentication, a gmail notification is generated with his face photo. In case of a fire at home, the system also generates a gmail message which can be further channelized through IFTTT.

Therefore, the work has three features:

Home Monitoring: In the home monitoring system, we have a temperature sensor, the temperature sensor value can be remotely checked with a think speak channel. So the values can be seen as a graph remotely.
Home Safety and Alarm system: When the temperature sensor value exceeds above 40, then the notification is generated to your gmail and also locally. By receiving this mail, you can take the appropriate decision, you can further forward this mail to many other channels using IFTTT.
Home Control: where we implement Lighting and fan control system (small DC fan and small LED series light!) using our mobiles with Mqtt protocol.

Home Security

Fig: Block Diagram of the project

The HASSI framework offers opening of the door lock using your face. So, user needs facial verification in order to be able to open the lock. Any intruder trying to unlock the home without facial verification will be snapped and the picture along with alarm will be sent to owner's gmail account.

We are going to use C# as the frontend for user interaction with the physical world and physical world's gateway to the cloud. We will use Arduino C program in Arduino to communicate with this C# UI with serial communication.

Now these are the ways in which the software works.

When you sit in front of the system, the software is going to authenticate user through face recognition. For that, you need to register your face and you also require administrative password. You give the correct administrative password, then only your face is registered. It shows that you are in the frame. So immediately after administrative authentication, you can go with facial verification. Facial recognition is required for opening your home door lock.

If your face is not recognized, then the software is displayed unknown message. The software is giving a voice message that some unknown person is trying to open the door. And the software also sends a message to gmail id by adding the photo of the unauthorized person. Therefore, that software not only generates a voice message but also takes a photograph at the same time.

Before you start any home control system through the software, you need to give the Arduino port. Arduino is connected with the PC with the virtual port. Different virtual port will be there, but the software will show the connected port.

When you start monitoring the temperature, the software will show the temperature with a simple linechart right on your C# UI form.

The data is also available in the ThingSpeak channel, which displays temperature values with a graph so that you can monitor it remotely from anywhere on the world.

If the sensed temperature crosses 40 degree Celsius mark (which can be simulated by taking a candle stick and holding it nearer to the temperature sensor), then software displays the message locally on the software and also a message will be sent to the gmail account. A buzzer is triggered locally so that anyone at home knows about the event.

We are communicating with the MQTT protocol. It is a publish-subcribe based protocol. You can publish a message from anywhere into a message broker, which is asynchronously pushed to all the peers which have subscribed to the channel where message is published.

Our C# client is going to receive the message and based on that, it sends a signal to Arduino about On and Off control of light, fan and open and close control for the door.

Hardware Fabrication

Our work is a combination of hardware and software. The hardware component specifications are as follows:

LED light array. It is 27 watt so it requires a battery connection for driving it. We use 12V 4.5 Amps battery to drive LED array.
In the fan module, we use CPU fan.
One Gear motor for synthesizing open and close of the gate.
To generate notification, we use temperature sensor LM 35 and one buzzer.
Set of relays to control light and fan.

Circuit Schematic

Figure: Complete Circuit Diagram (Click to enlarge)

You can learn about rigging up a 12v power supply from here.

We use the Relay board that we had developed for Urobi.

The above diagram is the main circuit of the project. Arduino is connected to a relay module. A Bulb and a fan is controlled through relays. The electrical unit is connected to main line through relay. When relay is active, the electrical unit is switched on, drawing more current. When the relay is turned off, the electrical load draws no current. LM 35 is connected to A5 for sensing temperature value. The sensed value is sent to PC through serial port which is mitigated to ThingSpeak channel.

The first relay is for controlling the direction of the lock gate. Its input is +-12v. When this relay is turned off, its output is +12 and when turned on, output is -12v. The output of this relay is connected to second relay. This relay's output is connected to lock. So if both relay 1 and relay 2 are active, gate will move anticlockwise, if only relay 2 is active, gate 2 will move clockwise.

relay3 and 4 drives a light and fan respectively. These relays are given input supply from 12v 4.5A battery.

As we have used 12v relay which cannot be driven by Microcontroller's 5v supply, relays are coupled through MCTE2E optocoupler.

Hardware modules are as follows:

Figure: 12 volt power supply

Figure: Relay board

Figure: LED light array, LM 35 and buzzer

Coding

Arduino Code

Let us first look at our Arduino sketch. bulbPin and fanPin are the pins with which the relays corresponding to bulb and fan are connected. tSensor is the analog pin where temperature sensor LM 35 is connected. lockPin is the relay connected to door lock. dpdt is the directional control relay which either outputs +12v or -12v depending upon its state. When both dpdt and lockPin are made high, motor with door lock moves in clockwise direction. When dpdt is off and lockPin is high, lock motor rotates in anti clock wise direction, simulating lock close condition.

As Arduino would be receiving serial data from our C# client, Serial.begin() is called to initialize serial port communication. When C# sends data to Arduino, Serial.available() will be non zero, if so, character values are converted to numbers. For example, PC sends command '1' which is converted to 1 by Arduino by subtracting 48 from the ASCII value of the received character.

Commands 1 and 2 correspond to Bulb High and Low respectively.

Commands 3 and 4 correspond to Fan High and Low respectively.

Commands 5 and 6 correspond to Lock and Unlock respectively. Note that when either of locking or unlocking operations are initiated, the motor rotates for about 300ms and then stops. This is to calibrate only a small movement of the motor for locking and unlocking.

Command 0 corresponds to switching off all the home equipments. It is helpful when the user is going outside. He can close all his electrical appliances with a single command.

int bulbPin=9;
int fanPin=11;
int tSensor=5;
int buzzPin=8;
int dpdt=13;
int lockPin=12;
int ref=0;

void setup() {
  analogReference(INTERNAL);
  // initialize digital pin 13 as an output.
  Serial.begin(19200);
  pinMode(bulbPin, OUTPUT);
  pinMode(fanPin,OUTPUT);
  pinMode(buzzPin,OUTPUT);
  pinMode(dpdt,OUTPUT);
  pinMode(lockPin,OUTPUT);
  digitalWrite(lockPin,LOW);
  digitalWrite(dpdt,LOW);
  digitalWrite(bulbPin,LOW);
  digitalWrite(fanPin, LOW);
  digitalWrite(buzzPin,LOW);
  Serial.println(ref);
}

// the loop function runs over and over again forever
bool on=false;
float avEn=0;
int count=0;
void loop() {

  int reading=analogRead(tSensor);
  float tempC = reading / 9.31;
  avEn=avEn+tempC;
  //Serial.println(tempC);
  count++;
  if(tempC>40)
  {
  // Serial.println("High");
   digitalWrite(buzzPin,HIGH);
 }
 else
 {
  digitalWrite(buzzPin,LOW);
 }
  if(Serial.available())
  {
   int r=Serial.read();
   r=r-48;
   //1= light on 2=light off 3 fan on 4 fan off 0 all off
   switch(r)
   {
    case 1: digitalWrite(bulbPin,HIGH);
    break;
    case 2: digitalWrite(bulbPin,LOW);
    break;
    case 3: digitalWrite(fanPin,HIGH);
    break;
    case 4: digitalWrite(fanPin,LOW);
    break;
    case 5: // Lock
    digitalWrite(dpdt,HIGH);
    digitalWrite(lockPin,HIGH);
    delay(3000);
    digitalWrite(lockPin,LOW);
    digitalWrite(dpdt,LOW);
    break;

    case 6: //unlock 
    digitalWrite(lockPin,HIGH);
    delay(5000);
    digitalWrite(lockPin,LOW);
    break;
    case 0: digitalWrite(fanPin, LOW);
            digitalWrite(bulbPin,LOW);
     break;
   }
  }
  
  delay(100);              // wait for 100 ms
}

C# Code

We have developed our code on top of this awesome RealTime Face recognition article of Sergio.

Figure: Form Design (Click to zoom)

When the form is loaded, our Mqtt client is connected with global mqtt broker iot.eclipse.org (or test.mosquitto.org). If the Mqtt client named mc is successfully connected to broker, then it subscribes to a channel by name rupam/MyHome (you must change the name as many readers may be trying to use the same channel simultaneously, causing unexpected behaviour to your commands). So, whenever a command is received by the mqtt, clientmc_MqttMsgPublishReceived event handler is called.

We provide manual buttons for all the operations: Bulb On, Bulb Off, Fan On, Fan Off, All Off, Lock and Unlock.

At the same time, we loop through all the available serial ports and load the port names in the combobox in GroupBox with label "connect to Arduino".

MqttClient mc = null;
    Color ac = Color.Blue;
    string topic = "rupam/MyHome"; 
private void FrmPrincipal_Load(object sender, EventArgs e)
        {
            string[] ports = SerialPort.GetPortNames();
            for (int i = 0; i < ports.Length; i++)
            {
                cmbPort.Items.Add(ports[i]);
            }
            try
            {
                mc = new MqttClient("test.mosquitto.org");
                mc.Connect("RUPAM");
                mc.Subscribe(new string[] { topic }, new byte[] { (byte)0 });
                mc.MqttMsgPublishReceived += mc_MqttMsgPublishReceived;
                label5.Text = "Connected to test.mosquitto.org in channel rupam/MyHome";
                label5.BackColor = Color.Green;
            }
            catch
            {
                label5.Text = "No Connection with Server-Check Internet Connection";
                label5.BackColor = Color.Red;
            }
            ac = btnConnect.BackColor;
        }

Once the serial port is connected, you can test the operations by clicking these buttons before trying out Mqtt based remote control. When a particular button is clicked, its color is changed to Green. When the other button is clicked corresponding to this operation, button's color is replaced with the original color. So when Bulb on is clicked, Bulb off becomes gray and bulb on is green, when bulb off is clicked, that button will be green and button with label bulb on will be back to its actual color.

private void btnBulbOn_Click(object sender, EventArgs e)
        {
            serialPort1.WriteLine("1");
            btnBulbOn.BackColor = Color.Green;
            btnBulbOff.BackColor = ac;
        }

        private void btnBulbOff_Click(object sender, EventArgs e)
        {
            serialPort1.WriteLine("2");
            btnBulbOn.BackColor = ac;
            btnBulbOff.BackColor = Color.Green;
        }

        private void btnFanOn_Click(object sender, EventArgs e)
        {
            serialPort1.WriteLine("3");
            btnFanOn.BackColor = Color.Green;
            btnFanOff.BackColor = ac;
        }

        private void btnFanOff_Click(object sender, EventArgs e)
        {
            serialPort1.WriteLine("4");
            btnFanOn.BackColor = ac;
            btnFanOff.BackColor = Color.Green;
        }

        private void btnAllOff_Click(object sender, EventArgs e)
        {
            serialPort1.WriteLine("0");
            btnFanOff.BackColor = ac;
            btnFanOn.BackColor = ac;
            btnBulbOn.BackColor = ac;
            btnBulbOff.BackColor = ac;
        }

For both Lock and unlock buttons, user face verification is needed. Once user's face is verified, Label4 will have a tag name with that of the verified username, else it will have a tag called Unknown. In case user's face is unknown, a photo will be captured and will be sent to an entered email along with the face in front of the camera. Gmail Message is sent with attachment using a method called SendMail in EmailSend class.

You need to set your mail id and password in EmailSend's SendMail method to be able to send a mail from your mail id.

You need to also follow the official gmail tutorial to allow less secured apps to access your mail id:

 public class EmailSend
    {
        static Stopwatch stTime = new Stopwatch();
        public static void SendMail(string emailTo, string subject,
                                    string body,string attachmentPath)
        {
            if (!stTime.IsRunning || stTime.Elapsed.TotalMilliseconds > 10000 || 
                                     subject.Contains("Alarm"))
            {
                string emailFrom = "YourMailID@gmail.com";
                string password = "yourgmailPassword";
                string smtpAddress = "smtp.gmail.com";
                int portNumber = 587;
                bool enableSSL = true;

                using (MailMessage mail = new MailMessage())
                {
                    mail.From = new MailAddress(emailFrom);
                    mail.To.Add(emailTo);
                    mail.Subject = subject;
                    mail.Body = body;
                    mail.IsBodyHtml = true;
                    // Can set to false, if you are sending pure text.
                    if (attachmentPath.Length > 2)
                    {
                        mail.Attachments.Add(new Attachment(attachmentPath));
                    }

                    using (SmtpClient smtp = new SmtpClient(smtpAddress, portNumber))
                    {
                        smtp.Credentials = new NetworkCredential(emailFrom, password);
                        smtp.EnableSsl = enableSSL;
                        smtp.Send(mail);
                    }
                }
                if (!stTime.IsRunning)
                {
                    stTime.Start();
                }
                else
                {
                    stTime.Reset();
                    stTime.Restart();
                }
            }
        }
    }

 private void btnLock_Click(object sender, EventArgs e)
        {
            if (!label4.Text.Contains("Unknown"))
            {
                serialPort1.WriteLine("5");
                btnUnlock.BackColor = ac;
                btnLock.BackColor = Color.Green;
                grpDoor.Enabled = false;                
            }
            else
            {
                //// Send a Mail
                (new System.Speech.Synthesis.SpeechSynthesizer()).SpeakAsync
                ("Unauthorized Attempt!!!");

                String fileName = ".\\Unknown\\Image_" + DateTime.Now.ToString
                       ("yyyyMMdd_hhmmss") + ".jpg";//Save the filename first on 

                imageBoxFrameGrabber.Image.Save(fileName);
                EmailSend.SendMail("receiversmailID@gmail.com", 
                "Unknown Person", "Unknown person attempting unlock " + 
                 DateTime.Now, fileName);
                ///////
            }            
        }

The serial port object by name serialPort1 is added with an event handler called serialPort1_DataReceived. Whenever there is a data in the port, it is captured through a delegate ( because it is an independent thread which cannot use UI elements).

 private void serialPort1_DataReceived(object sender, SerialDataReceivedEventArgs e)
        {
            string s = serialPort1.ReadLine();
            this.BeginInvoke(MyDlg, s);
        }

Note that Arduino only sends Temperature data to C# application. Therefore, only temperature data is received by this application. This data is sent to ThingSpeak channel and is analyzed for threshold >40. If the temperature exceeds the threshold, an email is sent to the user.

void Display(string s)
        {            
            try
            {
                double en = double.Parse(s);
                chart1.Series[0].Points.AddY(en);
                if (en > 40)
                {
                    EmailSend.SendMail("receiversMailId@gmail.com", 
                    "Fire at home Alarm","Fire at Home Alarm "+DateTime.Now ,"");
                }
                //string s = UpdateThingSpeakData(new string[] { "40", "40" }, 
                //"C606GJW98AWMN9P9");
                string s1 = UpdateThingSpeakData(new string[] 
                            { en.ToString() }, "MB2RQE6KCT9D670V");
            }
            catch
            {
            }
        }

In Mqtt event handler, we check for the command, convert the character command to number and then appropriately click the corresponding button by calling button click event handlers.

void mc_MqttMsgPublishReceived(object sender, 
     uPLibrary.Networking.M2Mqtt.Messages.MqttMsgPublishEventArgs e)
        {
            //throw new NotImplementedException();
            //MessageBox.Show(GetString(e.Message));
            string s = "";
            try
            {
                byte b = e.Message[0];
                //e.Retain = false;
                s = (b - 48).ToString();
            }
            catch
            {
                return;
            }
            this.Invoke((MethodInvoker)delegate
            {
                if (s.Equals("1"))
                {
                    btnBulbOn_Click(btnBulbOn, new EventArgs());
                }
                if (s.Equals("2"))
                {
                    btnBulbOff_Click(btnConnect, new EventArgs());
                }
                if (s.Equals("3"))
                {
                    btnFanOn_Click(btnFanOn, new EventArgs());
                }
                if (s.Equals("4"))
                {
                    btnFanOff_Click(btnFanOff, new EventArgs());
                }
                if (s.Equals("0"))
                {
                    btnAllOff_Click(btnAllOff, new EventArgs());
                }
                if (s.Equals("5"))
                {
                    if (grpDoor.Enabled)
                    {
                        btnLock_Click(btnAllOff, new EventArgs());
                    }
                    else
                    {
                        (new System.Speech.Synthesis.SpeechSynthesizer()).SpeakAsync
                        ("Unauthorized Attempt!!!");
                    }
                }
                if (s.Equals("6"))
                {
                    if (grpDoor.Enabled)
                    {
                        btnUnlock_Click(btnAllOff, new EventArgs());
                    }
                    else
                    {
                        (new System.Speech.Synthesis.SpeechSynthesizer()).SpeakAsync
                        ("Unauthorized Attempt!!!");

                        String fileName = ".\\Unknown\\Image_" + 
                        DateTime.Now.ToString("yyyyMMdd_hhmmss") + 
                        ".jpg";//Save the filename first on                 
                        
                        imageBoxFrameGrabber.Image.Save(fileName);
                        EmailSend.SendMail("maneeshanmc27@gmail.com", 
                        "Unknown Person", "Unknown person attempting unlock " + 
                         DateTime.Now, fileName);
                    }
                }
            });
        }

You can download MyMqtt App from Google play. Open settings in the app and give broker address as iot.eclipse.org or test.mosquitto.org (whatever you have set in the form). Now publish to Channel rupam/MyHome.

You need to publish 1 for bulb on, 2 for bulb off and so on.

Results

First to become authentic user needs to register his name on the software through face recognition. It requires an administrative permission. If permission granted, then user is considered as authorized.

Figure: First step of user registration

Figure: admin password for user face registration

Figure: Admin permission granted

Figure: User registered

Now Arduino is connected with the system with a virtual line. Select the port.

Figure: Selection of Arduino port

Figure: Started home control temperature unit

Figure: Temperature data also available in the thinkspeak

Now to increase the temperature sensor value, you can hold a fire stick nearer to the LM-35 (Temperature sensor).

Figure: Increasing the temperature sensor value

Figure: Temperature is increasing showing on the graph

When temperature will increase and if it crossed the threshold value, then the notification will send to gmail id.

Figure: gmail notification for the temperature notification

Now if a unknown person wants to open gate, then the software is going to display unknown. Unknown persons are those who have not registered their name.

Figure: Display unknown to user

Not only is display unknown to the software but also the photo of that unknown person will be sent to the gmail address so that user can take some action.

Figure: Email with unknown person photograph

Conclusion

Home automation has been one of the most popular areas of research for both hobby projects as well as for industrial IoT development. However, home automation has always been considered separately in comparison to home monitoring and security system. In this project, we have combined safety, security, monitoring and control system with IoT. The safety system ensures that in case of a fire, user is always notified as early as possible. The security system ensures that unauthorized face cannot break into the home, controlling system ensures that user can control lights and fans from his mobile and monitoring system ensures that user monitors his home temperature remotely which is helpful if he has plants at home which are sensitive to temperature. Results show that the system meets all the prerequisites and generates real time data and alert. This system can be further improved by incorporating other features like automatic climate control based on temperature monitoring, automatic lighting control based on light value monitoring.

So, even though HASSI is simple, low cost, it is still extremely powerful. You can build more complicated home automation projects based on this framework.

HASSI- Home Automation, Safety and Security with IoT Using Arduino