Raspberry PI: Lights, Camera, Action, and Off-site Storage

Also learn

There are several components of the tutorial that can be used individually, some of which are important skills on any Linux system. If you jump directly to these, be sure to install the applicable additional packages first. Quickly jump to these sections from the following list:

• use memory as if it were disk space, reducing disk access (any Linux system)
• public key authenticated ssh and tunneling VNC (any Linux system)
• using the PI camera (PI only)
• controlling GPIO pins with simple bash commands and scripts (Pi only)
• and several others

Estimated time to complete

• Beginners: a few days
• Experienced: a few hours

Target capabilities

• Collect movie clips when motion is detected
• PIR based, enabling motion detection in the dark
• Turn on a room light when recording a movie
• Transfer the movies to a secure, offsite location for storage
• No accumulating disk space usage
• Easy on SDD access
• Stable service for long periods without a reboot
• Automatic recovery after a power failure
• Compact, lightweight, low power
• Simple programming with low overhead
• Entirely user controlled with secure remote administration
• Periodic reporting of external IP address
• Opensource, fully-transparent hardware and software with no black-box firmware updates to potentially brick the device
• 100% re-taskable in an indefinite number of ways if you get bored
• Fun
• Inexpensive

My first build was five month ago. The unit has been faithfully serving me from 700 miles away, without a reboot for over 2 months... except for a power failure, which it recovered from. Twice a day, it sends me a friendly reminder that it's on the job by sending me a movie together with its external IP address.

Background

Prerequisites

Materials

Here are the parts used in the project:

• Raspberry PI 2, Model B
• PI camera board
• PI power supply
• 8Gb SDD with NOOBS
• PIR (motion) sensor
• Powerswitch Tail II
• Pig-Tail Cables
• A few resistors, LEDs, and a switch for initial testing
• A usb mouse/keyboard for setup only
• An HDMI connector for setup only (I used: HMDI-to-VGA adaptor)

You can buy a less expensive PI and SDD, perhaps saving as much as $30, but the PI 2 – with its 1 Gb of memory – makes for a robust project... and you can't beat the simplicity of the NOOBS setup. Once setup, the system can be entirely administered by ssh, so the keyboard, mouse, and HDMI connector can be used elsewhere. I prefer a hard-wired network interface for this application because it is substantially more reliable on a remote system that runs unattended, perhaps for months at a time.

This monitoring cam is best run as a fire-and-forget device: when a movie or picture is collected, it is consumed in some way and is overwritten by subsequent movies. (Naturally, you can change this behavior.) Without fire-and-forget, the SDD could fill up (or be compromised) in the interval of time between user interventions. To consume the movie, the device is programmed to transfer the information offsite when an event occurs. In this project, I'll use FTP to transfer the movie to another computer. If you don't have an FTP account somewhere, you can set one up on another computer. Alternatively, modify the code to send the movies to a cloud service or use scp, email, a network share, etc.

I have tagged this tutorial as Intermediate only because you must first be familiar with your Raspberry PI, the Raspian OS, and very basic electronics. Install and work with the Raspian OS, configure/use the camera, use the terminal, etc. Shortly after your first yawn, you will be ready to move ahead.

A note about notation: in some of the commands below, I have used the notation <something>, eg pi@<Pi's IP>. When you see this, it means replace the entire <something> (including < and >) with your corresponding content, e.g., pi@192.168.0.17.

Minimal Raspian OS

While the PI is off, connect the camera, SDD, mouse/keyboard and HDMI monitor, then plug it in. You may already have a working install that is loaded with packages and configurations/customizations, but I recommend that you start with a new SDD and install afresh. During setup:

• change password
• enable boot to desktop
• internationalization>language, keyboard, time zone
• enable camera
• advanced>enable SSH
• advanced>set hostname

Later, after the machine is rebooted, you can use sudo raspi-config to make other configuration changes. You now have a just-what's-needed system and, starting at the same place and taking the same steps, we should end up in the same place. I will follow the process here on my PI as well. When system files are edited, we will first make a backup file using: sudo cp <thePath/theOriginalFile> <thePath/theBackupFile>. If you have to recover, you run: sudo mv <thePath/theBackupFile> <thePath/theOriginalFile>. Finally, don't use sudo unless it's printed here.

Once booted into Rasbian, configure your network with a static IP.

Additional packages

Update the OS and install some needed packages:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install wget lynx ffmpeg mplayer x11vnc omxplayer

Using memory as if it were disk space

To ease the burden on the SDD card, perhaps over years of service, and to increase the efficiency of movie recording we want to use memory for temporary movie storage. Make a place for it:

mkdir ~/volatile

Tell the OS to assign that place to memory:

sudo cp /etc/fstab /etc/fstab.backUp
gksu leafpad /etc/fstab

Carefully, precisely, add the following new line to the end:

tmpfs /home/pi/volatile tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=20M 0 0

Save, quit, and reboot. You should now have a directory named volatile in your home folder that points to memory and that can hold up to 20 MB of file data. To test it, run:

touch ~/volatile/helloMemory

Go to your file manager and see that the new file, named helloMemory, is in the volatile directory. Then, reboot and verify that it is no longer there. If it's gone, then all is well. We will write all of our movies to this volatile folder, deleting them as soon as they are consumed.

SSH and, optionally, tunneled VNC

Being able to ssh into the PI from elsewhere is essential for remote administration. I prefer to tunnel a VNC connection over ssh so I can work with the PI desktop... as though I were sitting there with a mouse, keyboard, and monitor.

The default configuration for ssh is provided in the file /etc/ssh/sshd_config. We'll use it for now but be sure your PI does not expose port 22 to the internet yet (eg via a router that forwards port 22 to your PI's IP address or by a provider/device that treats the PI as demilitarized zone.) Before facing the internet, we need to lock it down better.

On another LAN computer, one that you will be connecting from (called a client), install an ssh client and a VNC viewer. I use Linux on my client and I use open-ssh for my ssh connections and remmina for my VNC viewer. With many Linux distros, open-ssh is already installed by default, so the only command is:

sudo apt-get install remmina 

There are other approaches, including ones that allow connections from Windows computers, but x11vnc with remmina is fast, lightweight, and rich in features. If you use a commercial OS on your client machine, you'll have to do some research (putty is a good, cross-platform ssh client). A better alternative when you don't have a native Linux machine to install VirtualBox on the client and create a Debian Linux VBox machine with a bridged adaptor.

Now (assuming a Linux client), test the connection to the PI from the client:

ssh -L 5092:localhost:5900 pi@<Pi's IP> 

This creates an ssh terminal session on the PI that tunnels port 5900 on the PI to your client at port 5902. From that terminal session, start x11vnc:

x11vnc -ncache -display :0 # must be run on each new session

That opens port 5900 on the PI as a VNC port. To get the desktop from the client, open remmina on the client machine and connect to the session using the following profile: protocol VNC, server 127.0.0.1:5092, Color depth True color, Quality Medium. If all went well, you now have the PI desktop sitting in a window on the client machine… with copy/paste and camera viewing ability.

Always reboot/shutdown the PI gracefully. From an ssh terminal session:

sudo reboot -h && exit

or:

sudo shutdown -h now && exit

Before you expose your PI to the internet by opening a port on your router, create an ssh key pair, configure ssh to use PubkeyAuthentication, and set PasswordAuthentication to no. Here are the basic steps:

On the PI, create a hidden folder named .ssh:

mkdir ~/.ssh

On the client machine (not the PI), create an ssh key:

ssh-keygen -t rsa -b 4096 -C "$(whoami)@$(hostname)-$(date -I)" #open-ssh required

When asked for a location, enter: /home/<user>/.ssh/piClient_rsa. I set a password in case someone gets physical access to my machine (it's not a login password). The command creates a keypair and a separate file containing the public part of the key in the client's ~/.ssh folder.

Then, from the client, transfer the public part of the key to the pi.

ssh-copy-id -i ~/.ssh/piClient_rsa.pub pi@<PI's IP> # PI's password when asked

On the PI, open the hidden folder ~/.ssh. There should be an authorized_keys file there that contains the client's public key. Next, edit the /etc/ssh/sshd_config file:

sudo cp /etc/sshd_config /etc/sshd_config.backUp
gksu leafpad /etc/ssh/sshd_config

Below are the only uncommented lines in my sshd_config.

Port 22
Protocol 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
UsePrivilegeSeparation yes
KeyRegenerationInterval 1h
ServerKeyBits 2048
LoginGraceTime 1m
PermitRootLogin no
StrictModes yes
PubkeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys
IgnoreRhosts yes
RhostsRSAAuthentication no
HostbasedAuthentication no
PermitEmptyPasswords no
ChallengeResponseAuthentication no
PasswordAuthentication no
AcceptEnv LANG LC_*
Subsystem sftp /usr/lib/openssh/sftp-server

With this, PasswordAuthentication is no longer used.

Finally, on the PI, change some permissions or it won't work:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/*

Although of controversial benefit, you may also want to change the port from the default (22) to a non-standard number greater than 1024. Obscuring the ssh port in this way will discourage inexperienced bad guys from hammering your router with fruitless password attempts. (You are not exposed to successful simple-password attacks because you are smart and use public key authentication.) Keep port 22 if you are on an potentially hostile local network. The best idea is to keep port 22 on the PI and have the router forward WAN requests to an obscure port, say 1141, to port 22 on the PI.

Reboot or restart ssh on the PI:

sudo service ssh restart

Note below: If you set a password when you created the key, your session will ask you to enter it. This is the password required to read the key, not the system password. (Think of it as a second layer of security.) Now, back to the client. Ssh into the PI using the ssh key for access:

ssh -i ~/.ssh/piClient_rsa pi@<PI's IP>

To tunnel VNC over ssh with key authentication:

ssh -i ~/.ssh/piClient_rsa -L 5092:localhost:5900 pi@<IpForPi> 

(If you changed the default port, include the switch -p <yourPort#>).

To read more about ssh and ssh keys, visit the following two links. They are Arch Wiki pages but they apply to Raspian as well, except the install and daemon content. Raspian enables the daemon by default and ssh is already installed.

• https://wiki.archlinux.org/index.php/Secure_Shell
• https://wiki.archlinux.org/index.php/SSH_keys.

[optional] Force HDMI resolution

When the HDMI connection is unplugged at boot, the Raspian OS defaults to low resolution. Under normal use, my HDMI connection is disconnected and, when using VNC, I prefer a higher resolution.

sudo cp /boot/config.txt /boot/config.txt.backUp
gksu leafpad /boot/config.txt

After the line that reads NOOBS Auto-generated Settings, make it read just this:

# NOOBS Auto-generated Settings:
hdmi_force_hotplug=1
config_hdmi_boost=4
start_x=1
gpu_mem=128
#Modified below to enable higher resolution when hdmi is unplugged
disable_overscan=1
hdmi_group=2
hdmi_mode=51

Save, quit, restart.

Test the camera

Note: Neither raspivid nor omxplayer display video over VNC. To see the movie, you need to convert it to mp4 and show it with mplayer.

Open a terminal from the PI desktop and run:

raspivid -t 15000 -o ~/volatile/video.h264 -fps 4 -w 600 -h 400 -rot 180 -p '10,10,600,400'

That will collect 15 seconds of video from the cam at 4 frames per second and save it in ~/volatile/video.h264. The -w and -h flags set the image dimensions. The -rot flag rotates it and the -p string sets a preview window. Adding the -n flag suppresses the preview during the recording to conserve resources.

If the PI is connected to an HMDI monitor, you can view the movie with: omxplayer /volatile/video.h264 --fps 4, but omxplayer won't render in a VNC client. Instead, run:

ffmpeg -r 4 -i volatile/video.h264 -vcodec copy video.mp4
mplayer video.mp4 -fps 4
rm volatile/video.h264

The first command created a 4-fps, mp4 movie from the volatile/video.h264 file and stored it in your home directory as video.mp4. The mplayer command played the movie. Once consumed, the original movie, volatile/video.h264 file, was removed.

Experiment with other raspivid frame rates, etc. I set my duration and frame rate based on my real world application requirements. I only need 15 seconds of video and 4 fps creates a movie that is less than 1 Mb, making for a fast upload. If you are recording energetic, furtive wildlife, boost the frame rate. (I mostly record a certain slow-moving mammal.)

Configuring the pins

First, join the gpio group:

sudo gpasswd -a pi gpio

Log out, then log in again for the membership to complete.

We'll use two pins, GPIO 23 and GPIO 24. GPIO 23 will be an input (a sensor pin) that will sense when the motion detector is activated. GPIO 24 will be an output (a signal pin) that will trigger a room light. We want these to be configured at boot. We'll do everything with simple bash scripts. First, we'll make a folder for all of them.

mkdir ~/bashScripts

Then, make a script to configure the pins:

leafpad ~/bashScripts/configPins

Enter the following into the file, save, and quit:

#!/bin/bash
cd /sys/class/gpio/
echo "23" > export
echo "in" > gpio23/direction
echo "24" > export
echo "out" > gpio24/direction
chown root:gpio gpio24/value
chmod 660 gpio24/value

Make it executable:

chmod +x ~/bashScripts/configPins

Now, we need to make the PI run the configPins script when it boots:

sudo cp /etc/rc.local /etc/rc.local.backUp
gksu leafpad /etc/rc.local

Make the last two lines read:

sh /home/pi/bashScripts/configPins
exit 0

Save, quit, reboot. GPIO 23 will always be an input and 24 will always be an output when you boot unless you later remove the line you added in rc.local.

Electronics

Before we wire the final project, it's best do make a test circuit and verify that the pins work as expected. First, let's look at the physical location of the pins on the board. The following picture is credited to http://elinux.org/RPi_Low-level_peripherals#P2_header.

Note the orientation of the board in the picture. The even pins are along the right-hand side, ascending from top to bottom (2, 4, 6, etc.):

• Pin 4 is +5V
• Pin 6 is ground (GND)
• Pin 16 is GPIO 23
• Pin 18 is GPIO 24

The top, left-hand pin is Pin 1 (+3V).

Never let a lead from a 5V pin (e.g., pin 4) make a connection to any of the other pins except GND, and even then, only when a high-value resistor is between them. For our purposes, we won't let any pin connect directly to any other pin. All of our pins, both in the test circuit and in the final circuit, will always connect to a device. In the test circuit the device is a resistor or resistor-LED combination. Here's the test circuit's schematic:

avconv

I used a 2.2K resistor between pin 4 and pin 6 for the (red) power indicator LED. The resistor for the (blue) pin-18 LED is 570 Ohm. I also put a 47 Ohm resistor in the switch circuit (not really needed, except for editorial consistency).

Pin 2's (red) LED acts like a power indicator (it should be on).

The blue LED shows the state of pin 18 (GPIO 24). To turn it on, run:

echo "1" > /sys/class/gpio/gpio24/value

Turn it off with:

echo "0" > /sys/class/gpio/gpio24/value

In the test circuit, pin 16 (GPIO 23) senses whether the switch is closed. To test it, run:

while [ true ]; do cat /sys/class/gpio/gpio23/value; done

The terminal will stream 0s until you close the switch, at which point it will stream 1s.

The project circuit looks like this:

Note that the orange wire connected to pin 1 (+3V) in the test circuit has been removed in the final circuit because the 3 volt signal is now provided by the PIR's yellow lead. You want the PIR jumper to be in the H position (trigger-then-release rather than trigger-and-hold http://learn.adafruit.com/downloads/pdf/pir-passive-infrared-proximity-motion-sensor.pdf). I keep the time-on screw fully counter-clockwise (very short)... just enough to trip a new motion event. Tweak the sensitivity as you desire. I make all adjustments with the PI off.

Run:

while [ true ]; do cat /sys/class/gpio/gpio23/value; done

If nothing is moving for a few moments, you will see a stream of 0s until you wave your hand over the PIR, at which point it streams 1s.

Note on the Powerswitchtail II that the ground wire connects to the negative terminal. On my switchtail, I jumpered the negative terminal to the ground terminal as well. With the switchtail plugged into a lamp, run:

echo "1" > /sys/class/gpio/gpio24/value # on

echo "0" > /sys/class/gpio/gpio24/value # off

Finally, The Code

PIR triggered lights and camera only

Create a new bash script named litesCamera:

leafpad ~/bashscripts/litesCamera

Make it read:

#!/bin/bash
echo control c to quit
while [ true ]; do    
   sleep 0.5
   state=$(cat /sys/class/gpio/gpio23/value)
   if [[ $state -eq 1 ]]; then
     if [ -e ~/volatile/video.h264 ]; then
        rm ~/volatile/video.h264
     fi
     echo "1" > /sys/class/gpio/gpio24/value
     raspivid -t 15000 -o ~/volatile/video.h264 -fps 4 -w 600 -h 600 -rot 180 -n
     echo "0" > /sys/class/gpio/gpio24/value
     echo control c to quit
   fi
done

That translates to: if the PIR sees motion, delete the last movie if it is there, turn on the lights, record a 15 second movie, turn off the lights, and wait for another signal from the PIR. (Try to figure out why the file is explicitly removed instead of just overwritten.)

Make the script executable:

chmod +x ~/bashscripts/litesCamera

Run it:

~/bashscripts/litesCamera

When the PIR activates, a movie will be recorded in memory at ~/volatile/video.h264. You can process this video just like we did above under Test the camera. Beware, if the PIR trips again, the recorded video will be overwritten as you work.

FTP the movies off-site on-the-fly

Let's say you have an FTP account on another computer located at ftp://example.com and it has a folder named myVideos on it. Let's also say your user name is userName and your password is passWd.

Create a script named litesCameraFtp.

leafpad ~/bashScripts/litesCameraFtp

Make it read:

#!/bin/bash
path=~/volatile/
suffix=MyVideo.h264
################## Enter your ftp url and user:pwd here ################
secretUrl='ftp://example.com/myVideos/'
user=userName:passWd

while [ true ]; do
   sleep 0.5
   state=$(cat /sys/class/gpio/gpio23/value)
   if [[ $state -eq 1 ]]; then
     now=$(date +"%b_%d_%T")    
     url=$secretUrl$(echo ${now//:/_})$suffix
     if [ -e ~/volatile/video.h264 ]; then
        rm ~/volatile/video.h264
     fi
     echo "1" > /sys/class/gpio/gpio24/value
     raspivid -t 15000 -o ~/volatile/video.h264 -fps 4 -w 600 -h 600 -rot 180  -n
     echo "0" > /sys/class/gpio/gpio24/value
     curl -T ~/volatile/video.h264 $url --user $user
   fi
done

Don't forget to put in your real secretUrl and user in the script above. In an insecure LAN environment or physical setting, hard-coding the ftp user:pwd presents risks and further lockdown is needed. (What are some of the ways? Is simple ftp ever very secure?)

Make it executable:

chmod +x ~/bashScripts/litesCameraFtp

Run it:

~/bashscripts/litesCameraFtp

Then trigger a movie, watch the terminal for ftp completion, and verify that it's on your ftp site.

Finally, there is an inherent uncertainty with a negative result. Periodically, we'll send a movie to reassure us that the device is alive. This movie will include the PI's external IP in the file name. That way, we will have the current IP for remote administration tasks.

Make a bashscript named litesCameraAction in the ~/bashscripts folder.

leafpad ~/bashScripts/litesCameraAction

Make it read:

#!/bin/bash
path=~/volatile/
suffix=MyVideo.h264
################## Enter your ftp url and user:pwd here ################
secretUrl='ftp://example.com/myVideos'
user=userName:passWd

iteration=0
while [ true ]; do
   let iteration="iteration+1"
   sleep 0.5
   now=$(date +"%b_%d_%T")    
   state=$(cat /sys/class/gpio/gpio23/value)
   if [[ $state -eq 1 ]]; then   now=$(date +"%b_%d_%T")
        url=$secretUrl$(echo ${now//:/_})$suffix  
        if [ -e ~/volatile/video.h264 ]; then
           rm ~/volatile/video.h264
        fi
        echo "1" > /sys/class/gpio/gpio24/value
        raspivid -t 15000 -o ~/volatile/video.h264 -fps 4 -w 600 -h 600 -rot 180 -n
        echo "0" > /sys/class/gpio/gpio24/value
        curl -T ~/volatile/video.h264 $url --user $user
   fi
   if [[ $iteration -gt 20000 ]]; then
        ip=$(lynx --dump checkip.dyndns.org | grep -Eo '[0-9\.]+')    
        ipSuffix=$(echo ip)$ip$(echo _)$suffix        
        iteration=0
        if [ -e ~/volatile/video.h264 ]; then
            rm ~/volatile/video.h264
        fi
        echo "1" > /sys/class/gpio/gpio24/value
        raspivid -t 15000  -o ~/volatile/video.h264 -fps 4 -w 600 -h 600 -rot 180 -n
        echo "0" > /sys/class/gpio/gpio24/value
        curl -T ~/volatile/video.h264 $secretUrl$(echo ${now//:/_})$ipSuffix --user $user
    fi
done

Make it executable:

chmod +x ~/bashScripts/litesCameraAction

In the script, the IP is checked every 20000 cycles and results in only a few checks per day on my PI. Be sure your check rate does not abuse dyndns.org. Test it with:

~/bashScripts/litesCameraAction

Make the litesCameraAction script run at login:

Add an autostart for the script (this method assumes you will be booting to the desktop):

leafpad ~/.config/lxsession/LXDE-pi/autostart

Make it read:

@lxterminal --command="bashScripts/litesCameraAction"

Save, quit, and log-out/log-in for a full system test.

Important

Do not use this device in medical applications or in any other application demanding high reliability. Don't use these devices in areas where there may be flammable gases/vapors or in other hazardous environments.

I warrant only that I have built this project, documented the process to the best of my ability, and tested it over an extended period of time.

The only high voltages are inside the PI's encapsulated power supply and the Powerswitch II device's housing. If you make a mistake in hooking-up your components, you may destroy the PI and/or PIR. My PI is very slightly warm to the touch (a good sign). Consider heat sinks for the chips if yours runs hotter. Usb devices, HMDI adaptors, and other peripherals generate heat. If you put your PI in a case/box, make sure it allows for adequate air flow. Take further steps to enhance network and physical security if confidential information could be exposed by the device.

Although links are provided to component distributors, I don't represent them or warrant their products. I did purchase my parts from them and I was pleased with their products and services. With the other web links, the safety and content of their information is their responsibility alone.

Some Teasers

The PI is able to sense the outside world and respond to it with outside-world actions. It can interact with its surroundings. Invent!

• What are six reasons to use a PID instead of pixel-differences to detect motion?
• How can we make the litesCameraAction script shorter by never repeating identical lines?
• How can we make the device only record nocturnal animals?
• How can we make it text us when an event occurs?
• Are there more ways to cut down on SDD access?
• Why does the author use Hello Kitty boxes?

History

• New: August 13, 2015