Project SentrINO: An Internet-of-Things enabled security system
This project showcase the various skills learned in the labs of ELEC 291, a second year project design course. For this project, students in groups of six were free to choose what we created as long as it interacted with the internet and embodied the
principles of "Internet of Things".
My group chose to implement a home security robot that autonomously reacts to its environment and be manually controlled over the internet; this was accomplished using a Raspberry Pi, Arduino, a USB webcam, a webserver, and various electronic
In order to keep the wires organized, having the breadboards on the platform was essential since it had to be in close proximity to all of the hardware. As a result, the Pi had to be kept separate from the platform to free up space for the breadboards;
however this worked to our advantage because it allowed us to connect external devices such as a monitor and a keyboard to the Pi easily.
The Arduino was used to integrate all of the other hardware components; this included a keypad, an LCD, an LED, a Piezo buzzer, and the servo motors. The Arduino was incompatible with other USB cables and audio plugs so the webcam and
the speakers were connected directly to the Pi. As for other external devices, we had a keyboard and a monitor connected to the Pi so that we were able to program on the Pi.
We initially powered the Arduino from the Pi, but found that this causes power issues which conseuqently affected the communication between the two modules. In the end we had to give the Arduino its own power source from its DC power
jack through an adapter, after this change the system ran smoothly.
This part of the project included creating a video stream from the webcam, a server to host the web page, a listener script to listen for command from the remote interface and a file server to show saved webcam recordings.
The webpage has onscreen buttons and keyboard control which in turn control the Raspberry Pi, and by extension the Arduino. Furthermore, the user can send text messages from the website that is relayed to the audio speaker by text-to-speech.
The images are taken from the camera through OpenCV, an open source computer vision library. The processed video stream is then broken up into individual JPEG frames where we used the Flask web framework to create an MJPEG stream; this
stream can be embedded as any regular image on the web page except it automatically updates with new frames, becoming a image livestream.
As with all livestreams, latency is an important factor to keep in mind. The latency was due to our wireless internet connection being a phone’s hotspot. Coupled with low transfer speeds around 50kb/s, it was important for us to make
every byte count. First, the image is reduced from (640px x 480px) to (100px x 67px) before being sent to the website. The quality of the image is greatly reduced, but we got better framerates and smoothness in the end.
The webpage where the video feed is displayed is simple. It contains the video stream, with buttons overlayed to control the webcam orientation. When a button is pressed, a XHR request is sent to the listener script running on the Raspberry
Pi. The keyboard arrow keys also call on these same requests. The listener script then translates this requests to serial commands to send to the arduino.
The listener script is also a Flask application; there is a text box also on the website, where the user can submit text that is to be spoken on the speaker. The text is transferred over an XHR request as well to the same listener script
running on the Pi. The listener script then uses the pyttsx text to speech engine to output the speech from the Pi’s audio jack.
When the alarm is triggered, a video recording is started with a timestamp as the name of the video file. The video consists of the processed frames before they are compressed and set to the web server so that the quality of the image is preserved. The
camera will continue to record until two seconds have elapsed after the enemy was last detected. The file created is then hosted in a file server accessible through the website and the video can be downloaded directly from the webpage.
To improve the user experience we’ve added a text display LCD; when the alarm is raised the LCD screen will prompt the user to enter the alarm code in addition to displaying what the user has typed in so far. We originally planned to
display which mode the Sentrino was in (ie: Autonomous tracking mode or Wifi controlled mode) however due to the way we had set up the wifi commands there was no way to distinguish between movement commands sent from either mode as they both used
the same commands.
When an intruder is detected from the autonomous tracking the web server will send a command to raise the alarm. The alarm consist of a yellow LED being turned on for visual indication, along with a piezo buzzer beeping for audio indication.
The keypad must be used to manually disable the alarm; we implemented this action in case the SentrINO colour recognition registered a false positive.
Alarm Mode Software
The keypad software consist of a 16 character buffer array. When the user presses a key, it will be appended to the next open array index position. If the buffer array has reached capacity the software will automatically check whether the keycode entered
was correct before resetting the keycode buffer to prevent an overflow. We had chosen the keycode length to be of length 16 because it was long enough to cover one row of the LCD screen; which made the screen appear more organized.
The keypad contained a pound key [#] which was used for the user to submit the their keycode if the correct keycode was less than 16 characters. The keypad also contained an asterisk [*] key, which we implemented as a backspace so the
user can undo their previous key press. We could have switched the key definition but we’d decided to stick with the asterisks for the undo functionality; this was because people often attach an asterisk in front of what they meant to type after
sending a text message with a typo.
Voip.ms offers a local phone number for around $1/month with free SMS texting. Using their REST API, we were able to send text alerts when in alarm mode. The video processing script would first send a request to the website script which
would in turn send a request to voip.ms’ servers to send a SMS. Parameters in this GET request included the username, password, destination number and message.