Off-site backup of CCTV using the cloud

Having a camera security-system (“CCTV”) can be great for loss prevention, deterring crime, and providing physical evidence when something goes amiss. Many companies use surveillance cameras connected to a DVR device to store video on-site. Of course, having the DVR in a lock-box can only go so far to protect against catastrophic accidents, and sometimes it’s desirable to have offsite backups.

In this post I’ll demonstrate how a small company could use an Amazon EC2 instance to offload some processing and storage, in addition to the in-house video recording they may be using. Since I’m aiming for the AWS free tier, I’ll be looking to use no more than 30GB for OS and storage. To accomplish this, later on we’ll offload processed videos (hoping to stay under the 15GB monthly “free tier” limit). All this sweeping the deck, to say that the cameras will be doing once-per-minute snapshots (time lapse).

Doing a time lapse won’t provide perfect video in the event of catastrophe, but it’s better than nothing.

Demo setup: 9 IP cameras on an isolated subnet with the DVR hardware. The cameras are a mix of TRENDnet TV-IP321PI and TV-IP310PI cameras, and the DVR captures using the camera video streams.

The cameras will be configured to send a once-per-minute snapshot to the router via FTP, which will redirect the traffic over the IPsec VPN set up in a previous post. The reason I chose to point the cameras at the router is because it’s much easier to change one redirect, than it is to change 9 cameras, should the offsite host be changed.

Screenshot from 2016-05-07 22-39-26

After pointing all 9 cameras to the router’s IP, the router can be made to forward all FTP traffic to the EC2 instance:

/ip firewall nat add chain=dstnat action=dst-nat to-addresses=172.31.48.1 protocol=tcp src-address=172.21.1.0/24 dst-address=172.21.1.1 dst-port=21 log=no log-prefix="" comment=AWS place-before=1

Following this, vsftpd is installed and the following changes made in vsftpd.conf: write_enable=YES

Finally, a user “camera” is added with the same password as set in the cameras, and port 21 is allowed through the server’s firewall (not AWS!)

The images are uploaded into folders such as:
$ ls
FrontDoor OutbackLeft OutbackRight WHBackLeft WHBackRight WHDockLeft WHDockRight WHFront WHGlassDoor

The files are named in the following format:
172.21.1.249_01_20160507221853_TIMING.jpg
172.21.1.249_01_20160507221953_TIMING.jpg
172.21.1.249_01_20160507222053_TIMING.jpg
172.21.1.249_01_20160507222153_TIMING.jpg

Now there will be 1440 or so images per camera. Uncompressed this will take anywhere from 130MB to 200MB depending on megapixels and other factors, which means a script to encode daily videos would be especially fitting.

#!/bin/bash
cd /home/camera/
for i in `ls -d [A-Z]*/`
do
    C=0
    YESTERDAY=`date +%Y%m%d --date="1 day ago"`
    for j in `ls -1 $i*$YESTERDAY*`
    do
        mv $j $i`printf "%04d" $C`.JPG
        C=$((C+1))
    done
    # /usr/bin/avconv -i $i%04d.JPG -vcodec mpeg4 -qscale:v 7 _archive/$YESTERDAY${i:0:-1}.avi
    /usr/bin/avconv -i $i%04d.JPG -vcodec libx264 -crf 24 _archive/$YESTERDAY${i:0:-1}.avi
   rm $i*.JPG
done

The above script dives into each of the camera directories, renames all of the images from the day before into sequentially-numbered JPG’s for conversion, and calls avconv (feel free to use ffmpeg), encoding a video with the day’s images into an _archive folder. This script can be called every day at midnight using a cron job.

Notice there are two options for encoders: The older mpeg4 “Xvid” encoder makes slightly larger files but is less CPU-intensive and takes a third the time as libx264. Since this is running on a dedicated EC2 instance, libx264 is used.

The resulting files are 17 to 35MB on average for mpeg4, and smaller for h.264 encoding. This amounts to about 268MB per day for mpeg4 (source: ~/_archive$ for i in {1..31}; do ls -l `date +%Y%m%d --date="$i days ago"`* |awk '{s += $5} END{print s/1000000}'; done | awk '{s += $1} END{print s/NR}' 😉

Overall, using libx264 could offer about a half a year of these time-lapses for a free-tier EC2 instance with 20GB usable storage, with ten 1.3MP cameras capturing once per minute. This capacity could further be increased by periodically retrieving the images like so:

scp camera@50.1.1.1:_archive/*.avi ~/_archive/ && ssh camera@50.1.1.1 -f 'rm _archive/*.avi'

Of course, this requires adding an ssh keypair for the camera user, but that’s probably the most trivial aspect of this setup, for a seasoned linux user!