Playing with my Raspberry Pi

It's been a while, and a few updates:  I got a pair of Raspberry Pi 3's on my desk and a Foscam exterior dome camera hanging over my garage now.  One of the Pi's is acting as a server to store the images for me on a custom built attached storage array (I've been busy!).  This post is not about those four projects though....it's about a piece of scripting that I found myself writing today.  See, the camera just dumps the files into a very flat directory structure, resulting in thousands of images and video files per day.  As you can imagine, this can build up to tens or hundreds of thousands if I'm not proactively moving the files into a structured directory tree, which until today, has been a manual process.  I got tired of that, and knew I could write a bash script to do it for me.  It only took me four hours, and here is the result:

#!/bin/bash
### Script to move output files from a Foscam exterior camera into a structured directory tree instead of storing them all in only the two default folders
### Adapted from a script found on the web at http://unix.stackexchange.com/questions/261888/help-creating-script-to-move-files-based-on-date-or-filename
### Modifed with information from http://stackoverflow.com/questions/26881104/extract-date-from-a-file-name-in-unix-using-shell-scripting
### and with information from http://stackoverflow.com/questions/12230762/insert-characters-into-a-string-in-bash
### Written by:
### Christopher M Goodrich, CISSP, ISSEP, ISSMP
### phaldor@comcast.net

### Lets set some variables to the base directories.  These should be changed to the base directories that you are using for the Foscam recording location
### and the top level folder where you want to store the organized files
BASEDIR=[insert folder here]   ### Top level storage location (we are going to make directories under this with this script)
RECDIR=[insert folder here]   ### Location where your Foscam is saving the recorded content

cd ${RECDIR}/record       ### Start with the motion recording files

for file in alarm_*.mkv       ### we are going to loop for .mkv files produced by the motion alarm
do
 ### Foscam uses alarm motion recording file format of alarm_YYYYMMDD-HHMMSS.mkv so
 ### get the year, month, and day out of filename
 rawdate=$(echo ${file} | awk -F'[_-]' '{print $2}')  ### sets variable rawdate to just the 8 digit date from the filename
 date=$(echo ${rawdate} | sed 's/^\(.\{4\}\)\(.\{2\}\)/\1-\2-/') ### sets variable date to the format 'YYYY-MM-DD'
# date=${date}-       ### adds ending '-' (ending hyphen is important)
 year=$(echo ${date} | cut -d"-" -f1)    ### sets varialbe year to the first four digits in variable date
 month=$(echo ${date} | cut -d"-" -f2)    ### sets variable month to the next two digits in variable date
 day=$(echo ${date} | cut -d"-" -f3)    ### sets variable day to the last two digits in variable date

 PARENTDIR=${BASEDIR}/${year}-${month}
 MIDDIR=${PARENTDIR}/${day}     ### create the variable for the parent folder (year and month) and storage folder (day)
 STOREDIR=${MIDDIR}/Motion-record

 if [ -d ${PARENTDIR} ]      ### if the parent directory exists
 then
  if [ -d ${MIDDIR} ]     ### if the storage directory exists
  then
   if [ -d ${STOREDIR} ]    ### if the motion-record subfolder exists
   then
    mv ${file} ${STOREDIR}   ### move the file
   else      ### the sub-folder doesn't exist
    mkdir ${STOREDIR}   ### create it
    mv ${file} ${STOREDIR}   ### move the file
   fi      ### close the if statement
  else       ### the storeage directory doesn't exist
   mkdir -p ${STOREDIR}    ### create the storage directory and the sub-folder that the file will reside in
   mv ${file} ${STOREDIR}    ### then move the file
  fi       ### close the if statement
 else        ### the parent directory doesn't exist
  mkdir -p ${STOREDIR}     ### create the directory tree to store motion recorded media
  mv ${file} ${STOREDIR}     ### then move the file
 fi        ### close the if statement
done         ### close the for loop

for file in schedule_*.mkv      ### we are going to loop for .mkv files produced by the scheduled recording feature
do
 ### Foscam uses scheduled recording file format of schedule_YYYYMMDD-HHMMSS.mkv so
 ### get the year, month, and day out of the filename
 rawdate=$(echo ${file} | awk -F'[_-]' '{print $2}')  ### sets variable rawdate to just the 8 digit date from the filename
 date=$(echo ${rawdate} | sed 's/^\(.\{4\}\)\(.\{2\}\)/\1-\2-/') ### sets variable date to the format 'YYYY-MM-DD'
# date=${date}-       ### adds ending '-' (ending hyphen is important)
# rawdate=$(echo ${file} | grep -Eo '[[:digit:]]{8}')  ### sets variable rawdate to just the 8 digit date from the filename
# date=$(echo ${rawdate:0:4}-${rawdate:4:2}-${rawdate:6:2}-) ### sets variable date to the format 'YYYY-MM-DD-" (ending hyphen is important)
 year=$(echo ${date} | cut -d"-" -f1)    ### sets varialbe year to the first four digits in variable date
 month=$(echo ${date} | cut -d"-" -f2)    ### sets variable month to the next two digits in variable date
 day=$(echo ${date} | cut -d"-" -f3)    ### sets variable day to the last two digits in variable date

 PARENTDIR=${BASEDIR}/${year}-${month}
 MIDDIR=${PARENTDIR}/${day}     ### create the variable for the parent folder (year and month) and storage folder (day)
 STOREDIR=${MIDDIR}/Schedule-record     ### create the variables for the parent folder (year and month) and storage folder (day)

 if [ -d ${PARENTDIR} ]      ### if the parent directory exists
 then
  if [ -d ${MIDDIR} ]     ### if the storage directory exists
  then
   if [ -d ${STOREDIR} ]    ### if the schedule-record subfolder exists
   then
    mv ${file} ${STOREDIR}   ### move the file
   else      ### the sub-folder doesn't exist
    mkdir ${STOREDIR}   ### create it
    mv ${file} ${STOREDIR}   ### move the file
   fi      ### close the if statement
  else       ### the storeage directory doesn't exist
   mkdir -p ${STOREDIR}    ### create the storage directory and the sub-folder that the file will reside in
   mv ${file} ${STOREDIR}    ### then move the file
  fi       ### close the if statement
 else        ### the parent directory doesn't exist
  mkdir -p ${STOREDIR}     ### create the directory tree to store scheduled recorded media
  mv ${file} ${STOREDIR}     ### then move the file
 fi        ### close the if statement
done         ### close the for loop


cd ${RECDIR}/snap       ### switch directory of focus to the snap folder

for file in MDAlarm_*.jpg      ### we are goinig to loop for .jpg files produced by the snapshot feature during motion events
do
 ### Foscam uses a file format of MDAlarm_YYYYMMDD-HHMMSS.jpg for taking snapshots during a motion alarm event so
 ### get the year, month, and day out of the filename
 rawdate=$(echo ${file} | awk -F'[_-]' '{print $2}')  ### sets variable rawdate to just the 8 digit date from the filename
 date=$(echo ${rawdate} | sed 's/^\(.\{4\}\)\(.\{2\}\)/\1-\2-/') ### sets variable date to the format 'YYYY-MM-DD'
# date=${date}-       ### adds ending '-' (ending hyphen is important)
# rawdate=$(echo ${file} | grep -Eo '[[:digit:]]{8}')  ### sets variable rawdate to just the 8 digit date from the filename
# date=$(echo ${rawdate:0:4}-${rawdate:4:2}-${rawdate:6:2}-) ### sets variable date to the format 'YYYY-MM-DD-" (ending hyphen is important)
 year=$(echo ${date} | cut -d"-" -f1)    ### sets varialbe year to the first four digits in variable date
 month=$(echo ${date} | cut -d"-" -f2)    ### sets variable month to the next two digits in variable date
 day=$(echo ${date} | cut -d"-" -f3)    ### sets variable day to the last two digits in variable date

 PARENTDIR=${BASEDIR}/${year}-${month}
 MIDDIR=${PARENTDIR}/${day}     ### create the variable for the parent folder (year and month) and storage folder (day)
 STOREDIR=${MIDDIR}/Motion-snap     ### create the variable for the parent folder (year and month) and storage folder (day)

 if [ -d ${PARENTDIR} ]      ### if the parent directory exists
 then
  if [ -d ${MIDDIR} ]     ### if the storage directory exists
  then
   if [ -d ${STOREDIR} ]    ### if the motion-snap subfolder exists
   then
    mv ${file} ${STOREDIR}   ### move the file
   else      ### the sub-folder doesn't exist
    mkdir ${STOREDIR}   ### create it
    mv ${file} ${STOREDIR}   ### move the file
   fi      ### close the if statement
  else       ### the storeage directory doesn't exist
   mkdir -p ${STOREDIR}    ### create the storage directory and the sub-folder that the file will reside in
   mv ${file} ${STOREDIR}    ### then move the file
  fi       ### close the if statement
 else        ### the parent directory doesn't exist
  mkdir -p ${STOREDIR}     ### create the directory tree to store motion recorded media
  mv ${file} ${STOREDIR}     ### then move the file
 fi        ### close the if statement
done         ### close the for loop


for file in Schedule_*.jpg      ### we are goinig to loop for .jpg files produced by the scheduled snapshot feature
do
 ### Foscam uses a file format of Schedule_YYYYMMDD-HHMMSS.jpg for taking regularly scheduled snapshots so
 ### get the year, month, and day out of the filename
 rawdate=$(echo ${file} | awk -F'[_-]' '{print $2}')  ### sets variable rawdate to just the 8 digit date from the filename
 date=$(echo ${rawdate} | sed 's/^\(.\{4\}\)\(.\{2\}\)/\1-\2-/') ### sets variable date to the format 'YYYY-MM-DD'
# date=${date}-       ### adds ending '-' (ending hyphen is important)
# rawdate=$(echo ${file} | grep -Eo '[[:digit:]]{8}')  ### sets variable rawdate to just the 8 digit date from the filename
# date=$(echo ${rawdate:0:4}-${rawdate:4:2}-${rawdate:6:2}-) ### sets variable date to the format 'YYYY-MM-DD-" (ending hyphen is important)
 year=$(echo ${date} | cut -d"-" -f1)    ### sets varialbe year to the first four digits in variable date
 month=$(echo ${date} | cut -d"-" -f2)    ### sets variable month to the next two digits in variable date
 day=$(echo ${date} | cut -d"-" -f3)    ### sets variable day to the last two digits in variable date

 PARENTDIR=${BASEDIR}/${year}-${month}
 MIDDIR=${PARENTDIR}/${day}     ### create the variable for the parent folder (year and month) and storage folder (day)
 STOREDIR=${MIDDIR}/Schedule-snap     ### create the variable for the parent folder (year and month) and storage folder (day)

 if [ -d ${PARENTDIR} ]      ### if the parent directory exists
 then
  if [ -d ${MIDDIR} ]     ### if the storage directory exists
  then
   if [ -d ${STOREDIR} ]    ### if the schedule-snap subfolder exists
   then
    mv ${file} ${STOREDIR}   ### move the file
   else      ### the sub-folder doesn't exist
    mkdir ${STOREDIR}   ### create it
    mv ${file} ${STOREDIR}   ### move the file
   fi      ### close the if statement
  else       ### the storeage directory doesn't exist
   mkdir -p ${STOREDIR}    ### create the storage directory and the sub-folder that the file will reside in
   mv ${file} ${STOREDIR}    ### then move the file
  fi       ### close the if statement
 else        ### the parent directory doesn't exist
  mkdir -p ${STOREDIR}     ### create the directory tree to store motion recorded media
  mv ${file} ${STOREDIR}     ### then move the file
 fi        ### close the if statement
done         ### close the for loop

Pretty handy little script there, and it processed the 26k files I had lying in the directories with a fair amount of speed!  You will notice that I have some extra lines in there that are commented out.  I tried a couple different methods before I found one that worked, and I failed to remove the extra lines before posting this, I was just that excited!  #HappyDance