Saveconfig Backup scripts

Introduction

saveconfig are two small Shell and TCL/Expect scripts which create a backup of configuration files of network devices. The typical usage is for Cisco hardware like switches, routers, wireless controllers and wireless access points but the script can be extended to create also backup files for any other device which supports the protocols SSH, SCP, SFTP and so on.

Adding ciphers to SSHD

Instead of using TFTP to copy configuration files I rewrote the script to use SCP. I noticed that Cisco's IOS uses some type of encryption which was disabled in GNU/Linux Debian 8 Jessie, therefore a copy run scp did not work. I've added two lines at the end of my /etc/ssh/sshd_config to have older Ciphers, Macs and KexAlgorithms enabled. My last lines of the sshd_config look like this:


Ciphers 3des-cbc,blowfish-cbc,cast128-cbc,arcfour,arcfour128,arcfour256,aes128-cbc,
aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr,
aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com

Macs hmac-sha1,hmac-sha1-96,hmac-sha2-256,hmac-sha2-512,hmac-md5,hmac-md5-96,hmac-ripemd160,
hmac-ripemd160@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha1-etm@openssh.com,
hmac-sha1-96-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,
hmac-md5-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-ripemd160-etm@openssh.com,
umac-64-etm@openssh.com,umac-128-etm@openssh.com

KexAlgorithms curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,
ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1,
diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
Adding older ciphers in sshd_config

After this change the SSH daemon must be restarted. I'm doing this with the command:


root@linux-devel:~# service sshd restart
Restarting SSH daemon

Requirements

The scripts require a running GNU/Linux server, in my case I use Debian GNU/Linux. By starting with a minimal configuration the following packets must be installed:


root@linux-devel:~# aptitude install expect
root@linux-devel:~# aptitude install openssh-server
root@linux-devel:~# aptitude install atftpd
root@linux-devel:~# aptitude install diffutils
root@linux-devel:~# aptitude install bsd-mailx
Adding packages for the script

The scripts require a directory structure on /srv/tftp as shown below. The reason for that is to use a control script (spawn script) to parallelize the backup process. In an environment with about 1000 network devices the script will run about 20 hours if a serial approach is used. By the parallel approach the script runs about 1 hour and 30 minutes.

The directory diff contains the changes from today's backup to yesterday's backup. The directory log contains the log files which are useful in case something goes wrong.


root@linux-devel:~# tree -d /srv/tftp/
/srv/tftp/
├── archive
│     └── 2017-10-13
├── config
│     ├── aironet
│     │     ├── diff
│     │     └── log
│     ├── catalyst
│     │     ├── diff
│     │     └── log
│     ├── ciscoasa
│     │     ├── diff
│     │     └── log
│     └── ciscowlc
│            ├── diff
│            └── log
├── scripts
│     ├── saveconfig
│            └── inc
└── temp
Directory structure

Script execution

The script saveconfig-backup.sh does the real backup job, this means it logs in to a device and executes commands. The script starts with the Shebang #!/usr/bin/expect to identify itself as an expect script, then I defined some variables which are required for functions. The script accepts command line arguments, i.e. to do a backup of my switch hardware I use:


root@linux-devel:/srv# /srv/tftp/scripts/saveconfig-backup.sh catalyst
Execute with command line parameter

Script header

The script header with the variable definition can look like this:


#!/usr/bin/expect
################################################################################
##
## saveconfig-backup.sh
## --------------------
## Saves text-based configuration files. Currently supports:
## - Cisco Catalyst Switches
## - Cisco Aironet Standalone Access Points
## - Cisco Wireless LAN Controller
## - Cisco ASA Firewall
##
## Requires CSV file in the format: ipv4;hostname;type;username;password;enable
## 
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28   str     re-writing and modularization of code
##
################################################################################
##
## GLOBAL-VARIABLES
## ----------------
## DEVICE            Device-Prefix as command line argument
## COPYTFTP          The path to your TFTP server for the device
## COPYSCP           The path to your SCP server for the device
## LOCALTFTP         The path on your server for the configs
## SERVERIP          The server IP address
## INPUTPATH         The path on your server for the scripts
## TODAY             Get today's date in ISO format
## YESTERDAY         Calculate yesterday's date in ISO format
##
################################################################################
set DEVICE [lindex $argv 0]
set COPYSCP "scp://saveconfig:saveconfig@172.30.100.202//srv/tftp/config"
set LOCALTFTP "/srv/tftp/config"
set SERVERIP "172.30.100.202"
set INPUTPATH "/srv/tftp/scripts/saveconfig/"
set TODAY [exec date "+%Y-%m-%d"]
set YESTERDAY [exec date --date "-1 day" "+%Y-%m-%d"]
set timeout 15
set SCRIPTNAME [info script]
# log_user          Expect's log_user command. 0 = no output, 1 = output
# log_file          Expect's log_file command.
log_user 0
log_file -noappend $LOCALTFTP/$DEVICE/log/$TODAY-$DEVICE.log
#
# Define colored lines
#
set colDefault "\033\[0;39m"
set colBegin "\033\[0;96mBEGIN\033\[0;39m"
set colCompleted "\033\[0;96mCOMPLETED\033\[0;39m"
set colSkipped "\033\[0;96mSKIPPED\033\[0;39m"
set colFailed "\033\[0;31mFAILED\033\[0;39m" 
set colSuccess "\033\[0;32mSUCCESS\033\[0;39m"
Script variable definitions

Script modularization

To add functionality for additional devices I've modularized the device specific functions. They are loaded with the following code:


#
# Include device specific functions
#
source [file join [file dirname $SCRIPTNAME] inc/cisco-catalyst.sh]
source [file join [file dirname $SCRIPTNAME] inc/cisco-aironet.sh]
source [file join [file dirname $SCRIPTNAME] inc/cisco-asa.sh]
source [file join [file dirname $SCRIPTNAME] inc/cisco-wlc.sh]
Include device specific code

The next step is to write a function for a Cisco switch. The function takes the variable $IPV4 and $NAME and imports the global variables defined above. Note that those variables are read in by a CSV file. A simple error handling is implemented by a catch-block.

In the catch block the script connects to the Cisco switch with SSH. To avoid any questions I'm jusing the option UserKnownHostsFile=/dev/null to disable OpenSSH Known Hosts file and StrictHostKeyChecking=no to disable OpenSSH Host key checking. The reason doing so is that the script should run in a non-interactive mode without any prompts scheduled by a cron job.

Function for a Cisco switch

The next lines are typically Cisco IOS commands, expects shows it's power here to automate the things. When I log in to one of my switches I always want to know that a configuration backup was done. Therefore I set today's date in the exec banner. To avoid interaction with the switch I set file prompt quiet which suppresses confirmation prompts. Because I did some changes in the configuration by changing the exec banner I am saving those with wr mem. This comes also handy in case you configured something and forgot to save your changes. Next step is to copy run tftp or to copy run scp the configuration file and a logout.

If the above function is not working, I send an information line to the shell. To know what has changed between today and yesterday I use diff and write the output to a file.


################################################################################
##
## Function for Cisco Catalyst Switches
##
## This file is part of saveconfig.
##
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28    str     modularization of code
##
################################################################################
proc cisco_catalyst { IPV4 NAME USERNAME PASSWORD ENABLE} {
    #
    # Import global variables into function
    #
    global DEVICE
    global COPYSCP
    global LOCALTFTP
    global TODAY
    global YESTERDAY
    # Import global color lines
    global colDefault
    global colBegin
    global colCompleted
    global colSkipped
    global colFailed
    global colSuccess
    #
    # Execute the device commands with error handling
    #
    if { [catch {
        spawn ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $USERNAME@$IPV4
        expect "assword:"     { send "$PASSWORD\n" }
        if { $ENABLE!="" }    { expect ">" { send "enable\n" } expect "assword:" { send "$ENABLE\n" } }
        expect "#"            { send "conf t\n" }
        expect "#"            { send "no banner exec\n" }
        expect "#"            { send "banner exec ^$TODAY - BACKUP DONE^\n" }
        expect "#"            { send "file prompt quiet\n" }
        expect "#"            { send "exit\n" }
        expect "#"            { send "wr mem\n" }
        expect "OK"           { send "copy run $COPYSCP/$DEVICE/$TODAY.$NAME.cfg\n\n\n" } 
        expect "copied"       { send "logout\n" }
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colSuccess $NAME / $IPV4.\n"
    #
    # Catch errors
    #
    } errorid] } {
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed $NAME / $IPV4.\n"
    }
    set status [catch {
        exec diff -y -W 200 --suppress-common-lines $LOCALTFTP/$DEVICE/$TODAY.$NAME.cfg \
        $LOCALTFTP/$DEVICE/$YESTERDAY.$NAME.cfg > $LOCALTFTP/$DEVICE/diff/$TODAY-$NAME.txt
    } result]
}
################################################################################
Function for a Cisco switch

Function for a Cisco standalone Access Point

Similar to a Cisco switch a Cisco standalone access point can be backed up. Note: to backup the configuration of controller based access points a different approach is required.


################################################################################
##
## Function for Cisco Aironet Standalone Access Points
##
## This file is part of saveconfig.
##
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28    str     modularization of code
##
################################################################################
proc cisco_aironet { IPV4 NAME USERNAME PASSWORD ENABLE} {
    #
    # Import global variables into function
    #
    global DEVICE
    global COPYSCP
    global LOCALTFTP
    global TODAY
    global YESTERDAY
    # Import global color lines
    global colDefault
    global colBegin
    global colCompleted
    global colSkipped
    global colFailed
    global colSuccess
    #
    # Execute the device commands with error handling
    #
    if { [catch {
        spawn ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $USERNAME@$IPV4
        expect "assword:"     { send "$PASSWORD\n" }
        if { $ENABLE!="" }    { expect ">" { send "enable\n" } expect "assword:" { send "$ENABLE\n" } }
        expect "#"            { send "conf t\n" }
        expect "#"            { send "no banner exec\n" }
        expect "#"            { send "banner exec ^$TODAY - BACKUP DONE^\n" }
        expect "#"            { send "file prompt quiet\n" }
        expect "#"            { send "exit\n" }
        expect "#"            { send "wr mem\n" }
        expect "OK"           { send "copy run $COPYSCP/$DEVICE/$TODAY.$NAME.cfg\n\n\n" } 
        expect "copied"       { send "logout\n" }
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colSuccess $NAME / $IPV4.\n"
    #
    # Catch errors
    #
    } errorid] } {
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed $NAME / $IPV4.\n"
    }
    set status [catch {
        exec diff -y -W 200 --suppress-common-lines $LOCALTFTP/$DEVICE/$TODAY.$NAME.cfg \
        $LOCALTFTP/$DEVICE/$YESTERDAY.$NAME.cfg > $LOCALTFTP/$DEVICE/diff/$TODAY-$NAME.txt
    } result]
}
################################################################################
Function for a Cisco Standalone Access Point

Function for a Cisco ASA Firewall

A Cisco ASA Firewall can be backed up with the same function too.


################################################################################
##
## Function for Cisco ASA Firewalls
##
## This file is part of saveconfig.
##
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28    str     modularization of code
##
################################################################################
proc cisco_asa { IPV4 NAME USERNAME PASSWORD ENABLE} {
    #
    # Import global variables into function
    #
    global DEVICE
    global COPYSCP
    global LOCALTFTP
    global TODAY
    global YESTERDAY
    # Import global color lines
    global colDefault
    global colBegin
    global colCompleted
    global colSkipped
    global colFailed
    global colSuccess
    #
    # Execute the device commands with error handling
    #
    if { [catch {
        spawn ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $USERNAME@$IPV4
        expect "assword:"     { send "$PASSWORD\n" }
        if { $ENABLE!="" }    { expect ">" { send "enable\n" } expect "assword:" { send "$ENABLE\n" } }
        expect "#"            { send "conf t\n" }
        expect "#"            { send "no banner exec\n" }
        expect "#"            { send "banner exec ^$TODAY - BACKUP DONE^\n" }
        expect "#"            { send "file prompt quiet\n" }
        expect "#"            { send "exit\n" }
        expect "#"            { send "wr mem\n" }
        expect "OK"           { send "copy /noconfirm run $COPYSCP/$DEVICE/$TODAY.$NAME.cfg\n\n\n" } 
        expect "copied"       { send "logout\n" }
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colSuccess $NAME / $IPV4.\n"
    #
    # Catch errors
    #
    } errorid] } {
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed $NAME / $IPV4.\n"
    }
    set status [catch {
        exec diff -y -W 200 --suppress-common-lines $LOCALTFTP/$DEVICE/$TODAY.$NAME.cfg \
        $LOCALTFTP/$DEVICE/$YESTERDAY.$NAME.cfg > $LOCALTFTP/$DEVICE/diff/$TODAY-$NAME.txt
    } result]
}
################################################################################
Function for a Cisco ASA Firewall

Function for a Cisco Wireless Controller

A Cisco Wireless Controller can be backed up with this function. In addition the installed licenses on the wireless controller are backed up too.


################################################################################
##
## Function for Cisco Wireless Controller
##
## This file is part of saveconfig.
##
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28    str     modularization of code
##
################################################################################
proc cisco_wlc { IPV4 NAME USERNAME PASSWORD ENABLE} {
    #
    # Import global variables into function
    #
    global DEVICE
    global COPYSCP
    global LOCALTFTP
    global TODAY
    global YESTERDAY
    global SERVERIP
    # Import global color lines
    global colDefault
    global colBegin
    global colCompleted
    global colSkipped
    global colFailed
    global colSuccess
    #
    # Execute the device commands with error handling
    #
    if { [catch {
        spawn ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $USERNAME@$IPV4
        expect "ser:"          { send "$USERNAME\n" }
        expect "assword:"      { send "$PASSWORD\n" }
        expect ">"             { send "save config\n" }
        expect "ave"           { send "y\n" }
        expect "aved"          { send "transfer upload mode sftp\ntransfer upload datatype config\ntransfer upload filename $TODAY.$NAME.cfg\ntransfer upload path /srv/tftp/config/$DEVICE/\ntransfer upload serverip $SERVERIP\ntransfer upload username Cisco\ntransfer upload password Cisco\ntransfer upload start\n" }
        expect "start? (y/N)   { send "y\n" }
        expect "successfully." { send "license save tftp://$SERVERIP//srv/tftp/config/$DEVICE/$TODAY.$NAME.lic\n" }
        expect ">"             { send "logout\n" }
        expect "now? (y/N)"    { send "y\n" }
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colSuccess $NAME / $IPV4.\n"
    #
    # Catch errors
    #
    } errorid] } {
        send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed $NAME / $IPV4.\n"
    }
    set status [catch {
        exec diff -y -W 200 --suppress-common-lines $LOCALTFTP/$DEVICE/$TODAY.$NAME.cfg \
        $LOCALTFTP/$DEVICE/$YESTERDAY.$NAME.cfg > $LOCALTFTP/$DEVICE/diff/$TODAY-$NAME.txt
    } result]
}
################################################################################
Function for a Cisco ASA Firewall

As shown above with three different device types any other device using command line or any other method like TELNET, FTP, etc. can be used. It is just a matter to determine the device type (in a CSV file) and to write the corresponding function with expect.

Bringing it all together

The next lines are required to start the backup function. The script requires CSV files as input containing the IPv4 address, hostname, device type, username, password and enable password. The script opens the CSV files, read their content into an array and closes the files. In case of errors, the script writes an informational message.


#
# Starting backup
#
send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colBegin $DEVICE.\n"
#
# Open CSV files
#
if { [catch {
    set fileid [open $INPUTPATH/$DEVICE.csv]
    set content [read $fileid]
close $fileid
#
# Catch errors
#
} errorid]
} {
    send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed, cannot open file: $INPUTPATH/$DEVICE.csv.\n"
}
Open CSV file

Now the real job begins. The script takes the content of the array and split it up into lines. With a foreach function each line is split into variables separated by semicolon which can be used in the script.

With the content of the CSV files in variables the device type can be determined. The device type variable read from the CVS determines the script function called for the device. This includes also some error handling in case the CSV file can't be parsed.


#
# Determine device type from CSV and start the proper function
#
if { [catch {
    # split into line numbers
    set hosts [split $content "\n"]
    # iterate through lines
    foreach rec $hosts {
        # split lines with ; into data
        set fields [split $rec ";"]
        # and assign variables
        lassign $fields ipv4 name type username password enable
        # determine device type and start backup job function
        if {$type == "catalyst"} {
            set running [cisco_catalyst $ipv4 $name $username $password $enable]
        }
        if {$type == "aironet"} {
            set running [cisco_aironet $ipv4 $name $username $password $enable]
        }
        if {$type == "ciscoasa"} {
            set running [cisco_asa $ipv4 $name $username $password $enable]
        }
        if {$type == "ciscowlc"} {
            set running [cisco_wlc $ipv4 $name $username $password $enable]
        }
    }
#
# Catch errors
#
} errorid]
} {
    send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colFailed, cannot parse file: $INPUTPATH/$DEVICE.csv.\n"
}
#
# Ending backup
#
send_user "[exec date "+%Y-%m-%d %H:%M:%S"] saveconfig-backup: $colCompleted $DEVICE.\n"
Determine device type and call function

Device script testing

This is how a CSV file looks like. It has an IPv4 address, a hostname, the device type, username, password and enable password. Because the CSV files contains a password they should be protected (chmod):


root@linux-devel:~# cat /srv/tftp/scripts/saveconfog/catalyst.csv
127.0.0.1;swt-test1;catalyst;Cisco;Cisco;Cisco
Example CSV file

The script can be tested now and returns the following output:


root@linux-devel:~# /srv/tftp/scripts/saveconfig/saveconfig-backup.sh catalyst
2017-10-29 20:02:11 saveconfig-backup: BEGIN catalyst.
2017-10-29 20:02:21 saveconfig-backup: SUCCESS swt-test1 / 127.0.0.1.
2017-10-29 20:02:56 saveconfig-backup: COMPLETED catalyst.
root@linux-devel:~#
Testing backup of a single device

To verify if the script did it's job the exec banner can be verified.


root@linux-devel:~# ssh -l Cisco 127.0.0.1
Password:
2017-10-29 - BACKUP DONE
swt-test1>logout
Connection to 127.0.0.1 closed.
Verifying Cisco Switch banner

The config file directory should now contain the configuration files.


root@linux-devel:~# ls -l /srv/tftp/config/catalyst/
total 32
-rw-r--r-- 1 nobody nogroup 8538 Oct 28 21:37 2017-10-28.swt-test1.cfg
-rw-r--r-- 1 nobody nogroup 8538 Oct 29 20:02 2017-10-29.swt-test1.cfg
drwxrwxrwx 2 nobody nogroup 4096 Oct 29 17:44 diff
drwxrwxrwx 2 nobody nogroup 4096 Oct 29 17:44 log
Verifying config directory

In addition to the console output the script writes the same messages into a log file. Note: The log file can contain the complete dialog with Cisco's IOS commands if the variable log_user is set to 1. This will cost additional disk space but is useful for troubleshooting.


root@linux-devel:~# cat /srv/tftp/config/catalyst/log/2017-10-29-catalyst.log
2017-10-29 20:02:11 saveconfig-backup: BEGIN catalyst.
2017-10-29 20:02:21 saveconfig-backup: SUCCESS swt-test1 / 127.0.0.1.
2017-10-29 20:02:56 saveconfig-backup: COMPLETED catalyst.
Verifying log file

The generated diff file contains the lines between yesterday's and today's configuration:


root@linux-devel:~# cat /srv/tftp/config/catalyst/diff/2017-10-29-catalyst-swt-001.txt
! Last configuration change at 19:02:20 utc Sun Oct 29 2017 by Cisco |
! Last configuration change at 20:37:12 utc Sat Oct 28 2017 by Cisco
! NVRAM config last updated at 19:02:21 utc Sun Oct 29 2017 by Cisco |
! NVRAM config last updated at 20:37:13 utc Sat Oct 28 2017 by Cisco
banner exec 2017-10-29 - BACKUP DONE |
banner exec 2017-10-28 - BACKUP DONE
Verifying Diff file

Spawn script

The second script is a so-called spawn script which starts parallel processes of the saveconfig-backup.sh script.

The script starts with the Shebang #!/bin/bash to identify itself as a bash script. The script has a DEVICE array which contains the device types (i. e. the CSV files). Also some date and timestamp variables are set, for example the variable ARCHIVEDATE is used to create a tarball of all configuration files.

In the next step the script iterates through the array and starts the child processes of the saveconfig-backup.sh script. A for loop waits until all child processes are completed.


#!/bin/bash
################################################################################
##
## saveconfig-spawn.sh
## --------------------
## Spawns saveconfig-backup.sh processes
##
## License : EUROPEAN UNION PUBLIC LICENCE v. 1.2
##
## 2017-10-28   str     re-writing and modularization of code
##
################################################################################
##
## GLOBAL-VARIABLES
## ----------------
## RCPT             Email recepient
## DEVICE()         Array with device types
## TODAY            Todays date
## TIMESTAMP        Timestamp for script
## ARCHIVEDATE      Archivedate 
##
################################################################################
RCPT="str"
DEVICE=(catalyst aironet ciscoasa ciscowlc)
TODAY=`date +%Y-%m-%d`
TIMESTAMP=`date '+%Y-%m-%d %H:%M:%S'`
ARCHIVEDATE=`date +%Y-%m-%d -d "15 days ago"`
#
# Iterate through DEVICE() Array to start parallel saveconfig-backup processes
#
echo "$TIMESTAMP saveconfig-spawn: BEGIN parallel processes."
for i in "${DEVICE[@]}"
do
    { /srv/tftp/scripts/saveconfig/saveconfig-backup.sh $i & } 2>/dev/null;
    echo "$TIMESTAMP saveconfig-spawn: starting parallel process for: $i, pid:" $!
done
#
# wait for processes
#
for job in `jobs -p`
    do wait $job
done
echo "$TIMESTAMP saveconfig-spawn: COMPLETED processes completed."
Spawn script, creating child processes

Handling log files and email report

To handle log files and an email report the log files of each device are concatenated into a temporary report file. With the log files concatenated a wc (word count) can count the occurence of the keywords FAILED or SUCCESS. A small information email is send out (Example: local account on my development system) and then the script deletes the temporary report file.


#
# Concatenate log files
#
echo "$TIMESTAMP saveconfig-spawn: BEGIN concatenate logfiles."
for i in "${DEVICE[@]}"
do
    cat /srv/tftp/config/$i/log/$TODAY-$i.log >> /srv/tftp/temp/$TODAY-saveconfig-report.log
done
echo "$TIMESTAMP saveconfig-spawn: COMPLETED Concatenate logfiles."
#
# Send email report
#
echo "$TIMESTAMP saveconfig-spawn: BEGIN send email report."
countfailed=$(grep -i 'FAILED' /srv/tftp/temp/$TODAY-saveconfig-report.log | wc -l)
countsuccess=$(grep -i 'SUCCESS' /srv/tftp/temp/$TODAY-saveconfig-report.log | wc -l)

mailx -s "Daily Backup Report ($TODAY)" $RCPT <<ENDOFMAIL
Hi,
This is the email with the Daily Backup Report.

My Report summary:
FAILED: $countfailed
SUCCESS: $countsuccess

Backup, log and diff files:
See details in /srv/tftp/config
Thank you
ENDOFMAIL
rm /srv/tftp/temp/$TODAY-saveconfig-report.log
echo "$TIMESTAMP saveconfig-spawn: COMPLETED send email report."
Spawn script, log files and email report

Handling archiving and cleanup

After 15 days the configuration files are archived. The script creates a directory with the archive date and then it copies the configuration files into a temporary directory to tar and gzip them. After this is completed the script does a cleanup. Note: For some commands like mv it's useful to suppress the output with 2> /dev/null, especially when no files are there to archive.


#
# Archiving and cleanup
#
echo "$TIMESTAMP saveconfig-spawn: BEGIN archiving $ARCHIVEDATE."
mkdir /srv/tftp/archive/$ARCHIVEDATE 2> /dev/null
for i in "${DEVICE[@]}"
do
    mv /srv/tftp/config/$i/$ARCHIVEDATE* /srv/tftp/temp/ 2> /dev/null
    mv /srv/tftp/config/$i/diff/$ARCHIVEDATE* /srv/tftp/temp/ 2> /dev/null
    mv /srv/tftp/config/$i/log/$ARCHIVEDATE* /srv/tftp/temp/ 2> /dev/null
    tar -cf /srv/tftp/archive/$ARCHIVEDATE/$ARCHIVEDATE.$i.tar /srv/tftp/temp/* 2> /dev/null
    gzip -9 -f /srv/tftp/archive/$ARCHIVEDATE/$ARCHIVEDATE.$i.tar 2> /dev/null
    rm /srv/tftp/temp/* 2> /dev/null
done
echo "$TIMESTAMP saveconfig-spawn: COMPLETED archiving $ARCHIVEDATE."
Spawn script, archiving and cleanup

Spawn script testing

By starting a test (catalyst and one switch only) the script creates a child process for saveconfig-backup.sh, concatenates the log files, sends out an email report and does it's archiving.


root@linux-devel:~# /srv/tftp/scripts/saveconfig/saveconfig-spawn.sh
2017-10-29 22:43:47 saveconfig-spawn: BEGIN parallel processes.
2017-10-29 22:43:47 saveconfig-spawn: starting parallel process for: catalyst, pid: 376
2017-10-29 22:43:47 saveconfig-backup: BEGIN catalyst.
2017-10-29 22:43:53 saveconfig-backup: SUCCESS swt-test1 / 127.0.0.1.
2017-10-29 22:43:53 saveconfig-backup: COMPLETED catalyst.
2017-10-29 22:43:47 saveconfig-spawn: COMPLETED processes completed.
2017-10-29 22:43:47 saveconfig-spawn: BEGIN concatenate logfiles.
2017-10-29 22:43:47 saveconfig-spawn: COMPLETED Concatenate logfiles.
2017-10-29 22:43:47 saveconfig-spawn: BEGIN send email report.
2017-10-29 22:43:47 saveconfig-spawn: COMPLETED send email report.
2017-10-29 22:43:47 saveconfig-spawn: BEGIN archiving 2017-10-14.
2017-10-29 22:43:47 saveconfig-spawn: COMPLETED archiving 2017-10-14.
root@linux-devel:~#
Executing the spawn script

A check of my development system's local email account shows the report email with the summary of FAILED and SUCCESS devices.


str@linux-devel:/root$ mail
Mail version 8.1.2 01/15/2001.  Type ? for help.
"/var/mail/str": 6 messages 6 unread
>U  1 root@linux-devel.  Sat Oct 28 04:01   29/865   Daily Backup Report (2017-10-28)
 U  2 root@linux-devel.  Sat Oct 28 04:23   29/865   Daily Backup Report (2017-10-28)
 U  3 root@linux-devel.  Sat Oct 28 04:30   29/865   Daily Backup Report (2017-10-28)
 U  4 root@linux-devel.  Sat Oct 28 04:33   29/865   Daily Backup Report (2017-10-28)
 U  5 root@linux-devel.  Sun Oct 29 03:00   29/865   Daily Backup Report (2017-10-29)
 U  6 root@linux-devel.  Sun Oct 29 03:03   29/865   Daily Backup Report (2017-10-29)
& 6
Message 6:
From root@linux-devel.local.virt Sun Oct 29 03:03:15 2017
Envelope-to: str@linux-devel.local.virt
Delivery-date: Sun, 29 Oct 2017 03:03:15 +0100
To: str@linux-devel.local.virt
Subject: Daily Backup Report (2017-10-29)
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
From: root 
Date: Sun, 29 Oct 2017 03:03:15 +0100

Hi,
This is the email with the Daily Backup Report.

My Report summary:
FAILED: 3
SUCCESS: 4

Backup, log and diff files:
See details in /srv/tftp/config
Thank you

& q
Saved 1 message in /home/str/mbox
Held 5 messages in /var/mail/str
str@linux-devel:/root$ 
Email report

By verifying the /srv/tftp/config directory the following files are created:


root@linux-devel:~# tree /srv/tftp/config/
/srv/tftp/config/
├── aironet
│   ├── diff
│   │   └── 2017-10-29-ap-test1.txt
│   └── log
│       └── 2017-10-29-aironet.log
├── catalyst
│   ├── diff
│   │   ├── 2017-10-28-swt-test1.txt
│   │   └── 2017-10-29-swt-test1.txt
│   └── log
│       ├── 2017-10-28-catalyst.log
│       └── 2017-10-29-catalyst.log
├── ciscoasa
│   ├── diff
│   │   └── 2017-10-29-asa-test1.txt
│   └── log
│       └── 2017-10-29-ciscoasa.log
└── ciscowlc
    ├── diff
    │   └── 2017-10-29-wlc-test1.txt
    └── log
        └── 2017-10-29-ciscowlc.log
Directory structure with config files

By verifying the /srv/tftp/archive directory the following files are created


root@linux-devel:~# tree /srv/tftp/archive/
/srv/tftp/archive/
└── 2017-10-13
│   ├── 2017-10-13.aironet.tar.gz
│   ├── 2017-10-13.catalyst.tar.gz
│   ├── 2017-10-13.ciscoasa.tar.gz
│   └── 2017-10-13.ciscowlc.tar.gz
└── 2017-10-14
     ├── 2017-10-14.aironet.tar.gz
     ├── 2017-10-14.catalyst.tar.gz
     ├── 2017-10-14.ciscoasa.tar.gz
     └── 2017-10-144.ciscowlc.tar.gz
Archived files

Download

saveconfig.zip

SHA256sum: 2ec7d9732f631b7aa7bfdc1fb8016aa5b2ebbb628237da6e7b98f5d775473363

If you're using this script, if you find errors or if you just want to leave a comment do not hesitate to get in touch with me by email: stefan [AT] thierolf [DOT] org.

Search my web site