Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
tutorial:adm:backups [2019/03/25 18:58] fiserp [Configuration backups] |
tutorial:adm:backups [2020/03/20 09:23] fiserp [Application backups] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
When it comes to backup, the CzechIdM deployments consists of three parts: | When it comes to backup, the CzechIdM deployments consists of three parts: | ||
Line 8: | Line 8: | ||
Depending on your deployment, there can also be sets of scripts, the Vault and so on. But those are highly deployment-specific. | Depending on your deployment, there can also be sets of scripts, the Vault and so on. But those are highly deployment-specific. | ||
+ | <note important> | ||
===== Repository backups ===== | ===== Repository backups ===== | ||
- | Using PostgreSQL allows us to do the '' | + | Using PostgreSQL allows us to do the '' |
- | <file bash enc_backup.sh> | + | |
- | #!/bin/bash | + | |
- | + | ||
- | # ********************************** READ ME ********************************** | + | |
- | # | + | |
- | # General: | + | |
- | # Script is intended to do encrypted backups of whatever you implement in parts | + | |
- | # "do the dump" and "pack the dump". The result of your doing should be a tar | + | |
- | # archive called " | + | |
- | # script will take care of everything else. Presumed shell is BASH. | + | |
- | # | + | |
- | # Output of the script is saved into BACKUP_LOC directory in an encrypted form. | + | |
- | # Each backup consists of two files - symmetric key and public key. Because en- | + | |
- | # cryption is done by openssl, which cannot process an arbitrary file directly | + | |
- | # with RSA, files are first encrypted with random 32B key using AES-256-CBC. | + | |
- | # This 32B key is encrypted with RSA public key which is stored on the machine. | + | |
- | # Private RSA key SHOULD NOT be found anywhere on the same machine. If it was, | + | |
- | # you could do plain backups and not bother with this at all and security would | + | |
- | # be the same. | + | |
- | # | + | |
- | # Needed binaries and builtins: | + | |
- | # test, | + | |
- | # | + | |
- | # Setup: | + | |
- | # 1) Create separate system user to run this script, do not run it as root. | + | |
- | # 2) Generate public-private key pair of at least 2048b: | + | |
- | # openssl genrsa -out backups-rsa-key 2048 | + | |
- | # openssl rsa -in backups-rsa-key -out backups-rsa-key.pub \ | + | |
- | # | + | |
- | # 3) The backups-rsa-key file contains private key, store it in the keepass | + | |
- | # or somewhere safe. Do not leave it on the machine! | + | |
- | # 4) Move backups-rsa-key.pub to BACKUP_ROOT, | + | |
- | # name it as you wish and set RSA_ENC_KEY_FILE accordingly. | + | |
- | # 5) Fill in the "do the dump" and "pack the dump" parts of the script to suit | + | |
- | # your needs. | + | |
- | # 6) Adjust other settings in the script as needed. Ensure that service user | + | |
- | # used for dumping the DB, LDAP, whatever is dedicated to this and has | + | |
- | # read-only privileges! This is IMPORTANT! | + | |
- | # 7) Run the script as a cronjob. Preferred setting is in the crontab, not in the | + | |
- | # /etc/ | + | |
- | # | + | |
- | # Recovering backups: | + | |
- | # Backups are stored in BACKUP_LOC as a pair of files. One file is an actual | + | |
- | # backup encrypted symmetrically. The other file is a symmetric key for the | + | |
- | # specific backup. (New symmetric key is generated for each backup run.) | + | |
- | # Symmetric key is encrypted with RSA. | + | |
- | # | + | |
- | # To recover backups, do the following: | + | |
- | # 1) Get you backups, we will call them " | + | |
- | # 2) Get your private RSA key " | + | |
- | # 3) Decrypt the AES key, you will obtain " | + | |
- | # openssl rsautl -decrypt -inkey backups-rsa-key \ | + | |
- | # -in key.bin.e -out key.bin | + | |
- | # 4) Decrypt the actual backup, you will get a tarball: | + | |
- | # openssl enc -d -aes-256-cbc -in data.tgz.e -out data.tgz \ | + | |
- | # -pass file: | + | |
- | # 5) Extract the tarball: | + | |
- | # tar xf data.tgz | + | |
- | # ***************************************************************************** | + | |
- | # | + | |
- | # TODO: | + | |
- | # * better backups naming | + | |
- | # * something like .d directory where backup scripts will lay to make whole | + | |
- | # thing a bit more modular | + | |
- | # * add actions like " | + | |
- | # | + | |
- | # | + | |
- | # Revision history: | + | |
- | # 2017-05-16 | + | |
- | # * removed hardwired LDAP variables (original script was for LDAP backups) | + | |
- | # * removed hardwired lockfile name | + | |
- | # * PASS_FILE made optional | + | |
- | # * backup timestamp with granularity to seconds instead of hours | + | |
- | # 2016-02-25 | + | |
- | # * first version of the script | + | |
- | + | ||
- | # basic setup | + | |
- | export PATH="/ | + | |
- | #directory where everything happens | + | |
- | #should be empty except for backup scripts, keys and BACKUP_LOC folder | + | |
- | BACKUP_ROOT="/ | + | |
- | #hic sunt backupes | + | |
- | BACKUP_LOC=" | + | |
- | #lockfile | + | |
- | RUN_LOCK=" | + | |
- | BACKUP_PREFIX=" | + | |
- | BACKUP_SUFFIX=" | + | |
- | BACKUP_AES_KEY_PREFIX=" | + | |
- | BACKUP_AES_KEY_SUFFIX=" | + | |
- | #files with public RSA key and password file | + | |
- | #if the password file is not needed, leave the setting blank or the file nonexistent | + | |
- | PASS_FILE=" | + | |
- | RSA_ENC_KEY_FILE=" | + | |
- | #backups retention period | + | |
- | BACKUP_KEEP_DAYS=" | + | |
- | + | ||
- | # setup runtime variables | + | |
- | NOW=$(date +" | + | |
- | BACKUP_FILE_NAME=" | + | |
- | BACKUP_AES_KEY_FILENAME=" | + | |
- | + | ||
- | # check root, must not run as root | + | |
- | if test " | + | |
- | echo " | + | |
- | exit 1 | + | |
- | fi | + | |
- | + | ||
- | # check lock | + | |
- | if test -e " | + | |
- | echo " | + | |
- | exit 1 | + | |
- | fi | + | |
- | + | ||
- | # check binaries we need | + | |
- | if test ! -x `which tar`; then | + | |
- | echo "' | + | |
- | exit 1 | + | |
- | fi | + | |
- | if test ! -x `which openssl`; then | + | |
- | echo "' | + | |
- | exit 1 | + | |
- | fi | + | |
- | + | ||
- | # check files privileges (euid==owner, | + | |
- | if [ -f " | + | |
- | if test ! $(stat -c %a " | + | |
- | echo "File ${PASS_FILE} has incorrect permissions (should be 400) or owner/group (should be `stat -c %U ${0}`)." >& | + | |
- | exit 1 | + | |
- | fi | + | |
- | fi | + | |
- | if test ! $(stat -c %a " | + | |
- | echo "File ${RSA_ENC_KEY_FILE} has incorrect permissions (should be 400) or owner/group (should be `stat -c %U ${0}`)." | + | |
- | exit 1 | + | |
- | fi | + | |
- | + | ||
- | #create lock so we cannot run it more than once | + | |
- | touch " | + | |
- | + | ||
- | #cd to our working dir | + | |
- | cd " | + | |
- | + | ||
- | #generate symmetric key here and push it (asymmetrically encrypted) into a file. this file will accompany symmetrically encrypted tar | + | |
- | #we use aes-256 to encrypt our dumps so we need 32*8=256b symmetric key | + | |
- | SYM_KEY=`openssl rand -base64 32` | + | |
- | + | ||
- | #encrypt the symmetric key | + | |
- | openssl rsautl -encrypt -pubin -inkey " | + | |
- | chmod 600 current_key.bin.e | + | |
- | + | ||
- | #do the dump | + | |
- | # say we run the actual backup and create dump1.dmp, dump2.dmp and dump3.dmp here | + | |
- | + | ||
- | #pack the dump | + | |
- | #tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | + | |
- | tar --remove-files -czf current_backup.tgz dump1.dmp dump2.dmp dump3.dmp | + | |
- | chmod 600 current_backup.tgz | + | |
- | + | ||
- | #encrypt the dump with current symmetric key, also add a pinch of salt | + | |
- | openssl enc -aes-256-cbc -salt -in " | + | |
- | #remove unencrypted dump and key | + | |
- | rm -f current_backup.tgz | + | |
- | + | ||
- | #move encrypted things to backup_loc | + | |
- | mv current_backup.tgz.e " | + | |
- | mv current_key.bin.e " | + | |
- | + | ||
- | #clean up backups older than $BACKUP_KEEP_DAYS days | + | |
- | find " | + | |
- | find " | + | |
- | + | ||
- | #we have finished, remove lock | + | |
- | rm -f " | + | |
- | + | ||
- | exit 0 | + | |
- | </ | + | |
The script needs a public-private keypair to be set up. When making a backup, a symmetric key will be generated. This symmetric key is used to encrypt the actual backup. Symmetric key is then asymmetrically encrypted by the public key and stored alongside the backup. For recovery, you need the backup and its corresponding symmetric key. When recovering, first, you have to decrypt the symmetric key with the private key generated earlier. Then you will use symmetric key to restore the data backup. | The script needs a public-private keypair to be set up. When making a backup, a symmetric key will be generated. This symmetric key is used to encrypt the actual backup. Symmetric key is then asymmetrically encrypted by the public key and stored alongside the backup. For recovery, you need the backup and its corresponding symmetric key. When recovering, first, you have to decrypt the symmetric key with the private key generated earlier. Then you will use symmetric key to restore the data backup. | ||
For instructions about keypair initialization, | For instructions about keypair initialization, | ||
- | When you obtain the repository backup, you can restore the repository: | + | When you obtain the repository backup, |
- Stop the identity manager container. | - Stop the identity manager container. | ||
- Backup current repository somewhere else - in case you need to check some data later. | - Backup current repository somewhere else - in case you need to check some data later. | ||
- Delete all data from the repository / drop the repository itself. This depends on the backup created - if you have database creation statements in there and such. | - Delete all data from the repository / drop the repository itself. This depends on the backup created - if you have database creation statements in there and such. | ||
- | - Restore the data from the pgdump with '' | + | - Restore the data from the pgdump with '' |
- Start the identity manager. | - Start the identity manager. | ||
- | |||
- | <note important> | ||
**Script deployment** | **Script deployment** | ||
Line 229: | Line 53: | ||
#do the dump | #do the dump | ||
# say we run the actual backup and create dump1.dmp, dump2.dmp and dump3.dmp here | # say we run the actual backup and create dump1.dmp, dump2.dmp and dump3.dmp here | ||
+ | # STRONGLY ADVISED TO GZIP YOUR BACKUPS, SCRIPT DOES NOT DO THAT FOR YOU !!! | ||
+ | |||
#pack the dump | #pack the dump | ||
#tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | #tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | ||
- | tar --remove-files -czf current_backup.tgz dump1.dmp dump2.dmp dump3.dmp | + | tar --remove-files -cf current_backup.tar PUT-YOUR-FILES-HERE |
+ | chmod 600 current_backup.tar | ||
</ | </ | ||
- | And change them to (expected name of the czechidm database is " | + | And change them to (expected name of the czechidm database is '' |
<code bash> | <code bash> | ||
#do the dump | #do the dump | ||
- | pg_dump --create --dbname=idm > idm.sql | + | pg_dump --create |
#pack the dump | #pack the dump | ||
#tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | #tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | ||
- | tar --remove-files -czf current_backup.tgz | + | tar --remove-files -czf current_backup.tgz |
</ | </ | ||
Line 311: | Line 138: | ||
In some cases, CzechIdM is not deployed with frontend and backend bundled together in the '' | In some cases, CzechIdM is not deployed with frontend and backend bundled together in the '' | ||
+ | |||
+ | ===== Restoring IdM application ===== | ||
+ | This is a basic DR howto for restoring the identity manager in case you lose it. It does not deal with other disaster scenarios (i.e. IdM bulk-propagating bad data to other systems). |