Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
tutorial:adm:backups [2017/05/29 07:55] 127.0.0.1 external edit |
tutorial:adm:backups [2020/03/20 10:13] fiserp [Restoring IdM application] |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Server preparation - Backup and Recovery ====== | ||
+ | When it comes to backup, the CzechIdM deployments consists of three parts: | ||
+ | * Identity manager' | ||
+ | * Application' | ||
+ | * Configuration folders under ''/ | ||
+ | |||
+ | Depending on your deployment, there can also be sets of scripts, the Vault and so on. But those are highly deployment-specific. | ||
+ | |||
+ | <note important> | ||
+ | ===== Repository backups ===== | ||
+ | Using PostgreSQL allows us to do the '' | ||
+ | |||
+ | The script needs a public-private keypair to be set up. When making a backup, a symmetric key will be generated. This symmetric key is used to encrypt the actual backup. Symmetric key is then asymmetrically encrypted by the public key and stored alongside the backup. For recovery, you need the backup and its corresponding symmetric key. When recovering, first, you have to decrypt the symmetric key with the private key generated earlier. Then you will use symmetric key to restore the data backup. | ||
+ | For instructions about keypair initialization, | ||
+ | |||
+ | When you obtain the repository backup, **you can restore the repository**: | ||
+ | - Stop the identity manager container. | ||
+ | - Backup current repository somewhere else - in case you need to check some data later. | ||
+ | - Delete all data from the repository / drop the repository itself. This depends on the backup created - if you have database creation statements in there and such. | ||
+ | - Restore the data from the pgdump with '' | ||
+ | - Start the identity manager. | ||
+ | |||
+ | **Script deployment** | ||
+ | |||
+ | This is a standard deployment scenario. First, create the directory structure and setup the script and keys: | ||
+ | < | ||
+ | mkdir -p / | ||
+ | chown -Rf postgres: | ||
+ | chmod 700 / | ||
+ | # deploy the script above: | ||
+ | vim / | ||
+ | chown root: | ||
+ | chmod 750 / | ||
+ | # create public-private keypair on YOUR machine, NOT on the CzechIdM server | ||
+ | # copy ONLY the public key to the CzechIdM server, STORE the private key SAFELY | ||
+ | scp publickey.key czechidm-server:/ | ||
+ | chown root: | ||
+ | chmod 440 / | ||
+ | </ | ||
+ | |||
+ | Then, configure the backup script: | ||
+ | <code bash> | ||
+ | BACKUP_ROOT="/ | ||
+ | BACKUP_LOC=" | ||
+ | BACKUP_PREFIX=" | ||
+ | BACKUP_AES_KEY_PREFIX=" | ||
+ | RSA_ENC_KEY_FILE=" | ||
+ | </ | ||
+ | |||
+ | Plain version of the script does not come with commands for making the actual backup, script serves as a wrapper which will encrypt whatever you need. Therefore, you have to check the script if there is actually a backup command specified and, if not, fill it in. If you want, you can add other things to the encrypted backup. See the script for lines: | ||
+ | <code bash> | ||
+ | #do the dump | ||
+ | # say we run the actual backup and create dump1.dmp, dump2.dmp and dump3.dmp here | ||
+ | # STRONGLY ADVISED TO GZIP YOUR BACKUPS, SCRIPT DOES NOT DO THAT FOR YOU !!! | ||
+ | |||
+ | |||
+ | #pack the dump | ||
+ | #tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | ||
+ | tar --remove-files -cf current_backup.tar PUT-YOUR-FILES-HERE | ||
+ | chmod 600 current_backup.tar | ||
+ | </ | ||
+ | And change them to (expected name of the czechidm database is '' | ||
+ | <code bash> | ||
+ | #do the dump | ||
+ | pg_dump --create -Z 9 --dbname=czechidm > czechidm.sql.gz | ||
+ | |||
+ | #pack the dump | ||
+ | #tar usage "tar [parameters] archive_name file1 [file2 file3 ...]" | ||
+ | tar --remove-files -czf current_backup.tgz czechidm.sql.gz | ||
+ | </ | ||
+ | |||
+ | Last thing to do is to set up a cronjob. Here we set up a daily backup to 04:00. It may be useful to change the time the backup is done because there may be batch jobs running inside the identity manager. Set the job to run under the postgres user. | ||
+ | < | ||
+ | crontab -l -u postgres | ||
+ | 00 04 * * * / | ||
+ | </ | ||
+ | |||
+ | ===== Configuration backups ===== | ||
+ | Everything about the identity manager lives in ''/ | ||
+ | * Losing repository password is not a problem (you can always set the new password in postgres) but it is a complication. | ||
+ | * Losing '' | ||
+ | |||
+ | For simplicity reasons (and also because all the files count up to few kilobytes in size) we recommend backing up whole configuration folder. **It is vital that this backup is encrypted, as it contains the confidential storage key.** | ||
+ | |||
+ | **Implementation** | ||
+ | * For implementation, | ||
+ | * Create new instance of the script. | ||
+ | * Ideally generate separate RSA keypair for it. | ||
+ | * Do not pack the '' | ||
+ | ===== Application backups ===== | ||
+ | CzechIdM is a Java application distributed as a WAR archive. This archive is deployed inside a Apache Tomcat container. For recovery of the application, | ||
+ | * If you use standard application release without any added modules and customizations, | ||
+ | * If you customized identity manager' | ||
+ | * If you still want to backup the '' | ||
+ | #!/bin/bash | ||
+ | |||
+ | BACKUP_ROOT="/ | ||
+ | BACKUP_DIR=" | ||
+ | LOCKFILE=" | ||
+ | BACKUP_KEEP_DAYS=" | ||
+ | |||
+ | NOW=$(date +" | ||
+ | |||
+ | if [ -f " | ||
+ | echo " | ||
+ | exit 1 | ||
+ | fi | ||
+ | touch " | ||
+ | |||
+ | tar cpfz " | ||
+ | find " | ||
+ | rm " | ||
+ | exit 0 | ||
+ | </ | ||
+ | |||
+ | When doing the recovery, simply put the CzechIdM WAR archive into the same folder as you would normally do and restart the Tomcat container. | ||
+ | |||
+ | **Script deployment** | ||
+ | |||
+ | If you really want to use the '' | ||
+ | < | ||
+ | mkdir -p / | ||
+ | # this is because we usually do repository backup to sibling directories in the tree | ||
+ | chown postgres: | ||
+ | chown root:root / | ||
+ | chmod 700 / | ||
+ | # deploy the script above: | ||
+ | vim / | ||
+ | chown root:root / | ||
+ | chmod 750 / | ||
+ | </ | ||
+ | At last, set up a cronjob: | ||
+ | < | ||
+ | crontab -l -u root | ||
+ | 50 03 * * * / | ||
+ | </ | ||
+ | |||
+ | In some cases, CzechIdM is not deployed with frontend and backend bundled together in the '' | ||
+ | |||
+ | ===== Restoring IdM application ===== | ||
+ | < | ||
+ | This is a basic DR howto for restoring the identity manager in case you lose it. It does not deal with other disaster scenarios. | ||
+ | |||
+ | If you backup your environment in some other way, virtual machine snapshots for example, use your DR procedures. | ||
+ | </ | ||
+ | |||
+ | When the application is lost - due to HW or virtualization failure, human error or due to security compromise, you can restore it using backups and documentation. In this case, we show how to restore everything on the clean operating system installation. | ||
+ | - Install the operating system. | ||
+ | - Configure the OS according to your internal standards. | ||
+ | - Configure the OS according to [[https:// | ||
+ | - Deploy and configure the CzechIdM according to [[https:// | ||
+ | - When creating a database user and CzechIdM database in the PostgreSQL, use credentials you already used before the failure. Restore the database from backup, for example '' | ||
+ | - **Do not** create brand new configuration in ''/ | ||
+ | - **Do not** download new '' | ||
+ | - Disable all new outgoing connections from the IdM machine **except for communication between your station and IdM server**. | ||
+ | - This way, the IdM will not start to communicate with end systems until you check its data is consistent. | ||
+ | - But you will still be able to access the web UI. | ||
+ | - Start the Tomcat container and wait for the identity manager to deploy. | ||
+ | - Log into the application as an administrator (use locally-authenticated account - any account that was granted '' | ||
+ | - Disable LRTs, kill all those that are running. | ||
+ | - Check data in the application: | ||
+ | - Allow outgoing connections from the IdM machine. | ||
+ | - Test connections to all end systems, reprovision some users to end systems. Check event and provisioning queues for any errors and resolve them if needed. | ||
+ | - Test your general use-cases / UAT tests to make sure the application works as intended. | ||
+ | - Schedule LRTs. |