Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
tutorial:adm:server_os_updates [2019/12/17 07:36]
fiserp [Things to consider] 		tutorial:adm:server_os_updates [2019/12/17 07:53]
fiserp [Solving issues]
Line 39: Line 39:
     - Prepare backup and restore procedures.     - Prepare backup and restore procedures.
     - Identify which LRTs can be safely killed when running.     - Identify which LRTs can be safely killed when running.
-    - Make a checklist with timing information to determine the length of the maintenance.+    - Make a checklist with timing estimates to determine the length of the maintenance.
   - Perform the update   - Perform the update
     - Begin the maintenance.     - Begin the maintenance.
 +    - Disable monitoring system notifications.
     - (If you use hot snapshots, make one.)     - (If you use hot snapshots, make one.)
     - Make sure no user or external application can access the IdM.     - Make sure no user or external application can access the IdM.
Line 49: Line 50:
     - Stop the IdM.     - Stop the IdM.
     - Disable automatic start of the IdM on OS start.     - Disable automatic start of the IdM on OS start.
-    - (If you use cold snapshots, turn of the machine and make one.)+    - (If you use cold snapshots, turn off the machine and make one.)
     - (If you do not use snapshots, make a backup of the IdM database and store it off-machine.)     - (If you do not use snapshots, make a backup of the IdM database and store it off-machine.)
     - Make backup of ``/boot``, ``/etc``, list of processes ``ps -ef`` and list of network services ``netstat -tulnp`` (or ``ss -tulnp``). Those dumps will help you check if all the services started. You can also recover some settings from backups in case something goes wrong (in a minor way) - you will not need to roll back whole snapshot.     - Make backup of ``/boot``, ``/etc``, list of processes ``ps -ef`` and list of network services ``netstat -tulnp`` (or ``ss -tulnp``). Those dumps will help you check if all the services started. You can also recover some settings from backups in case something goes wrong (in a minor way) - you will not need to roll back whole snapshot.
     - Perform the update (e.g. ``yum update``).     - Perform the update (e.g. ``yum update``).
-    - Reboot the affected services or the whole machine if necessary.+    - Restart affected services or reboot the whole machine if necessary.
     - When the machine is up, check ``dmesg`` and ``/var/log/{messages,syslog}`` or analogous files for your OS.     - When the machine is up, check ``dmesg`` and ``/var/log/{messages,syslog}`` or analogous files for your OS.
     - Check running processes and network services whether everything started properly.     - Check running processes and network services whether everything started properly.
       - Namely PostgreSQL and HTTPd should be up and running. Those are parts of IdM deployment.       - Namely PostgreSQL and HTTPd should be up and running. Those are parts of IdM deployment.
     - If everything is ok, start the IdM service.     - If everything is ok, start the IdM service.
-    - Enable autostart of IdM service upon OS start. 
     - Check IdM logs whether it started successfuly.     - Check IdM logs whether it started successfuly.
     - Log into the IdM and test connection to end systems (configuration form for the system, green button "Test connector").     - Log into the IdM and test connection to end systems (configuration form for the system, green button "Test connector").
     - Check your testing use-cases.     - Check your testing use-cases.
 +    - Enable autostart of IdM service upon OS start.
 +    - (If there were changes to the database (e.g. PostgreSQL major version upgrade), make a backup of the upgraded database.)
     - Allow users to access the IdM.     - Allow users to access the IdM.
 +    - Enable monitoring system notifications.
     - End the maintenance.     - End the maintenance.
   - Wrap-up   - Wrap-up
Line 72: Line 75:
 <note>For Windows OSes, the update process is roughly the same. For checking services, status of the system and system logs, use the Event Viewer and Server Manager.</note> <note>For Windows OSes, the update process is roughly the same. For checking services, status of the system and system logs, use the Event Viewer and Server Manager.</note>
  
-==== Solving issues ====+==== Resolving issues ====
 For maintenance actions, it is necessary to: For maintenance actions, it is necessary to:
   * Know how long each task will take and to measure the task duration when actually performing them.   * Know how long each task will take and to measure the task duration when actually performing them.
Line 80: Line 83:
   * Know how long (at worst) the whole rollback will take (rollback time **RT**).   * Know how long (at worst) the whole rollback will take (rollback time **RT**).
   * Have a maintenance window that spans at least **MT**+**RT** with some extra time **ET**.   * Have a maintenance window that spans at least **MT**+**RT** with some extra time **ET**.
-    * You are not able to safely perform the maintenance in shorter window, there is simply not enough time. If something goes wrong, you need at most **RT** time to perform the rollback! +    * You are not able to safely perform the maintenance in shorter window, there is simply not enough time. If something goes wrong, you will need **RT** time to perform the rollback! 
-    * If you do not have any **ET**, if anything goes wrong you have to perform rollback procedure. Therefore, **ET** gives you some time you can spend on solving the issue so you can carry on with updates. +    * When you have no **ET**, if anything goes wrong you have to perform rollback procedure. Therefore, **ET** gives you some time you can spend on solving the issue so you can carry on with updates.
- +
-You should have a rollback procedure that can safely restore the deployment. This depends on your environment. +
- +
-Fortunately, in most cases it simply means restoring the snapshot of the virtual machine. After restoring the snapshot, you have to perform tests (with test use-cases) to confirm the rollback was performed correctly. +
-Minor issues can be generally resolved with the help of ``/boot`` and ``/etc`` backups you created before updating the OS.+
  
-If IdM installation gets hit, you can debug the configuration or restore it from periodic backup. Since IdM is not installed from OS packages, this basically never happens.+  * You should have a rollback procedure that can safely restore the deployment. 
 +    * This depends on your environment and on the way you updated OS packages. 
 +  * Fortunately, in most cases it simply means restoring the snapshot of the virtual machine. 
 +    * After restoring the snapshot, you have to perform tests (with test use-cases) to confirm the rollback was performed correctly. 
 +    * Minor issues can be generally resolved with the help of ``/boot`` and ``/etc`` backups you created before updating the OS. 
 +  * If IdM installation gets hit, you can debug the configuration or restore it from periodic backup. Since IdM is not installed from OS packages, this basically never happens.
  • by fiserp