Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
tutorial:adm:server_os_updates [2019/12/16 14:14]
fiserp [Things to consider]
tutorial:adm:server_os_updates [2019/12/16 15:09]
fiserp [Performing the OS update]
Line 29: Line 29:
       * Apache HTTPD. Deployment should be stable and no special care is needed. We recommend to have a backup of vhost configuration.       * Apache HTTPD. Deployment should be stable and no special care is needed. We recommend to have a backup of vhost configuration.
     * Windows-based installations have all deployment components installed by-hand and therefore are not really susceptible to break by OS updates. But this also means you have to update all deployment components manually.     * Windows-based installations have all deployment components installed by-hand and therefore are not really susceptible to break by OS updates. But this also means you have to update all deployment components manually.
 +  * Finding bugs
 +    * It is for the best to have at least two environments - test env. and production env.
 +    * Update the test environment first, then leave it running for at least one week. If no bugs are found by then, you can update the production environment. The one week provides minimal safe time frame where some of the bugs can manifest (e.g. memleaks).
 +    * Define use-cases that are important for your deployment. Before and after the update, test if those use-cases work.
 +
 +==== Performing the OS update ====
 +  - Preparations
 +    - Prepare testing use-cases.
 +    - Prepare backup and restore procedures.
 +    - Identify which LRTs can be safely killed when running.
 +    - Make a checklist with timing information to determine the length of the maintenance.
 +  - Perform the update
 +    - Begin the maintenance.
 +    - (If you use hot snapshots, make one.)
 +    - Make sure no user or external application can access the IdM.
 +    - Log into the IdM as administrator and check if there are some LRTs running.
 +      - If they are not, continue.
 +      - If they are, either stop those LRTs or let them finish. This depends on your deployment.
 +    - Stop the IdM.
 +    - Disable automatic start of the IdM on OS start.
 +    - (If you use cold snapshots, turn of the machine and make one.)
 +    - (If you do not use snapshots, make a backup of the IdM database and store it off-machine.)
 +    - Make backup of ``/boot``, ``/etc``, list of processes ``ps -ef`` and list of network services ``netstat -tulnp`` (or ``ss -tulnp``). Those dumps will help you check if all the services started. You can also recover some settings from backups in case something goes wrong (in a minor way) - you will not need to roll back whole snapshot.
 +    - Perform the update (e.g. ``yum update``).
 +    - Reboot the affected services or the whole machine if necessary.
 +    - When the machine is up, check ``dmesg`` and ``/var/log/{messages,syslog}`` or analogous files for your OS.
 +    - Check running processes and network services whether everything started properly.
 +      - Namely PostgreSQL and HTTPd should be up and running. Those are parts of IdM deployment.
 +    - If everything is ok, start the IdM service.
 +    - Enable autostart of IdM service upon OS start.
 +    - Check IdM logs whether it started successfuly.
 +    - Log into the IdM and test connection to end systems (configuration form for the system, green button "Test connector").
 +    - Check your testing use-cases.
 +    - Allow users to access the IdM.
 +    - End the maintenance.
 +  - Wrap-up
 +    - Update documentation if necessary.
 +    - Perform maintenance analysis and update your procedures if necessary.
 +    - Update your test cases if necessary.
 +    - After about a week, check system logs to make sure all components work as expected.
 +
 +<note>For Windows OSes, the update process is roughly the same. For checking services, status of the system and system logs, use the Event Viewer and Server Manager.</note>
  • by fiserp