Differences

This shows you the differences between two versions of the page.

--- tutorial:adm:server_preparation_tmp [2020/03/12 12:29]
urbanl [HTTPd installation and configuration]
+++ tutorial:adm:server_preparation_tmp [2020/07/24 08:19]
fiserp [Database server installation - CentOS8]
@@ Line 1: / Line 1: @@
-<note important>temp pro centos 8
+<note important>Instalation pro CentOS8
+**This tutorial is under development, DO NOT USE.**
 Author: Ludek Urban
@@ Line 5: / Line 7: @@
-====== Server preparation - Linux ======
+====== Server preparation - Linux - CentOS8 ======
 {{tag>installation java tomcat quickstart "apache httpd"}}
-This tutorial shows how to prepare the server for test or production usage of CzechIdM. If you are looking for much quicker way of how to start the CzechIdM, use the demo setup described here [[:getting-started|]]
+This tutorial shows how to prepare the server for test or production use of CzechIdM. If you are looking for much quicker way of how to start the CzechIdM, use the demo setup described here [[:getting-started|]]
 ===== Basic system setup =====
-  * 1 server (can be virtualized) for all: backend, frontend and database.
+  * 1 server (can be virtualized) for everything: backend, frontend and database.
-  * OS Linux with EPEL repository enabled - CENTOS, basic network enabled installation
+  * OS Linux with EPEL repository enabled - CentOS, basic network enabled installation
-    * It is possible to use Debian but you have to adjust the installation guide a little. We tested CzechIdM installation on Stretch.
+    * It is possible to use Debian (we tested on Stretch) or other distributions, but you have to adjust steps in this guide accordingly.
-  * PostgreSQL - installed from a new repository
+  * PostgreSQL 12.x - installed from OS packages.
-  * Java - distribution repository (OpenJDK 1.8)
+  * Java 11 - installed from OS packages.
-  * Apache Tomcat - manually installed into /opt/tomcat
+  * Apache Tomcat 8.5.x - installed manually into ''/opt/tomcat''.
-  * Services start via systemd in OS
+  * Apache HTTPd 2.4.x - installed from OS packages. Can be replaced by nGinx.
-  * Services run under dedicated user (non-privileged one)
+  * All services start via systemd.
+  * Each service runs under dedicated non-privileged user.
 ===== Instalation and software configuration =====
-Prerequisities - Basic installation of CentOS 7
+Prerequisities - Basic installation of CentOS 8
 <code bash>
 # EPEL installation
-yum clean all
+dnf clean all
-yum install -y epel-release
+dnf -y install epel-release
-yum update -y
+dnf update -y
 # other recommended packages installation
-yum install -y net-tools nano wget mc vim-enhanced screen sysstat bzip2 ssmtp bash-completion lsof haveged nmap zip unzip psmisc telnet policycoreutils-python
+dnf -y install mc haveged nmap screen sysstat telnet net-tools nano wget vim-enhanced bzip2 bash-completion lsof zip unzip psmisc policycoreutils-python-utils tar
 # enable haveged after OS start
 systemctl start haveged.service
 systemctl enable haveged.service
-# remove unnecessary software
-yum remove -y postfix
-systemctl stop avahi-daemon.socket avahi-daemon.service
-systemctl disable avahi-daemon.socket avahi-daemon.service
-yum remove -y avahi-autoipd avahi
 # set the hostname
 hostnamectl set-hostname FQDN_server_name
@@ Line 45: / Line 45: @@
 </code>
--!CHANGED
+===== PostgreSQL  =====
-When installing to centos8, check and install these packages:
+<note tip>If you are installing CzechIdM on Microsoft SQL Server, please follow [[tutorial:adm:mssql_database_support|this tutorial]].</note>
-<code>
+We install PostgreSQL 12 database binaries and set database data directory to ''/data'' not ''/var/lib'' which is the default option.
-# check installed packages. It's recommanded to have them installed.
+==== Database server installation - CentOS8 ====
-yum list installed  net-tools nano wget  vim-enhanced bzip2 bash-completion lsof zip unzip psmisc policycoreutils-python-utils
-# other recommended packages installation
-yum install -y mc haveged nmap screen sysstat telnet
-</code>
-When installing on Debian, install these packages:
-<code>
-screen dnsutils sysstat lsof haveged nmap tcpdump traceroute tcptraceroute curl iptables-persistent
-</code>
-===== PostgreSQL -!CHANGED =====
-<note tip>If you are install CzechIdM on Sql server, please follow [[tutorial:adm:mssql_database_support|this tutorial]].</note>
-CentOS8 default repository version of PostgreSQL is 10 but IdM not support that version. In our tutorial, we will install newer version 12. Moreover, we install database data into /data not /var/lib which is the default option.
-==== Database server installation - CentOS8 -!CHANGED ====
   * Software installation on CentOS8(versions can vary):
 <code bash>
 # enable module postgres 12
-yum module enable postgresql:12
+dnf module enable postgresql:12
-yum install -y postgresql-server postgresql-contrib postgresql-libs
+dnf -y install postgresql-server postgresql-contrib postgresql-libs
 </code>
-  * create new system directory:
+  * create new directory for database data:
 <code bash>
-mkdir -p /data/pgsql/12/data/
 mkdir -p /data/pgsql/12/data/
 chown -R postgres:postgres /data/pgsql/
@@ Line 79: / Line 65: @@
 </code>
-  * Copy of the configuration file for systemd, in which we will make change of directory for data:
+  * Copy the PostgreSQL's systemd unit to the /etc:
 <code bash>
@@ Line 91: / Line 77: @@
 </code>
-  * In the file ''~postgres/.bash_profile'' change the variable PGDATA value to:
+  * In the file ''/var/lib/pgsql/.bash_profile'' (bash profile for postgres user) change the variable PGDATA to:
 <code>
@@ Line 108: / Line 94: @@
 <code bash>
-/usr/bin/postgresql-setup --initdb --unit postgresql
+postgresql-setup --initdb --unit postgresql
 </code>
 Change SELINUX labels:
 <code>
-chcon -Rt postgresql_db_t pgsql/
+chcon -Rt postgresql_db_t /data/pgsql/
 chcon -Rt postgresql_log_t /data/pgsql/12/data/log/
 </code>
@@ Line 155: / Line 141: @@
 </code>
-==== Database server installation - Debian Stretch ====
-Install the database from OS packages:
+==== DB server configuration ====
-<code>
-apt-get install postgresql-9.6
-</code>
-We will move the database - create directory structure:
-<code>
-mkdir -p /data/pgsql/9.6/data/
-chown -R postgres:postgres /data/pgsql/
-chmod -R 700 /data/pgsql
-</code>
-Create the file .bash\_profile in postgres user's home (default /var/lib/postgresql) with following contents:
-<code>
-PGDATA=/data/pgsql/9.6/data
-</code>
-Stop the database:
-<code>
-systemctl stop postgresql
-</code>
-Move database directory (run this as root):
-<code>
-mv /var/lib/postgresql/9.6/main/* /data/pgsql/9.6/data/
-</code>
-In the PostgreSQL configuration file /etc/postgresql/9.6/main/postgresql.conf set the data\_directory property to:
-<code>
-data_directory = '/data/pgsql/9.6/data'
-</code>
-Enable and start the database:
-<code>
-systemctl start postgresql
-systemctl enable postgresql
-</code>
-==== DB server configuration -!CHANGED ====
 First of all, enable the password authentication.
@@ Line 224: / Line 179: @@
 Restart DB: ''systemctl restart  postgresql.service''
-For Debian installation, edit those configuration files instead:
-<code>
-/etc/postgresql/12/main/pg_hba.conf
-/etc/postgresql/12/main/postgresql.conf
-</code>
 <note>If you install the database to a different server than the CzechIdM application itself (Tomcat etc.), don't forget to configure PostgreSQL to allow remote SSL connection from that server.</note>
-===== Java - CentOS8 -! CHANGED =====
+===== Java - CentOS8 =====
 Java must be installed before Tomcat start. It is recommended to use OpenJDK (at least 1.11) from standard OS repository.
@@ Line 247: / Line 196: @@
 </file>
-===== Java - Debian =====
-Java must be installed before Tomcat start. It is recommended to use OpenJDK (at least 1.8) from standard OS repository.
-Installation:
-<code bash>
-apt-get install openjdk-8-jdk-headless openjdk-8-jre-headless
-</code>
-Then create the file ''/etc/profile.d/java.sh'' with following:
-<file bash java.sh>
-[ -d /usr/lib/jvm/java-1.8.0-openjdk-amd64 ] && export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
-</file>
 ===== Tomcat =====
-  * Create a new group and add user for the tomcat to run under (for Debian, use /usr/sbin/nologin in the useradd):
+  * Create a new group and add user for the tomcat to run under:
 <code>
@@ Line 269: / Line 205: @@
 useradd -r -s /bin/nologin -g tomcat -d /opt/tomcat tomcat
 getent passwd tomcat
-tomcat:x:995:993::/opt/tomcat:/bin/nologin
+#tomcat:x:995:993::/opt/tomcat:/bin/nologin
 </code>
@@ Line 351: / Line 287: @@
   * Tomcat will be started under user ''tomcat:tomcat'' a will use java installed in ''/usr/lib/jvm/java-1.8.0-openjdk''.
-  * For Debian, change the JAVA\_HOME to ''JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64''.
   * After every systemd configuration change it is necessary to reload:
@@ Line 433: / Line 368: @@
 It is advised to follow these steps for production usage:
-  * Remove unnecessary aplications that comes with Tomcat:
+  * Remove unnecessary applications that comes with Tomcat:
 <code bash>
@@ Line 445: / Line 380: @@
 <Server port="-1" shutdown="SHUTDOWN">
 </code>
--! CHANGED
   * Make Tomcat listen only on localhost:
     * In the ''/opt/tomcat/current/conf/server.xml'' add the ''address="127.0.0.1"'' property to configuration of ''8080'' port.
-    * In same file configure ajp port(''8009'') to look like this:
-    <Connector protocol="AJP/1.3"
+  * Set the ''maxSwallowSize'' for the HTTP/1.1 connector:
+    * In the ''/opt/tomcat/current/conf/server.xml'', locate the configuration for port 8080 and add the ''maxSwallowSize="-1"'' property therein.
+  * In same file configure ajp port(''8009'') to look like this:
+<code>
+<Connector protocol="AJP/1.3"
                 address="127.0.0.1"
                 secretRequired="true"
@@ Line 456: / Line 396: @@
                 port="8009"
                 redirectPort="8443" />
+</code>
   * Do not show aplication server version:
@@ Line 485: / Line 425: @@
 <file txt tomcat>
 /opt/tomcat/current/logs/catalina.out {
-	rotate COUNT
+	rotate 90
 	daily
 	dateext
@@ Line 509: / Line 449: @@
   * Adjust particular SELinux labels. Example ([[https://access.redhat.com/solutions/39006|here]]).
 </note>
-Please note that on Debian, the log is not rotate during the first day, but after the second day.
@@ Line 529: / Line 467: @@
 </code>
-On Debian install those packages and allow modules:
+HTTPd basic configuration:
-<code>
-apt-get install apache2 libapache2-mod-security2 modsecurity-crs
-a2enmod ssl
-a2enmod proxy
-a2enmod proxy_ajp
-a2enmod proxy_http
-a2enmod security2
-a2enmod rewrite
-a2enmod headers
-</code>
-HTTPd basic configuration -!CHANGED:
 Change MPM to worker (lower system requirements) - in the file ''/etc/httpd/conf.modules.d/00-mpm.conf'' comment all lines but mod\_mpm\_worker.so:
@@ Line 573: / Line 499: @@
 </code>
-Virtualhost configuration to forward the communication from port 80 to 443. Add following section and change string 'server' to the real servername in the file ''/etc/httpd/conf.d/vhost-redirect.conf'' (or ''/etc/apache2/sites-available/vhost-redirect.conf'' for Debian):
+Virtualhost configuration to forward the communication from port 80 to 443. Add following section and change string 'server' to the real servername in the file ''/etc/httpd/conf.d/vhost-redirect.conf'':
 <code xml>
 <VirtualHost _default_:80>
@@ Line 581: / Line 507: @@
 </code>
-Set the  proxy in the virtualhost for https (443/tcp) - at the end of the file ''/etc/httpd/conf.d/ssl.conf'' (or ''/etc/apache2/sites-available/ssl.conf'' for Debian) add following before ending "tag" VirtualHost:
+Set the  proxy in the virtualhost for https (443/tcp) - at the end of the file ''/etc/httpd/conf.d/ssl.conf'' add following before ending "tag" VirtualHost:
 <code>
+  Protocols       h2 https/1.1
   ProxyRequests     off
   ProxyPreserveHost on
@@ Line 613: / Line 540: @@
 </code>
 <note>In some cases older clients (i.e. IE10 and older, Java6, etc.) will not be able to communicate with IdM. If this is your case, you may need to slacken the cipher settings a bit.</note>
-On Debian, create symlinks to sites-enabled:
-<code>
-cd /etc/apache2/sites-enabled
-ln -s ../sites-available/vhost-redirect.conf 01vhost-redirect.conf
-ln -s ../sites-available/ssl.conf 02ssl.conf
-</code>
 Syntax check before httpd restart:
@@ Line 631: / Line 551: @@
 </code>
+Allow in SELINUX to httpd connect to network:
+<code>
+/usr/sbin/setsebool -P httpd_can_network_connect 1
+</code>
 Enable httpd after OS start:
 <code bash>
@@ Line 637: / Line 562: @@
 ===== mod_security configuration =====
-Mod_security files locations (on CentOS7):
+Mod_security files locations (on CentOS8):
   * Audit log: ''/var/log/httpd/modsec\_audit.log''
   * Directory with activated rules: ''/etc/httpd/modsecurity.d/activated\_rules/''
-  * basic configuration file for mod\_security: ''/etc/httpd/modsecurity.d/modsecurity\_crs\_10\_config.conf''
+  * basic configuration file for mod\_security: '' /etc/httpd/modsecurity.d/activated_rules/REQUEST-901-INITIALIZATION.conf''
   * The file for chosen rules deactivation: ''/etc/httpd/conf.d/ssl.conf''
@@ Line 656: / Line 581: @@
 ==== Disabling mod_security rules ====
-In the file ''/etc/httpd/conf.d/ssl.conf'' (or ''/etc/apache2/sites-available/ssl.conf'' for Debian) deactivate following rules and set their logging:
+These rules are disabled for modsec_crs 3.0.
+In the file ''/etc/httpd/conf.d/ssl.conf'' deactivate following rules and set their logging:
 <code xml>
 <IfModule mod_security2.c>
-        SecRuleRemoveById 981173
+        SecRuleRemoveById 942430
-        SecRuleRemoveById 960015
+        SecRuleRemoveById 942431
-        SecRuleRemoveById 950109
+        SecRuleRemoveById 920300
+        SecRuleRemoveById 920230
         # Allow Czech signs
-        SecRuleRemoveById 981318
+        SecRuleRemoveById 942110
-        SecRuleRemoveById 981242
+        SecRuleRemoveById 942330
-        SecRuleRemoveById 960024
+        SecRuleRemoveById 942460
-        SecRuleRemoveById 981245
+        SecRuleRemoveById 942260
         # Too restrictive for login format
-        SecRuleRemoveById 960035
+        SecRuleRemoveById 920440
         # Needed by Websockets
         <Location "/idm/api/v1/websocket-info/">
-                SecRuleRemoveById 970901
+                SecRuleRemoveById 950100
         </Location>
-        # These break Certificate Authority module
-	<Location "/idm/api/v1/crt/certificates/action/validate">
-		SecRuleRemoveById 960915
-		SecRuleRemoveById 200003
-	</Location>
-	# Modsec can throw false positives on some files due to multipart boundary check
-	<Location "/idm/api/v1/attachments/upload">
-		SecRuleRemoveById 960915
-		SecRuleRemoveById 200003
-	</Location>
         # do not log request/response body
         SecAuditLogParts ABFHZ
@@ Line 694: / Line 611: @@
 </code>
-==== mod_security configuration - CentOS7  ====
+==== mod_security configuration - CentOS8  ====
-In the file /etc/httpd/modsecurity.d/modsecurity\_crs\_10\_config.conf, find the rule with id=900012 and add support for content\_type=application/json, application/hal+json and text/plain on the line starting with tx.allowed\_request\_content\_type, then allow PUT DELETE and PATCH methods on the line with tx.allowed\_methods.
+In the file /etc/httpd/modsecurity.d/activated_rules/REQUEST-901-INITIALIZATION.conf
-Whole rule after the changes looks like this:
+  * find the rule 900200 and add methods PUT DELETE and PATCH on the line with tx.allowed\_methods. It look like this after change:
 <code>
-SecAction \
+# Default HTTP policy: allowed_methods (rule 900200)
-  "id:'900012', \
+SecRule &TX:allowed_methods "@eq 0" \
-  phase:1, \
+    "id:901160,\
-  t:none, \
+    phase:1,\
-  setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE', \
+    pass,\
-  setvar:'tx.allowed_request_content_type=application/hal+json|application/json|text/plain|application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf', \
+    nolog,\
-  setvar:'tx.allowed_http_versions=HTTP/0.9 HTTP/1.0 HTTP/1.1', \
+    setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
-  setvar:'tx.restricted_extensions=.asa/ .asax/ .ascx/ .axd/ .backup/ .bak/ .bat/ .cdx/ .cer/ .cfg/ .cmd/ .com/ .config/ .conf/ .cs/ .csproj/ .csr/ .dat/ .db/ .dbf/ .dll/ .dos/ .htr/ .htw/ .ida/ .idc/ .idq/ .inc/ .ini/ .key/ .licx/ .lnk/ .log/ .mdb/ .old/ .pass/ .pdb/ .pol/ .printer/ .pwd/ .resources/ .resx/ .sql/ .sys/ .vb/ .vbs/ .vbproj/ .vsdisco/ .webinfo/ .xsd/ .xsx/', \
-  setvar:'tx.restricted_headers=/Proxy-Connection/ /Lock-Token/ /Content-Range/ /Translate/ /via/ /if/', \
-  nolog, \
-  pass"
 </code>
-==== mod_security configuration - Debian ====
+  * find the rule 900220 and add support for content\_type=application/json, application/hal+json and text/plain on the line starting with tx.allowed\_request\_content\_type, after change:
-Enable mod\_security configuration:
-<code>
-cd /etc/modsecurity
-cp modsecurity.conf-recommended modsecurity.conf
-</code>
-Uncomment following rules in the ''/etc/modsecurity/crs/crs-setup.conf'' and change them accordingly (add allowed content types and allowed HTTP methods):
 <code>
-SecAction \
+# Default HTTP policy: allowed_request_content_type (rule 900220)
- "id:900200,\
+SecRule &TX:allowed_request_content_type "@eq 0" \
-  phase:1,\
+    "id:901162,\
-  nolog,\
+    phase:1,\
-  pass,\
+    pass,\
-  t:none,\
+    nolog,\
-  setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
+    setvar:'tx.allowed_request_content_type=application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf|application/json|text/plain|application/hal+json'"
-SecAction \
- "id:900220,\
-  phase:1,\
-  nolog,\
-  pass,\
-  t:none,\
-  setvar:'tx.allowed_request_content_type=application/x-www-form-urlencoded|multipart/form-data|text/xml|application/xml|application/x-amf|application/json|text/plain|application/hal+json'"
 </code>
@@ Line 778: / Line 678: @@
 </code>
-===== Workaround for slow HTTPD shutdown =====
-In some RHEL/CentOS versions Apache HTTPD shutsdown or restarts itself very slowly. It is caused by [[https://bugzilla.redhat.com/show_bug.cgi?id=906321]].
-Workaround is to edit '''/usr/lib/systemd/system/httpd.service''' and add the option:
-<code>
-KillMode=none
-</code>
-Then reload systemd:
-<code>
-systemctl daemon-reload
-</code>
-It is absolutely correct to create new versions of unity in /etc, that has the option:
-<code>
-cp /usr/lib/systemd/system/httpd.service /etc/systemd/system/httpd.service
-vim /etc/systemd/system/httpd.service # add parametr KillMode=none
-systemctl daemon-reload
-</code>
-The patch of httpd should come soon so the first option is OK too.
-===== SSO =====
-If you want to enable SSO to CzechIdM, additional configuration must be done with mod\_auth\_kerb. See [[tutorial:adm:sso_ad_domain#configure_apache_httpd_-_linux|SSO installation guide]] for more details.
-====== INSTALACTNI NAVOD ======
-<note important>Tato cast se vlozi do instalacniho navodu pro IdM</note>
-==== 2. JDBC driver installation - CentOS8 ====
-**CentOS**
-Install the package with PostgreSQL JDBC driver:
-<code bash>
-yum install -y postgresql-jdbc
-</code>
-allow Tomcat to use the driver:
-<code bash>
-ln -s /usr/share/java/postgresql-jdbc.jar /opt/tomcat/current/lib/
-</code>
-==== Application properties ====
-  * The most important file is **/opt/czechidm/etc/application-production.properties** (application-PROFILE.properties, where the PROFILE is the profile you run the IdM under). You can use most of the file as-is, there is a bit of configuration needed though. This is a template file:
-<file properties application-production.properties>
-# Doc: https://wiki.czechidm.com/devel/dev/configuration/backend
-idm.pub.app.instanceId=idm-primary
-idm.pub.app.stage=production
-spring.datasource.url=jdbc:postgresql://localhost:5432/czechidm
-spring.datasource.username=czechidm
-spring.datasource.password=********** TODO *********
-spring.datasource.driver-class-name=org.postgresql.Driver
-spring.datasource.validationQuery=SELECT 1
-spring.datasource.test-on-borrow=true
-spring.jpa.generate-ddl=false
-spring.jpa.hibernate.ddl-auto=none
-flyway.enabled=true
-scheduler.properties.location=quartz-production.properties
-logging.config=/opt/czechidm/etc/logback-spring.xml
-idm.sec.core.demo.data.enabled=false
-# attachments will be stored under this path.
-# new directories for attachment will be created in this folder (permissions has to be added)
-# System.getProperty("user.home")/idm_data will be used if no path is given
-idm.sec.core.attachment.storagePath=/opt/czechidm/data
-# configuration property for default backup
-idm.sec.core.backups.default.folder.path=/opt/czechidm/backup
-idm.pub.security.allowed-origins=http://localhost
-# Generate JWT token security string as "cat /dev/urandom | tr -dc 'a-z0-9' | head -c VALUE" where VALUE can be from 1 to 255.
-# We recommend the VALUE to be at least 25.
-idm.sec.security.jwt.secret.token=********** TODO *********
-idm.sec.security.jwt.expirationTimeout=36000000
-# Cipher secret key for crypt values in confidential storage
-# for crypt values is used secretKey or secretKey defined by file - secretKeyPath
-#cipher.crypt.secret.key=XXXXXXXXXXXXXXXX
-cipher.crypt.secret.keyPath=/opt/czechidm/etc/secret.key
-# Defaults for: emailer.*
-# test.enabled=true means mail WILL NOT be sent
-idm.sec.core.emailer.test.enabled=true
-# http://camel.apache.org/mail.html
-idm.sec.core.emailer.protocol=smtp
-idm.sec.core.emailer.host=something.tld
-idm.sec.core.emailer.port=25
-# idm.sec.core.emailer.username=czechidm@domain.tld
-# idm.sec.core.emailer.password=password
-idm.sec.core.emailer.from=czechidm@localhost
-# Default user role will be added automatically, after an identity is logged in
-# could contains default authorities and authority policies configuration
-# for adding autocomplete or all record read permission etc.
-idm.sec.core.role.default=userRole
-# Admin user role
-idm.sec.core.role.admin=superAdminRole
-# Max file size of uploaded file. Values can use the suffixed "MB" or "KB" to indicate a Megabyte or Kilobyte size.
-spring.servlet.multipart.max-file-size=100MB
-spring.servlet.multipart.max-request-size=100MB
-</file>