Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
tutorial:dev:creating_a_development_environment [2019/07/30 12:51] – link to external frontend build instructions poulm | tutorial:dev:creating_a_development_environment [2025/04/11 09:10] (current) – [Working-around IdM rebuild when developing backend module] fiserp | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Creating a development environment ====== | ====== Creating a development environment ====== | ||
- | This text is intended as a tutorial for CzechIdM developers with the goal to set up IDE, build CzechIdM in it and run the system. If you prefer quick glance at the code or need to start quickly, you can use [[tutorial: | + | This text is intended as a tutorial for CzechIdM developers with the goal to set up IDE, build CzechIdM in it and run the system. If you prefer quick glance at the code or need to start quickly, you can use [[.: |
====== 1. Install Java and Maven ====== | ====== 1. Install Java and Maven ====== | ||
Line 7: | Line 7: | ||
First install JDK and Maven: | First install JDK and Maven: | ||
- | Download Java 1.8 Oracle JDK (or OpenJDK), minor version **at least 101** (it's necessary due to the certificate of Nexus repository, which uses Let's encrypt and its support was added in version 101). The guide was tested | + | Download |
Install Maven from your system packages, at least version 3.1 is required. | Install Maven from your system packages, at least version 3.1 is required. | ||
+ | |||
<code bash> | <code bash> | ||
yum install maven | yum install maven | ||
mvn -v | mvn -v | ||
+ | |||
+ | |||
</ | </ | ||
Note: If you installed Java separately from your system libraries, you should set the correct Java home for Maven in the file " | Note: If you installed Java separately from your system libraries, you should set the correct Java home for Maven in the file " | ||
< | < | ||
+ | |||
export JAVA_HOME=/ | export JAVA_HOME=/ | ||
+ | |||
</ | </ | ||
+ | |||
This way Maven will always use this JDK. | This way Maven will always use this JDK. | ||
Line 27: | Line 33: | ||
<code bash> | <code bash> | ||
git clone https:// | git clone https:// | ||
+ | |||
+ | |||
</ | </ | ||
- | Check branch: | + | Check branch: |
<code bash> | <code bash> | ||
git branch | git branch | ||
+ | |||
+ | |||
</ | </ | ||
Line 39: | Line 49: | ||
<code bash> | <code bash> | ||
git checkout -b develop origin/ | git checkout -b develop origin/ | ||
+ | |||
+ | |||
</ | </ | ||
Line 45: | Line 57: | ||
<code bash> | <code bash> | ||
git checkout -b personal/ | git checkout -b personal/ | ||
+ | |||
+ | |||
</ | </ | ||
Line 51: | Line 65: | ||
====== 3. Install PostgreSQL ====== | ====== 3. Install PostgreSQL ====== | ||
- | CzechIdM in " | + | CzechIdM in " |
+ | |||
+ | You have to folow one of these next steps. Choose one which fits you more. | ||
+ | |||
+ | - PostgreSQL to computer 3.1. | ||
+ | - Run docker image with postgres part 3.2. | ||
+ | |||
+ | ===== 3.1 Install PostgreSQL to computer ===== | ||
Install necessary packages: | Install necessary packages: | ||
+ | |||
<code bash> | <code bash> | ||
yum install postgresql postgresql-server postgresql-init | yum install postgresql postgresql-server postgresql-init | ||
+ | |||
+ | |||
</ | </ | ||
Fedora 25 packages: | Fedora 25 packages: | ||
+ | |||
<code bash> | <code bash> | ||
dnf install postgresql-server postgresql-contrib | dnf install postgresql-server postgresql-contrib | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | Ubuntu 22.04 LTS packages: | ||
+ | |||
+ | <code bash> | ||
+ | apt install postgresql postgresql-contrib | ||
+ | |||
+ | |||
</ | </ | ||
First run of the service: | First run of the service: | ||
+ | |||
<code bash> | <code bash> | ||
systemctl start postgresql.service | systemctl start postgresql.service | ||
+ | |||
+ | |||
</ | </ | ||
- | If the following error occurs: "Job for postgresql.service failed. See ' | + | If the following error occurs: "Job for postgresql.service failed. See ' |
- | Try initializing the PostgreSQL first and start the database again: | + | |
<code bash> | <code bash> | ||
postgresql-setup --initdb --unit postgresql | postgresql-setup --initdb --unit postgresql | ||
+ | |||
+ | |||
</ | </ | ||
Set automatic start of the service when starting OS: | Set automatic start of the service when starting OS: | ||
+ | |||
<code bash> | <code bash> | ||
systemctl enable postgresql.service | systemctl enable postgresql.service | ||
+ | |||
+ | |||
</ | </ | ||
Set the configuration file for the database to accept login and password (change METHOD from ident to md5) | Set the configuration file for the database to accept login and password (change METHOD from ident to md5) | ||
+ | |||
< | < | ||
vim / | vim / | ||
+ | |||
+ | in Ubuntu it is: | ||
+ | vim / | ||
+ | |||
+ | in Windows it is: | ||
+ | < | ||
# TYPE DATABASE | # TYPE DATABASE | ||
Line 92: | Line 141: | ||
# IPv6 local connections: | # IPv6 local connections: | ||
host | host | ||
+ | |||
</ | </ | ||
Restart the service to load the changes: | Restart the service to load the changes: | ||
+ | |||
< | < | ||
systemctl restart postgresql.service | systemctl restart postgresql.service | ||
+ | |||
</ | </ | ||
- | Create the database: | + | Create the database |
- | (Make sure you have installed and allowed en_US locale in your OS) | + | |
+ | * Version 13 is for IdM 13 and develop (IdM 14) - bcv\_idm\_13 | ||
+ | * Version 10 is for IdM 12 - 10 - bcv\_idm\_10 | ||
+ | * Below version 10 - bcv\_idm\_storage | ||
+ | * (Make sure you have installed and allowed en_US locale in your OS) | ||
< | < | ||
su postgres -c psql postgres | su postgres -c psql postgres | ||
CREATE USER idmadmin PASSWORD ' | CREATE USER idmadmin PASSWORD ' | ||
- | CREATE DATABASE "bcv_idm_storage" WITH OWNER idmadmin ENCODING ' | + | CREATE DATABASE "bcv_idm_13" WITH OWNER idmadmin ENCODING ' |
+ | CREATE DATABASE " | ||
+ | |||
+ | </ | ||
+ | |||
+ | ===== 3.2 Run postgreSQL in docker container ===== | ||
+ | |||
+ | Installation instructions for **Docker** | ||
+ | |||
+ | After installation of docker. Use next command which will create postgres container with user and password which is used for development. | ||
+ | < | ||
+ | |||
+ | docker run --name postgresql -e POSTGRES_USER=idmadmin -e POSTGRES_PASSWORD=idmadmin -p 5432:5432 postgres: | ||
+ | |||
+ | </ | ||
+ | |||
+ | Connect to docker container and create the databases with names: | ||
+ | |||
+ | * Version 13 is for IdM 13 and develop (IdM 14) - bcv\_idm\_13 | ||
+ | * Version 10 is for IdM 12 - 10 - bcv\_idm\_10 | ||
+ | * Below version 10 - bcv\_idm\_storage | ||
+ | * (Make sure you have installed and allowed en_US locale in your OS) | ||
+ | * \l command will return list of databases make sure you have properly set owner of databases! | ||
+ | |||
+ | < | ||
+ | docker exec -it postgresql /bin/bash | ||
+ | psql -U idmadmin | ||
+ | |||
+ | CREATE DATABASE " | ||
+ | CREATE DATABASE " | ||
+ | |||
+ | \l | ||
+ | |||
+ | </ | ||
+ | |||
+ | After running command " | ||
+ | < | ||
+ | List of databases | ||
+ | | ||
+ | s privileges | ||
+ | -----------------------+----------+----------+------------+------------+-------- | ||
+ | --------------- | ||
+ | | ||
+ | | ||
+ | </ | ||
+ | |||
+ | Database will be available on address: | ||
+ | |||
+ | < | ||
+ | | ||
</ | </ | ||
Line 112: | Line 218: | ||
The tutorial is different for each IDE. Select one and continue with a certain tutorial below: | The tutorial is different for each IDE. Select one and continue with a certain tutorial below: | ||
- | * [[devel: | + | * [[:devel: |
- | * [[devel: | + | * [[:devel: |
In the IDE start the Tomcat server with CzechIdM deployed. | In the IDE start the Tomcat server with CzechIdM deployed. | ||
Line 119: | Line 225: | ||
In browser access [[http:// | In browser access [[http:// | ||
- | > CzechIdM API is running | + | > CzechIdM API is running > > If you can see this message, API is running |
- | > | + | |
- | > If you can see this message, API is running | + | |
====== 5. Frontend development environment ====== | ====== 5. Frontend development environment ====== | ||
- | <note important> | + | <note important> |
+ | |||
+ | ===== Select IDE for frontend environment ===== | ||
+ | |||
+ | You can use any text editor or JavaScript IDE. We recommend to use ATOM | ||
+ | |||
+ | * See [[: | ||
====== 6. Run backend and frontend together ====== | ====== 6. Run backend and frontend together ====== | ||
Line 139: | Line 248: | ||
cd CzechIdM/ | cd CzechIdM/ | ||
gulp | gulp | ||
+ | |||
+ | |||
</ | </ | ||
- | A new browser window or tab will open with [[https:// | + | A new browser window or tab will open with [[https:// |
The frontend is fully started after you see in following line in log: | The frontend is fully started after you see in following line in log: | ||
Line 147: | Line 258: | ||
<code bash> | <code bash> | ||
[BS] 1 file changed (app.js) | [BS] 1 file changed (app.js) | ||
+ | |||
+ | |||
</ | </ | ||
After the application is running, log in: | After the application is running, log in: | ||
+ | |||
* username: admin | * username: admin | ||
* password: admin | * password: admin | ||
- | <note tip>If the app shows error with permission, log out</ | + | <note tip>If the app shows error with permission, log out</ |
- | <note tip>To use CzechIdM in more windows or tabs for development purposes go to [[http:// | + | |
Congratulations! Your CzechIdM development environment is ready to use. | Congratulations! Your CzechIdM development environment is ready to use. | ||
- | |||
====== Common development errors & tips ====== | ====== Common development errors & tips ====== | ||
Line 164: | Line 276: | ||
If Tomcat server fails to start, try following: | If Tomcat server fails to start, try following: | ||
- | | + | |
+ | | ||
* Check that PostgreSQL server is running. | * Check that PostgreSQL server is running. | ||
+ | |||
<code bash> | <code bash> | ||
systemctl status postgresql.service | systemctl status postgresql.service | ||
+ | |||
+ | |||
</ | </ | ||
- | | + | |
+ | | ||
< | < | ||
+ | |||
spring.datasource.url=jdbc: | spring.datasource.url=jdbc: | ||
spring.datasource.username=idmadmin | spring.datasource.username=idmadmin | ||
spring.datasource.password=idmadmin | spring.datasource.password=idmadmin | ||
+ | |||
</ | </ | ||
+ | |||
use the following command and type the password when asked for it. | use the following command and type the password when asked for it. | ||
+ | |||
<code bash> | <code bash> | ||
psql -h 127.0.0.1 -U idmadmin bcv_idm_storage | psql -h 127.0.0.1 -U idmadmin bcv_idm_storage | ||
+ | |||
+ | |||
</ | </ | ||
+ | |||
* Check that you can read from the tables of the database. As in the previous point, connect to the PostgreSQL database and try to select data from any of the IdM tables. If you have insufficient access rights, check the owner and grants for the schema " | * Check that you can read from the tables of the database. As in the previous point, connect to the PostgreSQL database and try to select data from any of the IdM tables. If you have insufficient access rights, check the owner and grants for the schema " | ||
* Check the error message in the server output console. | * Check the error message in the server output console. | ||
Line 184: | Line 308: | ||
==== Common error messages ==== | ==== Common error messages ==== | ||
- | '' | + | '' |
- | '' | + | '' |
< | < | ||
+ | |||
Offenders: | Offenders: | ||
-> ... idm-core-impl-7.7.0-SNAPSHOT.jar!/ | -> ... idm-core-impl-7.7.0-SNAPSHOT.jar!/ | ||
-> ... idm-core-impl-7.7.0-SNAPSHOT.jar!/ | -> ... idm-core-impl-7.7.0-SNAPSHOT.jar!/ | ||
+ | |||
</ | </ | ||
+ | |||
Just increase the version in the name of one of the scripts, clean & build the project again. | Just increase the version in the name of one of the scripts, clean & build the project again. | ||
- | |||
=== Could not create Vfs.Dir from url === | === Could not create Vfs.Dir from url === | ||
+ | |||
**This isn't error just warning**. You may don't interest about this warning. Warning is only when you start local development environment and you configuration for ic connectors path isn't correctly setup. Dont worry about this warning. | **This isn't error just warning**. You may don't interest about this warning. Warning is only when you start local development environment and you configuration for ic connectors path isn't correctly setup. Dont worry about this warning. | ||
Line 203: | Line 330: | ||
org.reflections.ReflectionsException: | org.reflections.ReflectionsException: | ||
either use fromURL(final URL url, final List< | either use fromURL(final URL url, final List< | ||
+ | |||
</ | </ | ||
+ | |||
+ | ===== Frontend build fails ===== | ||
+ | |||
+ | ==== Out of memory ==== | ||
+ | |||
+ | If your gulp build fails with the following error: | ||
+ | |||
+ | < | ||
+ | [INFO] [18:43:00] Starting ' | ||
+ | [INFO] [18:45:30] Finished ' | ||
+ | [INFO] [18:45:30] Finished ' | ||
+ | [INFO] | ||
+ | [INFO] <--- Last few GCs ---> | ||
+ | [INFO] | ||
+ | [INFO] 192087 ms: Scavenge 1282.6 (1434.8) -> 1282.1 (1434.8) MB, 3.8 / 0 ms (+ 3.8 ms in 9 steps since last GC) [allocation failure]. | ||
+ | [INFO] 192095 ms: Scavenge 1282.6 (1434.8) -> 1282.2 (1434.8) MB, 3.6 / 0 ms (+ 3.3 ms in 8 steps since last GC) [allocation failure]. | ||
+ | [INFO] 192102 ms: Scavenge 1282.6 (1434.8) -> 1282.1 (1434.8) MB, 3.7 / 0 ms (+ 2.9 ms in 7 steps since last GC) [allocation failure]. | ||
+ | [INFO] | ||
+ | [INFO] | ||
+ | [INFO] <--- JS stacktrace ---> | ||
+ | [INFO] Cannot get stack trace in GC. | ||
+ | [ERROR] FATAL ERROR: Scavenger: semi-space copy | ||
+ | [ERROR] Allocation failed - process out of memory | ||
+ | [ERROR] 1: node:: | ||
+ | |||
+ | </ | ||
+ | |||
+ | Try following: | ||
+ | |||
+ | * Go to '' | ||
+ | * If you build BE+FE of a project together, check the '' | ||
+ | < | ||
+ | |||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | </ | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | |||
+ | * try adding '' | ||
+ | |||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | </ | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | < | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | |||
+ | ==== Wrong layout and NPM version ==== | ||
+ | |||
+ | If your frontend looks like this: [[https:// | ||
+ | |||
+ | * there is package-lock.json file in the frontend app module and it looks like this: [[https:// | ||
+ | * NPM version which you use when building frontend (or FE+BE) is at least 6. | ||
+ | < | ||
+ | |||
+ | npm -v | ||
+ | 6.4.1 | ||
+ | |||
+ | </ | ||
+ | |||
+ | ====== Developing on Appliance ====== | ||
+ | |||
+ | This section contains howtos and hacks for more effective development on IAM Appliance. | ||
+ | |||
+ | ===== Working-around IdM rebuild when developing backend module ===== | ||
+ | |||
+ | When you are developing a backend module and need to debug it directly on the Appliance, each '' | ||
+ | To speed things up you can, technically, | ||
+ | |||
+ | <note warning> | ||
+ | |||
+ | **Remarks** | ||
+ | * If you are debugging a frontend module, this howto **will not work** because the frontend rebuild will be skipped. | ||
+ | * If you forget to properly persist your changes (see steps below) you are creating rather nasty surprise for anybody who re-creates the '' | ||
+ | |||
+ | **Steps** | ||
+ | - (you start developing, add all your modules into proper folders in the appliance, restart the '' | ||
+ | - At this point, you have a basis of the application you are going to develop further. You can even have some custom frontend parts put in. | ||
+ | - Suppose you want to put another JAR into you application or replace one that is already here. (here we work with '' | ||
+ | - Copy new / modified files directly into the container. < | ||
+ | Successfully copied 28.2kB to czechidm:/ | ||
+ | - In case you need to delete something, remove it directly from the container. **This is the way you now have to handle conflicting libraries, by the way.**< | ||
+ | -rw-r--r--. 1 root root 26277 Apr 10 13:05 ssh-connector-1.0.1.jar | ||
+ | |||
+ | [root@appliance70 ~]# docker exec czechidm rm -fv / | ||
+ | removed '/ | ||
+ | |||
+ | [root@appliance70 ~]# docker exec czechidm ls -l / | ||
+ | (empty listing)</ | ||
+ | - Restart the IdM container. < | ||
+ | [root@appliance70 ~]# systemctl restart iam-czechidm</ | ||
+ | - Let the IdM start as usual. Because you did not change anything in the appliance directories (under ''/ | ||
+ | - Repeat the previous step as long as you need to develop / debug your backend module. | ||
+ | - Persist the changes. | ||
+ | - Once you are done working, you need to persist all changes you made. Gather all modules you were working on and put them into directories under ''/ | ||
+ | - Stop, drop and start the IdM container. < | ||
+ | [root@appliance70 ~]# docker rm czechidm | ||
+ | [root@appliance70 ~]# systemctl start iam-czechidm</ | ||
+ | - Let the rebuild finish and wait for IdM to start. | ||
+ | - Test that changes you made are really there. |