CSV Connector - how to synchronize identities
Do you have an HR system that exports data into CSV files? You can try our brand new CSVConnConnector and easily transfer data into CzechIdM! This connector was developed for easy synchronization of CSV files into our system. The connector is mostly used for synchronization. Provisioning methods (update, create, delete) are implemented, too, but are not advisable for now.
In this tutorial, we will show you how to populate CzechIdm with identities from a CSV file given below as an example:
Example file
If you want to follow this tutorial step by step, please copy these lines to a text editor and save the content as a CSV file (example.csv).
uid;username;firstname;lastname;desc 1;karamel;mel;kara;clovek 1 2;velbloud;bloud;vel;clovek 2 3;sakajavi;javi;saka;clovek 3 4;rucnik;nik;ruc;clovek 4 5;apache;che;apa;clovek 5 6;lalaalala;;lala;clovek 6 7;nugatek;tek;nuga;clovek 7 8;marenka;renka;ma;clovek 8 9;hodbod;bod;hod;clovek 9 10;blabol;bol;bla;clovek 10 11;karambol;bol;kara;clovek 11
As you can see, the CSV file contains a header.
Dependency to CzechIdm
First of all, you need to enable the CSV connector in CzechIdM
Import .jar file into CzechIdM library
1) Please, download this LINK to your computer.
2) Then copy it to YourServer(tomcat)/webapps/idm/WEB-INF/lib/
3) You also need to modify /webapps/idm/WEB-INF/lib/idm-ic-9.0.0-SNAPSHOT.jar and edit module-ic.properties here. It should look like this.
ic.localconnector.packages=net.tirasa.connid,eu.bcvsolutions.idm.vs.connector,eu.bcvsolutions.idm.connectors.csv
4) You are now ready to do the next steps.
Creating a new system
Now you go back to CzechIdM. First of all, go to Systems and here Add a new system. Fill in the name. The other options such as Password policies, Description or Checkboxes below are optional. You can just skip them or read more on the subject in generic system connection. After saving the previous step you will see the (left-hand side) menu.
Configuration
Now we can reach Configuration page, where we need to set all properties for our CSV file. Look at the picture below: First, we need to set our CSV Connector (connId), then all other needed information will pop up. Follow the instructions for each property.
- Separator - character between two different columns
- Source path - path to the CSV file, which needs to be imported
- Checkbox included - if the first line of CSV file includes the header
- Header - if the file doesn't include the header, we need to write the header here
- Identifier - Which column holds the identifier
- Name identifier - This column is not mandatory for synchronization, you will need it for provisioning as secondary id.
- Synchronization token - token for repeated synchronization (not reconciliation) - usually DateTime
After we filled all information needed, just click save and Test connector and see if everything went all right. If some exception pops up, follow the instruction given. If you are following my example just fill every single item the same as shown in the picture.
Scheme
When the connection works, we need to specify what attributes (CSV columns) will be synchronized. We call this a Scheme. Just click on the button to generate a scheme and then a new schema should appear at the bottom of the page.
2) The schema should be created below for the respective object (in our example, it is ACCOUNT - the basic type of synchronized objects)
3) Click on the magnifying glass and the following page will pop up
4) You can see that all items from our header were created as attributes here (desc, firstname, lastname…). You wonder what does the \NAME\ stand for? In the configuration window, there is the option Name, which is exactly what the \NAME\ is (again in our example, it is the username).
5) Please check if every single item from your header is present, only then proceed with the following steps in this tutorial.
Mapping
We have our schema created but now we need to map all items in the schema to actual attributes of Identity in CzechIdM. 1) Fill all needed attributes according to this picture if you follow my example.
- Operation type - type of operation. Synchronization for input data into CzechIdm and Provisioning for writing data into our CSV file (example.csv).
- Mapping name - the name which will be then used in synchronization/provisioning according to operation type.
- Object name - currently we have just account
- Entity type - Type of entity which will be created - can be Identity (our example), Tree, Role and so on. Look at the list given for other.
2) Click Save and continue. 3) Now the blank table is shown at the bottom of this page. There is option add so we click on that. 4) Attribute mapping detail - this is the page where you chose the item from the schema and actually map it to the attribute of the object, we have chosen in the previous step. First, find Attribute in schema in the table of options for this select box. Name following will be set automatically. Warning: if we change our mind and set Attribute in schema to different option later, it won't change name automatically. 5) Attribute in CzechIdM and checkboxes with other info about attribute. 6) Fill all select boxes as shown in the pictures. We won't use transformations so according to the last picture we leave these two boxes blank. 7) Make sure you mapped all attributes you need - the table should look like this.
For our example, we chose one of the items - lastname. All other items will be similar. Just for uid we need to choose select box Identifier and Extended attribute. Then just type to IdM key - uid.
If you want to know more about this topic, there is also the whole tutorial on mapping attributes in this LINK.
Synchronization
It starts with the addition of new synchronization.
We do not use synchronization tokens (timestamps), so every time we will synchronize all data = reconciliation.
1) We just simply have to set Allowed and Reconciliation - first, attribute only says that it can be run and second says that no sync. The token is given.
2) Choose your Name and Set of mapped attributes you created in previous steps.
3) Correlation attribute is the attribute which should connect both attribute in CzechIdM and item in the example.csv.
4) There are more options but in this case, we won't change anything. For a brief explanation of what to set here follow this LINK.
See the log
If we want to make sure that our Identities were made, we can look into our identities or we can look into the log of our Synchronization.
1) Click on the magnifying glass and then in the top of the page find Logs and follow to this page.
2) Now the result should say: "Some number" Created Entities as shown in the picture (we created 11 identities).
3) We can go further and finds out which identities were created. Just click again on the magnifying glass.
4) You will see the log in the top part of the page but we will look at the bottom as the picture shows. There click on the magnifying glass.
5) Now you should see the same as is shown in the picture.
This is the basic usage of this connector and now you can try also filters. Just set all filters by your choice and synchronize! To be able to use filters follow please to this tutorial.
Provisioning
Potential issues
When testing the connector you can see get an error "Connection test failed: Can't read given path. Check if can be read or if it is right!" If that happens make sure your csv file is in a directory to which tomcat has permission to read. You can move the file to the czechidm/data folder which should solve the issue.