Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
tutorial:adm:connector_csv_-_how_to_connect_csv_connector [2018/08/28 11:47]
klementm [Provisioning]
tutorial:adm:connector_csv_-_how_to_connect_csv_connector [2019/06/12 06:56] (current)
doischert [Potential issues]
Line 1: Line 1:
 +===== CSV Connector - how to synchronize identities =====
  
 +Do you have an HR system that exports data into CSV files? You can try our brand new CSVConnConnector and easily transfer data into CzechIdM!
 +This connector was developed for easy synchronization of CSV files into our system. The connector is mostly used for synchronization. Provisioning methods (update, create, delete) are implemented, too, but are not advisable for now.
 +
 +In this tutorial, we will show you how to populate CzechIdm with identities from a CSV file given below as an example:
 +
 +<note warning>
 +The CSV Connector is only meant for reconciliation!
 +</note>
 +
 +=== Example file ===
 +
 +If you want to follow this tutorial step by step, please copy these lines to a text editor and save the content as a CSV file (example.csv).
 +
 +<code>
 +uid;username;firstname;lastname;desc
 +1;karamel;mel;kara;clovek 1
 +2;velbloud;bloud;vel;clovek 2
 +3;sakajavi;javi;saka;clovek 3
 +4;rucnik;nik;ruc;clovek 4
 +5;apache;che;apa;clovek 5
 +6;lalaalala;;lala;clovek 6
 +7;nugatek;tek;nuga;clovek 7
 +8;marenka;renka;ma;clovek 8
 +9;hodbod;bod;hod;clovek 9
 +10;blabol;bol;bla;clovek 10
 +11;karambol;bol;kara;clovek 11
 +</code>
 +
 +As you can see, the CSV file contains a header.
 +
 +=== Dependency to CzechIdm ===
 +First of all, you need to enable the CSV connector in CzechIdM
 +
 +== Import .jar file into CzechIdM library ==
 +
 +1) Please, download this [[http://nexus.bcvsolutions.eu/repository/maven-public-releases/eu/bcvsolutions/idm/connectors/csv/csv-connector/1.0.0/csv-connector-1.0.0.jar|LINK]] to your computer.
 +
 +2) Then copy it to YourServer(tomcat)/webapps/idm/WEB-INF/lib/
 +
 +3) You also need to modify /webapps/idm/WEB-INF/lib/idm-ic-9.0.0-SNAPSHOT.jar and edit module-ic.properties here. It should look like this.
 +<note warning>Versions may vary!</note>
 +
 +<code>
 +ic.localconnector.packages=net.tirasa.connid,eu.bcvsolutions.idm.vs.connector,eu.bcvsolutions.idm.connectors.csv
 +</code>
 +
 +4) You are now ready to do the next steps.
 +
 +
 +==== Creating a new system ====
 +
 +Now you go back to CzechIdM. First of all, go to //Systems// and here //Add// a new system.
 +{{ :tutorial:adm:0basic.png?600 |}} 
 +Fill in the name. The other options such as **Password policies**, **Description** or **Checkboxes below** are optional. You can just skip them or read more on the subject in [[tutorial:adm:systems#basic_information| generic system connection]].
 +{{ :tutorial:adm:sys1.png?100 |}}
 +After saving the previous step you will see the (left-hand side) menu.
 +
 +==== Configuration ====
 +
 +Now we can reach **Configuration** page, where we need to set all properties for our CSV file. Look at the picture below:
 +{{ :tutorial:adm:sys2.png?600 |}}
 +First, we need to set our CSV Connector (connId), then all other needed information will pop up. Follow the instructions for each property.
 +{{ :tutorial:adm:0configu.png?600 |}}
 +  * **Separator** - character between two different columns
 +  * **Source path** - path to the CSV file, which needs to be imported
 +  * **Checkbox included** - if the first line of CSV file includes the header
 +  * **Header** - if the file doesn't include the header, we need to write the header here
 +  * **Identifier** - Which column holds the identifier
 +  * **Name identifier** - This column is not mandatory for synchronization, you will need it for provisioning as secondary id.
 +  * **Synchronization token** - token for repeated synchronization (not reconciliation) - usually DateTime
 +After we filled all information needed, just click **save** and **Test connector** and see if everything went all right. If some exception pops up, follow the instruction given. If you are following my example just fill every single item the same as shown in the picture.
 +
 +==== Scheme ====
 +
 +When the connection works, we need to specify what attributes (CSV columns) will be synchronized. We call this a **Scheme**. Just click on the button to generate a scheme and then a new schema should appear at the bottom of the page.
 +
 +1) Click on //Generate schema//
 +{{ :tutorial:adm:0generate.png?600 |}}
 +
 +2) The schema should be created below for the respective object (in our example, it is ACCOUNT - the basic type of synchronized objects)
 +{{ :tutorial:adm:0objects.png?600 |}}
 +
 +3) Click on the magnifying glass and the following page will pop up
 +{{ :tutorial:adm:0schemecreated.png?600 |}}
 +
 +4) You can see that all items from our header were created as attributes here (desc, firstname, lastname...). You wonder what does the \__NAME\__ stand for? In the configuration window, there is the option **Name**, which is exactly what the \__NAME\__ is (again in our example, it is the username).
 +
 +5) Please check if every single item from your header is present, only then proceed with the following steps in this tutorial.
 +
 +==== Mapping ====
 +
 +We have our schema created but now we need to map all items in the schema to actual attributes of Identity in CzechIdM.
 +{{ :tutorial:adm:0mapping1.png?600 |}}
 +1) Fill all needed attributes according to this picture if you follow my example.
 +  * **Operation type** - type of operation. Synchronization for input data into CzechIdm and Provisioning for writing data into our CSV file (example.csv).
 +  * **Mapping name** - the name which will be then used in synchronization/provisioning according to operation type.
 +  * **Object name** - currently we have just account
 +  * **Entity type** - Type of entity which will be created - can be Identity (our example), Tree, Role and so on. Look at the list given for other.
 +
 +2) Click Save and continue.
 +{{ :tutorial:adm:0mapped.png?600 |}}
 +3) Now the blank table is shown at the bottom of this page. There is option add so we click on that.
 +{{ :tutorial:adm:0mappingdetail.png?600 |}}
 +4) Attribute mapping detail - this is the page where you chose the item from the schema and actually map it to the attribute of the object, we have chosen in the previous step. First, find **Attribute in schema** in the table of options for this select box. **Name** following will be set automatically. **Warning**: if we change our mind and set **Attribute in schema** to different option later, it won't change name automatically.
 +{{ :tutorial:adm:0mappingdetail2.png?600 |}}
 +5) Attribute in CzechIdM and checkboxes with other info about attribute.
 +{{ :tutorial:adm:0mappingdetail3.png?600 |}}
 +6) Fill all select boxes as shown in the pictures. We won't use transformations so according to the last picture we leave these two boxes blank.
 +{{ :tutorial:adm:0mappedalready.png?600 |}}
 +7) Make sure you mapped all attributes you need - the table should look like this.
 +
 +For our example, we chose one of the items - **lastname**. All other items will be similar. Just for **uid** we need to choose select box //Identifier// and //Extended attribute//. Then just type to **IdM key** - uid.
 +
 +If you want to know more about this topic, there is also the whole tutorial on mapping attributes in this [[https://wiki.czechidm.com/devel/dev/system/system-mapping?s[]=mapping#attribute_mapping|LINK]].
 +
 +==== Synchronization ====
 +
 +It starts with the addition of new synchronization.
 +{{ :tutorial:adm:0syncadd.png?600 |}}
 +
 +We do not use synchronization tokens (timestamps), so every time we will synchronize all data = reconciliation.
 +
 +{{ :tutorial:adm:0syncdetailset.png?600 |}}
 +1) We just simply have to set **Allowed** and **Reconciliation** - first, attribute only says that it can be run and second says that no sync. The token is given.
 +
 +2) Choose your **Name** and **Set of mapped attributes** you created in previous steps.
 +
 +3) **Correlation attribute** is the attribute which should connect both attribute in CzechIdM and item in the example.csv.
 +{{ :tutorial:adm:0entityset.png?600 |}}
 +
 +4) There are more options but in this case, we won't change anything. For a brief explanation of what to set here follow this [[tutorial:adm:how_to_identity_sync#synchronization|LINK]].
 +
 +5) After save and continue we can run our **Synchronization**.
 +{{ :tutorial:adm:0syncrun.png?600 |}}
 +
 +=== See the log ===
 +If we want to make sure that our Identities were made, we can look into our identities or we can look into the log of our **Synchronization**.
 +
 +1) Click on the **magnifying glass** and then in the top of the page find **Logs** and follow to this page.
 +{{ :tutorial:adm:0syncentitycreated.png?600 |}}
 +
 +2) Now the result should say: "Some number" Created Entities as shown in the picture (we created 11 identities).
 +
 +3) We can go further and finds out which identities were created. Just click again on the magnifying glass.
 +{{ :tutorial:adm:0syncactlog.png?600 |}}
 +
 +4) You will see the log in the top part of the page but we will look at the bottom as the picture shows. There click on the magnifying glass.
 +{{ :tutorial:adm:0syncitemslog.png?600 |}}
 +
 +5) Now you should see the same as is shown in the picture.
 +
 +
 +This is the basic usage of this connector and now you can try also filters. Just set all filters by your choice and synchronize! To be able to use filters follow please to this [[devel:documentation:synchronization:dev:synchronization#connector_synchronization_vs_my_own_filter|tutorial]].
 +
 +==== Provisioning ====
 +
 +<note warning>
 +Provisioning is not supported yet!
 +</note>
 +
 +==== Potential issues ====
 +
 +When testing the connector you can see get an error "Connection test failed: Can't read given path. Check if can be read or if it is right!" If that happens make sure your csv file is in a directory to which tomcat has permission to read. You can move the file to the czechidm/data folder which should solve the issue.