====== Systems - AD: Groups synchronization ======

This tutorial is intended as a guide for administrators that want to load AD groups into CzechIdM (either one time or as a scheduled job).

<note important>This tutorial is for IdM 9 and 10. In version 11 you can use a wizard for groups synchronization configuration - [[:devel:documentation:wizards:ad_group|wizard tutorial]]</note>

You will learn

  * how to connect an AD system for groups synchronization
  * how to use a groups sync workflow
  * how to prepare users to be able to assign them IdM roles by their AD groups

===== Before you start =====

==== Adding Active Directory connector ====

Since CzechIdM 9.2, the [[https://github.com/bcvsolutions/ad-connector|forked ConnId AD connector]] is bundled inside CzechIdM by default. You can use it out of hand.

==== System for managing AD users ====

Loading AD groups to IdM is usually done when you want to manage the group membership of the AD users by IdM. So connecting the system for managing AD users is a logical step before you start to synchronize the groups.

If you followed the [[.:manage_ad|tutorial for managing AD users]], you have the necessary configuration of the **AD users**  system mostly prepared. Specifically:

  * the attribute ''Base contexts for group entry searches''  contains all containers in AD where the groups are located. (Or it's empty and ''Root suffixes''  cover all those containers.)
  * the attribute ''ldapGroups''  is set in the connector configuration, in the schema and in the provisioning mapping with the MERGE strategy

However, it's a common request to do **initial**  loading of the group membership from AD. This topic will be covered later. FIXME synchronization of AD users with mapped distinguishedName to EAV of identity, so the [[..:dev:ad_groups_sync_workflow|groups synchronization workflow]] can resolve membership.

==== Automatic creation of automatic roles ====

The synchronization of AD groups can also create some automatic roles based on the position of the groups in AD. These are specific options of the [[..:dev:ad_groups_sync_workflow|groups synchronization workflow]] and it's not often used for typical setup. However, if you want to use it, make sure to create [[.:eav|EAVs]] for IdmTreeNode, IdmIdentity and IdmIdentityContract, so this EAVs can be used to create automatic roles. IdmTreeNode for an automatic role by organization and the others for an automatic role by attributes.

===== Create system =====

  * Go to **Systems**  in the left menu and then click on **Add**.

{{  .:wfad01.png  }}

  * Fill in name of a system. And click on **Save and continue**.
  * In tab **Configuration**  choose AD connector (net.tirasa.connid.bundles.ad.ADConnector)

{{  .:wfad02.png  }}

===== Connector configuration =====

On this page fill in these important values:

  * **Server hostname**  - IP address of ad server or hostname
  * **Server port**  - on this port will server listen
  * **Principal**  - with this username connector will connect to the AD system, this user has to have enough rights to reads groups
  * **Principal password**  - password of the "principal" account
  * **Root suffixes**  - there should be DNs of **Base contexts**, groups outside of these "paths" will be ignored. Content of **Root suffixes**  could be same as **Base contexts**  or just put in domain.
  * **Entry object classes**  - List of all objectClasses groups have in AD. It is necessary to find just groups. With wrong settings, it could find even users. Usual values: top, group (every value on a single line)
  * **Group search scope**  - Default subtree. Options: object, onelevel or subtree. It means where it will search for groups. As a **subtree**, a search will start on paths in **Base context**  and it will search in every **Organization Unit**  in this path. FIXME All behave the same on the current version, so other options can't be used: <del>**onelevel**  ("onlevel" is a typo) will search just one **OU**, where distinguished names of **Base context**  points to and the last **object**  means, in **Base context**  there are DNs of groups we want to synchronize.</del>
  * **Custom group search filter**  - this enables additional filter for groups, which will be searched for. You can use it e.g. to filter out roles with some specific substrings in their CN by using LDAP filter ''(&(!(cn=*Administrator*))(!(cn=*Auditor*)))''. However, you can't use a filter with wildcards by whole distinguishedName attributes (''distinguishedName'', ''member'', ''manager''  etc.). If you want to for example exclude a certain OU from searches use ''msDS-parentdistname''  attribute instead (available since Windows Server 2012), e.g. ''(!(msDS-parentdistname=OU=Excluded,DC=example,DC=tl))''.
  * **Base contexts for group entry searches**  - list of distinguished names (paths), where it will search for groups.
  * **Group members reference attribute**  - a name of the attribute, which indicates membership. It contains whole DNs of users.
  * **useVlvControls**  - have to be enabled - this is only supported option
  * **pageSize**  - number, it should be lower than maximum page size limit in AD, which is by default 1000. Recommended: 100.
  * **vlvSortAttribute**  - this should be identifier with sorting properties. Recommended for groups is cn. **DO NOT**  use **distinguishedName**  or any other unindexed attribute or you'll end up with "[LDAP: error code 12 - 0000217A: SvcErr: DSID-03140414, problem 5010 (UNAVAIL_EXTENSION), data 0];" error!
  * **Uid Attribute for groups**  - unique identifier, recommended is objectGUID.
  * **Object classes to synchronize**  - Based on this filled object classes, groups to synchronized will be found. Content is usually same as **Entry object classes**.
<note tip>**When you configure the system for the first time, root suffix should lead to the top container (e.g. DC=domain,DC=local), so the system schema can be correctly generated**</note>

<note tip> In user provisioning system's configuration **Base context of groups**  should be filled too, for correctly provisioning memberships</note> <note tip> In user provisioning system's schema and mapping should have attribute memberOf/ldapGroups and **Strategy**  as "Merge".</note> <note warning> If there are more than 10000 groups in AD and "Base contexts for group entry searches" is set for DC=AD,DC=FIRMA,DC=CZ(root OU). LDAP: error code 12 - 000020EF: SvcErr: DSID-03140552, problem 5010 (UNAVAIL_EXTENSION), data 0

workaround/solution: separate ldap search with "Base context for group entry searches" and divide it into smaller searches(each line with one OU):

  * OU=001OU,OU=FIRMA,DC=ad,DC=FIRMA,DC=cz
  * OU=002OU,OU=FIRMA,DC=ad,DC=FIRMA,DC=cz
  * OU=003OU,OU=FIRMA,DC=ad,DC=FIRMA,DC=cz
  * OU=004OU,OU=FIRMA,DC=ad,DC=FIRMA,DC=cz
  * OU=005OU,OU=FIRMA,DC=ad,DC=FIRMA,DC=cz

Another way to solve this problem is by using "Custom group search filter" in the system configuration. </note>


===== Connector's mapping =====

  * Firstly in **Scheme**  tab generate a schema with a green button. If there is some exception, you have probably mistake in the configuration of the connector.

{{  .:systems_-_ad:schema_ad_groups.png?900  }}

  * Then in **Mapping**  tab create new mapping - synchronization (\_\_GROUP\_\_ (Object name), Role (Entity type)).

{{  .:systems_-_ad:mapping_ad_groups.png?900  }}

  * Now we will map just 4 attributes. Click on green add button like on picture below and this fill in:
<code>

| Attribute in schema | Name               | Attribute          | IdM key            |
| __NAME__ (__GROUP__)| DN(__NAME__)       | extended           | distinguished_name |
| name (__GROUP__)    | name               | entity             | name               |
| name (__GROUP__)    | name-code          | entity             | code               |
| __UID__ (__GROUP__) | __UID__            | identifier         |   |

</code>

{{  .:systems_-_ad:mapping_ad_groups_2.png?900  }}

  * In **Synchronization**  tab create new synchronization.

{{  .:wfad06.png  }}

  * Enable **Allowed**  and **Reconcilation**. Fill **Name, Set of mapped attributes**  and then **Correlation attribute**  as '\_\_UID\_\_'.
  * Bellow there are 4 possibilities on state when synchronization starts (Linked, Not linked, Missing entity, Missing account).
      * **Linked**  - it's like update, group is in the AD and also in IdM, but it is possible in the AD could be some change, so usually **Action**  is "Update entity"
      * **Not Linked**  - this means the group is in the AD and also in IdM, but in IdM was not created by synchronization, so it does not have an account on this system.
      * **Missing entity**  - in other words - create action - group is in the AD, but in IdM it is not. It could be newly created in the AD, so it is not yet in IdM or it could be already erased in IdM. But this situation only supports "Ignore" and "Create entity" action.
      * **Missing account**  - or "delete" - group is in IdM, but missing in AD. Groups are synchronized from the AD, and this situation usually means group was deleted in the AD, so in IdM we expect to be erased as well.
  * In each of these possibilities there is a select box "Workflow". It is for selecting additional steps, which will be done. For example, when groups are synchronized, we want them to be in **Role catalog**  or some of them as an **automatic role by organization**. And this is done in a workflow. You could write your own, but we have one, which covers basic actions with a just little bit of adjusting ([[..:dev:ad_groups_sync_workflow|]]).

{{  .:wfad07.png  }}{{  .:wfad08.png  }}

===== Synchronization of groups =====

At this point configuring of synchronization is complete. Save this synchronization and run it. It should smoothly create a catalog, new roles and maybe even some automatic roles. If provisioning of memberships will fail do not forget to try "ldapGroups" attribute.

<note tip>If you synchronize groups with resolving users membership, the connector doesn't support groups with more than 1000 members (by default). If you need more, you must (temporarily) increase MaxPageSize in the AD configuration.</note>

==== Editing groups in Active Directory ====

CzechIdM managing membership of users in Active Directory groups, editing of groups is controlled by administrators directly in AD, you need to link these edits with IDM. If you will don't follow correct steps, you will end with following error in provisioning of users with incorrectly edited AD group:

<note tip>
<code>

org.identityconnectors.framework.common.exceptions.ConnectorException: javax.naming.NameNotFoundException:
[LDAP: error code 32 - 0000208D: NameErr: DSID-03100288, problem 2001 (NO_OBJECT), data 0, best match of:'OU=Groups,DC=test_company,DC=local'];
remaining name 'CN=My_test_group,OU=Groups,DC=test_company,DC=local'

</code>

This error means that CzechIdM can not find DistinguishedName set in assigned role for any group in Active Directory. This group could be renamed, moved or deleted. If you come across a mentioned error, just delete items in provisioning queue for users, go through the specified tutorial and resave stuck users when it's finished. </note>

==== 1) Rename or move group in Active Directory ====

Synchronization must be started after each time you **rename**  a group or **move**  group to another organization unit. Otherwise provisioning of any user who is a member of the modified group will fail with following error in provisioning queue.

==== 2) Delete group in Active Directory or move group from CzechIdM scope ====

If you want to delete role or move it from IDM scope:

  * Make sure that no users have assigned role for this group and that the role is not used as automatic role.
  * Then you can remove group from AD and **remove role from managed attributes**.

If you deleted groups or moved from IDM scope and you will try provisioning of users with linked role before synchronization of roles, provisioning will not be successful.

You will recognize this situation by error mentioned in the note above and also if you will run synchronization of groups, in log of synchronization you will have some items in the state **Missing account**.

**To correctly remove group and role:**

  * Open synchronization item with **Missing account**  state and copy **Entity ID**  from item. In most cases ID is ObjectGUID of the group.
  * Go to **Account on system**  on system for Groups and paste Entity ID into filter. By opening found item, you can see **role**  for missing group.
  * Make sure that you remove this role from all users.
  * Remove the role from IDM.
  * Remove group from AD.
  * Go to system for AD User → Attributes mapping → Mapping for provisioning and click on attribute **ldapGroups**  → go to tab **Controlled values**  → In section **Attributes controlled in past**, you will see the group → delete it

<note warning> If you will not perform last step and role was just moved from scope of IDM, because you want to manage this role without IDM → **IDM will still remove group managed users!**  </note>

===== Tips =====

==== CREATE NEW GROUP IN ACTIVE DIRECTORY ====

You can create a new security group in Active Directory with the Apache Directory Studio by following these steps:

  - Select an existing group
  - Right click on the group name → New → New entry
  - Check the "Use existing entry as template" and click Next
  - Object classes: Write "group" and click Add → group and top are added to "Selected object classes" → Next
  - Distinguished Name: Set the value of RDN to your choice → Next
  - A warning is displayed - click Cancel
  - Set instanceType = 4
  - Set sAMAccountName to your choice (right click → Edit values)
  - Delete values (right click → Delete values) of these attributes:
      - nTSecurityDescriptor
      - objectCategory
      - member (if you don't want to copy members)
      - sAMAccountType

{{.:new_entry_attributes.png?400}}

Finally, click Finish