You are viewing the documentation for an outdated or unreleased devel version.
This page is also available in versions: 9.5, 9.7 (current), devel

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
devel:documentation:adm:configuration:events [2019/04/26 09:32]
tomiskar [Features]
devel:documentation:adm:configuration:events [2019/05/13 08:49] (current)
tomiskar
Line 1: Line 1:
-====== Events - processing of events ====== +{{page>​..:​..:​architecture:dev:events}}
- +
-{{tag> event architecture configuration processor}} +
- +
- +
-===== Event types ===== +
- +
-Every module can publish their own event types. Basic (core) event types +
-  * ''​CREATE''​ - synchronous,​ new entity is created +
-  * ''​UPDATE''​ - synchronous,​ entity is updated +
-  * ''​DELETE''​ - synchronous,​ entity is removed +
-  * ''​NOTIFY''​ - [[#​asynchronous_event_processing|asynchronous]],​ notify about entity is changed (CU). +
-  * ''​EAV\_SAVE''​ - synchronous,​ some entity extended attribute is modified (CUD) - it is synonym for ''​NOTIFY''​ type => notify about entity extended attribute is changed (only extended attribute is changed). When entity is saved together with extended attributes, then only source event (e.g. CREATE) on owner entity is published (=> eav attributes are saved after owner entity is saved in once process line - e.g. CREATE). +
- +
-Different (and custom) event types can be used for different entities. +
- +
-Event is published with content and specific event type (e.g. ''​CREATE'',​ ''​PASSWORD''​). Processors can be registered by content type and event type - e.g. process event with ''​IdmIdentityDto''​ content and event type ''​CREATE''​ => other event contents and other event types will not be processed by this processor. +
- +
-===== Entities with event support ===== +
- +
-Supported events for individual entities: +
-  * ''​core'':​ +
-    * ''​IdmAuthorizationPolicyDto''​ - authoriyation policies +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmAutomaticRoleAttributeDto''​ - automatic role by attribute +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmAutomaticRoleAttributeRuleDto''​ - rule of automatic role by attribute +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmIdentityDto''​ - operation with the identity +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​EAV\_SAVE'',​ ''​NOTIFY''​ +
-      * adds event type ''​PASSWORD''​ - changes password +
-      * adds event type ''​PASSWORD\_EXPIRED''​ - password expires +
-    * ''​IdmProfileDto''​ - operation with the identity profile +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmPasswordDto''​ - operation with the CzechIdM password +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * event is propagated when identity log in (e.g. last login date is changed) +
-    * ''​IdmTokenDto''​ - operation with the CzechIdM token +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * event is propagated when token is created, disabled (=> updated), deleted - e.g. when identity log in / out (token type ''​cidmst''​) +
-    * ''​IdmRoleDto''​ - operation with the role +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​EAV\_SAVE'',​ ''​NOTIFY''​ +
-    * ''​IdmRoleGuaranteeDto''​ - operation with the role guarantee by identity +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmRoleGuaranteeRoleDto''​ - operation with the role guarantee by role +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmRoleRequestDto''​ - role requests +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * adds event type ''​EXCECUTE''​ - execute role request (//i know EXECUTE!, but it's too late ...//) +
-      * supports event type ''​NOTIFY''​ - is published, when request is completely approved or executed - provisioning listen this event mainly. +
-    * ''​IdmRoleCatalogueDto''​ - operation with the role catalogue +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​NOTIFY''​ +
-    * ''​IdmRoleCompositionDto''​ - operation with the business roles +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmIdentityRoleValidRequestDto''​ - role starts to be valid +
-      * adds event type ''​IDENTITY\_ROLE\_VALID''​. +
-    * ''​IdmIdentityRoleDto''​- assigning a role to the user +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * supports base event type ''​NOTIFY''​ - when parent role request event is propagated, then provisioning listener skips this elementary event processing and waits to role request is completed - listen ''​NOTIFY''​ on ''​IdmRoleRequestDto''​. +
-    * ''​IdmIdentityContractDto''​ - labor-law relation +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​EAV\_SAVE'',​ ''​NOTIFY''​ +
-    * ''​IdmContractPositionDto''​ - other contract position +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​NOTIFY''​ +
-    * ''​IdmRoleTreeNodeDto''​ - automatic role by tree structure +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​PasswordChangeDto''​ - used in password prevalidation only +
-      * adds event type ''​PASSWORD\_PREVALIDATION''​ - evaluates registered password policies - show policies setting before password is changed. +
-    * ''​IdmContractGuaranteeDto''​ - manually added guarantee to contract +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​NOTIFY''​ +
-    * ''​IdmTreeTypeDto''​ - tree structure type +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmTreeNodeDto''​ - tree structure node +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​EAV\_SAVE'',​ ''​NOTIFY''​ +
-    * ''​IdmPasswordPolicyDto''​ - password policy +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​  +
-    * ''​IdmLongRunningTaskDto''​ - long running task +
-      * adds event type ''​END''​ - long running task ended. +
-    * ''​IdmEntityEventDto''​ - persisted event +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * adds event type ''​EXECUTE''​ - executes persisted event. +
-    * ''​IdmEntityStateDto''​ - persisted entity / event state +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmFormInstanceDto''​ - eav attribute values from single definition. +
-      * supports base event types ''​UPDATE''​ - update event type is used for saving eav attributes (=> CUD form values) +
-    * ''​IdmCodeListDto''​ - code lists +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​IdmCodeListItemDto''​ - code list items +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-  * ''​acc'':​ +
-    * ''​AccAccountDto''​ - Accounts on target system +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-      * adds event type ''​START''​ - starts provisioning for given account. +
-    * ''​AccIdentityAccountDto''​ - Identity accounts on target system +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​SysSystemDto''​ - System in ACC module +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​SysSystemMappingDto''​ - Mapping between system and his mapping of provisioning or sync +
-      * supports base event types ''​CREATE'',​ ''​UPDATE'',​ ''​DELETE''​ +
-    * ''​SysSchemaAttributeDto''​ - Connector schema on system +
-      * supports base event types ''​DELETE''​ +
-    * ''​SysProvisioningOperationDto''​ - execute provisioning operation. Look out, persisting provisioning operation into queue itself doesn'​t support events. Events are added for executing operations from queue: +
-      * adds event type ''​CREATE''​ - execute provisioning for ''​CREATE''​ operation +
-      * adds event type ''​UPDATE''​ - execute provisioning for ''​UPDATE''​ operation +
-      * adds event type ''​DELETE''​ - execute provisioning for ''​DELETE''​ operation +
-      * adds event type ''​CANCEL''​ - cancels provisioning operation +
-    * ''​AbstractSysSyncConfigDto''​ (''​SysSyncConfigDto'',​ ''​SysSyncContractConfigDto'',​ ''​SysSyncIdentityConfigDto''​) - synchronization +
-      * adds event type ''​START''​ - starts synchronization +
-      * adds event type ''​START\_ITEM''​ - starts synchronization of one item (~entity) +
-      * adds event type ''​CANCEL''​ - cancels synchronization +
-    * ''​SysSyncItemLogDto''​ - synchronization item  +
-  * ''​vs'':​ +
-    * ''​VsRequestDto''​ - Request for account change in virtual system  +
-      * adds event type ''​EXCECUTE''​ - executes request +
-  * ''​rpt'':​ +
-    * ''​RptReportDto''​ - generate report +
-      * adds event type ''​GENERATE''​ - generates request +
- +
- +
-page has been created directly in the application on the module page for an overview of all entity types and event types migrating through event processing. All the registered processors including the configuration are listed there: +
- +
-{{:​navrh:​entity-event-processors.png|}} +
- +
- +
-===== Asynchronous event processing ===== +
- +
-CzechIdM 8.0.0 ushered in a new feature - asynchronous event processing. New event type ''​NOTIFY''​ was added, all previous events (''​CREATE'',​ ''​UPDATE'',​ ''​DELETE'',​ ''​EAV_SAVE''​ etc.) are still synchronous.  +
- +
-<note important>Entity ''​DELETE''​ event is still synchronous.</​note>​ +
- +
- +
-==== Features ==== +
- +
-=== Event priority === +
- +
-Before event is persisted into queue, then event priority is evaluated, priority types: +
-  * ''​IMMEDIATE''​ - immediate ~ synchronously. Event will be executed synchronously. +
-  * ''​HIGH''​ - asynchronously (7 / 10 in one cycle, batch size can be [[..:..:application_configuration:dev:backend#​entity_events|configured]]) +
-  * ''​NORMAL''​ - asynchronously (3 / 10 in one cycle, batch size can be [[..:​..:​application_configuration:​dev:​backend#​entity_events|configured]]) +
- +
-Priority can be set to event manually or registered processors can vote about event'​s priority - see [[#​asyncentityeventprocessor|AsyncEntityEventProcessor]] - the highest priority is used. +
- +
-=== Execute date === +
- +
-Execute date can be set to event manually. Event with priority ''​HIGH''​ or ''​NORMAL''​ will be processed after given date. Can be used for events, which could be executed sometimes "in night"​. +
- +
- +
-=== Remove duplicate events === +
- +
-When internal scheduled task for executing event from queue is processed, then duplicate events are removed.  +
- +
-Duplicate event is event with the same: +
-  * owner  +
-  * event type +
-  * event properties +
-  * original source (embedded dtos and audit fields are ignored). +
- +
-Older duplicate events are removed - the newest event is used. Events are processed by priority in bulk, default bulk size can be [[..:​..:​application_configuration:​dev:​backend#​entity_events|configured]] => duplicates are removed only in this bulk (not configurable for now, see future development). Bulk size is designed this way, because events are processed by priority - event with ''​HIGH''​ priority should not wait too long for another bulk is begin to process. Remove duplicates should be redesigned from scratch - remove duplicates through whole queue. +
- +
- +
-=== Notification === +
- +
-Notification about registered asynchronous processors is prepared, when an asynchronous event is published. Notification is send into topic ''​core:​event''​ - uses console log by default and is send to currently logged identity - e.g. identity is saved, but provisioning will be executed asynchronously. Localization for asynchronous processors was added on frontend (see key ''​acc:​processor.identity-save-processor''​). +
- +
-==== Configuration ==== +
- +
-  * [[..:​..:​application_configuration:​dev:​backend#​scheduler|Scheduler]] - configure internal scheduled tasks for processing events from queue. +
-  * [[..:​..:​application_configuration:​dev:​backend#​entity_events|Event processing]] - configure asynchronous event processing +
-  * [[#​entityeventstartprocessor|Processors]] - configure entity event processors. +
- +
-===== Predefined processors order ===== +
- +
-  * **0** - basic / core functionality - operation ''​save'',​ ''​delete''​ etc. +
-  * **50** - save eav, which are send together with owner'​s dto +
-  * **100** - automatic roles computation. +
-  * **1000** - after ''​save''​ - e.g. sends notifications. +
-  * **10000** - publish ''​NOTIFY''​ event about entity is changed. +
-  * **-1000** - before ''​delete''​ provisioning (before identity role is deleted). +
-  * **Identity**:​ +
-    * **-2000** - validate password in acc module - checks all system password policies and idm default policy => all policies are evaluated in one request. If acc module is enabled, then core password validation processor can be disabled. +
-    * **-1000** - validate password in core module - checks idm default policy. +
-    * **100** - persist password +
-  * **Contract**:​ +
-    * **100** - Automatic roles recount while identity contract is saved, updated or deleted / disabled. +
-    * **200** - Contract [[..:​..:​identities:​dev:​contractual-relationship#​invalid_cr|exclusion,​ end and enable]]. +
-  * **LongRunningTask**:​ +
-    * **100** - execute scheduled long running tasks, which depends on currently ended scheduled task. +
-  * **Provisioning**:​ +
-    * **-5000** - check disabled system +
-    * **-1000** - compute attributes for provisioning (read attribute values from target system) +
-    * **-500** - check readonly system +
-    * **0** - execute provisioning (create / update / delete) +
-    * **1000** - execute ''​after''​ provisioning actions (e.g. sends notifications) +
-    * **5000** - archive processed provisioning operation. +
- +
-Other orders can be found directly in application,​ see **supported event types**. +
- +
-===== Processor configuration ===== +
- +
-Processors can be configured through ``Configurable`` interface by standard [[..:​..:​application_configuration:​dev:​backend#​entity_event_processors|application configuration]]. +