{{tag>audit logging siem}}

===== Audit logging for SIEM tools =====

==== Log message structure ====

The IdM provides a special type of audit logs intended for processing in SIEM tools. The structure of the audit log message is fixed. It starts with **date and time** of the event logging. It is followed by the **severity level** of the log message. In case of audit logging there is always used ''INFO'' level. Following number is the **duration** from the start of the application in milliseconds and the standard part ends with the execution **thread name**. The audit log dedicated part always starts with the **hierarchical key** defining the logged action. It consists of ''AUDIT'' keyword at the topmost level then followed by the type of the operation or event which is in the example below ''ROLE_ASSIGNMENT'' and ends with the action specification - ''CREATE'' in this case. All levels are dot separated. The audit log message consists of named attributes providing detailed information.

   * **result**  - Result of the operation. Value is either SUCCESS or FAIL.
  * **targetName**  - User friendly name/code of the target entity.
  * **targetUUID**  - UUID of target entity. Unambiguous identifier of the entity in the IdM.
  * **subjectName**  - User friendly name/code of the subject entity which plays an important role in the operation.
  * **subjectUUID**  - UUID of subject entity. Unambiguous identifier of the entity in the IdM.
  * **performedByName**  - Username of the identity who performs the logged action.
  * **performedByUUID**  - UUID of of the identity who performs the logged action. Unambiguous identifier of the identity in the IdM.
  * **transactionUUID**  - UUID which serves as binding of the logged information with the particular event in the IdM.
  * **detail**  - Holds additional information. It is a message in case of an exception. Or additional information such as request state or LRT state.
Some attribute values may stay blank if not available or there is nothing to fill in.

<code>
2021-08-23 11:49:32.142  INFO 759183 --- [event-task-executor-2] AUDIT.ROLE_ASSIGNMENT.CREATE.log : result:[SUCCESS] targetName:[ferda] targetUUID:[08b12f0e-c353-4e74-9793-cac38233a26e] subjectName:[LoggedRole] subjectUUID:[bc95d7c2-1a6e-4eec-a102-59484fde5c48] performedByName:[admin] performedByUUID:[773b5f9f-a4b3-4e60-bb86-ffd26c519db5] transactionUUID:[1f14d999-ea2b-44d6-b24f-b5ac198d512f] detail:[]

</code>

Such log lines can be conveniently parsed with GROK and other regex engines. We provide general GROK pattern below.

<code>
%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:loglevel}\s+%{NUMBER:application_run_time}\s+---\s+\[%{DATA:application_thread}\] AUDIT.%{DATA:object_type}\.%{DATA:audited_action}\.log\s+:\s+result:\[%{DATA:action_result}\]\s+targetName:\[%{DATA:target_name}\]\s+targetUUID:\[%{DATA:target_uuid}\]\s+subjectName:\[%{DATA:subject_name}\]\s+subjectUUID:\[%{DATA:subject_uuid}\]\s+performedByName:\[%{DATA:performed_by}\]\s+performedByUUID:\[%{DATA:performed_by_uuid}\]\s+transactionUUID:\[%{DATA:transaction_uuid}\]\s+detail:\[%{DATA:detail}\]

</code>

==== Logged events ====

**Identity** \\ Operation on: ''IDENTITY'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'', ''PASSWORD''  (change) \\ Target: identity \\ Datail contains ENABLED/DISABLED state if changed or the exception message if occurs

**Role** \\ Operation on: ''ROLE'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'' \\ Target: role \\ Detail: the exception message if occurs

**Change of role permission** \\ Operation on: ''ROLE_PERMISSION'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'' \\ Target: role the permission is assigned to \\ Subject: permission entity \\ Detail: the exception message if occurs

**Role assignment** \\ Operation on: ''ROLE_ASSIGNMENT'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'' \\ Target: identity the role is assigned to \\ Subject: role \\ Detail: the exception message if occurs

**Role request** \\ Operation on: ''ROLE_REQUEST'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'', ''EXECUTE'', ''REFRESH_SYSTEM_STATE'' \\ Target: role request \\ Detail: contains the state of the role request or the exception message if occurs

**Long running task** \\ Operation on: ''LRT'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'' \\ Target: long running task entity \\ Detail: state of the LRT or the exception message if occurs

**Property configuration** \\ Operation on: ''CONFIGURATION'' \\ Action: ''CREATE'', ''UPDATE'', ''DELETE'' \\ Target: configuration property entity \\ Detail: the exception message if occurs

**Login operations** \\ Operation on: ''LOGIN'' \\ Action: ''LOGIN'', ''LOGOUT'', ''USER_SWITCH'' \\ Target: identity who is logging in/out \\ Subject: original identity in case of USER_SWITCH \\ Detail: the exception message if occurs \\ //Additional Info//: Only authentication against IdM is currently logged.

==== Audit logger configuration ====

The audit logger is turned off by default. It can be turned on via dedicated properties. Configuration properties start with the common part ''idm.sec.core.logger.AUDIT''  and follow the same hierarchy as keys specifying logged action. User is hence able to control what will be logged very granularly. Audit logging uses ''INFO''  severity level of the slf4j logger (which is internally used). Therefore setting the property value to ''INFO''  or other more verbose log level such as ''TRACE'', ''DEBUG''  or ''ALL''  turns the logger on. The more general setting can be overridden by more specific setting. Example: setting properties as follows ''idm.sec.core.logger.AUDIT=INFO''  and ''idm.sec.core.logger.AUDIT.IDENTITY=ERROR''  causes that all events are logged except ''IDENTITY''  the level of which is set to ''ERROR''.

<note warning>The audit logging has to be turned on by explicitly set property either in the IdM property file or via IdM UI. The configuration property which is automatically created based on the logger logback.xml file does not suffice. It is caused by an optimization mechanism used in the audit logger implementation. </note>

=== Logback file configuration ===

The logback.xml configuration file should have overridden formatting pattern in order to increase the maximum length of the %logger{<length>} attribute. The length has to be large enough not to shorten the log key (''AUDIT.ROLE_ASSIGNMENT.CREATE.log''  see the example above). The topmost level of the audit logger setting should be set by default to a level which disables logging e.g. ''<logger name="AUDIT" level="ERROR"/>''  (see below). This line needs to be added into all ''<springProfile….>''  sections of the logback file. Added or edited logback file parts should look like this.

logback-spring.xml

<code xml>
   <configuration>
     <!-- !!!BEWARE!!! The specification of the LOG PATTERNS overrides the default configuration and increases the maximum length of the %logger{<size>} attribute.
     It is neccessary for correct function of the AUDIT logging feature (redmine ticket #2717). If AUDIT logger key is longer then the set limit it gets shortened
     and SIEM software is not able to parse logs properly. -->
     <property name="CONSOLE_LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %5level %relative --- [%thread] %logger{60}.%M : %msg%n"/>
     <property name="FILE_LOG_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} %5level %relative --- [%thread] %logger{60}.%M : %msg%n"/>
     .
     .
     .
     <springProfile name=<PROFILE_NAME>>
        .
        .
        .
        <logger name="AUDIT" level="ERROR"/>
     </springProfile>


</code>