{{tag> topic module idm-tech technical_asset csv import lrt eav }}

====== ImportTechnicalAssetsFromCSVExecutor ======

Long Running Task (LRT) that bulk-creates or updates **Technical Assets** in CzechIdM by reading a semicolon-separated CSV file. Each CSV row represents one technical asset. Besides the scalar attributes of the asset the task imports its guarantors and holders and its extended (EAV) attributes. It validates the whole row before it writes anything and supports dry-run mode.

<badge>@since 16.1.0</badge>

The task is the writing counterpart of the generic ''generic-entity-report'' export: there is no other bulk write path for technical assets, because the standard IdM export and import does not cover them and they cannot be synchronised from a target system.

===== Version =====

^Version  ^Compatible with product  ^Notes  |
|16.1.0  |16.0.0  |First implementation of the task.  |

===== How It Works =====

  * The task reads the attached CSV file line by line (first line = header).
  * For each data row it looks up an existing Technical Asset by the value in the **code column** (matched against ''code'', which is globally unique).
  * Depending on whether the asset exists and which behaviour flags are set, the asset is **created**, **updated** or **skipped**.
  * Before anything is written, the whole row is validated in one gate. Every problem found in the row is reported together and the row is aborted without a partial write.
  * Optional columns assign guarantors and holders, and pairs of prefixed columns import extended attributes.
  * All item-level results are written to the LRT task log.

<note important>
An unmapped column means "keep the current value", but an **empty cell of a mapped column clears the value**. This is the opposite of the technical account import, so read [[#empty_cells|Empty Cells]] before you configure the task.
</note>

===== CSV Format =====

^Property  ^Value  |
|Encoding  |UTF-8  |
|Column separator  |'';'' (configurable)  |
|First row  |Header with column names  |
|Multi-values  |<nowiki>|</nowiki> by default (configurable)  |
|Date format  |''yyyy-MM-dd'' (ISO 8601)  |

<note>
Only the **code column** is mandatory. All other columns are optional - the task reads a column only when the corresponding parameter is configured **and** the column exists in the CSV header. Configuring a column name that is absent from the header aborts the whole task with an error listing the available columns.
</note>

The recommended column names follow the asset attributes of the generic report export, so an export can be edited and fed back in: `code`, `name`, `description`, `externalId`, `externalCode`, `validFrom`, `validTill`, `disabled`, `pam`, `zone`.

===== Task Parameters =====

==== File and Format ====

^Parameter  ^Type  ^Required  ^Default  ^Description  |
|''importFile''  |Attachment  |Yes  |-  |CSV file to import  |
|''separator''  |String  |No  |'';''  |Column separator character  |
|''encoding''  |String  |No  |''utf-8''  |File encoding  |

==== Column Mapping ====

Each of the **column** parameters below is the exact name of a column in the CSV header (case-sensitive).

^Parameter  ^Description  |
|''codecolumn''  |**Required.** Asset code, the pairing key. Globally unique. An empty cell fails the row.  |
|''namecolumn''  |Asset name. Mandatory in the database: required when creating a new asset and an empty cell **fails the row** instead of clearing the value.  |
|''descriptioncolumn''  |Free-text description.  |
|''externalidcolumn''  |External identifier. Must be unique; the task also rejects a duplicate occurring twice within one file.  |
|''externalcodecolumn''  |External code. Same uniqueness rule as ''externalidcolumn''.  |
|''validfromcolumn''  |Validity start date (''yyyy-MM-dd'').  |
|''validtillcolumn''  |Validity end date (''yyyy-MM-dd''). The resulting validity must not end before it starts.  |
|''disabledcolumn''  |Inactive flag. `1`, `true` and `yes` mean inactive, `0`, `false` and `no` mean active; matching is case-insensitive and the cell is trimmed. Any other value fails the row. Disabling an existing asset is gated by ''candisable'' - see [[#disabling_an_asset|Disabling an Asset]].  |
|''pamcolumn''  |PAM flag, same parser as ''disabledcolumn''.  |
|''zonecolumn''  |Security zone of the asset - see [[#zone|Zone]].  |
|''guarantoridentitycolumn''  |Guarantors given by **username**, multi-value.  |
|''guarantorrolecolumn''  |Guarantors given by **role code**, multi-value.  |
|''holderidentitycolumn''  |Holders given by **username**, multi-value.  |
|''holderrolecolumn''  |Holders given by **role code**, multi-value.  |
|''eavcodeprefix''  |Prefix of the columns carrying the code of an extended attribute - see [[#extended_attributes|Extended Attributes]].  |
|''eavvalueprefix''  |Prefix of the columns carrying the value of an extended attribute.  |
|''multivalueseparator''  |Character separating values in multi-value columns. Default <nowiki>|</nowiki>.  |

<note tip>
In the UI a **guarantor** manages the asset, a **holder** uses it. The Czech labels are `garant` and `disponent`.
</note>

==== Behaviour Flags ====

All flags are of type Boolean and default to ''false''.

^Parameter  ^Description  |
|''cancreate''  |Allow creating new assets. When ''false'', rows whose code is not found are skipped.  |
|''canoverwrite''  |Allow updating existing assets. When ''false'', existing assets are skipped.  |
|''candisable''  |Allow switching an existing asset to inactive. Without it the transition to ''disabled = true'' fails the row. The flag exists because disabling an asset cascades to its technical accounts.  |
|''canchangezone''  |Allow changing the zone of an existing asset, clearing it included.  |
|''replacerelations''  |Synchronise guarantors and holders to the exact state given by the CSV, that is remove the relations the CSV does not list. Without it the task only adds.  |
|''validatezone''  |Check the zone value against the items of the ''security-zones'' code list. An unknown value fails the row.  |

===== Empty Cells =====

The value is trimmed first, so a cell containing only spaces behaves as empty.

  * An **unmapped column** means the value is not touched.
  * An **empty cell of a mapped column** clears the value, and it clears it to ''null'', not to an empty string.

There are three deliberate exceptions.

^Column  ^Unmapped  ^Empty cell  ^Note  |
|''code''  |not possible, it is required  |row fails  |pairing key, NOT NULL in the database  |
|''name''  |row fails for a new asset  |row fails  |NOT NULL in the database, it cannot be cleared  |
|''description'', external identifiers, validity, ''zone''  |not changed  |cleared  | |
|''disabled'', ''pam''  |not changed  |**not changed**  |exception - a primitive boolean has no ''null''  |
|the four relation columns  |not changed  |**not changed**; with ''replacerelations'' it removes every relation of that type  |exception  |

<note tip>
Clearing to ''null'' matters for ''externalId'' and ''externalCode''. They are business-unique, so two assets holding an empty string would collide, while two assets holding ''null'' do not.
</note>

===== Processing Logic per Row =====

<code>
read code from codecolumn; empty  ->  row fails
find the asset by code
  |- NOT FOUND + cancreate = false   ->  log CSV_IMPORT_ASSET_SKIPPED (NOT_EXECUTED), stop
  |- FOUND + canoverwrite = false    ->  log CSV_IMPORT_ASSET_SKIPPED (NOT_EXECUTED), stop

validatePreSave - one gate, nothing is written yet
  |- lengths of code, name, zone and the external identifiers; description
  |- name present for a new asset, not cleared for an existing one
  |- validFrom and validTill parsable, validity not inverted
  |- externalId / externalCode free, both in the database and within the file
  |- disabled and pam recognised boolean values
  |- transition to disabled = true on an existing asset requires candisable
  |     and the number of accounts the cascade will disable is counted here
  |- zone change requires canchangezone; with validatezone also a code list lookup
  |- every guarantor and holder username and role code resolved
  |- every extended attribute code present in the main form definition
  |- for an existing asset with linked accounts: the assignment check script
  |- any problem  ->  all problems joined into one CSV_IMPORT_ASSET_ERROR, row aborted

write
  |- save the asset (this is what triggers the cascade when it is being disabled)
  |- guarantors and holders: add only, or synchronise when replacerelations is on
  |- extended attribute values
</code>

<note>
Every reference in the row is resolved **before** the first write. An unknown username, role code or extended attribute aborts the row with ''CSV_IMPORT_ASSET_ERROR'' - the asset is neither created nor updated. All problems found in the row are reported in a single message and the import continues with the following rows.
</note>

===== Multi-Value Columns =====

The four relation columns can carry several values separated by the multi-value separator (default `|`). Whitespace around each value is trimmed.

<code>
jnovak|pdvorak|mmaly
</code>

<note tip>
Enter the separator as a plain character (`|`, `,`, `;`, `#`). The task escapes it for its internal regular expression itself, so a regex metacharacter is matched literally and must **not** be pre-escaped by the operator. An already escaped value would be escaped again on the next run and splitting would break.
</note>

===== Guarantors and Holders =====

The asset has four independent relation agendas: guarantors given by identity, guarantors given by role, holders by identity and holders by role. Each mapped column is processed on its own.

^''replacerelations''  ^Behaviour  |
|''false'' (default)  |Add only. Relations listed in the CSV are added if missing; relations that the CSV does not list are kept. An empty cell does nothing.  |
|''true''  |The relations of that type are synchronised to the CSV. Relations missing from the CSV are removed, and an **empty cell of a mapped column removes all relations of that type**.  |

A relation that already exists is not duplicated, so repeated runs are idempotent. Assigning a guarantor or a holder does **not** grant the identity the ''techAssetGuarantorRole'' or ''techAssetHolderRole'' role; the task does not touch role assignments.

===== Extended Attributes =====

Values of extended (EAV) attributes are imported from pairs of prefixed columns. With ''eavcodeprefix'' set to `eavCode` and ''eavvalueprefix'' to `eavValue` the task reads `eavCode1` and `eavValue1`, then `eavCode2` and `eavValue2`, and so on for as long as both columns of the pair exist in the header. The number of pairs is not limited.

<code csv>
code;eavCode1;eavValue1;eavCode2;eavValue2
CORE-INFRA;costCentre;4711;environment;PROD
</code>

  * The values always go into the **main** form definition of the technical asset. There is no parameter for choosing another definition.
  * An attribute code that is not in the main definition **fails the row**.
  * An empty value **clears** the attribute, consistently with the rule for ordinary columns.
  * A multi-valued attribute is split by the multi-value separator.
  * A pair whose code cell is empty is skipped - the value has nowhere to go.

<note>
The value is handed over as a string, so for attributes that are not textual (date, number) the outcome depends on the conversion done by the core. A conversion failure fails that row.
</note>

===== Zone =====

The zone is an attribute of the asset. Two independent things guard it.

  * ''canchangezone'' gates any change of the zone on an existing asset, clearing it included. Without the flag such a row fails.
  * ''validatezone'' additionally checks the value against the items of the ''security-zones'' code list. The code list is created empty by the product and filled by the customer, so with an empty code list only an empty value passes.

Whether the resulting zone is acceptable for the accounts of the asset is a separate question, decided by the assignment check script below.

===== Assignment Check Script =====

Whenever a row updates an existing asset that has technical accounts assigned, the pair (asset, account) is validated by a configurable Groovy script through ''TechnicalAssetService.isAssetAssignable''. The script is named by the private configuration property ''idm.sec.tech.accountAssetAssignment.checkScript''.

<code properties>
# Script deciding whether a technical asset may be assigned to an account.
# Default: nothing is enforced out of the box.
idm.sec.tech.accountAssetAssignment.checkScript=techAccountAssetAssignmentAllowAll
</code>

  * ''techAccountAssetAssignmentAllowAll'' - the shipped default, always returns ''true''.
  * ''techAccountAssetAssignmentZoneMatch'' - requires the asset zone and the account zone to match; an asset without a zone is always accepted.

The check is evaluated on a candidate copy of the asset with the whole CSV row already applied, against every AccAccount the asset is assigned to. Because the script may decide by any attribute of the asset, the check runs for every row that updates an existing asset, not only when the zone column is mapped. When the script rejects a pair, the row fails with ''CSV_IMPORT_ASSET_ERROR'' before anything is written; the same happens in dry-run.

<note>
The verdict is the one ''TechnicalAssetAssignmentCheckProcessor'' would give when the asset is saved. The task asks the script itself so that the row fails early, with a readable reason, and so that a dry run reports it too.
</note>

===== Disabling an Asset =====

<note warning>
Switching an asset to inactive **cascades to all its technical accounts**: each of them is disabled and provisioning is triggered for it. One CSV row can therefore turn into N provisioning operations against target systems. The cascade is one-way - re-enabling the asset does not re-enable the accounts, and this task does not do it either.
</note>

The transition to ''disabled = true'' on an existing asset is allowed only with ''candisable''. When it happens, the item log of that row carries the number of accounts the cascade disabled, and the result code of the row is ''CSV_IMPORT_ASSET_UPDATED_WITH_CASCADE''. Accounts that were already disabled are neither touched nor counted. A newly created asset has no accounts yet, so creating it as inactive is not gated.

===== Dry-Run Mode =====

The task supports CzechIdM's built-in dry-run flag. When dry-run is enabled:

  * No asset is created or updated, no relation is changed and no extended attribute value is written.
  * Rows that would be created or updated are logged with the ''WOULD_BE'' result codes and state ''NOT_EXECUTED''.
  * The full validation still runs, so a dry run reports the same row failures a real run would hit, a rejection by the assignment check script included.
  * The size of the disabling cascade is reported as well, because it is counted during validation.
  * Would-be operations are written to the application log at INFO level (prefix ''[DRY-RUN]'').
  * The task counter still increments, so the row count can be verified.

<note tip>
To execute a dry run, enable the **Dry run** toggle when scheduling the task in the UI, or set ''dryRun = true'' on the ''IdmLongRunningTaskDto'' before calling ''LongRunningTaskManager.executeSync()'' in code.
</note>

===== Result Codes =====

^Code  ^State  ^Counted as  ^Description  |
|''CSV_IMPORT_ASSET_CREATED''  |''EXECUTED''  |successItemCount  |Asset was created.  |
|''CSV_IMPORT_ASSET_UPDATED''  |''EXECUTED''  |successItemCount  |Asset was updated.  |
|''CSV_IMPORT_ASSET_UPDATED_WITH_CASCADE''  |''EXECUTED''  |successItemCount  |Asset was updated and the disabling cascade switched off N technical accounts.  |
|''CSV_IMPORT_ASSET_WOULD_BE_CREATED''  |''NOT_EXECUTED''  |warningItemCount  |Dry-run: the asset would be created.  |
|''CSV_IMPORT_ASSET_WOULD_BE_UPDATED''  |''NOT_EXECUTED''  |warningItemCount  |Dry-run: the asset would be updated.  |
|''CSV_IMPORT_ASSET_WOULD_BE_UPDATED_WITH_CASCADE''  |''NOT_EXECUTED''  |warningItemCount  |Dry-run: the asset would be updated and the cascade would switch off N accounts.  |
|''CSV_IMPORT_ASSET_SKIPPED''  |''NOT_EXECUTED''  |warningItemCount  |Row skipped (''cancreate = false'' or ''canoverwrite = false'').  |
|''CSV_IMPORT_ASSET_ERROR''  |''EXCEPTION''  |failedItemCount  |Row failed. This is the single code for every row-level failure; all problems found in the row are joined into one reason.  |

<note>
The counters follow the operation state, not the HTTP status: ''EXECUTED'' and ''CREATED'' count as success, ''EXCEPTION'' as failed and everything else, including ''NOT_EXECUTED'', as a warning. A skipped row is therefore a warning here, while the technical account import counts it as a failure - the difference is deliberate.
</note>

===== Examples =====

==== Minimal CSV Example ====

Creating assets with a code and a name only.

<code csv>
code;name
CORE-INFRA;Core Infrastructure
DATABASES;Databases
MAIL;Mail Platform
</code>

Task configuration:

^Parameter  ^Value  |
|''codecolumn''  |`code`  |
|''namecolumn''  |`name`  |
|''cancreate''  |''true''  |

==== Full CSV Example ====

<code csv>
code;name;description;externalId;validFrom;validTill;zone;disabled;pam;guarantors;holders;eavCode1;eavValue1
CORE-INFRA;Core Infrastructure;Shared infrastructure;EXT-1;2024-01-01;;ZONE_A;false;true;jnovak|pdvorak;mpolak;costCentre;4711
DATABASES;Databases;Database platform;EXT-2;2024-01-01;2026-12-31;ZONE_B;false;false;mpolak;jnovak;costCentre;4712
</code>

Task configuration:

^Parameter  ^Value  |
|''codecolumn''  |`code`  |
|''namecolumn''  |`name`  |
|''descriptioncolumn''  |`description`  |
|''externalidcolumn''  |`externalId`  |
|''validfromcolumn''  |`validFrom`  |
|''validtillcolumn''  |`validTill`  |
|''zonecolumn''  |`zone`  |
|''disabledcolumn''  |`disabled`  |
|''pamcolumn''  |`pam`  |
|''guarantoridentitycolumn''  |`guarantors`  |
|''holderidentitycolumn''  |`holders`  |
|''eavcodeprefix''  |`eavCode`  |
|''eavvalueprefix''  |`eavValue`  |
|''multivalueseparator''  |<nowiki>|</nowiki>  |
|''cancreate''  |''true''  |
|''canoverwrite''  |''true''  |
|''canchangezone''  |''true''  |

<note tip>
The blank ''validTill'' cell in the first row **clears** the end of validity, it does not keep it. To keep the current value, leave the whole ''validtillcolumn'' parameter unconfigured.
</note>

===== See Also =====

  * [[.:importtechacclrt|]] - the sibling task importing Technical Accounts.