# Persistence configuration

Transformer’s built-in Persistence Service provides a persistent key-value store, shareable between nodes in a Transformer cluster. The configuration items here are used to define how persistence databases and files are accessed and controlled.

The Persistence Service is disabled by default. See [Activating the Persistence Service 6.2](transformer-activating-the-persistence-service.md).

The Persistence Service 6.2 persists data to a relational database server or to the local filesystem, but only a relational database server is supported as a persistence store in production environments. File-backed persistence (persisting data to the local filesystem) is provided as an option of convenience for development environments, but it is unsupported in production environments.

**⚠️ WARNING**\
File-backed persistence is unsupported in production environments, and, from version 7 of the Persistence Service, file-backed persistence cannot be used in Transformer clusters.

All the nodes of a Transformer cluster can persist data and share it. If data is persisted to files, each Transformer in the cluster maintains its own persistence file, but the Transformer also sends a copy of its persisted data to every other Transformer so that they can write it to their own persistence files as well. If data is persisted to a database, there’s just one copy of the database that’s shared by all the Transformers in the cluster.

If you’re persisting the data to a database (rather than a file), besides defining the relevant configuration items that are documented here, you’ll also need to set up the database, giving it a name, defining the table and columns needed to store the persisted data, and defining the usernames(s) and access permissions that will allow Transformer to write and read the data.

Add your persistence configuration to the file `<Framework-root>/global_config/overrides/PersistenceService/Transformer/etc/persistence.conf`, where `<Framework-root>` is your Deployment Framework’s top-level directory.

**📌 NOTE**\
In the following documentation, the term "this Transformer" means the Transformer for which you are defining the configuration, and "this node" means the cluster node that "this Transformer" represents.

## add-database-params

`add-database-params` specifies parameters for the database or file used to persist data.

**Syntax:**

```
add-database-params
   xref:init-string[init-string]    [string]
   xref:driver-name[driver-name]    [string]
   xref:username[username]       [string]
   xref:password[password]       [string]
   xref:table-name[table-name]     [string]
   xref:columns[columns]        [array of strings]
   xref:extra-params[extra-params]   [array of strings]
end-add-database-params
```

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `<a name="columns"></a>columns` | array of strings | +[none]+ | A space-separated list of the names of the database columns where the persisted data is stored. _This option is not needed when persisting to a file._ Each piece of data is stored as a key-value pair. The names of the database columns should be listed in this order: &lt;key column> &lt;value column>. For example, to persist keys to a column called 'pers_key' and values to a column called 'pers_val', use the configuration below: |
```
columns pers_key pers_val
` WARNING: If you’re implementing your own Transformer module that persists data to the persistence database, make sure the keys you store have names that are unique to your module and don’t clash with the keys used by other modules, such as the [Watchlist Service](configure-transformer-to-host-synchronised-watchlists.md). |
| `<a name="driver-name"></a>driver-name` | string | +[none]+ | The name of the JDBC driver; for example, `com.mariadb.jdbc.Driver` _This option’s not needed when persisting to a file._ |
| `<a name="extra-params"></a>extra-params` | array of strings | +[implementation dependent]+ | A list of additional parameters that are used by this Transformer’s implementation of the Java `Persistor` interface. _This option’s not needed when persisting to a file._ The type, number, usage and default values of the parameters in the list are dependent on implementation of `Persistor` that’s used; see the API documentation for the `[Persistor,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/com/caplin/transformer/persistence/Persistor.html)` interface in the _[Transformer Persistence API for Java,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/)_. For example, the basic JDBC implementation of `Persistor` (`[JdbcPersistorImpl,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/com/caplin/transformer/persistence/JdbcPersistorImpl.html)`) that’s provided with, and used by, Transformer’s **Persistence Config blade** provides for two extra parameters: the Java debug level, and the number of pooled connections (the latter has a default value). Also see [`persistence-classid`](#persistence-classid). |
| `<a name="init-string"></a>init-string` | string | +[none]+ | Initialisation string for the persistence database or file. * If data is persisted to a file, this is the directory name and path of the file. In the Persistence Config blade the default file path and name is `${ccd}/persistence`, where `${ccd}` is the Transformer’s #c<mark>urrent [underline]</mark><mark>c</mark><mark>onfiguration [underline]</mark><mark>d</mark>#irectory. * If data is persisted to a database, this is the JDBC URL of the database; for example:<br> `jdbc:mariadb://localhost/mydatabase` |
| `<a name="password"></a>password` | string | +[none]+ | The database access password associated with [`username`](#username). _This option’s not needed when persisting to a file._ |
| `<a name="table-name"></a>table-name` | string | +[none]+ | The name of the database table where the persisted data is stored. _This option’s not needed when persisting to a file._ |
| `<a name="username"></a>username` | string | +[none]+ | The username that allows access to the database. _This option’s not needed when persisting to a file._ |

## disable-cluster-replication

**📌 NOTE**\
Persistence Service 6.2 only. The Persistence Service 7 does not replicate persisted values between nodes in a cluster.

`disable-cluster-replication` when ``TRUE,``prevents this Transformer from _sending to_ the other Transformers in the cluster data that it has already persisted.

When persistence is file-based, don’t specify `disable-cluster-replication`. The default value of `FALSE` then applies, and, although each Transformer in the cluster maintains its own persistence file, this Transformer sends a copy of its persisted data to every other Transformer so that they can also write it to their own persistence files.

Conversely, when a (shared) persistence database is used, you should set `disable-cluster-replication` to `TRUE` on every node in the cluster to prevent multiple copies of the same data being persisted to the (shared) database. You should set [`dont-persist-cluster-messages`](#dont-persist-cluster-messages) to `TRUE` as well.

**Syntax:** `disable-cluster-replication <boolean>`

**Type:** boolean

**Default value:** `FALSE`

## dont-persist-cluster-messages

**📌 NOTE**\
Persistence Service 6.2 only. The Persistence Service 7 does not replicate persisted values between nodes in a cluster.

`dont-persist-cluster-messages` when `TRUE`, ensures that messages _received from_ another Transformer in the cluster are not persisted on _this_ Transformer.

When persistence is file-based, don’t specify `dont-persist-cluster-messages`. The default value of `FALSE` then applies, so that this Transformer saves to its own persistence file all the persisted data that other Transformers in the cluster send to it.

Conversely, when a (shared) persistence database is used, you should set `dont-persist-cluster-messages` to `TRUE` on every node in the cluster, to prevent the same data being persisted to the database multiple times (the data’s then persisted solely by the Transformer where it originated). You should set [`disable-cluster-replication`](#disable-cluster-replication) to `TRUE` as well.

**Syntax:** `dont-persist-cluster-messages <boolean>`

**Type:** boolean

**Default value:** `FALSE` (messages from another cluster node are always persisted on _this_ node)

## enable-file-database

**⚠️ WARNING**\
File-backed persistence is unsupported in production environments, and, from version 7 of the Persistence Service, file-backed persistence cannot be used in Transformer clusters.

`enable-file-database` when `TRUE`, enables file-based persistence.

**Syntax:** `enable-file-database <boolean>`

**Type:** boolean

**Default value:** `FALSE` (data is persisted to a database)

## logfile

`logfile` specifies the name of the file in which events about persisting data are recorded.

The filename can contain the parameters `%a` and `%h` At run time, `%a` is replaced by the Transformer’s application name (see the DataSource configuration item [`application-name`](../datasource/datasource-run-time-environment-configuration.md#application-name)), and `%h` is replaced by the host name of the machine on which the Transformer is running.

**Syntax:** `logfile <log-file-name>`

**Type:** string

**Default value:** `persistence.log`

## log-level

`log-level` specifies the severity of the errors and events that Transformer’s persistence module writes to its log file (see [`logfile`](#logfile)).

**Syntax:** `log-level <log-level-name>`

**Type:** string

**Default value:** `INFO`

**Values accepted:**

| log-level-name | DESCRIPTION |
| --- | --- |
| `DEBUG` | Reports all errors and events. |
| `INFO` | Reports events and information regarding normal operation of the persistence module, and all errors included in the `WARN`, `NOTIFY`, `ERROR` and `CRIT` log levels. |
| `WARN` | Reports minor errors and all errors included in the `NOTIFY`, `ERROR` and `CRIT` log levels. |
| `NOTIFY` | Reports errors regarding data corruptions and all errors included in the `ERROR` and `CRIT` log levels. |
| `ERROR` | Reports serious errors and all errors included in the `CRIT` log level. |
| `CRIT` | Reports critical errors that prevent the persistence module from running. |

## persistence-classid

`persistence-classid` specifies the Java class ID of the implementation of the Java `[Persistor,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/com/caplin/transformer/persistence/Persistor.html)` interface that provides this Transformer’s database persistence capability. There’s a basic JDBC implementation of `Persistor` that’s provided with, and used by, Transformer’s **Persistence Config blade**; it’s called `[JdbcPersistorImpl,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/com/caplin/transformer/persistence/JdbcPersistorImpl.html)`. (For more about `Persistor` and `JdbcPersistorImpl`, see the _[Transformer Persistence API for Java,opts="nofollow"](https://docs.caplin.com/developer/api/transformer_persistence/6/)_ documentation.)

The class ID is conventionally the package name of the Java class (with . replaced by /), plus the class name itself (see the example below).

***_Syntax:_ ***`persistence-classid <java-class-id>`

**_Type:_** string

**_Default value:_** (none; but `persistence-classid` is only required if [`enable-file-database`](#enable-file-database) is `FALSE`)

**_Example:_**

```
persistence-classid com/caplin/transformer/persistence/JdbcPersistorImpl
```

If you’ve implemented your own Persistance Adapter, you must:

* Specify your custom Persistance Adapter class in an [`add-javaclass`](../datasource/datasource-java-jvm-configuration.md#add-javaclass) configuration item.
* Specify the same class id for the Persistance Adapter in both the [`class-id`](../datasource/datasource-java-jvm-configuration.md#class-id) option of the [`add-javaclass`](../datasource/datasource-java-jvm-configuration.md#add-javaclass) item and the `persistence-classid` item.

---

**See also:**

* [Activating the Persistence Service](transformer-activating-the-persistence-service.md)
* [Transformer clustering features and concepts](transformer-clustering.md)
* Reference: [Transformer Clustering configuration](transformer-clustering-configuration.md)
* How can I ... [Configure Transformer to host synchronised watchlists](configure-transformer-to-host-synchronised-watchlists.md)