# Charting: cache and validate intraday data

This page provides information on the configuration items for caching intraday data, including options for conditional caching based on validation rules.

## max-age

Maximum amount of time (in days) to keep caching a user-requested symbol after it was last discarded.

**Default value**: 10

## cache-directory

Directory to store the cached charting data files. In the default configuration for the Charting Module, these files are stored in a directory `charts` relative to Transformer’s working directory. For a Transformer deployed under a [Caplin Deployment Framework](../deployment-framework/index.md), this value resolves to `servers/Transformer/charts`.

**Default value**: `${ccd}/charts`

**Example**: `cache-directory ${ccd}/var/charts`

## add-fieldset

Defines how a Transformer handles a set of instruments and which fields should be cached for that type. See [here](transformer-charting-blade-examples-add-fieldset.md) for examples showing `add-fieldset` in use.

There are two ways of matching a fieldset to an instrument: using a set of regular expressions that match the subject of the instrument (using `include-pattern` and `exclude-pattern`), or using the contents of the "RECORDTYPE" field.

The RECORDTYPE field is a field available on **RICs** (Refinitiv Instrument Code) requested from Refinitiv that identifies the asset class and type of instrument. See [record type codes](transformer-intraday-parameter-codes-and-values.md).

**Syntax**:

```
add-fieldset
    exclude-pattern  [value]
    include-pattern  [value]
    recordtype       [value]
    add-field        [field] [size] [flags] [source-field] [group-code]
    cache-field      [value]
    rules            [value]
    pipeline-file    [value]
    pipeline-func    [value]
end-fieldset
```

**💡 TIP**\
If you use `include-pattern` and `exclude-pattern` configuration to match a fieldset, you should always put the excludes before the includes as shown above.

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| `recordtype` | int | -- | The code identifying the record type. See [record type codes](transformer-intraday-parameter-codes-and-values.md#recordtype). |
| `add-field` | string array | -- | A calculated field that is based on the value of a field in the update (`source-field`, below). You can define multiple `add-field` options per `add-fieldset` stanza. This option takes 5 parameters: * `field` — The name of the field * `size` — The size of the field in characters (used in calculating the required disk storage). If a field is bigger than this it will be truncated. * `flags` — Flags to specify which type of caching should be used. For a list of valid flags, see [Charting: add-fieldset options](transformer-intraday-parameter-codes-and-values.md). * `source-field` — The name of the field that is the source of the data. * `group-code` — The name of the group in which this field is included. |
| `cache-field` | string array | -- | Space-separated list of fields holding details such as the instrument’s full name and permissioning information. |
| `exclude-pattern` | string | -- | Regex (first match) |
| `include-pattern` | string | -- | Regex (first match) |
| `rules` | string array | -- | A Reverse Polish notation (RPN) expression to validate an incoming price update before the fieldset caches it. If the RPN expression returns a non-zero value, the price update is not cached. You can define multiple `rules` options per `add-fieldset` stanza. You can also write validation rules using Lua and JavaScript. See the `pipeline-file` and `pipeline-func` options below. For more information on writing RPN expressions for `rules`, see [Charting: add-fieldset option values](transformer-intraday-parameter-codes-and-values.md). For example validation rules in RPN and Lua, see [Charting: add-fieldset examples](transformer-charting-blade-examples-add-fieldset.md#validation). IMPORTANT: Do not include both `rules` and `pipeline-*` options in the same `add-fieldset` stanza. |
| `pipeline-file` | string | -- | Name of a Transformer pipeline script file (Lua or JavaScript) used to validate an incoming price update before the fieldset caches it. Requires the `pipeline-func` option. For example validation rules in RPN and Lua, see [Charting: add-fieldset examples](transformer-charting-blade-examples-add-fieldset.md#validation). IMPORTANT: Do not include both `rules` and `pipeline-*` options in the same `add-fieldset` stanza. |
| `pipeline-func` | string | -- | Name a Transformer pipeline function (Lua or JavaScript), used to validate an incoming price update before the fieldset caches it. Requires the `pipeline-file` option. For example validation rules in RPN and Lua, see [Charting: add-fieldset examples](transformer-charting-blade-examples-add-fieldset.md#validation). IMPORTANT: Do not include both `rules` and `pipeline-*` options in the same `add-fieldset` stanza. |

### Defining the instruments to cache

The `add-fieldset` stanza defines the fields to cache for an instrument or set of instruments.

To define the instruments that an `add-fieldset` stanza applies to, use the following three configuration options:

* `recordtype`: the instrument’s record-type
* `include-pattern`: the instrument’s RTTP subject
* `exclude-pattern`: the RTTP subject of the instrument

### Validating instrument values

You can refine the applicability of an `add-fieldset` stanza by specifying validation rules for specific fields in an update. If the validation rules are not met, then the fields defined in the `add-fieldset` stanza are not cached.

For example, you could reject updates in which the value of a specific field is zero or outside a percentage difference from the previously cached value.

To define validation rules for an `add-fieldset` stanza, use one of two sets of configuration options below:

* `rules`: a reverse Polish notation (RPN) expression.
* `pipeline-func` and `pipeline-file`: the name of a pipeline function and the name of the script file in which the function is defined.

For example validation rules in RPN and Lua, see [Charting: add-fieldset examples](transformer-charting-blade-examples-add-fieldset.md#validation).

## add-interval

`add-interval` defines a caching period.

**Syntax**:

```
add-interval
    name                 [string]
    cache-interval       [integer]
    cache-size           [integer]
    historic-routing     [string] [string]
end-interval
```

| Option | Type | Default | Description |
| --- | --- | --- | --- |
| name | string | - | - |
| cache-size | integer | - | Number of data points in the cache |
| cache-interval | integer | - | Interval, in seconds, between data-points. Specify an interval of 0 to cache every change in value. |
| historic-routing | string array | - | - |

### Examples

In the example below, all requests for `1m` (one minute) will be satisfied by the intraday cache within the Charting blade. The cache will contain 1000 data points and will be updated every 60 seconds.

```
add-interval
    name                 1m
    cache-interval       60
    cache-size           1000
    historic-routing     %1 file
end-interval
```

In the example below, any requests to chart an instrument under the `/I` directory for `1d` (one day) will be forwarded onto a datasource request. So for example, when requesting `/I/VOD.L` with a 1d period from the charting blade, a request will be made for `/TS1/dVOD.Ld` to obtain the charting data.

```
add-interval
   name                  1d
   historic-routing      /I/%1  /TS1/d
end-interval
```