# Data services configuration

These DataSource configuration items define a DataSource application’s [data services](datasource-data-services.md).

When the DataSource application requests an object (such as subscribing to a subject), the data services system determines, through the configuration, which peers the request goes to instead of you having to specify this explicitly in the application’s code. It also prevents a request from going to any peer more than once.

Of course, if your DataSource application is an [Integration Adapter](../platform-architecture/putting-data-into-the-platform.md) that only sends data to its peers, you don’t need to define any data services for it.

**📌 NOTE**\
Data services can’t be configured in Java-based DataSource applications.

**📌 NOTE**\
"_this DataSource application_" means the DataSource application for which you are defining the configuration. "_DataSource peer_" or "_peer_" means a DataSource application that this DataSource application communicates with.

For some examples of how data services are configured, see the [data services features and concepts page](datasource-data-services.md).

## add-data-service

`add-data-service` specifies a data service that this DataSource application can use.Through the [add-source-group](#add-data-service--add-source-group) option and the [add-priority](#add-data-service--add-source-group--add-priority) option within `add-source-group`, it allows you to specify multiple peers that provide data for the service, load balance the provision of data across multiple peers, and configure alternate peers that your DataSource application can failover to. For examples of these setups, see the [data services features and concepts page](datasource-data-services.md).

<a name="default-data-service"></a>**Default data service**

If you don’t define at least one data service for your DataSource application, then the DataSource uses a default data service that requests each subject request from all static and dynamic peers of the DataSource.

The default data service contains a single-priority source group for each peer, with no minimum requirements for peer connectivity.

**Default data service**

```
add-data-service

  service-name         default
  required-state       down
  include-pattern      ^/ ①

  add-source-group ②
    required           FALSE
    add-priority
        remote-label   __peer_label__
    end-priority
  end-source-group

end-data-service
```
1. Matches all subject requests
2. Repeated for each static and dynamic peer

**Use in:** C

**Syntax**

```
add-data-service

  xref:affinity-1[affinity]                        [array of strings]
  xref:disable-auto-status[disable-auto-status]             [boolean]
  xref:discard-timeout[discard-timeout]                 [float]
  xref:exclude-pattern[exclude-pattern]                 [array of strings]
  xref:include-pattern[include-pattern]                 [array of strings]
  xref:min-source-groups[min-source-groups]               [integer]
  xref:nodata-ignore-request-timeout[nodata-ignore-request-timeout]   [boolean]
  xref:object-map-1[object-map]                      [pattern from] [pattern to]
  xref:rebalance-enable[rebalance-enable]                [boolean]
  xref:rebalance-time[rebalance-time]                  [float]
  xref:request-timeout[request-timeout]                 [float]
  xref:required-state[required-state]                  [string]
  xref:service-name[service-name]                    [string]
  xref:service-type[service-type]                    [string]
  xref:throttle-time[throttle-time]                   [float]

  xref:add-source-group-option[add-source-group]
      …
  end-source-group

end-data-service
```

**Options**

* **add-source-group**

  **Type:** group item

  The [add-source-group](#add-data-service--add-source-group) group item specifies, via the [add-priority](#add-data-service--add-source-group--add-priority) option, one or more DataSource peers that provide the data for the data service.

  * By using multiple [add-source-group](#add-data-service--add-source-group) items, you can specify multiple sources of data for the data service.
  * By using multiple [add-priority](#add-data-service--add-source-group--add-priority) options within an [add-source-group](#add-data-service--add-source-group) item, you can specify alternative sets of peers to failover to.

  For examples of these setups, see the [data services features and concepts page](datasource-data-services.md)
* **affinity**

  **Type:** array of strings | **Default:** _null_

  **Deprecated** from version 6.2.6+ of Liberator and version 6.2.5+ of Transformer; use the [affinity](#affinity-2) option within the [add-source-group](#add-data-service--add-source-group) group item instead.
* **disable-auto-status**

  **Type:** boolean | **Default:** `FALSE`

  If you don’t want this DataSource application to send [subject status](datasource-subject-status.md) changes for this data service to subscribing peers, set `disable-auto-status` to `TRUE` You’d typically do this in a Transformer, to prevent it from sending subject status changes unnecessarily on to subscribing Liberators.
* **discard-timeout**

  **Type:** float | **Default (Liberator):** value of `active-discard-timeout` | **Default (all others):** `-1.0`

  Specifies the time in seconds for which this DataSource application holds on to the data for a subscribed subject once the last client application has unsubscribed from that data. After this time, the object is deleted from the DataSource application’s cache, and the DataSource application sends a discard instruction to the data service’s peer(s) to cancel the subscription.

  For Liberator only:

  * This option overrides the setting of the global time out for active objects, `active-discard-timeout`.
  * If an object obtained by this data service was specified through the Liberator’s [add-object](../liberator/liberator-object-configuration-part-1.md#add-object) configuration item, the [discard-timeout](../liberator/liberator-object-configuration-part-1.md#discard-timeout) option of the `add-object` (if any) overrides this `discard-timeout`.
* **exclude-pattern**

  **Type:** array of strings | **Default:** _null_

  One or more regular expressions that determine subjects for which this data service must not provide data.

  This option is optional. Use it in conjunction with [include-pattern](#include-pattern).

  The options [include-pattern](#include-pattern) and `exclude_pattern` are applied in the order they are defined, and the first matching rule wins. So, define all your `exclude_pattern` options _before_ their associated [include-pattern](#include-pattern) options.

  **Using exclude-pattern to refine include-pattern**

  Match all subjects beginning /FX/MAJOR, but exclude /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP:

  ```
  exclude-pattern ^/FX/MAJOR/AUDUSD ^/FX/MAJOR/AUDGBP ①
  include-pattern ^/FX/MAJOR/ ②
  ```
  1. This data service does _not_ provide data for subjects /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP
  2. This data service provides data for subjects beginning /FX/MAJOR/

  Note that if the `exclude-pattern` was specified _after_ the `include-pattern` rule, then the rules would not exclude /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP. For either of those two subjects, the first matching rule (the winning rule) would be the `include-pattern`.

  You can have multiple occurrences of `exclude-pattern` in the same `add-data-service` item, so the above example could be written as:

  ```
  exclude-pattern ^/FX/MAJOR/AUDUSD
  exclude-pattern ^/FX/MAJOR/AUDGBP
  include-pattern ^/FX/MAJOR/
  ```
* **include-pattern**

  **Type:** array of strings | **Default:** `^/`

  One or more regular expressions that determine the subjects for which this data service provides data.

  For example: `include-pattern ^/FX/MAJOR/ ^/FX/MINOR/` specifies that this data service is to provide data for subjects that begin with `/FX/MAJOR/` or `/FX/MINOR/`

  You can have multiple occurrences of `include-pattern` in the same `add-data-service` item, so the above example could be written as:

  ```
  include-pattern ^/FX/MAJOR/
  include-pattern ^/FX/MINOR/
  ```

  You should normally specify an `include-pattern` option, otherwise the data service will return updates for all subjects (that is, subjects starting with `/`) that the service’s DataSource peers can provide.

  Also see the [exclude-pattern](#exclude-pattern) option.
* **min-source-groups**

  **Type:** integer | **Default:** `1` | **Since:** C DataSource API 7.1.6 (Liberator 7.1.5, Transformer 7.1.5)

  The minimum number of source groups within this data service that must have a status of 'up' for this data service to have a status of 'up'.
* **nodata-ignore-request-timeout**

  **Type:** boolean | **Default:** `FALSE`

  A DataSource peer that provides a data service can send back a "no data" response if there is no data available for a requested subject. The data service then discards any outstanding requests for the subject.

  The `nodata-ignore-request-timeout` option applies when multiple peers can supply requested data for a service.

  When this option is `TRUE`, if any peer providing the service sends back a "no data" response, this response is _immediately_ propagated from the DataSource API to the custom code of the DataSource application - the [request-timeout](datasource-datasource-peers-configuration-part-1.md#request-timeout)s (or [source-request-timeout](datasource-datasource-peers-configuration-part-2.md#source-request-timeout)s) for the remaining peers are ignored.

  When this option is `FALSE`, if a "no data" response is received from a peer providing the service, the data service continues wait for the data from any of the other peers, applying the [request-timeout](datasource-datasource-peers-configuration-part-1.md#request-timeout)s (or [source-request-timeout](datasource-datasource-peers-configuration-part-2.md#source-request-timeout)s), until either the data is returned by at least one peer, or all the peers have timed out or returned a "no data" response.
* **object-map**

  **Type:** string array | **Default:** | **Since:** Liberator 8

  Maps an object name to a different object name _before_ data-service routing occurs.

  This option has the same syntax and behaviour as Liberator’s [`object-map`](../liberator/liberator-object-configuration-part-2.md#object-map) configuration item.
* **request-timeout**

  **Type:** float | **Default:** _value of [service-request-timeout](#service-request-timeout)_

  Specifies the timeout in seconds for all subscription requests sent to this data service. If the DataSource peers providing the service don’t respond within this time, the requested subject is assumed to not be available and the application sends a discard message for the subject to the peers.

  A value of `0` means no timeout is set and this DataSource application then waits indefinitely for this data service to respond to a request. Use this setting in Liberator if you want to allow all Liberator’s subscription requests for this data service to be [open subscriptions](../liberator/liberator-open-subscriptions.md) (see How can I... [Keep subscriptions open when the peer is down](../liberator/liberator-keep-subscriptions-open-when-the-peer-is-down.md)).

  `request-timeout` overrides the global [service-request-timeout](#service-request-timeout) configuration item.
* **rebalance-enable**

  **Type:** boolean | **Default:** _null_ | **Since:** DataSource for C (DSDK) 7.1.21

  Enable or disable automatic rebalancing of workload in source groups that don’t have source affinity enabled.

  When rebalancing is enabled and a peer in a priority group changes its status to UP, after a set time ([`rebalance-time`](#rebalance-time)) the data service discards subscriptions to peers in the priority group and re-requests them, balancing the requests evenly over peers in the priority group.

  Overrides the global item [service-rebalance-enable](#service-rebalance-enable).
* **rebalance-time**

  **Type:** float | **Default:** _null_ | **Since:** DataSource for C (DSDK) 7.1.21

  Set the time to wait (in seconds) after a peer in a priority group (see [`add-priority`](datasource-data-services-configuration.md#add-priority)) sets its status to UP before rebalancing workload across all peers in the priority group.

  Overrides global item [service-rebalance-time](#service-rebalance-time).

  To enable service rebalancing, see [service-rebalance-enable](#service-rebalance-enable) and [rebalance-enable](#rebalance-enable).
* **required-state**

  **Type:** string | **Default:** `down`

  **📌 NOTE**\
  This option only applies to [Liberator](../liberator/index.md).

  The minimum state this data service must be in for Liberator to begin accepting client connections.

  Before version 7.1.19, Liberator ignores `required-state` after it has begun accepting connections.

  From version 7.1.19, Liberator can be configured to continue to enforce `required-state` after Liberator has begun accepting connections. If [`session-requires-services`](../liberator/liberator-session-configuration.md#session-requires-services) is set to `true`, then `required-state` is the minimum state this data service must be in for Liberator to accept _and_ retain client connections.

  **Valid values for required-state**

  * `down`, `optional`: Down (no source groups are up, or one or more required source groups are down). **Default**
  * `limited`: Limited (at least one source group is up, and all required source groups are up)
  * `ok`, `up`: Up (all source groups are up)
  * `once`: Has been in state `ok` at least once since Liberator was started.

  See also:

  * [`required`](datasource-data-services-configuration.md#required) option of `add-source-group` 
  * [Data services with minimum requirements for availability](datasource-data-services.md#data-services-with-minimum-requirements-for-peer-availability)
* **service-name**

  **Type:** string | **Default:** _null_

  The name of the data service.

  In deployments of the Caplin Platform with Discovery, providing DataSources that are configured with a matching service name in [`discovery-provide-service`](datasource-discovery-configuration.md#discovery-provide-service) automatically attempt peer connections to this DataSource. For more information, see [Peer discovery](../discovery/discovery-peer-discovery.md).
* **service-type**

  **Type:** string | **Default:** `active|broadcast`

  Specifies whether the DataSource peers that supply data for the service are of type active, broadcast, or both.

  Possible values are:

  * `active`: This data service is provided by active DataSource peers. An active DataSource keeps track of which [subjects](datasource-subjects-symbols-and-fields.md) have been [requested](datasource-subscriptions.md) and sends updates for those subjects only (see [Active subscription model](datasource-active-subscription-model.md)).
  * `broadcast`: This data service is provided by [broadcast](datasource-broadcasting-data.md) DataSource peers.
  * `active|broadcast`: This data service is provided by both active and broadcast peers.
  * `rest`: This data service provides a REST service, rebalancing is disabled for this service.
  * `contrib`: Identifies this data service as accepting contributions (serving [channels](../platform-architecture/channels.md)). Disables automatic rebalancing for this service. Automatically sets the `active` flag.

  You can use this option to override the [local-type](datasource-datasource-peers-configuration-part-1.md#local-type) option of a peer’s [add-peer](datasource-datasource-peers-configuration-part-1.md#add-peer) configuration. For example, assume a peer that supplies data for a data service is configured as an active peer and also is configured to broadcast data (that is, its ``add-peer`’s `local-type` option is set to `active|broadcast`). If you don’t want your data service to receive the peer’s broadcast data, set the ``add-data-service`’s `service-type` option to `active`.
* **throttle-time**

  **Type:** float | **Default:** `-1`| **Since:** Liberator 8

  Specifies a single [throttle](../liberator/liberator-throttling.md) level (that is the conflation period) for records and generic objects served to RTTP clients from this data service.

  When set to a value above `-1`, this option overrides the following Liberator throttling configuration for subjects served by the data service: [object-throttle-times](../liberator/liberator-throttling-configuration.md#object-throttle-times) and [add-object::throttle-times](../liberator/liberator-object-configuration-part-1.md#throttle-times).

  Only a single throttle level is supported.

### add-data-service :: add-source-group

`add-source-group` specifies, via the [add-priority](#add-data-service--add-source-group--add-priority) option one or more DataSource peers that provide the data for the data service defined by [add-data-service](#add-data-service). By using multiple `add-source-group items`, you can specify multiple sources of data for the data service. By using multiple [add-priority](#add-data-service--add-source-group--add-priority) options within an `add-source-group item`, you can specify alternative sets of peers to failover to. If you specify more than one peer within an [add-priority](#add-data-service--add-source-group--add-priority) item, subscriptions are load-balanced across them. For examples of these setups, see the [data services features and concepts page](datasource-data-services.md).

**Use in:** C

**Syntax**

```+quotes"
[gray]##add-data-service##
   [gray]##…##
   add-source-group
      xref:add-priority-option[add-priority]
        …
      end-priority
      …
      xref:affinity-2[affinity]         [array of strings]
      xref:contrib[contrib]          [boolean]
      xref:min-priorities[min-priorities]   [integer]
      xref:required[required]         [boolean]
      xref:retry-time[retry-time]       [float]
   end-source-group
   [gray]##…##
[gray]##end-data-service##
```

**Options**

* **add-priority**

  **Type:** group item

  Specifies one or more DataSource peers that this DataSource application can use to provide data for a data service. If you specify more than one peer in an `add-priority` option, [subscriptions](datasource-active-subscription-model.md) are load-balanced across those peers.

  You can also use multiple instances of `add-priority` to implement failover to alternative peers.

  **📌 NOTE**\
  Because the [remote-label](#remote-label) option of [add-priority](#add-data-service--add-source-group--add-priority) points to an [add-peer](datasource-datasource-peers-configuration-part-1.md#add-peer) configuration item, this DataSource application will first try to fail over to any alternative hosts that you’ve defined in the [addr](datasource-datasource-peers-configuration-part-1.md#addr) option of the `add-peer` item.

  For examples of load balancing and failover configurations, see the [data services features and concepts page](datasource-data-services.md).
* **affinity**

  **Type:** array of strings | **Default:** _null_

  Enables the alternative source affinity load-balancing algorithm for load-balanced adapters in this source group.

  **Syntax**: `affinity <affinity_key> <regular_expression>`

  For a full description of the syntax, guidelines on use, and a worked example, see [Load balancing adapters using source affinity](datasource-load-balance-adapters-using-source-affinity.md).
* **contrib**

  **Type:** boolean | **Default:** `FALSE`

  When set to `TRUE`, this option declares that peers listed in this source group provide contributions only (they don’t accept subscription requests).

  A source group containing contribution-only peers is a required addition to a data service that acts as a conduit for a messaging channel from one DataSource to another.

  For example, consider a data service on Transformer that routes a trade channel from Liberator to a trade adapter. Liberator is included in its own source group in the data service so that Transformer accepts Liberator as a valid source of data for the channel, and the `contrib` option of the source group is enabled to stop Transformer from routing subscription requests for the trade channel to Liberator.

  **A contribution-only source group in a Transformer data service**

  ```text
  add-data-service
    service name trade-channel
    include-pattern '^/PRIVATE/FX'

    add-source-group
      add-priority
        remote-label trade-adapter
      end-priority
    end source group

    add-source-group
      contrib
      add-priority
        remote-label liberator
      end-priority
    end-source-group
  end-data-service

  ```
* **min-priorities**

  **Type:** integer | **Default:** `1` | **Since:** C DataSource API 7.1.6, Liberator 7.1.5, and Transformer 7.1.5

  Minimum number of priority groups in this source group that must have a status of 'up' for this source group to have a status of 'up'.
* **required**

  **Type:** boolean | **Default:** `FALSE`

  When set to `TRUE`:

  * If data had been received from any peer in this data service and ***all*** of the peers in any required source group go down, then the data is marked as stale.
  * If the data service state is up or limited and then some peers go down such that the [minimum number of required priorities](#min-priorities) in this group are not up, then the data service state is changed to down.
  * If only one peer in this source group responds and that peer responds with [NODATA](../../caplin-platform/liberator/liberator-logging.md#nodata), then the DataSource API propagates the NODATA message to application code.

  When set to `FALSE`:

  * If data had been received from any peer in this data service and ***all*** of the peers in a non-required source group go down, then the data is marked as limited ( unless the data was already marked as stale ).
  * If the data service state is up and then some peers go down such that the [minimum number of required priorities](#min-priorities) in this group are not up, then the data service state is changed to limited. If the data service is in the down state then the state doesn’t change.
* **retry-time**

  **Type:** float | **Default:** `30.0`

  After finding that none of the peers defined by `add-priority` options in the source group have responded to a request, this DataSource application waits `retry-time` seconds before reissuing the request to the source group.

  **Valid values**

  * **DSDK &lt; 7.1.29:** Minimum value `1.0`. Any value &lt; `1.0` is rounded up to `1.0`.
  * **DSDK >= 7.1.29:** Minimum positive value `1.0`. Any positive value &lt; `1.0` is rounded up to `1.0`. Any value \&lt;= `0` disables the retry feature.

  The DataSource application issues the request to each of the peers in the source group in turn. Each request is timed out according to the setting of [`add-peer::request-timeout`](datasource-datasource-peers-configuration-part-1.md#request-timeout) or [`source-request-timeout`](datasource-datasource-peers-configuration-part-2.md#source-request-timeout). If none of the peers in the source group replies, the DataSource application waits `retry-time` and then tries each connection again in turn. It repeats this sequence until the master timeout for the service, defined by [`service-request-timeout`](#service-request-timeout) or [`add-data-service::request-timeout`](#request-timeout), expires. If [`add-data-service::request-timeout`](#request-timeout) is set to `-1`, the DataSource reissues the request indefinitely.

### add-data-service :: add-source-group :: add-priority

`add-priority` specifies one or more DataSource peers that this DataSource application can use to provide data for a data service. By using multiple `add-priority` options within an [add-source-group](#add-data-service--add-source-group) item, you can specify alternative sets of peers to failover to. If you specify more than one peer within an `add-priority` item, [subscriptions](datasource-active-subscription-model.md) are load-balanced across them; for an example of this, see the [data services features and concepts page](datasource-data-services.md).

**Use in:** C

**Syntax**

```+quotes"
[gray]##add-data-service##
  [gray]##...##
  [gray]##add-source-group##
    [gray]##...##
    add-priority
      xref:label[label]                   [array of strings]
      xref:min-peers[min-peers]               [integer]
      xref:remote-label[remote-label]            [array of strings]
      xref:remote-label-regex[remote-label-regex]      [string]
    end-priority
    [gray]##...##
  [gray]##end-source-group##
  [gray]##...##
[gray]##end-data-service##
```

**Options**

* **label**

  **Type:** array of strings | **Deprecated:** use `remote-label` instead

  One or more labels identifying DataSource peers that supply data for this DataService. Each label must match the [label](datasource-datasource-peers-configuration-part-1.md#label) option of an [add-peer](datasource-datasource-peers-configuration-part-1.md#add-peer) configuration item that defines a DataSource peer. If you specify more than one peer, either through multiple values of a `label` option or by multiple `label` options, subscriptions to the referenced peers are load-balanced across the peers.

  Also see [Note 1](#note-1).
* **min-peers**

  **Type:** integer | **Default:** `1` | **Since:** C DataSource API 7.1.6, Liberator 7.1.5, and Transformer 7.1.5

  The minimum number of peers in this priority group that must be connected for this priority group to have a status of 'up'.
* **remote-label**

  **Type:** array of strings

  One or more labels identifying DataSource peers that supply data for this DataService. Each label must match the [remote-label](datasource-datasource-peers-configuration-part-1.md#remote-label) option of an [add-peer](datasource-datasource-peers-configuration-part-1.md#add-peer) configuration item that defines a DataSource peer. 

  If you specify more than one peer, either through multiple values of a `remote-label` option or by multiple `remote-label` options, subscriptions to the referenced peers are load-balanced across the peers.

  Also see [Note 1](#note-1) below, and the [example](datasource-datasource-peers-configuration-part-1.md#example-of-datasrc-local-label-and-remote-label) in [DataSource peers configuration (part 1)](datasource-datasource-peers-configuration-part-1.md).
* **remote-label-regex**

  **Type:** string | **Default:** _null_ | **Since:** DataSource for C (DSDK) 7.1.21, Liberator 7.1.21, Transformer 7.1.13

  A single [regular expression](https://en.wikipedia.org/wiki/Regular_expression) (POSIX ERE) that describes the labels ([datasource-local-label](datasource-datasource-peers-configuration-part-2.md#datasrc-local-label)) of DataSource peers that supply data for this DataService. 

  This option is available only for DataSources connected to Discovery (see [discovery-addr](datasource-discovery-configuration.md#discovery-addr)).

  Only one `remote-label-regex` option may be specified per priority group. If multiple `remote-label-regex` options are specified for a priority group, only the final option applies.

  Subject requests to the peers defined by `remote-label-regex` are load balanced using the standard workload-based algorithm.

<a name="note-1"></a>Note 1: Platform blades, and Liberator and Transformer, that run under the [Caplin Platform Deployment Framework](../deployment-framework/index.md) use the deprecated [label](#label) option in their configuration, rather than [remote-label](#remote-label). This will change in a future release of the Framework.

## add-provided-data-service

**Use in:** C, Java

The `add-provided-data-service` configuration option allows an adapter to provide data service configuration to a remote Datasource application. 

For adapters written using DataSource for Java it is also possible to programatically set this configuration.

**Syntax**

```
add-provided-data-service
    xref:apply-if-label[apply-if-label]          [regex of labels that this service should be applied to]
    xref:discard-timeout[discard-timeout]         [float]
    xref:exclude-pattern-2[exclude-pattern]         [regex to exclude]
    xref:include-pattern-2[include-pattern]         [regex to include]
    xref:object-map-2[object-map]              [pattern from] [pattern to]
    xref:open-subscriptions[open-subscriptions]      [true|false]
    xref:remote-label-regex-2[remote-label-regex]      [regex matching peers]
    xref:required-state[required-state]          [down|optional|limited|ok|up|once]
    xref:service-name-2[service-name]            [string]
    xref:service-type[service-type]            [service-type-enum]
    xref:throttle-time-2[throttle-time]           [float value]
end-provided-data-service
```

**Options**

* **apply-if-label**

  **Type:** string | **Default:**

  A regular expression that restricts the provision of this data-service to DataSources with a matching label.

  The most common application for `apply-if-label` is an adapter that receives requests from Liberator routed via Transformer. In this scenario, the adapter needs to provision _both_ Liberator and Transformer with data service configuration.

  **Provided data services that route requests via Transformer**

  **Provided data service: liberator -> transformer**

  ```text
  add-provided-data-service
      service-name            blotters
      include-pattern         ^/FILTER/BLOTTER
      remote-label-regex      ^transformer[0-9]+
      apply-if-label          ^liberator[0-9]+
  end-provided-data-service
  ```

  **Provided data service: transformer -> myadapter**

  ```text
  add-provided-data-service
      service-name            blotters
      include-pattern         ^/BLOTTER
      remote-label-regex      ^myadapter[0-9]+
      apply-if-label          ^transformer[0-9]+
  end-provided-data-service
  ```
* **discard-timeout**

  **Type:** float | **Default (Liberator):** value of `active-discard-timeout` | **Default (all others):** `-1.0`
  -

  Specifies the time in seconds for which this DataSource application holds on to the data for a subscribed subject once the last client application has unsubscribed from that data. After this time, the object is deleted from the DataSource application’s cache, and the DataSource application sends a discard instruction to the data service’s peer(s) to cancel the subscription.

  For Liberator only:

  * This option overrides the setting of the global time out for active objects, [`active-discard-timeout`](../liberator/liberator-object-configuration-part-2.md#active-discard-timeout).
  * If an object obtained by this data service was specified through the Liberator’s [add-object](../liberator/liberator-object-configuration-part-1.md#add-object) configuration item, the [discard-timeout](../liberator/liberator-object-configuration-part-1.md#discard-timeout) option of the `add-object` (if any) overrides 
  this `discard-timeout`.
* **exclude-pattern**

  **Type:** array of strings | **Default:** _null_

  One or more regular expressions that determine subjects for which this data service must not provide data.

  This option is optional. Use it in conjunction with the [include-pattern](#include-pattern-2) option.

  For example, `include-pattern ^/FX/MAJOR/` specifies that this data service is to provide data for subjects that begin with `/FX/MAJOR/`.

  Then specifying `exclude-pattern ^/FX/MAJOR/AUDUSD ^/FX/MAJOR/AUDGBP` excludes the currency pairs `AUDUSD` and `AUDGBP` from the prices returned by the data service.

  You can have multiple occurrences of `exclude-pattern`, so the above example could be written as:

  ```
  exclude-pattern ^/FX/MAJOR/AUDUSD
  exclude-pattern ^/FX/MAJOR/AUDGBP
  ```

  **💡 TIP**\
  Because the configuration is applied on a first rule wins basis, you should always put the excludes before the includes as shown below.

  ```
  exclude-pattern ^/FX/MAJOR/AUDUSD
  exclude-pattern ^/FX/MAJOR/AUDGBP
  include-pattern ^/FX/MAJOR/
  ```
* **include-pattern**

  **Type:** array of strings | **Default:** `^/`

  One or more regular expressions that determine the subjects for which this data service provides data.

  For example: `include-pattern ^/FX/MAJOR/ ^/FX/MINOR/` specifies that this data service is to provide data for subjects that begin with `/FX/MAJOR/` or `/FX/MINOR/`.

  You can have multiple occurrences of `include-pattern`, so the above example could be written as:

  ```
  include-pattern ^/FX/MAJOR/
  include-pattern ^/FX/MINOR/
  ```

  You should normally specify an `include-pattern` option, otherwise the data service will return updates for all subjects (that is, subjects starting with `/`) that the service’s DataSource peers can provide.

  Also see the [exclude-pattern](#exclude-pattern-2) option.
* **object-map**

  **Type:** string array | **Default:** | **Applies to:** Liberator 8

  Maps an object name to a different object name _before_ data-service routing occurs.

  This option has the same syntax and behaviour as Liberator’s [`object-map`](../liberator/liberator-object-configuration-part-2.md#object-map) configuration item.
* **open-subscriptions**

  **Type:** boolean | **Default:** `FALSE`

  When set to true, enables [open subscriptions](../liberator/liberator-open-subscriptions.md) for this data service by setting the provided data service’s [request-timeout](#request-timeout) option to `-1`.

  When set to false (the default), the data service’s request timeout respects the value of the global [service-request-timeout](#service-request-timeout) configuration item.
* **remote-label-regex**

  **Type:** string | **Default:** _null_

  A single [regular expression](https://en.wikipedia.org/wiki/Regular_expression) (POSIX ERE) that describes the labels ([datasource-local-label](datasource-datasource-peers-configuration-part-2.md#datasrc-local-label)) of DataSource peers that supply data for this DataService. 
* **required-state**

  **Type:** string | **Default:** `down` | **Applies to:** Liberator

  The minimum state this data service must be in for Liberator to begin accepting client connections.

  Liberator can be configured to continue to enforce `required-state` after Liberator has begun accepting connections. If [`session-requires-services`](../liberator/liberator-session-configuration.md#session-requires-services) is set to `true`, then `required-state` is the minimum state this data service must be in for Liberator to accept _and_ retain client connections.

  **Valid values for required-state**

  * `down`, `optional`: Down (no source groups are up, or one or more required source groups are down). **Default**
  * `limited`: Limited (at least one source group is up, and all required source groups are up)
  * `ok`, `up`: Up (all source groups are up)
  * `once`: Has been in state `ok` at least once since Liberator was started.
* **service-name**

  **Type:** string | **Default:**

  The name of the data service. The service name is considered unique and should two adapters provide a configuration for the same name, then then only the latest configuration is used.
* **service-type**

  **Type:** string | **Default:** `active|broadcast`

  This option is used to specify the type of the data service and is analogous to [add-data-service::service-type](#service-type)

  Possible values are:

  * `active`: This data service is provided by active DataSource peers. An active DataSource keeps track of which [subjects](datasource-subjects-symbols-and-fields.md) have been [requested](datasource-subscriptions.md) and sends updates for those subjects only (see [Active subscription model](datasource-active-subscription-model.md)).
  * `broadcast`: This data service is provided by [broadcast](datasource-broadcasting-data.md) DataSource peers.
  * `active|broadcast`: This data service is provided by both active and broadcast peers.
  * `rest`: This data service provides a REST service, rebalancing is disabled for this service.
  * `contrib`: This data service provides contributions or trade channels. Rebalancing is disabled for this service.
* **throttle-time**

  **Type:** float |  **Default:** -1 | **Applies to:** Liberator 8

  Specifies a single [throttle](../liberator/liberator-throttling.md) level (that is the conflation period) for records and generic objects served to RTTP clients from this provided data service.

  When set to a value above `-1`, this option overrides the following Liberator throttling configuration for subjects served by the provided data service: [object-throttle-times](../liberator/liberator-throttling-configuration.md#object-throttle-times) and [add-object::throttle-times](../liberator/liberator-object-configuration-part-1.md#throttle-times).

  Only a single throttle level is supported.

## broadcast-cleanup-age

`broadcast-cleanup-age` specifies how old in minutes broadcast objects provided by this DataSource application need to be before they’re removed from the DataSource application’s cache. Every [broadcast-cleanup-period](#broadcast-cleanup-period) minutes past [broadcast-cleanup-time](#broadcast-cleanup-time), the application checks the cache and deletes any broadcast objects that are older than `broadcast-cleanup-age`.

This configuration item is ignored if [broadcast-cleanup-time](#broadcast-cleanup-time) isn’t set.

**Use in:** C

**Syntax:** `broadcast-cleanup-age <age-in-minutes>`

**Type:** integer

**Default value:** `-1` (no timeout set, so broadcast items are never removed from the cache)

## broadcast-cleanup-period

`broadcast-cleanup-period` specifies how often in minutes, relative to [broadcast-cleanup-time](#broadcast-cleanup-time), broadcast objects provided by this DataSource application are removed from the DataSource application’s cache. Every `broadcast-cleanup-period` minutes past `broadcast-cleanup-time`, the application checks the cache and deletes any broadcast objects that are older than [broadcast-cleanup-age](#broadcast-cleanup-age).

This configuration item is ignored if [broadcast-cleanup-time](#broadcast-cleanup-time) isn’t set.

**Use in:** C

**Syntax:** `broadcast-cleanup-period <time-interval-in-minutes>`

**Type:** integer

**Default value:** `1440` minutes (= 24 hours; that is, clean up every day at the time set by [broadcast-cleanup-time](#broadcast-cleanup-time).)

## broadcast-cleanup-time

`broadcast-cleanup-time` specifies when broadcast objects provided by this DataSource application are first removed from the DataSource application’s cache. This time is the number of minutes after midnight (local time) on the day the DataSource was started, or if it’s greater than or equal to 1440 (24 hours), it’s the number of minutes past midnight on a Sunday. At `broadcast-cleanup-time`, and subsequently every [broadcast-cleanup-period](#broadcast-cleanup-period) minutes past `broadcast-cleanup-time`, the application checks the cache and deletes any broadcast objects that are older than [broadcast-cleanup-age](#broadcast-cleanup-age).

By default, the DataSource application’s cache is not cleaned up on a timed basis.

**Use in:** C

**Syntax:** `broadcast-cleanup-time <time-past-midnight-in-minutes>`

**Default value:** `-1` (Don’t clean up the DataSource application’s cache on a timed basis, so [broadcast-cleanup-age](#broadcast-cleanup-age) and [broadcast-cleanup-period](#broadcast-cleanup-period) are ignored.)

**💡 TIP**\
We recommend that you configure a daily cleanup of broadcast objects. For example, set `broadcast-cleanup-time` to `0` and [broadcast-cleanup-period](#broadcast-cleanup-period) to `1440` mins (= 24 hrs), so that broadcast objects are cleaned up at midnight every day. Use [broadcast-cleanup-age](#broadcast-cleanup-age) to control how old the broadcast objects should be before they’re actually removed from the cache.

## cleanup-stale-timeout

`cleanup-stale-timeout` specifies when to clean up stale objects (such as subscriptions). When all the DataSource peers in a data service have been down for `cleanup-stale-timeout` seconds, all the objects that can no longer be updated by the service are deleted from this DataSource application’s cache.

**Use in:** C

**Syntax:** `cleanup-stale-timeout <timeout-in-seconds>`

**Type:** float

**Default value:** `-1.0` (no timeout)

## service-global-peer-balance

`service-global-peer-balance` determines how the workload-based load-balancing algorithm calculates the workload of peers in an [`add-priority`](#add-data-service--add-source-group--add-priority) set.

* When `service-global-peer-balance` is set to `false` (the default), peer workload is defined as the number of subscriptions the peer is serving to the data service.
* When `service-global-peer-balance` is set to `true`, peer workload is defined as the total number of subscriptions the peer is serving to _all_ data services on the DataSource.

For deployments in which peers serve more than one data service, set `service-global-peer-balance` to `true` to provide the load-balancing algorithm with a more accurate determination of peer workload.

This configuration item has no affect on the source-affinity load-balancing algorithm (`add-source-group:[affinity](#affinity-2)`).

**Example: load balancing with and without service-global-peer-balance**

Consider two data services that have a peer in common (PeerA):

**service1 (/SERVICE1)**

```
add-data-service
  service-name service1
  include-pattern /SERVICE1
  add-source-group
    add-priority
      remote-label PeerA
      remove-label PeerB
    end-priority
  end-source-group
end-data-service
```

**service2 (/SERVICE2)**

```
add-data-service
  service-name service2
  include-pattern /SERVICE2
  add-source-group
    add-priority
      remote-label PeerA
      remove-label PeerC
    end-priority
  end-source-group
end-data-service
```

Assume the following requests have been made to service1 (in this order):

1. /SERVICE1/A
2. /SERVICE1/B
3. /SERVICE1/C
4. /SERVICE1/D
5. /SERVICE1/E
6. /SERVICE1/F
7. /SERVICE1/G
8. /SERVICE1/H

…resulting in the following distribution of subscriptions:

| Data Service | Peer | Subscriptions | Total Subscriptions .2+.^ |
| --- | --- | --- | --- |
| service1 | PeerA | 4 | 4 |
| PeerB | 4 | 4 .2+.^ | service2 |
| PeerA | 0 | 4 | PeerC |

Now we request /SERVICE2/A. Because each peer in service2 has 0 service2 requests and PeerA is the 1st peer, PeerA gets the request even though it already has 4 service1 requests and PeerC has no requests at all.

If `service-global-peer-balance` were set to true, then peer C would get the new requests and the next 3 service2 requests until it had a total of 4 before service2 requests went to peer A.

**Use in:** C

**Since:** DataSource for C (DSDK) 7.1.28, Liberator 7.1.28

**Syntax:** `service-global-peer-balance`

**Type:** boolean

**Default value:** `false`

## service-rebalance-enable

`service-rebalance-enable` enables automatic rebalancing of workload in all data-service source groups that don’t have source affinity enabled.

To enable or disable rebalancing for a specific data service, use [`rebalance-enable`](#rebalance-enable), which overrides `service-rebalance-enable`.

When rebalancing is enabled and a peer in a priority group changes its status to UP, after a set time ([`service-rebalance-time`](#service-rebalance-time)) the data service discards subscriptions to peers in the priority group and re-requests them, balancing the requests evenly over peers in the priority group.

Service rebalancing is disabled by default and must be enabled globally (using `service-rebalance-enable`) or for specific data services (using [`rebalance-enable`](#rebalance-enable)) where required. The one exception to this is when no data services have been configured: in this one case, `service-rebalance-enable` defaults to `true`.

From DataSource 7.1.31 and DataSource 8.0.0, service rebalancing also applies to the [Default Data Service](#default-data-service).

**Use in:** C

**Since:** DataSource for C (DSDK) 7.1.21

**Syntax:** `service-rebalance-enable`

**Type:** boolean

**Default value:**

* No configured data services: `true`
* Configured data services: `false`

## service-rebalance-time

`service-rebalance-time` sets the time to wait (in seconds) before rebalancing workload after a peer in priority group changes its status to UP.

Service rebalancing is disabled by default. To enable service rebalancing, see [service-rebalance-enable](#service-rebalance-enable) and the `add-data-service` option [`rebalance-enable`](#rebalance-enable).

`service-rebalance-time` is overridden by the `add-data-service` option [`rebalance-time`](#rebalance-time)

From DataSource 7.1.31 and DataSource 8.0.0, service rebalancing also applies to the [Default Data Service](#default-data-service).

**Use in:** C

**Since:** DataSource for C (DSDK) 7.1.21

**Syntax:** `service-rebalance-time <seconds>`

**Type:** float

**Default value:** `5.0`

## service-request-timeout

`service-request-timeout` specifies the global request timeout in seconds for all data services.

This is the time that this DataSource application waits for a data service to respond to a request (for example, a subscription request). If the DataSource peers providing the service don’t respond within this time, the requested object is assumed to not be available and the application sends a discard message for the object to the peers.

A value of `0` or `-1` means no timeout is set and this DataSource application then waits indefinitely for a data service to respond to a request. Use this setting in Liberator if you want to allow all of the Liberator’s subscription requests to be [open subscriptions](../liberator/liberator-open-subscriptions.md) (see How can I... [Keep subscriptions open when the peer is down](../liberator/liberator-keep-subscriptions-open-when-the-peer-is-down.md)).

`service-request-timeout` is a global setting applied to all data services. You can override this timeout for a particular data service by setting the [request-timeout](#request-timeout) option of [add-data-service](#add-data-service).

**Use in:** C

**Syntax:** `service-request-timeout <timeout-in-seconds>`

**Type:** float

**Default value:** `10.0` seconds

---

**See also:**

* Reference: [DataSource peers configuration](datasource-datasource-peers-configuration-part-1.md)
* How can I... [Use timers in DataSource applications](datasource-use-timers-in-datasource-applications.md)
* How can I... [Enable heartbeats between DataSource applications](datasource-enable-heartbeats-between-datasource-applications.md)