Data services configuration

These DataSource configuration items define a DataSource application’s data services.

When the DataSource application requests an object (such as subscribing to a subject), the data services system determines, through the configuration, which peers the request goes to instead of you having to specify this explicitly in the application’s code. It also prevents a request from going to any peer more than once.

Of course, if your DataSource application is an Integration Adapter that only sends data to its peers, you don’t need to define any data services for it.

Data services can’t be configured in Java-based DataSource applications.
"this DataSource application" means the DataSource application for which you are defining the configuration. "DataSource peer" or "peer" means a DataSource application that this DataSource application communicates with.

For some examples of how data services are configured, see the data services features and concepts page.

add-data-service

add-data-service specifies a data service that this DataSource application can use.Through the add-source-group option and the add-priority option within add-source-group, it allows you to specify multiple peers that provide data for the service, load balance the provision of data across multiple peers, and configure alternate peers that your DataSource application can failover to. For examples of these setups, see the data services features and concepts page.

Default data service

If you don’t define at least one data service for your DataSource application, then the DataSource uses a default data service that requests each subject request from all static and dynamic peers of the DataSource.

The default data service contains a single-priority source group for each peer, with no minimum requirements for peer connectivity.

Default data service
add-data-service

  service-name         default
  required-state       down
  include-pattern      ^/ (1)

  add-source-group (2)
    required           FALSE
    add-priority
        remote-label   peer_label
    end-priority
  end-source-group

end-data-service
1 Matches all subject requests
2 Repeated for each static and dynamic peer

Use in: C

Syntax
add-data-service

  affinity                        [array of strings]
  disable-auto-status             [boolean]
  discard-timeout                 [float]
  exclude-pattern                 [array of strings]
  include-pattern                 [array of strings]
  min-source-groups               [integer]
  nodata-ignore-request-timeout   [boolean]
  object-map                      [pattern from] [pattern to]
  rebalance-enable                [boolean]
  rebalance-time                  [float]
  request-timeout                 [float]
  required-state                  [string]
  service-name                    [string]
  service-type                    [string]
  throttle-time                   [float]

  add-source-group
      …
  end-source-group

end-data-service
Options
add-source-group

Type: group item

The add-source-group group item specifies, via the add-priority option, one or more DataSource peers that provide the data for the data service.

  • By using multiple add-source-group items, you can specify multiple sources of data for the data service.

  • By using multiple add-priority options within an add-source-group item, you can specify alternative sets of peers to failover to.

For examples of these setups, see the data services features and concepts page

affinity

Type: array of strings | Default: null

Deprecated from version 6.2.6+ of Liberator and version 6.2.5+ of Transformer; use the affinity option within the add-source-group group item instead.

disable-auto-status

Type: boolean | Default: FALSE

If you don’t want this DataSource application to send subject status changes for this data service to subscribing peers, set disable-auto-status to TRUE You’d typically do this in a Transformer, to prevent it from sending subject status changes unnecessarily on to subscribing Liberators.

discard-timeout

Type: float | Default (Liberator): value of active-discard-timeout | Default (all others): -1.0

Specifies the time in seconds for which this DataSource application holds on to the data for a subscribed subject once the last client application has unsubscribed from that data. After this time, the object is deleted from the DataSource application’s cache, and the DataSource application sends a discard instruction to the data service’s peer(s) to cancel the subscription.

For Liberator only:

  • This option overrides the setting of the global time out for active objects, active-discard-timeout.

  • If an object obtained by this data service was specified through the Liberator’s add-object configuration item, the discard-timeout option of the add-object (if any) overrides this discard-timeout.

exclude-pattern

Type: array of strings | Default: null

One or more regular expressions that determine subjects for which this data service must not provide data.

This option is optional. Use it in conjunction with include-pattern.

The options include-pattern and exclude_pattern are applied in the order they are defined, and the first matching rule wins. So, define all your exclude_pattern options before their associated include-pattern options.

Example 1. Using exclude-pattern to refine include-pattern

Match all subjects beginning /FX/MAJOR, but exclude /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP:

exclude-pattern ^/FX/MAJOR/AUDUSD ^/FX/MAJOR/AUDGBP (1)
include-pattern ^/FX/MAJOR/ (2)
1 This data service does not provide data for subjects /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP
2 This data service provides data for subjects beginning /FX/MAJOR/

Note that if the exclude-pattern was specified after the include-pattern rule, then the rules would not exclude /FX/MAJOR/AUDUSD and /FX/MAJOR/AUDGBP. For either of those two subjects, the first matching rule (the winning rule) would be the include-pattern.

You can have multiple occurrences of exclude-pattern in the same add-data-service item, so the above example could be written as:

exclude-pattern ^/FX/MAJOR/AUDUSD
exclude-pattern ^/FX/MAJOR/AUDGBP
include-pattern ^/FX/MAJOR/
include-pattern

Type: array of strings | Default: ^/

One or more regular expressions that determine the subjects for which this data service provides data.

For example: include-pattern ^/FX/MAJOR/ ^/FX/MINOR/ specifies that this data service is to provide data for subjects that begin with /FX/MAJOR/ or /FX/MINOR/

You can have multiple occurrences of include-pattern in the same add-data-service item, so the above example could be written as:

include-pattern ^/FX/MAJOR/
include-pattern ^/FX/MINOR/

You should normally specify an include-pattern option, otherwise the data service will return updates for all subjects (that is, subjects starting with /) that the service’s DataSource peers can provide.

Also see the exclude-pattern option.

min-source-groups

Type: integer | Default: 1 | Since: C DataSource API 7.1.6 (Liberator 7.1.5, Transformer 7.1.5)

The minimum number of source groups within this data service that must have a status of 'up' for this data service to have a status of 'up'.

nodata-ignore-request-timeout

Type: boolean | Default: FALSE

A DataSource peer that provides a data service can send back a "no data" response if there is no data available for a requested subject. The data service then discards any outstanding requests for the subject.

The nodata-ignore-request-timeout option applies when multiple peers can supply requested data for a service.

When this option is TRUE, if any peer providing the service sends back a "no data" response, this response is immediately propagated from the DataSource API to the custom code of the DataSource application - the request-timeouts (or source-request-timeouts) for the remaining peers are ignored.

When this option is FALSE, if a "no data" response is received from a peer providing the service, the data service continues wait for the data from any of the other peers, applying the request-timeouts (or source-request-timeouts), until either the data is returned by at least one peer, or all the peers have timed out or returned a "no data" response.

object-map

Type: string array | Default: | Since: Liberator 8

Maps an object name to a different object name before data-service routing occurs.

This option has the same syntax and behaviour as Liberator’s object-map configuration item.

request-timeout

Type: float | Default: value of service-request-timeout

Specifies the timeout in seconds for all subscription requests sent to this data service. If the DataSource peers providing the service don’t respond within this time, the requested subject is assumed to not be available and the application sends a discard message for the subject to the peers.

A value of 0 means no timeout is set and this DataSource application then waits indefinitely for this data service to respond to a request. Use this setting in Liberator if you want to allow all Liberator’s subscription requests for this data service to be open subscriptions (see How can I…​ Keep subscriptions open when the peer is down).

request-timeout overrides the global service-request-timeout configuration item.

rebalance-enable

Type: boolean | Default: null | Since: DataSource for C (DSDK) 7.1.21

Enable or disable automatic rebalancing of workload in source groups that don’t have source affinity enabled.

When rebalancing is enabled and a peer in a priority group changes its status to UP, after a set time (rebalance-time) the data service discards subscriptions to peers in the priority group and re-requests them, balancing the requests evenly over peers in the priority group.

Overrides the global item service-rebalance-enable.

rebalance-time

Type: float | Default: null | Since: DataSource for C (DSDK) 7.1.21

Set the time to wait (in seconds) after a peer in a priority group (see add-priority) sets its status to UP before rebalancing workload across all peers in the priority group.

Overrides global item service-rebalance-time.

To enable service rebalancing, see service-rebalance-enable and rebalance-enable.

required-state

Type: string | Default: down

This option only applies to Liberator.

The minimum state this data service must be in for Liberator to begin accepting client connections.

Before version 7.1.19, Liberator ignores required-state after it has begun accepting connections.

From version 7.1.19, Liberator can be configured to continue to enforce required-state after Liberator has begun accepting connections. If session-requires-services is set to true, then required-state is the minimum state this data service must be in for Liberator to accept and retain client connections.

Valid values for required-state

  • down, optional: Down (no source groups are up, or one or more required source groups are down). Default

  • limited: Limited (at least one source group is up, and all required source groups are up)

  • ok, up: Up (all source groups are up)

  • once: Has been in state ok at least once since Liberator was started.

See also:

service-name

Type: string | Default: null

The name of the data service.

In deployments of the Caplin Platform with Discovery, providing DataSources that are configured with a matching service name in discovery-provide-service automatically attempt peer connections to this DataSource. For more information, see Peer discovery.

service-type

Type: string | Default: active|broadcast

Specifies whether the DataSource peers that supply data for the service are of type active, broadcast, or both.

Possible values are:

  • active: This data service is provided by active DataSource peers. An active DataSource keeps track of which subjects have been requested and sends updates for those subjects only (see Active subscription model).

  • broadcast: This data service is provided by broadcast DataSource peers.

  • active|broadcast: This data service is provided by both active and broadcast peers.

  • rest: This data service provides a REST service, rebalancing is disabled for this service.

  • contrib: Identifies this data service as accepting contributions (serving channels). Disables automatic rebalancing for this service. Automatically sets the active flag.

You can use this option to override the local-type option of a peer’s add-peer configuration. For example, assume a peer that supplies data for a data service is configured as an active peer and also is configured to broadcast data (that is, its add-peer's local-type option is set to active|broadcast). If you don’t want your data service to receive the peer’s broadcast data, set the add-data-service's service-type option to active.

throttle-time

Type: float | Default: -1| Since: Liberator 8

Specifies a single throttle level (that is the conflation period) for records and generic objects served to RTTP clients from this data service.

When set to a value above -1, this option overrides the following Liberator throttling configuration for subjects served by the data service: object-throttle-times and add-object::throttle-times.

Only a single throttle level is supported.

add-data-service :: add-source-group

add-source-group specifies, via the add-priority option one or more DataSource peers that provide the data for the data service defined by add-data-service. By using multiple add-source-group items, you can specify multiple sources of data for the data service. By using multiple add-priority options within an add-source-group item, you can specify alternative sets of peers to failover to. If you specify more than one peer within an add-priority item, subscriptions are load-balanced across them. For examples of these setups, see the data services features and concepts page.

Use in: C

Syntax
add-data-service
   …
   add-source-group
      add-priority
        …
      end-priority
      …
      affinity         [array of strings]
      contrib          [boolean]
      min-priorities   [integer]
      required         [boolean]
      retry-time       [float]
   end-source-group
   …
end-data-service
Options
add-priority

Type: group item

Specifies one or more DataSource peers that this DataSource application can use to provide data for a data service. If you specify more than one peer in an add-priority option, subscriptions are load-balanced across those peers.

You can also use multiple instances of add-priority to implement failover to alternative peers.

Because the remote-label option of add-priority points to an add-peer configuration item, this DataSource application will first try to fail over to any alternative hosts that you’ve defined in the addr option of the add-peer item.

For examples of load balancing and failover configurations, see the data services features and concepts page.

affinity

Type: array of strings | Default: null

Enables the alternative source affinity load-balancing algorithm for load-balanced adapters in this source group.

Syntax: affinity <affinity_key> <regular_expression>

For a full description of the syntax, guidelines on use, and a worked example, see Load balancing adapters using source affinity.

contrib

Type: boolean | Default: FALSE

When set to TRUE, this option declares that peers listed in this source group provide contributions only (they don’t accept subscription requests).

A source group containing contribution-only peers is a required addition to a data service that acts as a conduit for a messaging channel from one DataSource to another.

For example, consider a data service on Transformer that routes a trade channel from Liberator to a trade adapter. Liberator is included in its own source group in the data service so that Transformer accepts Liberator as a valid source of data for the channel, and the contrib option of the source group is enabled to stop Transformer from routing subscription requests for the trade channel to Liberator.

A contribution-only source group in a Transformer data service
add-data-service
  service name trade-channel
  include-pattern '^/PRIVATE/FX'

  add-source-group
    add-priority
      remote-label trade-adapter
    end-priority
  end source group

  add-source-group
    contrib
    add-priority
      remote-label liberator
    end-priority
  end-source-group
end-data-service
min-priorities

Type: integer | Default: 1 | Since: C DataSource API 7.1.6, Liberator 7.1.5, and Transformer 7.1.5

Minimum number of priority groups in this source group that must have a status of 'up' for this source group to have a status of 'up'.

required

Type: boolean | Default: FALSE

When set to TRUE:

  • If data had been received from any peer in this data service and all of the peers in any required source group go down, then the data is marked as stale.

  • If the data service state is up or limited and then some peers go down such that the minimum number of required priorities in this group are not up, then the data service state is changed to down.

  • If only one peer in this source group responds and that peer responds with NODATA, then the DataSource API propagates the NODATA message to application code.

When set to FALSE:

  • If data had been received from any peer in this data service and all of the peers in a non-required source group go down, then the data is marked as limited ( unless the data was already marked as stale ).

  • If the data service state is up and then some peers go down such that the minimum number of required priorities in this group are not up, then the data service state is changed to limited. If the data service is in the down state then the state doesn’t change.

retry-time

Type: float | Default: 30.0

After finding that none of the peers defined by add-priority options in the source group have responded to a request, this DataSource application waits retry-time seconds before reissuing the request to the source group.

Valid values

  • DSDK < 7.1.29: Minimum value 1.0. Any value < 1.0 is rounded up to 1.0.

  • DSDK >= 7.1.29: Minimum positive value 1.0. Any positive value < 1.0 is rounded up to 1.0. Any value <= 0 disables the retry feature.

The DataSource application issues the request to each of the peers in the source group in turn. Each request is timed out according to the setting of add-peer::request-timeout or source-request-timeout. If none of the peers in the source group replies, the DataSource application waits retry-time and then tries each connection again in turn. It repeats this sequence until the master timeout for the service, defined by service-request-timeout or add-data-service::request-timeout, expires. If add-data-service::request-timeout is set to -1, the DataSource reissues the request indefinitely.

add-data-service :: add-source-group :: add-priority

add-priority specifies one or more DataSource peers that this DataSource application can use to provide data for a data service. By using multiple add-priority options within an add-source-group item, you can specify alternative sets of peers to failover to. If you specify more than one peer within an add-priority item, subscriptions are load-balanced across them; for an example of this, see the data services features and concepts page.

Use in: C

Syntax
add-data-service
  ...
  add-source-group
    ...
    add-priority
      label                   [array of strings]
      min-peers               [integer]
      remote-label            [array of strings]
      remote-label-regex      [string]
    end-priority
    ...
  end-source-group
  ...
end-data-service
Options
label

Type: array of strings | Deprecated: use remote-label instead

One or more labels identifying DataSource peers that supply data for this DataService. Each label must match the label option of an add-peer configuration item that defines a DataSource peer. If you specify more than one peer, either through multiple values of a label option or by multiple label options, subscriptions to the referenced peers are load-balanced across the peers.

Also see Note 1.

min-peers

Type: integer | Default: 1 | Since: C DataSource API 7.1.6, Liberator 7.1.5, and Transformer 7.1.5

The minimum number of peers in this priority group that must be connected for this priority group to have a status of 'up'.

remote-label

Type: array of strings

One or more labels identifying DataSource peers that supply data for this DataService. Each label must match the remote-label option of an add-peer configuration item that defines a DataSource peer.

If you specify more than one peer, either through multiple values of a remote-label option or by multiple remote-label options, subscriptions to the referenced peers are load-balanced across the peers.

Also see Note 1 below, and the example in DataSource peers configuration (part 1).

remote-label-regex

Type: string | Default: null | Since: DataSource for C (DSDK) 7.1.21, Liberator 7.1.21, Transformer 7.1.13

A single regular expression (POSIX ERE) that describes the labels (datasource-local-label) of DataSource peers that supply data for this DataService.

This option is available only for DataSources connected to Discovery (see discovery-addr).

Only one remote-label-regex option may be specified per priority group. If multiple remote-label-regex options are specified for a priority group, only the final option applies.

Subject requests to the peers defined by remote-label-regex are load balanced using the standard workload-based algorithm.

Note 1: Platform blades, and Liberator and Transformer, that run under the Caplin Platform Deployment Framework use the deprecated label option in their configuration, rather than remote-label. This will change in a future release of the Framework.

add-provided-data-service

Use in: C, Java

The add-provided-data-service configuration option allows an adapter to provide data service configuration to a remote Datasource application.

For adapters written using DataSource for Java it is also possible to programatically set this configuration.

Syntax
add-provided-data-service
    apply-if-label          [regex of labels that this service should be applied to]
    discard-timeout         [float]
    exclude-pattern         [regex to exclude]
    include-pattern         [regex to include]
    object-map              [pattern from] [pattern to]
    open-subscriptions      [true|false]
    remote-label-regex      [regex matching peers]
    required-state          [down|optional|limited|ok|up|once]
    service-name            [string]
    service-type            [service-type-enum]
    throttle-time           [float value]
end-provided-data-service
Options
apply-if-label

Type: string | Default:

A regular expression that restricts the provision of this data-service to DataSources with a matching label.

The most common application for apply-if-label is an adapter that receives requests from Liberator routed via Transformer. In this scenario, the adapter needs to provision both Liberator and Transformer with data service configuration.

Example 2. Provided data services that route requests via Transformer
Provided data service: liberator → transformer
add-provided-data-service
    service-name            blotters
    include-pattern         ^/FILTER/BLOTTER
    remote-label-regex      ^transformer[0-9]+
    apply-if-label          ^liberator[0-9]+
end-provided-data-service
Provided data service: transformer → myadapter
add-provided-data-service
    service-name            blotters
    include-pattern         ^/BLOTTER
    remote-label-regex      ^myadapter[0-9]+
    apply-if-label          ^transformer[0-9]+
end-provided-data-service
discard-timeout

Type: float | Default (Liberator): value of active-discard-timeout | Default (all others): -1.0 -

Specifies the time in seconds for which this DataSource application holds on to the data for a subscribed subject once the last client application has unsubscribed from that data. After this time, the object is deleted from the DataSource application’s cache, and the DataSource application sends a discard instruction to the data service’s peer(s) to cancel the subscription.

For Liberator only:

  • This option overrides the setting of the global time out for active objects, active-discard-timeout.

  • If an object obtained by this data service was specified through the Liberator’s add-object configuration item, the discard-timeout option of the add-object (if any) overrides this discard-timeout.

exclude-pattern

Type: array of strings | Default: null

One or more regular expressions that determine subjects for which this data service must not provide data.

This option is optional. Use it in conjunction with the include-pattern option.

For example, include-pattern ^/FX/MAJOR/ specifies that this data service is to provide data for subjects that begin with /FX/MAJOR/.

Then specifying exclude-pattern ^/FX/MAJOR/AUDUSD ^/FX/MAJOR/AUDGBP excludes the currency pairs AUDUSD and AUDGBP from the prices returned by the data service.

You can have multiple occurrences of exclude-pattern, so the above example could be written as:

exclude-pattern ^/FX/MAJOR/AUDUSD
exclude-pattern ^/FX/MAJOR/AUDGBP
Because the configuration is applied on a first rule wins basis, you should always put the excludes before the includes as shown below. 
exclude-pattern ^/FX/MAJOR/AUDUSD
exclude-pattern ^/FX/MAJOR/AUDGBP
include-pattern ^/FX/MAJOR/
include-pattern

Type: array of strings | Default: ^/

One or more regular expressions that determine the subjects for which this data service provides data.

For example: include-pattern ^/FX/MAJOR/ ^/FX/MINOR/ specifies that this data service is to provide data for subjects that begin with /FX/MAJOR/ or /FX/MINOR/.

You can have multiple occurrences of include-pattern, so the above example could be written as:

include-pattern ^/FX/MAJOR/
include-pattern ^/FX/MINOR/

You should normally specify an include-pattern option, otherwise the data service will return updates for all subjects (that is, subjects starting with /) that the service’s DataSource peers can provide.

Also see the exclude-pattern option.

object-map

Type: string array | Default: | Applies to: Liberator 8

Maps an object name to a different object name before data-service routing occurs.

This option has the same syntax and behaviour as Liberator’s object-map configuration item.

open-subscriptions

Type: boolean | Default: FALSE

When set to true, enables open subscriptions for this data service by setting the provided data service’s request-timeout option to -1.

When set to false (the default), the data service’s request timeout respects the value of the global service-request-timeout configuration item.

remote-label-regex

Type: string | Default: null

A single regular expression (POSIX ERE) that describes the labels (datasource-local-label) of DataSource peers that supply data for this DataService.

required-state

Type: string | Default: down | Applies to: Liberator

The minimum state this data service must be in for Liberator to begin accepting client connections.

Liberator can be configured to continue to enforce required-state after Liberator has begun accepting connections. If session-requires-services is set to true, then required-state is the minimum state this data service must be in for Liberator to accept and retain client connections.

Valid values for required-state

  • down, optional: Down (no source groups are up, or one or more required source groups are down). Default

  • limited: Limited (at least one source group is up, and all required source groups are up)

  • ok, up: Up (all source groups are up)

  • once: Has been in state ok at least once since Liberator was started.

service-name

Type: string | Default:

The name of the data service. The service name is considered unique and should two adapters provide a configuration for the same name, then then only the latest configuration is used.

service-type

Type: string | Default: active|broadcast

This option is used to specify the type of the data service and is analogous to add-data-service::service-type

Possible values are:

  • active: This data service is provided by active DataSource peers. An active DataSource keeps track of which subjects have been requested and sends updates for those subjects only (see Active subscription model).

  • broadcast: This data service is provided by broadcast DataSource peers.

  • active|broadcast: This data service is provided by both active and broadcast peers.

  • rest: This data service provides a REST service, rebalancing is disabled for this service.

  • contrib: This data service provides contributions or trade channels. Rebalancing is disabled for this service.

throttle-time

Type: float | Default: -1 | Applies to: Liberator 8

Specifies a single throttle level (that is the conflation period) for records and generic objects served to RTTP clients from this provided data service.

When set to a value above -1, this option overrides the following Liberator throttling configuration for subjects served by the provided data service: object-throttle-times and add-object::throttle-times.

Only a single throttle level is supported.

broadcast-cleanup-age

broadcast-cleanup-age specifies how old in minutes broadcast objects provided by this DataSource application need to be before they’re removed from the DataSource application’s cache. Every broadcast-cleanup-period minutes past broadcast-cleanup-time, the application checks the cache and deletes any broadcast objects that are older than broadcast-cleanup-age.

This configuration item is ignored if broadcast-cleanup-time isn’t set.

Use in: C

Syntax: broadcast-cleanup-age <age-in-minutes>

Type: integer

Default value: -1 (no timeout set, so broadcast items are never removed from the cache)

broadcast-cleanup-period

broadcast-cleanup-period specifies how often in minutes, relative to broadcast-cleanup-time, broadcast objects provided by this DataSource application are removed from the DataSource application’s cache. Every broadcast-cleanup-period minutes past broadcast-cleanup-time, the application checks the cache and deletes any broadcast objects that are older than broadcast-cleanup-age.

This configuration item is ignored if broadcast-cleanup-time isn’t set.

Use in: C

Syntax: broadcast-cleanup-period <time-interval-in-minutes>

Type: integer

Default value: 1440 minutes (= 24 hours; that is, clean up every day at the time set by broadcast-cleanup-time.)

broadcast-cleanup-time

broadcast-cleanup-time specifies when broadcast objects provided by this DataSource application are first removed from the DataSource application’s cache. This time is the number of minutes after midnight (local time) on the day the DataSource was started, or if it’s greater than or equal to 1440 (24 hours), it’s the number of minutes past midnight on a Sunday. At broadcast-cleanup-time, and subsequently every broadcast-cleanup-period minutes past broadcast-cleanup-time, the application checks the cache and deletes any broadcast objects that are older than broadcast-cleanup-age.

By default, the DataSource application’s cache is not cleaned up on a timed basis.

Use in: C

Syntax: broadcast-cleanup-time <time-past-midnight-in-minutes>

Default value: -1 (Don’t clean up the DataSource application’s cache on a timed basis, so broadcast-cleanup-age and broadcast-cleanup-period are ignored.)

We recommend that you configure a daily cleanup of broadcast objects. For example, set broadcast-cleanup-time to 0 and broadcast-cleanup-period to 1440 mins (= 24 hrs), so that broadcast objects are cleaned up at midnight every day. Use broadcast-cleanup-age to control how old the broadcast objects should be before they’re actually removed from the cache.

cleanup-stale-timeout

cleanup-stale-timeout specifies when to clean up stale objects (such as subscriptions). When all the DataSource peers in a data service have been down for cleanup-stale-timeout seconds, all the objects that can no longer be updated by the service are deleted from this DataSource application’s cache.

Use in: C

Syntax: cleanup-stale-timeout <timeout-in-seconds>

Type: float

Default value: -1.0 (no timeout)

service-global-peer-balance

service-global-peer-balance determines how the workload-based load-balancing algorithm calculates the workload of peers in an add-priority set.

  • When service-global-peer-balance is set to false (the default), peer workload is defined as the number of subscriptions the peer is serving to the data service.

  • When service-global-peer-balance is set to true, peer workload is defined as the total number of subscriptions the peer is serving to all data services on the DataSource.

For deployments in which peers serve more than one data service, set service-global-peer-balance to true to provide the load-balancing algorithm with a more accurate determination of peer workload.

This configuration item has no affect on the source-affinity load-balancing algorithm (add-source-group:affinity).

Example: load balancing with and without service-global-peer-balance

Consider two data services that have a peer in common (PeerA):

service1 (/SERVICE1)
add-data-service
  service-name service1
  include-pattern /SERVICE1
  add-source-group
    add-priority
      remote-label PeerA
      remove-label PeerB
    end-priority
  end-source-group
end-data-service
service2 (/SERVICE2)
add-data-service
  service-name service2
  include-pattern /SERVICE2
  add-source-group
    add-priority
      remote-label PeerA
      remove-label PeerC
    end-priority
  end-source-group
end-data-service

Assume the following requests have been made to service1 (in this order):

  1. /SERVICE1/A

  2. /SERVICE1/B

  3. /SERVICE1/C

  4. /SERVICE1/D

  5. /SERVICE1/E

  6. /SERVICE1/F

  7. /SERVICE1/G

  8. /SERVICE1/H

…resulting in the following distribution of subscriptions:

Data Service Peer Subscriptions Total Subscriptions

service1

PeerA

4

4

PeerB

4

4

service2

PeerA

0

4

PeerC

0

0

Now we request /SERVICE2/A. Because each peer in service2 has 0 service2 requests and PeerA is the 1st peer, PeerA gets the request even though it already has 4 service1 requests and PeerC has no requests at all.

If service-global-peer-balance were set to true, then peer C would get the new requests and the next 3 service2 requests until it had a total of 4 before service2 requests went to peer A.

Use in: C

Since: DataSource for C (DSDK) 7.1.28, Liberator 7.1.28

Syntax: service-global-peer-balance

Type: boolean

Default value: false

service-rebalance-enable

service-rebalance-enable enables automatic rebalancing of workload in all data-service source groups that don’t have source affinity enabled.

To enable or disable rebalancing for a specific data service, use rebalance-enable, which overrides service-rebalance-enable.

When rebalancing is enabled and a peer in a priority group changes its status to UP, after a set time (service-rebalance-time) the data service discards subscriptions to peers in the priority group and re-requests them, balancing the requests evenly over peers in the priority group.

Service rebalancing is disabled by default and must be enabled globally (using service-rebalance-enable) or for specific data services (using rebalance-enable) where required. The one exception to this is when no data services have been configured: in this one case, service-rebalance-enable defaults to true.

From DataSource 7.1.31 and DataSource 8.0.0, service rebalancing also applies to the Default Data Service.

Use in: C

Since: DataSource for C (DSDK) 7.1.21

Syntax: service-rebalance-enable

Type: boolean

Default value:

  • No configured data services: true

  • Configured data services: false

service-rebalance-time

service-rebalance-time sets the time to wait (in seconds) before rebalancing workload after a peer in priority group changes its status to UP.

Service rebalancing is disabled by default. To enable service rebalancing, see service-rebalance-enable and the add-data-service option rebalance-enable.

service-rebalance-time is overridden by the add-data-service option rebalance-time

From DataSource 7.1.31 and DataSource 8.0.0, service rebalancing also applies to the Default Data Service.

Use in: C

Since: DataSource for C (DSDK) 7.1.21

Syntax: service-rebalance-time <seconds>

Type: float

Default value: 5.0

service-request-timeout

service-request-timeout specifies the global request timeout in seconds for all data services.

This is the time that this DataSource application waits for a data service to respond to a request (for example, a subscription request). If the DataSource peers providing the service don’t respond within this time, the requested object is assumed to not be available and the application sends a discard message for the object to the peers.

A value of 0 or -1 means no timeout is set and this DataSource application then waits indefinitely for a data service to respond to a request. Use this setting in Liberator if you want to allow all of the Liberator’s subscription requests to be open subscriptions (see How can I…​ Keep subscriptions open when the peer is down).

service-request-timeout is a global setting applied to all data services. You can override this timeout for a particular data service by setting the request-timeout option of add-data-service.

Use in: C

Syntax: service-request-timeout <timeout-in-seconds>

Type: float

Default value: 10.0 seconds

See also: