DataSource Prometheus metrics

This page lists the Prometheus metrics published by DataSource applications.

Prometheus metrics are not published by default. To enable this feature, see Publishing DataSource metrics in Prometheus format.

Contents:

C and .NET DataSource metrics
Java DataSource metrics

C and .NET DataSource metrics

This section describes Prometheus metrics published by .NET DataSource 8.0.0 applications and C DataSource 8.0.0+ applications.

Prometheus metrics are disabled by default. To enable Prometheus metrics in your DataSource application, set prometheus-port to a free port on the host.

The Prometheus metrics endpoint URI is /metrics.

For descriptions of additional metrics published by specific C DataSource applications, see the links below:

C and .NET: process metrics

C and .NET DataSource applications publish the following metrics on the operating system process that runs the DataSource.

process_cpu_seconds_total

Total user and system CPU time spent in seconds.

Type: Gauge

process_max_fds

The maximum number of file descriptors the process may have open (set by ulimit on Linux).

Type: Gauge

process_open_fds

Number of file descriptors the process has open.

Type: Gauge

process_resident_memory_bytes

Resident memory size in bytes.

Type: Gauge

process_start_time_seconds

Start time of the process (number of seconds since the start of the Unix epoch).

Type: Gauge

process_virtual_memory_bytes

Virtual memory size in bytes.

Type: Gauge

process_virtual_memory_max_bytes

Maximum amount of virtual memory available in bytes.

Type: Gauge

dsdk_process_fds

Number of file descriptors the process has open.

Type: Gauge

Labels
	`type`	File descriptor type. Possible values: `non_socket_fds`, `socket_fds`

dsdk_process_uptime_seconds_total

Process uptime in seconds.

Type: Counter

dsdk_process_cputime_seconds_total

Process total CPU time in seconds.

Type: Counter

Labels
	`type`	CPU time type. Possible values: `user`, `system`

dsdk_process_delayed_threads

Count of delayed threads that have been delayed for 5 seconds or more. Delayed threads are an indication that the DataSource is under high load or that threads have locked up.

Type: Gauge

dsdk_process_cpu_usage_percentage

Average CPU percentage used by the DataSource process in the period configured by the configuration option process-usage-period (default 10 seconds).

Type: Gauge

dsdk_process_total_cpu_usage_percentage

Total CPU percentage used by the DataSource process in the period configured by the configuration option process-usage-period (default 10 seconds).

This percentage can be greater than 100% if the DataSource is running with a system that has more than one core.

Type: Gauge

Since: DSDK 8.0.12

dsdk_process_num_cpus_total

The number of cores on the server.

Type: Counter

Since: DSDK 8.0.12

dsdk_process_thread_cpu_usage_percentage

Average CPU percentage used by the Caplin DataSource thread in the previous 500 milliseconds.

This metric is only provided for Caplin DataSource threads. It is not provided for threads started by third-party libraries - for instance any threads started by a JVM if the DataSource loads one.

Type: Gauge

Since: DSDK 8.0.5

Labels
	`thread_name`	the name of the DataSource thread

C and .NET: DataSource JVM metrics

Metrics for a JVM loaded by a DataSource.

These metrics are reported when a DataSource loads a JVM, i.e. when jvm_location is configured.

Since: DSDK 8.0.9 ( except where specified )

dsdk_jvm_gc_collection_seconds_count

The total number of garbage collections that have occurred.

Type: Gauge

Labels
	`gc`	the name of the memory manager

dsdk_jvm_gc_collection_seconds_sum

The approximate accumulated collection elapsed time in seconds.

Type: Gauge

Labels
	`gc`	the name of the memory manager

dsdk_jvm_memory_committed_bytes

The amount of memory in bytes that is committed for the Java virtual machine to use.

Type: Gauge

Labels
	`area`	the memory area: heap or non-heap

dsdk_jvm_memory_init_bytes

The amount of memory in bytes that the Java virtual machine initially requests from the operating system for memory management.

Type: Gauge

Labels
	`area`	the memory area: heap or non-heap

dsdk_jvm_memory_max_bytes

The maximum amount of memory in bytes that can be used for memory management..

Type: Gauge

Labels
	`area`	the memory area: heap or non-heap

dsdk_jvm_memory_used_bytes

The amount of used memory in bytes.

Type: Gauge

Labels
	`area`	the memory area: heap or non-heap

dsdk_jvm_threads_current

The current number of live threads including both daemon and non-daemon threads.

Type: Gauge

dsdk_jvm_threads_daemon

The current number of live daemon threads.

Type: Gauge

dsdk_jvm_threads_deadlocked

The of threads that are in deadlock waiting to acquire object monitors or ownable synchronizers.

Type: Gauge

dsdk_jvm_threads_deadlocked_monitor

The threads that are in deadlock waiting to acquire object monitors.

Type: Gauge

dsdk_jvm_threads_peak

The peak live thread count.

Type: Gauge

dsdk_jvm_threads_started_total

The total number of threads created and also started since the Java virtual machine started.

Type: Gauge

C and .NET: DataSource peer metrics

Metrics for DataSource peer connections.

To have peer metrics be reported under recognisable labels, add a remote-label option to the add-peer configuration of the connecting peer.

dsdk_peer_bytes_read_total

Total number of bytes read across all peer connections.

Type: Counter

Labels
	`remote_label`	the label of the DataSource peer

dsdk_peer_bytes_written_total

Total number of bytes written across all peer connections.

Type: Counter

Labels
	`remote_label`	the label of the DataSource peer

dsdk_peer_messages_read_total

Total number of messages read across all peer connections.

Type: Counter

Labels
	`remote_label`	the label of the DataSource peer
	`message_type`	the type of DataSource message. Possible values: `status`, `update`

dsdk_peer_messages_written_total

Total number of messages written across all peer connections.

Type: Counter

Labels
	`remote_label`	the label of the DataSource peer
	`message_type`	the type of DataSource message. Possible values: `request`, `discard`

dsdk_peer_pingpong_time_milliseconds

Average of the last ten heartbeat round-trip times.

Type: Gauge

Labels
	`remote_label`	the label of the DataSource peer

dsdk_peer_status

Status of the peer connection.

Type: Gauge

Since: DSDK 8.0.8

Values:

Peer connection is down

Peer connection is up

Labels
	`remote_label`	the label of the DataSource peer

dsdk_peer_active_subscriptions

Number of active subscriptions for the peer.

Type: Gauge

Since: DSDK 8.0.8

Labels
	`remote_label`	the label of the DataSource peer

C: DataSource data-service metrics

C DataSources publish the following metrics for each data service.

dsdk_data_service_messages_read_total

The total number of messages read from a data service.

Type: Counter

Labels
	`service_name`	the data-service name
	`message_type`	the type of message received. Possible values: `update`, `map`, `nodata`

dsdk_data_service_bytes_read_total

The total number of bytes read from a data service.

Type: Counter

Labels
	`service_name`	the data-service name

dsdk_data_service_bytes_written_total

The total number of bytes written to a data service (contributions).

Type: Counter

Labels
	`service_name`	the data-service name

dsdk_data_service_state

The state of a labelled data service. It has two possible values: 1 for up, 0 for down.

Type: Gauge

Labels
	`service_name`	the data-service name

dsdk_data_service_subscriptions

The number of open subscriptions associated with a data service.

Type: Gauge

Labels
	`service_name`	the data-service name

dsdk_data_service_response_time_milliseconds

The time taken for data services to respond to the first subscription request.

Type: Histogram

Labels
	`le`	the upper-bound of a histogram bucket.

Java DataSource metrics

This section describes Prometheus metrics published by all Java DataSources (8.0.0+) when prometheus-port is set.

The Prometheus metrics endpoint URI is /metrics. This can be customised for Java DataSources; see prometheus-endpoint.

Java: process metrics

Metrics for the Java virtual machine (JVM), process, and system running the DataSource.

The JVM metrics available may differ depending on the JVM and garbage collector.

jvm_buffer_count_buffers

An estimate of the number of buffers in the pool.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`id`	Buffer identifier. Examples: 'direct', 'mapped'.

jvm_buffer_memory_used_bytes

An estimate of the memory that the Java virtual machine is using for this buffer pool.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`id`	Buffer identifier. Examples: 'direct', 'mapped'.

jvm_buffer_total_capacity_bytes

An estimate of the total capacity of the buffers in this pool.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`id`	Buffer identifier. Examples: 'direct', 'mapped'.

jvm_gc_live_data_size_bytes

Size of old generation memory pool after a full GC.

Type: Gauge

Labels
	`app_name`	DataSource application name

jvm_gc_max_data_size_bytes

Max size of old generation memory pool.

Type: Gauge

Labels
	`app_name`	DataSource application name

jvm_gc_memory_allocated_bytes_total

Incremented for an increase in the size of the young generation memory pool after one GC to before the next.

Type: Counter

Labels
	`app_name`	DataSource application name

jvm_gc_memory_promoted_bytes_total

Count of positive increases in the size of the old generation memory pool before GC to after GC.

Type: Counter

Labels
	`app_name`	DataSource application name

jvm_gc_pause_seconds

Time spent in GC pause.

Type: Summary

Labels
	`action`	`end of major GC`, `end of minor GC`
	`cause`	`Metadata GC Threshold`, `Allocation Failure`

jvm_gc_pause_seconds_max

Time spent in GC pause

Type: Gauge

Labels
	`action`	`end of major GC`, `end of minor GC`
	`cause`	`Metadata GC Threshold`, `Allocation Failure`

jvm_memory_committed_bytes

The amount of memory in bytes that is committed for the Java virtual machine to use

Type: Gauge

Labels
	`app_name`	DataSource application name
	`area`	`heap`, `nonheap`
	`id`	memory block identifier. Examples: "G1 Eden Space", "G1 Old Gen"

jvm_memory_max_bytes

The maximum amount of memory in bytes that can be used for memory management

Type: Gauge

Labels
	`app_name`	DataSource application name
	`area`	`heap`, `nonheap`
	`id`	memory block identifier. Examples: "G1 Eden Space", "G1 Old Gen"

jvm_memory_used_bytes

The amount of used memory.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`area`	`heap`, `nonheap`
	`id`	memory block identifier. Examples: "G1 Eden Space", "G1 Old Gen"

jvm_threads_daemon_threads

The current number of live daemon threads.

Type: Gauge

Labels
	`app_name`	DataSource application name

jvm_threads_live_threads

The current number of live threads including both daemon and non-daemon threads.

Type: Gauge

Labels
	`app_name`	DataSource application name

jvm_threads_peak_threads

The peak live thread count since the Java virtual machine started or peak was reset.

Type: Gauge

Labels
	`app_name`	DataSource application name

jvm_threads_states_threads

The current number of threads having NEW state.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`state`	`runnable`, `blocked`, `waiting`, `timed-waiting`, `new`, `terminated`

process_cpu_usage

The "recent cpu usage" for the Java Virtual Machine process.

Type: Gauge

Labels
	`app_name`	DataSource application name

process_start_time_seconds

Start time of the process since Unix epoch.

Type: Gauge

Labels
	`app_name`	DataSource application name

process_uptime_seconds

The uptime of the Java virtual machine.

Type: Gauge

Labels
	`app_name`	DataSource application name

system_cpu_count

The number of processors available to the Java virtual machine.

Type: Gauge

Labels
	`app_name`	DataSource application name

system_cpu_usage

The "recent cpu usage" for the whole system.

Type: Gauge

Labels
	`app_name`	DataSource application name

system_load_average_1m

The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time.

Type: Gauge

Labels
	`app_name`	DataSource application name

Java: DataSource metrics

Metrics for Java DataSources.

datasource_cached_publisher_subscriptions

Type: Gauge

Labels
	`app_name`	DataSource application name
	`name`	Cached publisher identifier

datasource_cached_publisher_messages_total

Type: Gauge

Labels
	`app_name`	DataSource application name
	`name`	Cached publisher identifier

datasource_system_threads_delayed

Count of delayed threads that have been delayed for 5 seconds or more. Delayed threads are an indication that the DataSource is under high load or that threads have locked up.

Type: Gauge

Labels
	`app_name`	DataSource application name

Java: DataSource peer metrics

Metrics for Java DataSource peer connections.

datasource_bytes_read

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_bytes_written

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_discard_average

Average duration of discarded requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_discard_count

Total number of discarded requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Counter

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_discard_max

Duration in milliseconds of the longest discarded request from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_executor_queue_length

Total number of requests from a DataSource peer that are pending execution. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Counter

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_messages_read

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_netty_bytes_before_writable

How many bytes remain in underlying buffers until a thread can begin to process queued requests. See Channel.bytesBeforeWritable() for more information. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_netty_bytes_before_unwritable

How many bytes can be written to underlying buffers until a thread must add new requests to the queue rather than process them immediately. See Channel.bytesBeforeUnwritable() for more information. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_messages_written

Total number of messages written to a specific DataSource peer.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_ping_pong_time

Average of the last ten heartbeat round-trip times.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_request_average

Average duration of requests from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_request_count

Total number of requests made from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Counter

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

datasource_request_max

Duration in milliseconds of the longest request from a DataSource peer. To configure the update frequency for this metric, see peer-debug-metrics-interval.

Type: Gauge

Labels
	`app_name`	DataSource application name
	`peer_name`	DataSource peer identifier

See also: