# DataSource Prometheus metrics This page lists the Prometheus metrics published by DataSource applications. Prometheus metrics are not published by default. To enable this feature, see [Publishing DataSource metrics in Prometheus format](datasource-configure-prometheus-monitoring.md#datasource). ## DataSource for C (DSDK) metrics This section describes Prometheus metrics published by all C DataSource applications, including Liberator, Transformer, and the TREP Adapter, when [`prometheus-port`](datasource-datasource-monitoring-configuration-part-1.md#prometheus-port) is set. This feature requires DataSource for C 7.1.16+ (Liberator 7.1.15+, Transformer 7.1.10+, TREP Adapter 7.1.3+). The Prometheus metrics endpoint URI is `/metrics`. For descriptions of additional metrics published by specific C DataSource applications, see the links below: * [Liberator Prometheus metrics](../liberator/liberator-prometheus-metrics.md) * [Transformer Prometheus metrics](../transformer/transformer-prometheus-metrics.md) ### Process metrics C DataSource applications publish the following metrics on the operating system process that runs the DataSource. These metrics are only available on RHEL builds of C DataSources. * **process_cpu_seconds_total**\ [gauge] Total user and system CPU time spent in seconds. * **process_max_fds**\ [gauge] Maximum number of open file descriptors. * **process_open_fds**\ [gauge] Number of open file descriptors. * **process_start_time_seconds**\ [gauge] Start time of the process since the unix epoch in seconds. * **process_virtual_memory_bytes**\ [gauge] Virtual memory size in bytes. * **process_virtual_memory_max_bytes**\ [gauge] Maximum amount of virtual memory available in bytes. ### System metrics C DataSources publish the following CPU, memory, and process metrics, where _appname_ is the name of the DataSource application, as defined by the [`application-name`](datasource-run-time-environment-configuration.md#application-name) configuration item. * **__appname___system_cputime_total**\ [gauge] The total amount of time (seconds) the CPU has spent running the DataSource process. This value is the sum of **__appname___system_system_cputime_total** and **__appname___system_user_cputime_total**. * **__appname___system_system_cputime_total**\ [gauge] The total amount of time (seconds) the CPU has spent running kernel-space code for the DataSource process. * **__appname___system_user_cputime_total**\ [gauge] The total amount of time (seconds) the CPU has spent running user-space code for the DataSource process. * **__appname___system_process_uptime**\ [gauge] Elapsed 'wall clock' time (seconds) since the DataSource process was started. * **__appname___system_memory_usage**\ [gauge] The total amount of virtual memory (kilobytes) used by the DataSource process, as reported by the `VmRSS` value in `/proc//status`. * **__appname___system_cpu_usage**\ [gauge] Average CPU percentage used by the DataSource process in the period configured by the configuration option [`process-usage-period`](../datasource/datasource-datasource-monitoring-configuration-part-1.md#process-usage-period) (default 10 seconds). ### DataSource peer metrics Metrics for DataSource peer connections, where _appname_ is the name of the DataSource application, as defined by the [`application-name`](datasource-run-time-environment-configuration.md#application-name) configuration item. * **__appname___peer_global_bytes_read**\ [gauge] Total number of bytes read from all peer connections. * **__appname___peer_global_bytes_read_rate**\ [gauge] The rate data is received (bytes/second) from all peer connections, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___peer_global_bytes_written**\ [gauge] Total number of bytes written to peer connections. * **__appname___peer_global_bytes_written_rate**\ [gauge] The rate data is written (bytes/second) to all peer connections, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___peer_global_messages_read**\ [gauge] Total number of messages read from peer connections. * **__appname___peer_global_messages_read_rate**\ [gauge] The rate messages are received (messages/second) from all peer connections, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___peer_global_messages_written**\ [gauge] Total number of messages written to peer connections. * **__appname___peer_global_messages_written_rate**\ [gauge] The rate messages are written (messages/second) to all peer connections, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). ### Data service metrics C DataSources publish the following metrics for each [data service](datasource-data-services.md), where _appname_ is the name of the DataSource application as defined by [`application-name`](datasource-run-time-environment-configuration.md#application-name) and `servicename` is the name of the data service. * **__appname___service___servicename___average_response_time**\ [gauge] The average time (milliseconds) taken for the data service to respond to a subscription request, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___service___servicename___bytes_read**\ [gauge] Total number of bytes read from the data service. * **__appname___service___servicename___bytes_read_rate**\ [gauge] The rate at which data is received (bytes/second) from the data service, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___service___servicename___bytes_written**\ [gauge] Total number of bytes written to the data service. * **__appname___service___servicename___bytes_written_rate**\ [gauge] The rate at which data is written (bytes/second) to the data service, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___service___servicename___messages_read**\ [gauge] The total number of messages received from the data service. * **__appname___service___servicename___messages_read_rate**\ [gauge] The rate at which messages are received (messages/second) from the data service, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___service___servicename___messages_written**\ [gauge] The total number of messages written to the data service. * **__appname___service___servicename___messages_written_rate**\ [gauge] The rate at which messages are written (messages/second) to the data service, measured over the sample period defined by [`peer-monitor-interval`](../datasource/datasource-datasource-peers-configuration-part-2.md#peer-monitor-interval). * **__appname___service___servicename___response_time**\ [gauge] The time taken (seconds) for the data service to respond to the first subscription request. This value only changes if the object is discarded and requested again. * **__appname___service___servicename___state**\ [gauge] The state of the data service. It has two possible values: `1` for up, `0` for down. ## DataSource for Java metrics Prometheus metrics published by all Java DataSources (7.1.14+) when [`prometheus-port`](datasource-datasource-monitoring-configuration-part-1.md#prometheus-port) is set. The Prometheus metrics endpoint URI is `/metrics`. This can be customised for Java DataSources; see [`prometheus-endpoint`](datasource-datasource-monitoring-configuration-part-1.md#prometheus-endpoint). ### Java DataSource: JVM metrics Metrics for the Java virtual machine (JVM) running the DataSource. * **jvm_buffer_count_buffers**\ [gauge] An estimate of the number of buffers in the pool. * **jvm_buffer_memory_used_bytes**\ [gauge] An estimate of the memory that the Java virtual machine is using for this buffer pool Tags for `jvm_buffer_memory_used_bytes`: * `id="direct"` * `id="mapped"` * **jvm_buffer_total_capacity_bytes**\ [gauge] An estimate of the total capacity of the buffers in this pool * **jvm_gc_live_data_size_bytes**\ [gauge] Size of old generation memory pool after a full GC * **jvm_gc_max_data_size_bytes**\ [gauge] Max size of old generation memory pool * **jvm_gc_memory_allocated_bytes_total**\ [counter] Incremented for an increase in the size of the young generation memory pool after one GC to before the next * **jvm_gc_memory_promoted_bytes_total**\ [counter] Count of positive increases in the size of the old generation memory pool before GC to after GC * **jvm_gc_pause_seconds**\ [summary] Time spent in GC pause Tags for `jvm_gc_pause_seconds_count`: * `action="end of major GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Allocation Failure"` Tags for `jvm_gc_pause_seconds_sum`: * `action="end of major GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Allocation Failure"` * **jvm_gc_pause_seconds_max**\ [gauge] Time spent in GC pause Tags for `jvm_gc_pause_seconds_max`: * `action="end of major GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Metadata GC Threshold"` * `action="end of minor GC", cause="Allocation Failure"` * **jvm_memory_committed_bytes**\ [gauge] The amount of memory in bytes that is committed for the Java virtual machine to use Tags for `jvm_memory_committed_bytes`: * `area="heap", id="PS Survivor Space"` * `area="heap", id="PS Old Gen"` * `area="heap", id="PS Eden Space` * `area="nonheap", id="Metaspace"` * `area="nonheap", id="Code Cache"` * `area="nonheap", id="Compressed Class Space"` * **jvm_memory_max_bytes**\ [gauge] The maximum amount of memory in bytes that can be used for memory management Tags for `jvm_memory_max_bytes`: * `area="heap", id="PS Survivor Space"` * `area="heap", id="PS Old Gen"` * `area="heap", id="PS Eden Space` * `area="nonheap", id="Metaspace"` * `area="nonheap", id="Code Cache"` * `area="nonheap", id="Compressed Class Space"` * **jvm_memory_used_bytes**\ [gauge] The amount of used memory * **jvm_threads_daemon_threads**\ [gauge] The current number of live daemon threads * **jvm_threads_live_threads**\ [gauge] The current number of live threads including both daemon and non-daemon threads * **jvm_threads_peak_threads**\ [gauge] The peak live thread count since the Java virtual machine started or peak was reset * **jvm_threads_states_threads**\ [gauge] The current number of threads having NEW state Tags for `jvm_threads_states_threads`: * `state="runnable"` * `state="blocked"` * `state="waiting"` * `state="time-waiting"` * `state="new"` * `state="terminated"` * **process_cpu_usage**\ [gauge] The "recent cpu usage" for the Java Virtual Machine process * **process_start_time_seconds**\ [gauge] Start time of the process since unix epoch. * **process_uptime_seconds**\ [gauge] The uptime of the Java virtual machine * **system_cpu_count**\ [gauge] The number of processors available to the Java virtual machine * **system_cpu_usage**\ [gauge] The "recent cpu usage" for the whole system * **system_load_average_1m**\ [gauge] The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time # Java DataSource: peer metrics Metrics for DataSource peer connections, where _appname_ is the name of the DataSource application, as defined by the [`application-name`](datasource-run-time-environment-configuration.md#application-name) configuration item. * **__appname___discard.average**\ +[gauge]+ Average duration of discarded requests from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___discard.count**\ [counter] Total number of discarded requests from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___discard.max**\ [gauge] Duration in milliseconds of the longest discarded request from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___executor.queue.length**\ [counter] Total number of requests from a DataSource peer that are pending execution. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___netty.bytes.before.writable**\ [gauge] [Netty] How many bytes remain in underlying buffers until a thread can begin to process queued requests. See [`Channel.bytesBeforeWritable()`](https://netty.io/4.1/api/io/netty/channel/Channel.html#bytesBeforeWritable--) for more information. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___netty.bytes.before.unwritable**\ [gauge] [Netty] How many bytes can be written to underlying buffers until a thread must add new requests to the queue rather than process them immediately. See [`Channel.bytesBeforeUnwritable()`](https://netty.io/4.1/api/io/netty/channel/Channel.html#bytesBeforeUnwritable--) for more information. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___peer_global_bytes_read**\ [gauge] Total number of bytes read from all peer connections. * **__appname___peer_global_bytes_read_rate**\ [gauge] The rate data is received (bytes/second) from all peer connections (sample period: 1 second). * **__appname___peer_global_bytes_written**\ [gauge] Total number of bytes written to peer connections. * **__appname___peer_global_bytes_written_rate**\ [gauge] The rate data is written (bytes/second) to all peer connections (sample period: 1 second). * **__appname___peer_global_messages_read**\ [gauge] Total number of messages read from peer connections. * **__appname___peer_global_messages_read_rate**\ [gauge] The rate messages are received (messages/second) from all peer connections (sample period: 1 second). * **__appname___peer_global_messages_written**\ [gauge] Total number of messages written to peer connections. * **__appname___peer_global_messages_written_rate**\ [gauge] The rate messages are written (messages/second) to all peer connections (sample period: 1 second). * **__appname___request.average**\ [gauge] Average duration of requests from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___request.count**\ [counter] Total number of requests made from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). * **__appname___request.max**\ [gauge] Duration in milliseconds of the longest request from a DataSource peer. To configure the update frequency for this metric, see [`peer-debug-metrics-interval`](datasource-datasource-peers-configuration-part-2.md#peer-debug-metrics-interval). --- **See also:** * [`prometheus-port`](datasource-datasource-monitoring-configuration-part-1.md#prometheus-port) * [`prometheus-endpoint`](datasource-datasource-monitoring-configuration-part-1.md#prometheus-endpoint)