# Logging configuration These DataSource configuration items define what information a DataSource application can log and how the log files are automatically cycled. For an overview of how logging works, including more about **log levels** and **log file cycling**, see [Logging in DataSource applications](datasource-logging-in-datasource-applications.md). **📌 NOTE**\ "_this DataSource application_" means the DataSource application for which you are defining the configuration. "_DataSource peer_" or "_peer_" means a DataSource application that this DataSource application communicates with. ## add-log `add-log` provides configuration options that, for a particular type of log file, override one or more of the global logging configuration items. You specify the type of log file in the [name](#name) option, and add other options to customise the file’s log level, log monitor level, and log file cycling characteristics. **Use in:** C, Java **Syntax:** ``` add-log xref:name[name] [string] xref:level[level] [string] xref:maxsize[maxsize] [integer] xref:monitor-level[monitor-level] [string] xref:offset[offset] [integer] xref:period[period] [integer] xref:suffix[suffix] [string] xref:syslog-level[syslog-level] [string] xref:time[time] [integer] xref:use-parent-handlers[use-parent-handlers] [boolean] end-log ``` | Option | Type | Default | Description | | --- | --- | --- | --- | | `level` | string | [value of [log-level](#log-level)] | This option only applies to the Event log. It specifies the severity of the errors and events that this DataSource application reports to the log file for this type of log message. It overrides the [log-level](#log-level) item. For the list of valid values, see [log-level](#log-level). Also see the [syslog-level](#syslog-level) option. | | `maxsize` | integer | [value of [log-maxsize](#log-maxsize)] | The maximum size in bytes for this type of log file. The file size is checked every [log-cycle-period](#log-cycle-period) (or every [period](#period) option of this `add-log`, if defined), and the log file is [cycled](#logging-overview-ref) if it exceeds `maxsize`. A value of `0` means there’s no maximum size limit, but then the log file is always cycled every [period](#period) / [log-cycle-period](#log-cycle-period) regardless of its size. This option overrides the [log-maxize](#log-maxsize) item. | | `monitor-level` | string | [value of [log-monitor-level](datasource-datasource-monitoring-configuration-part-2.md#log-monitor-level)] | This option only applies to the Event log. It specifies the threshold at which log messages about this DataSource application’s events and errors are published to the monitoring subsystem. It overrides the [log-monitor-level](datasource-datasource-monitoring-configuration-part-2.md#log-monitor-level) item. For the list of valid values, see [log-monitor-level](datasource-datasource-monitoring-configuration-part-2.md#log-monitor-level). NOTE: This option is not available in Java-based DataSource applications. | | `name` | string | +[none]+ | The name (type) of log to which the rest of this ``add-log`’s options apply. Note that the [log cycle](#logging-overview-ref) filename for the log is constructed from a combination of the run-time filename implied by `name`, the value of [suffix](#suffix)/[log-cycle-suffix](#log-cycle-suffix), and the value of [offset](#offset)/[log-cycle-offset](#log-cycle-offset). If you’re writing your own C-based DataSource application, using the C DataSource API, you can define your own log files in addition to the standard ones. In that case, `name` must correspond to the name of a log object that you’ve created in the code. (In Java-based DataSource applications you can either log to the standard DataSource event log, or create your own logger when you create an instance of the `DataSource` class.) Caplin-supplied DataSource applications, such as [Liberator](../liberator/index.md) have a set of built-in log files with standard types that you can specify in `name`, as follows: * `event_log`: (All DataSource applications) The event log file. The run-time filename is defined by [event-log](#event-log). * `http_access_log`: (Liberator only) The HTTP access log file. The run-time filename is defined by Liberator’s [http-access-log](../liberator/liberator-http-configuration-part-1.md#http-access-log) configuration item. * `javaauth_log`: (Liberator only) The Java auth log file.The run-time filename is _auth-rttpd.log_ (this is the log file produced by [Java-based Liberator Permissioning Service blades](../deployment-framework/cdf-create-a-java-based-liberator-permissioning-service-blade.md)). * `news_log`: (Liberator only) The log file that stores news headlines. The run-time filename is defined by Liberator’s [news-log](../liberator/liberatornews-configuration.md#news-log) configuration item. * `object_log`: (Liberator only) The object log file that keeps a record of all request and discard commands for objects. The run-time filename is defined by Liberator’s [object-log](../liberator/liberator-session-configuration.md#object-log) configuration item. * `packet_log`: (All DataSource applications) The log file that records each packet received from the application’s DataSource peers. The run-time filename is defined by the [datasrc-pkt-log](datasource-datasource-peers-configuration-part-2.md#datasrc-pkt-log) configuration item. * `request_log`: (Liberator only) The log file that records RTTP requests made to Liberator. The run-time filename is defined by the Liberator’s [request-log](../liberator/liberator-session-configuration.md#request-log) configuration item. * `session_log`: (Liberator only) The log file that records messages about client connections, disconnections, logins and logouts. The run-time filename is defined by Liberator’s [session-log](../liberator/liberator-session-configuration.md#session-log) configuration item. * `xmlauth_log`: (Liberator only) The XMLauth log file. The run-time filename is defined by Liberator’s `auth-logfile` configuration item in the configuration for the XMLauth [auth module](../liberator/liberator-user-authentication-and-permissioning.md#how-does-liberator-implement-user-authentication-and-permissioning). | | `offset` | integer | [value of [log-cycle-offset](#log-cycle-offset)] | The number of minutes to take off the time before using it to create a cycled log filename suffix. The suffix is formatted according to [log-cycle-suffix](#log-cycle-suffix) or the [suffix](#suffix) option of this [add-log](#add-log). A value of -1 means the offset is the same as [log-cycle-period](#log-cycle-period) (or, if defined, the [period](#period) option of this `add-log`). This option overrides the [log-cycle-offset](#log-cycle-offset) item. It works in the same way as `log-cycle-offset`; for examples see [log-cycle-offset](#log-cycle-offset). | | `period` | integer | [value of [log-cycle-period](#log-cycle-period)] | The interval in minutes for [cycling](#logging-overview-ref) this type of log file. Each day, at every interval after the first log cycle defined by [log-cycle-time](#log-cycle-time) (or the [time](#time) option of this `add-log`), the log is checked to see if it should be cycled. If the [maxsize](#maxsize) option of this `add-log` is defined, or [log-maxsize](#log-maxsize) is defined, the log file is cycled if it exceeds the maximum allowed size. If neither of `maxsize` or `log-maxsize` is defined, the log file is always cycled. regardless of its size. This option overrides the [log-cycle-period](#log-cycle-period) item. For how to set up weekly log cycling for a particular type of log file, see the [time](#time) option. For log files that grow rapidly, a sensible configuration would be to set a suitable maximum size for the file (see the [maxsize](#maxsize) option), and set the `period` option to `5`, so the file is checked every five minutes. | | `suffix` | string | [value of [log-cycle-suffix](#log-cycle-suffix)] | A suffix format that’s passed to the `strftime()` system function; the resulting string is appended to the name of the cycled log file. For information about the possible format strings, see the LINUX/UNIX manual page for `strftime()` This option overrides the [log-cycle-suffix](#log-cycle-suffix) item. | | `syslog-level` | string | [value of [log-syslog-level](#log-syslog-level)] | **Since**: DataSource 6.2.4 `syslog-level` specifies the severity of the errors and events that this DataSource application reports to the operating system’s log for this type of log message. It overrides the [log-syslog-level](#log-syslog-level) item. The [log-level](#log-level) item, and [level](#level) option (if specified) take precedence over `syslog-level`. For example, if `syslog-level` is set to `DEBUG` but `log-level` is set to `INFO`, only log messages with severity `INFO` or higher go to the syslog. For the list of valid values, see [log-syslog-level](#log-syslog-level). NOTE: This option is not available in Java-based DataSource applications | | `time` | integer | [value of [log-cycle-time](#log-cycle-time)] | The time in minutes from midnight (local time) at which this DataSource application first checks this type of log file for [cycling](#logging-overview-ref). This option overrides the [log-cycle-time](#log-cycle-time) item. The subsequent times at which the application checks the log file for cycling is determined by [log-cycle-period](#log-cycle-period) (or the [period](#period) option of this [add-log](#period)). If neither the [maxsize](#maxsize) option of this `add-log` nor [log-maxsize](#log-maxsize) is defined, the log file is always cycled, regardless of its size. If `time` is greater than `1440` (`24` hours), the cycle time is taken from the start of the week (midnight on Sunday night). For example, a value of `1460` sets cycle time to `00:20` hrs on Monday. This allows you to cycle the log file weekly on a specific day if the `period` option is set to `10080` (7 days). For example, set `time` to `2880` (48 hours) and `period` to `10080`, and the log file is checked for cycling at midnight every Monday. To set weekly log cycling for all files, see [log-cycle-time](#log-cycle-time). If you don’t specify this option and you don’t specify [log-cycle-time](#log-cycle-time), the default value of `240` minutes (`4` hours) applies, so the log file is first checked for cycling at `04:00` hrs (`04:00` am) each day. | | `use-parent-handlers` | boolean | `FALSE` | This option applies to Java-based DataSource applications only. When set to `TRUE`, the application must send log records for this type of log file to its parent Logger (see [setUseParentHandlers](https://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html#setuseparenthandlers-28boolean-29) in the documentation of the Java `Logger` class). This option overrides the [log-use-parent-handlers](#log-use-parent-handlers) item. | ## event-log `event-log` specifies the file name of this DataSource application’s event Log. This log contains messages about starting up, shutting down, connections to DataSource peers, and other general and debug information. The log filename can contain the parameters `%a` and `%h` At run time, `%a` is replaced by the DataSource [application-name](datasource-run-time-environment-configuration.md#application-name), and `%h` is replaced by the host name of the machine on which this DataSource application is running. By convention, log filenames have the format: _<log type>-<application name>.log_ **Example for Liberator:** ``` event-log event-%a.log ``` In this example, because Liberator’s application name is "rttpd", the configuration item names the Liberator’s event log file as _event-rttpd.log_. **💡 TIP**\ Use [log-dir](#log-dir) to specify the directory where log files are to be created. **Use in**: C, Java **Syntax:** `event-log ` **Type:** string **Default value:** `event-%a.log` (in Transformer it’s just `%a.log`) ## log-cycle-offset `log-cycle-offset` specifies the number of minutes to take off the time before creating a [cycled](#logging-overview-ref) log filename suffix that’s formatted according to [log-cycle-suffix](#log-cycle-suffix). For example, assume `log-cycle-offset` is `10`, and cycled logs are given a filename suffix of the form _.hh-mm_ Then when the logs are cycled at 04:00 hrs, the resulting files all have the suffix _.03-50_ A value of `-1` means the offset is the same as [log-cycle-period](#log-cycle-suffix). For example, assume `log-cycle-period` is the default of `4` hours, and `log-cycle-offset` is `-1`. When the logs are cycled at 04:00 hrs on Tuesday, the value passed to ``strftime()``for formatting is 04:00 on _Monday_. This makes the timestamp suffix in the cycled log filenames more useful, as the suffix indicates the date and time from which the data in each file starts, rather than the date and time when the logs were cycled. You can override this setting for a particular type of log file by specifying the [offset](#offset) option of the [add-log](#add-log) configuration item. **Use in:** C, Java **Syntax:** `log-cycle-offset **Type:** integer **Default value:** `-1` (= value of [log-cycle-period](#log-cycle-period)) ## log-cycle-period `log-cycle-period` specifies the interval in minutes for [cycling logs](#logging-overview-ref). Each day, at every `log-cycle-period` after the first log cycle defined by [log-cycle-time](#log-cycle-time) (or by the [time](#time) option of an [add-log](#add-log) item), each log file is checked to see whether it should be cycled. If the [maxsize](#maxsize) option of the [add-log](#add-log) for the file is defined, or [log-maxsize](#log-maxsize) is defined, the log file is cycled if it exceeds the maximum allowed size. If neither of `maxsize` or `log-maxsize` is defined, the log file is always cycled, regardless of its size. You can override the `log-cycle-period` setting for a particular type of log file by specifying the [period](#period) option of the file’s [add-log](#add-log) configuration item. For log files that grow rapidly, a sensible configuration would be to set a suitable maximum size for the file (see the [maxsize](#maxsize) option of [add-log](#add-log)), and set the [period](#period) option of [add-log](#add-log) to `5`, so the file is checked every five minutes. For how to set up weekly log cycling, see [log-cycle-time](#log-cycle-time). **Use in:** C, Java **Syntax:** `log-cycle-period ` **Type:** integer **Default value:** `1440 minutes` (= `24` hours) ## log-cycle-suffix `log-cycle-suffix` defines a suffix format that’s passed to the `strftime()` system function; the resulting string is appended to [cycled](#logging-overview-ref) log files. You can override this setting for a particular type of log file by specifying the [suffix](#suffix) option of the [add-log](#add-log) configuration item. **💡 TIP**\ For information about the possible format strings, see the LINUX/UNIX manual page for `strftime()` **💡 TIP**\ We recommend that you don’t use a suffix format, such as `%T`, that includes seconds. This is because if the server is busy when the log file cycles, the create time of the log file may be out by a second, and this can prevent old (unwanted) log files from being overwritten. This can be a particular problem for packet log files, which can grow very large. **Use in:** C, Java **Syntax:** `log-cycle-suffix ` **Type:** string **Default value:** `%u` This appends the log filenames with a day number between 1 and 7, representing Monday (1) to Sunday (7). ## log-cycle-time `log-cycle-time` specifies the time in minutes from midnight (local time) at which log files are first checked for [cycling](#logging-overview-ref). You can override this setting for a particular type of log file by specifying the [time](#time) option of the [add-log](#add-log) configuration item. The subsequent times at which this DataSource application checks log files for cycling is determined by [log-cycle-period](#log-cycle-period) (or the [period](#period) option of [add-log](#period)). If neither the [maxsize](#maxsize) option of the [add-log](#add-log) configuration item for the log file nor [log-maxsize](#log-maxsize) is defined, the log file is always cycled, regardless of its size. You can override the `log-cycle-time` setting for a particular type of log file by specifying the [time](#time) option of the file’s [add-log](#add-log) configuration item. If `log-cycle-time` is greater than `1440` (24 hours), the cycle time is taken from the start of the week (midnight on Sunday night). For example, a value of `1460` sets cycle time to 00:20 hrs on Monday. This allows you to set weekly log cycling for all log files on a specific day if `log-cycle-period` is set to `10080` (7 days). For example, set `log-cycle-time` to `2880` (48 hours) and `log-cycle-period` to `10080`, and all log files are checked for cycling at midnight every Monday. To set weekly log cycling for an individual file, see the [time](#time) option of [add-log](#add-log). **Use in:** C, Java **Syntax:** `log-cycle-time ` **Type:** integer **Default value:** `240` minutes (= `4` hours) So the log files are first checked for cycling at `04:00` hrs (`04:00` am) every day. ## log-dir `log-dir` specifies the default directory (as an absolute or relative path) where this DataSource application’s log files are created. **Use in:** C, Java The directory path can contain the parameters `%r` and `%a` At run time, `%r` is replaced by the root directory ([application-root](datasource-run-time-environment-configuration.md#application-root)) under which this DataSource application runs, and `%a` is replaced by the DataSource [application-name](datasource-run-time-environment-configuration.md#application-name). **Syntax:** `log-dir ` **Type:** string **Default value:** `%r/var` ## log-level `log-level` specifies the severity of the errors and events that are reported to the log files when this DataSource application is running. You can override this setting for the Event log only by specifying the [level](#level) option of the [add-log](#add-log) configuration item. Also see [log-syslog-level](#log-syslog-level) and [syslog-facility](#syslog-facility). **Use in:** C, Java **Syntax:** `log-level ` **Type:** string **Default value:** `INFO` **Values accepted:** | log-level-name | CORRESPONDING JAVA LOG LEVEL | DESCRIPTION | | --- | --- | --- | | `DEBUG` | `FINE` | Reports all errors and events. | | `INFO` | `INFO` | Reports events and information regarding normal operation, and all errors included in the `WARN`, `NOTIFY`, `ERROR` and `CRIT` log levels. | | `WARN` | `WARNING` | Reports minor errors and all errors included in the `NOTIFY`, `ERROR` and `CRIT` log levels. | | `NOTIFY` | `WARNING` | Reports errors regarding data corruptions and all errors included in the `ERROR` and `CRIT` log levels. | | `ERROR` | `SEVERE` | Reports serious errors regarding network connections and all errors included in the `CRIT` log level. | | `CRIT` | `SEVERE` | Reports critical errors that prevent the DataSource application from running. | For Java-based DataSource applications only, the actual log level recorded against a log entry in the log file is the Java level. For example, if you specify a log-level setting of `NOTIFY`, the log file will contain just entries with Java log levels `WARNING` and `SEVERE`. **💡 TIP**\ `log-level` doesn’t set the threshold at which log messages about the DataSource application’s events and errors are published to the [monitoring subsystem](datasource-datasource-monitoring-and-management.md); to do this, use [log-monitor-level](datasource-datasource-monitoring-configuration-part-2.md#log-monitor-level). ## log-max-history `log-max-history` specifies the maximum number of log lines that are retained for viewing via JMX monitoring tools such as the [Caplin Management Console](../cmc/index.md). At any one time, up to `log-max-history` of the most recently generated log lines can be viewed. **Use in:** C **Syntax:** `log-max-history ` **Type:** integer **Default value:** `10` ## log-maxsize `log-maxsize` specifies the maximum log file size in bytes. The size of each log file is checked every [log-cycle-period](#log-cycle-period) (or every [period](#period) of the [add-log](#add-log) for the log file type, if defined), and a log file is cycled if it exceeds `log-maxsize`. A value of `0` means there’s no maximum size limit, but each log file is always cycled every [period](#period) / [log-cycle-period](#log-cycle-period) regardless of its size. You can override this setting for a particular type of log file by specifying the [maxsize](#maxsize) option of the [add-log](#add-log) configuration item. **Use in:** C, Java **Syntax:** `log-maxsize ` **Type:** integer **Default value:** `0` bytes (meaning that log files are cycled every time they are checked) ## log-syslog-level **📌 NOTE**\ This configuration item is only available in DataSource version 6.2.4 or later. `log-syslog-level` specifies the severity of the errors and events that are reported to the operating system’s log when this DataSource application is running. You can override this setting by specifying the [syslog-level](#syslog-level) option of the [add-log](#add-log) configuration item. The [log-level](#log-level) item, and [level](#level) option of [add-log](#add-log) (if specified) take precedence over `log-syslog-level`. For example, if `log-syslog-level` is set to `DEBUG` but `log-level` is set to `INFO`, only log messages with severity `INFO` or higher go to the syslog. Also see [syslog-facility](#syslog-facility) and [log-level](#log-level). **Use in:** C **Syntax:** `log-syslog-level ` **Type:** string **Default value:** `ERROR` **Values accepted:** | log-level-name | CORRESPONDING JAVA LOG LEVEL | DESCRIPTION | | --- | --- | --- | | `DEBUG` | `FINE` | Reports all errors and events. | | `INFO` | `INFO` | Reports events and information regarding normal operation and all errors included in the `WARN`, `NOTIFY`, `ERROR` and `CRIT` debug levels. | | `WARN` | `WARNING` | Reports minor errors and all errors included in the `NOTIFY`, `ERROR` and `CRIT` debug levels. | | `NOTIFY` | `WARNING` | Reports errors regarding data corruptions and all errors included in the `ERROR` and `CRIT` debug levels. | | `ERROR` | `SEVERE` | Reports serious errors regarding network connections and all errors included in the `CRIT` debug level. | | `CRIT` | `SEVERE` | Reports critical errors that prevent the DataSource application from running. | **💡 TIP**\ `log-syslog-level` doesn’t set the threshold at which log messages about the DataSource application’s events and errors are published to the [monitoring subsystem](datasource-datasource-monitoring-and-management.md); to do this, use [log-monitor-level](datasource-datasource-monitoring-configuration-part-2.md#log-monitor-level). ## log-use-parent-handlers `log-use-parent-handlers` specifies, for Java-based DataSource applications only, that the application must send log records to its parent Logger (see [setUseParentHandlers](https://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html#setuseparenthandlers-28boolean-29) in the documentation of the Java `Logger` class). The standard DataSource logger class that’s used when this item isn’t set is `com.caplin.datasource.DataSource`. You can override this setting for a particular type of log file by specifying the [use-parent-handlers](#use-parent-handlers) option of the [add-log](#add-log) configuration item. **Use in:** Java **Syntax:** `log-use-parent-handlers ` **Type:** boolean **Default value:** `FALSE` ## syslog-facility **📌 NOTE**\ This configuration item is only available in DataSource version 6.2.4 or later. `syslog-facility` specifies the syslog facility to use when this this DataSource application’s error and event messages are recorded in the operating system’s system log (syslog).The message severities to be logged are determined by [log-syslog-level](#log-syslog-level) and the [syslog-level](#syslog-level) option of the [add-log](#add-log) configuration item. **💡 TIP**\ Refer to your operating system manual for instructions on how to set up syslog to receive these messages. **Use in:** C **Syntax:** `syslog-facility ` **Type:** string **Default value:** `local6` ## Default log cycling behaviour If you don’t specify any log cycling configuration, then by defaultall log files are cycled at 04:00 hours each day, so that a separate log file for each type of log is created for each day. You can of course override this default setting for a particular type of log file by specifying the relevant options in an `add-log ... end-log` configuration entry. The default settings are equivalent to the following configuration: ``` # Cycle logs every time they are checked: log-maxsize 0 # Cycle logs for the first time at 4 hrs after midnight: log-cycle-time 240 # Cycle logs every 24 hrs after log-cycle-time: log-cycle-period 1440 # Cycled log filename has suffix 1 - 7 (representing Mon - Sun): log-cycle-suffix .%u # Cycled log filename has day number of the day when the logged data starts: log-cycle-offset -1 ``` **💡 TIP**\ This default configuration can create very large log files if your system has lots of fast moving data. It’s useful to have as much log data as possible in an individual file, but you should change the logging configuration appropriately if the files grow too large. --- **See also:** * How can I... [Manage and interpret log files](https://docs.caplin.com/developer/docs/HowToManageAndInterpretLogFiles_62.pdf) (PDF document) * [DataSource monitoring configuration](datasource-datasource-monitoring-configuration-part-1.md)