# View packet logs using logcat

The logs produced by most Caplin Platform components are simple text files that can be viewed using a suitable text display utility or text editor, such as the Linux commands `cat`, `more`, and `vim`. However, the packet logs are in binary format and must be viewed using the Caplin `logcat` utility. Here’s how to do this.

## Requirements

If the `logcat` command’s `-F` option is used, and the [fields file](datasource-field-definitions.md) referenced by the `-f` option contains the `CONFIG_BASE` variable, then you must enable the Deployment Framework’s command-line completion extension. See [Setting up dfw command completion](../deployment-framework/cdf-setting-up-dfw-command-completion.md).

From version 7.1 of the Caplin Platform, packet logs use a new binary format to accommodate high-resolution timestamps. Packet logs created by DataSource 7.1 components can only be read by logcat 7.1 or greater. For more information on high-resolution timestamps in Caplin Platform 7.1, see [Timestamp resolution](datasource-timestamp-resolution.md).

## Logcat locations

There’s a copy of the `logcat` utility in the `bin` directory of each installed [Liberator](../liberator/index.md), [Transformer](../transformer/index.md) and [Integration Adapter](../platform-architecture/putting-data-into-the-platform.md).

In the [Caplin Deployment Framework](../deployment-framework/index.md), you’ll find `logcat` in the following places:

* `<Framework-root>/servers/Liberator/bin`
* `<Framework-root>/servers/Transformer/bin`
* `<Framework-root>/kits/<Adapter-blade-name>/<Adapter-blade-name>/DataSource/bin`

where `<Framework-root>` is the Deployment Framework’s topmost directory.

**📌 NOTE**\
You can generally use any instance of `logcat` in your Framework to view any of the packet logs produced by any component running under the Framework. However, we recommend that you normally use the latest `logcat` whenever possible; that is, the `logcat` provided in the latest kit version out of all the component kits you’ve deployed in the Framework.

## Logcat syntax

You use `logcat` in a similar way to the standard Linux `cat` command.

**Syntax**

```
./logcat [options] _file_…
```

**Options**

* **`-h, --help`**\
Displays detailed information about the available logcat options.
* **`-F, --print-field-names`**\
Displays the names of the fields in each log record (the default is to just print the field numbers). The field names are obtained from the component’s [fields file](datasource-field-definitions.md) (see `-f` option).

  **📌 NOTE**\
  If the [fields file](datasource-field-definitions.md), or a file referenced by the fields file, uses the environment variable `CONFIG_BASE`, then you must enable the Deployment Framework’s command-line completion extension. See [Setting up dfw command completion](../deployment-framework/cdf-setting-up-dfw-command-completion.md).
* **`-f <fields-file>, --fields-file <fields-file>`**\
The location (directory path) and name of the fields file.

  The default setting, if you don’t provide a `-f` option, is `fields.conf` in the current directory.

  **📌 NOTE**\
  If the [fields file](datasource-field-definitions.md), or a file referenced by the fields file, uses the environment variable `CONFIG_BASE`, then you must enable the Deployment Framework’s command-line completion extension. See [Setting up dfw command completion](../deployment-framework/cdf-setting-up-dfw-command-completion.md).
* **`--print-info`**\
Displays the version, type and source of the log. The timezone reported is that of the local machine that the logs were written on.

  **Example**

  ```
  Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'
  ```
* **`-l, --print-flag-names`**\
Displays the flags that have been sent with each data object; for example `F_NONACTIVE|F_CREATEPARENT|F_CREATEOBJECT`

  For an explanation of what each flag means, see the Data Object Flags module of the [DataSource C (DSDK) API Documentation](https://docs.caplin.com/developer/api/datasource_c/latest/).
* **`-t <type>, --log-type <type>`**\
Tells `logcat` to interpret the supplied file as a particular type.

  This option takes an argument. Currently the only argument available is `packet` which makes `logcat` interpret the file as a packet log. Normally `logcat` determines the type of the binary log file by looking at a header record within the file. However, you’ll need to use the `-t` option if you’ve split the file into smaller pieces, because the header will be missing from the smaller file. See [Splitting packet logs into smaller files](#splitting-packet-logs-into-smaller-files), below.

  **Example**

  ```
  logcat -t packet mypacket.log
  ```
* **`-U <update-file>, --update-file <update-file>`**\
Specifies the name (and optionally the path) of a file to which `logcat` will write update rate statistics, based on an analysis of the supplied packet log file.

  The file will contain an entry for every 1 second period during which any message arrived. Each of these lines will contain:

  * A string showing the time
  * The UNIX time-stamp
  * The number of each type of message received (update, request, discard, nodata, status)
  * The number of update bytes for each one second period in the packet log.

  The peak update rate and the peak update byte rate will appear at the end of the file.

  You can specify both `-U` and `-P` in the same command. They are complementary: `-U` compiles update statistics across all peers and `-P` compiles update statistics for each peer.

  **Compiling statistics from a Liberator packet log**

  To compile update statistics for a Liberator packet log, `packet-rttpd.log`, run the command below:

  ```
  logcat -U update-file.txt packet-rttpd.log
  ```

  The update information for `packet-rttpd.log` is written into `update-file.txt`.
* **`-P, --peer-update-files`**\
**Available from:** DSDK 7.1.29, Liberator 7.1.29

  Compile update-rate statistics for each peer and write them to output files, one file per peer. The file names of output files are automatically generated and take the format `updates-__peer_name__`.

  Each output file contains a one-line entry for every 1 second period during which any message arrived. Each entry contains:

  * A string showing the time
  * The UNIX time-stamp
  * The number of each type of message received (update, request, discard, nodata, status)
  * The number of update bytes for each one second period in the packet log.

  The peak update rate and the peak update byte rate appear at the end of the file.

  You can specify both `-U` and `-P` in the same command. They are complementary: `-U` compiles update statistics across all peers and `-P` compiles update statistics for each peer.

  **Generating peer update statistics for a Liberator packet log**

  Consider a packet log, `packet-rttpd.log` for a Liberator connected to two peers: `transformer` and `permissioning`.

  To compile update statistics from the log file, run the command below:

  ```
  logcat -P packet-rttpd.log
  ```

  The command above generates two output files:

  * `updates-transformer`
  * `updates-permissioning`

  To compile update-rate statistics across all peers too, run the command below:

  ```
  logcat -U update-file.txt -P packet-rttpd.log
  ```

  The command above generates three output files:

  * `update-file.txt`
  * `updates-transformer`
  * `updates-permissioning`
* **`--log-version <version_num>`**\
Specifies the version of the log file, which is then checked against the log versions that this instance of `logcat` can handle.

  This option only has an effect when you pipe the log file to `logcat` from another tool, such as `tail`

  **Example**

  ```
  tail -f <packet-log-file> | logcat --log-version 7
  ```

  Since the log file header, which identifies the log version, isn’t usually available to `logcat` via `tail`, `logcat` takes the supplied `<version_num>` to be the log version instead and formats the file contents accordingly.

  If `<version_num>` is greater than the maximum log file version that this instance of `logcat` can handle, `logcat` terminates with a version warning message. So in the above example, if `logcat` cannot handle log versions greater than `5`, it displays the following message and then exits:

  ```
  Logcat: The Log 'Pipe' is of a type and version (DSBLog, p, 7) that I can't handle
  ```
* **`-x, --print-id`**\
Displays the [`datasrc-id`](datasource-datasource-peers-configuration-part-2.md#datasrc-id) of the peer referred to in a packet.

  By default, `logcat` displays the peer’s name as referred to by it’s [`datasrc-local-label`](datasource-datasource-peers-configuration-part-2.md#datasrc-local-label), if that is available.
* **`-z <timezone>, --timezone <timezone>`**\
Sets all times in the log to the specified timezone. To find the required timezone look in the system folder `zoneinfo`, typically found at `/usr/share/lib/zoneinfo` or `/usr/share/zoneinfo`

  In this folder there are sub folders with timezone information in them; for example, if there’s a subfolder called `US/Mountain`, set the timezone option as `-z US/Mountain`

### Usage examples

These examples are for a Liberator packet log. The current working directory is the Liberator’s `var` directory, where the packet logs are usually located. The default name of Liberator’s packet log is `packet-rttpd.log` — see the [`datasrc-pkt-log`](datasource-datasource-peers-configuration-part-2.md#datasrc-pkt-log) configuration item in [DataSource peers configuration (part 2)](datasource-datasource-peers-configuration-part-2.md).

**Printing information about a binary packet log**

Enter the command below:

```
../bin/logcat --print-info packet-rttpd.log
```

The output from the command is shown below:

```
Logcat: Log Type 'packet' Version 5 created by 'rttpd' in timezone 'Europe/London'
```

**Converting a binary packet log file to text**

Enter the command below:

```
../bin/logcat packet-rttpd.log
```

The output from the command is shown below (only the first few lines are shown):

```
2015/09/22-14:10:57.112 +0100: 0.0.0.0 < PEERINFO 102 transformer1 0  ACTIVE|CONTRIB Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, transformer-version=6.2.5-301153,
pipeline-module-loaded=true, pipeline-module-version=6.2.5-301153, format-module-loaded=true,
format-module-version=6.2.5-301153, autopub-module-loaded=true, autopub-module-version=6.2.5-301153,
cluster-module-loaded=true
2015/09/22-14:10:57.112 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:10:57.113 +0100: 127.0.0.1 > HEARTBEAT transformer1 15 0
2015/09/22-14:10:57.114 +0100: 127.0.0.1 < HEARTBEAT transformer1 15 0
2015/09/22-14:11:00.297 +0100: 0.0.0.0 < PEERINFO DemoDataSource1 DemoDataSource-johne-960 0 DemoDataSource1 ACTIVE|CONTRIB
Handshake data: datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > PEERINFO 101 liberator1 0  BROADCAST Handshake data:
datasrc-version=6.2.5-301125, hostname=johne-960, platform=i686-pc-win32, technology=DSDK, liberator-version=6.2.6-301198,
liberator-auth-module=openauth
2015/09/22-14:11:00.297 +0100: 127.0.0.1 > HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.298 +0100: 127.0.0.1 < HEARTBEAT DemoDataSource1 15 0
2015/09/22-14:11:00.392 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 1 8240 /EXAMPLES/BROADCAST/ABC 222 3
-11001=Value 0.243902 -11002=Value 0.149254 -11003=Value 0.294118
2015/09/22-14:11:00.396 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 2 8240 /EXAMPLES/BROADCAST/DEF 222 3
-11001=Value 1.#INF00 -11002=Value 0.144928 -11003=Value 0.416667
2015/09/22-14:11:00.397 +0100: 127.0.0.1 < DATAUPDATE2 DemoDataSource1 3 8240 /EXAMPLES/BROADCAST/GHI 222 3
-11001=Value 0.128205 -11002=Value 0.172414 -11003=Value 0.161290
...
```

The log entry timestamps in the example above have a resolution of one millisecond. DataSource 7.1 applications timestamp packet log entries to a resolution of one microsecond, and logcat 7.1 outputs log entry timestamps with a longer fractional second component (6 digits). For example: `2015/09/22-14:10:57.112578 +0100`

## Displaying the last part of the log file in real time

You can use the Linux `tail` command with `logcat` to display the last part of the log file and update the screen when more data appears.

**Example**

```
tail -f packet-rttpd.log | ../bin/logcat -t packet
```

## Splitting packet logs into smaller files

Packet logs can become very large. To make it easier to view a large packet log, you can split it into smaller files first, using the Linux `split` command.

**Example**

```
split -b 10m packet.log
```

This example splits the log into separate files of 10 megabytes each.

**🔥 CAUTION**\
The `split` command can produce a lot of files if you’re not careful with the size parameter.

When you use `logcat` to view one of the split files (other than the first one), you must tell it that the file is a packet log, because the file’s header will now be missing:

```
logcat -t packet packet-xab
```

---

**See also:**

* How can I... [Disable packet logging](datasource-disable-packet-logs-for-c-datasource.md)