Files
5fe06ac4d0 docs(P13d): reconcile Manage content drift — ABAC updates, telemetry, mmctl, health checks (#37482)
Ports content drift from the legacy Sphinx docs repo (mattermost/docs)
since the P13 migration fork point into the corresponding Administration
Guide > Manage MDX pages:

- ABAC v11.7/v11.8: public channel membership policies (advisory vs.
  hard-gate behavior), channel-level permission policies, policy
  simulation, and team-scoped membership policies in Team Settings
  (abac-channel-access-rules, abac-system-wide-policies,
  attribute-based-access-control)
- User Attributes: separate Display Name/Attribute name fields,
  backfill note, terminology updates
- Content flagging: quarantined message report generation and post
  deletion report documentation
- Autotranslation: translation target language and language-pair
  clarifications
- Generating support packet: sanitize typo fixes, removed stale
  ElasticsearchSettings.ClientKey reference
- Installing license key: Site Statistics -> System Statistics rename
- Team/channel members: SetChannelMembers bulk API note
- Telemetry: removed Rudder/Segment-specific reporting details
  following the telemetry stack migration
- Statistics: Site Statistics -> System Statistics rename
- Health check probes: corrected Go import path example
- mmctl: documented --workers flag for import process, renamed
  Custom Profile Attribute -> User Attribute in cpa command docs
- Logging: removed unused audit log file rotation settings from the
  Cloud defaults note

Co-authored-by: Hanzo Dev <[email protected]>
2026-07-14 11:42:37 +03:00

1212 lines
44 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: "Mattermost logging"
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
<PlanAvailability slug="all-commercial" />
<Important>
**From Mattermost v11, notification log settings have been consolidated into the standard console logs and mattermost.log file**. You can no longer disable notification logging without using advanced logging settings, as the main log level setting now controls both server and notification logs.
You can use the `AdvancedLoggingJSON` configuration with discrete notification log levels: `NotificationError`, `NotificationWarn`, `NotificationInfo`, `NotificationDebug`, and `NotificationTrace` to split notification logs into separate files and reduce troubleshooting noise. See the section below on [advanced logging](#advanced-logging) for details.
</Important>
Mattermost provides independent logging systems that can be configured separately with separate log files and rotation policies to meet different operational and compliance needs:
<table>
<colgroup>
<col style={{width: '45%'}} />
<col style={{width: '25%'}} />
<col style={{width: '30%'}} />
</colgroup>
<thead>
<tr>
<th>Logging Type &amp; Capture</th>
<th>Production Recommendation</th>
<th>Configuration Priority</th>
</tr>
</thead>
<tbody>
<tr>
<td><p><strong>Log Settings</strong></p><p>All general Mattermost server operations, errors, startup/initialization, API calls, and system events. From v11.0, also includes notification subsystem events.</p></td>
<td>Always enabled. Console: INFO, File: INFO</td>
<td><strong>High</strong> - Essential for operations</td>
</tr>
<tr>
<td><p><strong>Audit Log Settings</strong></p><p>Security and compliance events, user actions, API access, authentication events, and administrative changes.</p></td>
<td>Enable if compliance is required</td>
<td><strong>Medium</strong> - Based on regulatory needs</td>
</tr>
<tr>
<td><p><strong>Notification Log Settings</strong> <em>(applicable to v10.12 and earlier releases only)</em></p><p>Notification subsystem events, push notifications, email delivery, mobile notification processing.</p></td>
<td>Enable for notification troubleshooting</td>
<td><strong>Low</strong> - Enable when debugging issues</td>
</tr>
</tbody>
</table>
By default, all Mattermost plans write logs to both the console and to the `mattermost.log` file in a machine-readable JSON format for log aggregation tools. Mattermost Enterprise and Professional customers can additionally log directly to syslog and TCP socket destination targets. Audit logging is designed to be asynchronous to minimize performance impact.
System admins can customize the following logging options based on your business practices and needs by going to **System Console \> Environment \> Logging** or by editing the `config.json` file directly.
<Note>
From Mattermost v11.3, AdvancedLoggingJSON configuration includes enhanced validation that enforces proper separation between standard and audit log levels:
- Standard logging configurations reject audit-specific log levels (`audit-api`, `audit-content`, `audit-permissions`, `audit-cli`)
- Audit logging configurations reject standard log levels (`debug`, `info`, `warn`, `error`, `fatal`, `panic`, etc.)
- Configuration validation occurs at startup and when updating settings, preventing invalid log level combinations
</Note>
## Console logs
Console logs feature verbose debug level log messages for general activities that are written to the console using the standard output stream (stdout). You can customize console logs for general activities.
From Mattermost v11.0, notification logs are automatically included in the main console logs. See the [Logging configuration settings](/administration-guide/configure/environment-configuration-settings#output-logs-to-console) for details.
## File logs
File logs feature info level log messages for general activities including errors and information around startup, and initialization and webhook debug messages. The file is stored in `./logs/mattermost.log`, rotated at 100 MB, and archived to a separate file in the same directory. You can customize file logs for general activities.
From Mattermost v11.0, notification logs are automatically included in the main `mattermost.log` file. See the [Logging configuration settings](/administration-guide/configure/environment-configuration-settings#output-logs-to-file) for details.
<Tip>
Download the `mattermost.log` file locally by going to **System Console \> Reporting \> Server Logs**, and selecting **Download Logs**.
</Tip>
You can optionally output log records to any combination of [console](#console-target-configuration-options), [local file](#file-target-configuration-options), [syslog](#syslog-target-configuration-options), and [TCP socket](#tcp-target-configuration-options) targets, each featuring additional customization. See [Advanced Logging](#advanced-logging) for details.
## Define logging output
Define logging output for general activities in JSON format in the System Console by going to **Environment \> Logging \> Advanced Logging** or by editing the `config.json` file directly.
You can use the sample JSON below as a starting point.
<Tabs>
<TabItem value="v11-or-later" label="v11 or later">
``` JSON
{
"console1": {
"type": "console",
"format": "json",
"levels": [
{"id": 5, "name": "debug", "stacktrace": false},
{"id": 4, "name": "info", "stacktrace": false, "color": 36},
{"id": 3, "name": "warn", "stacktrace": false},
{"id": 2, "name": "error", "stacktrace": true, "color": 31},
{"id": 1, "name": "fatal", "stacktrace": true, "color": 31},
{"id": 0, "name": "panic", "stacktrace": true, "color": 31},
{"id": 10, "name": "stdlog", "stacktrace": false},
{"id": 300, "name": "NotificationError", "stacktrace": true, "color": 31},
{"id": 301, "name": "NotificationWarn", "stacktrace": false},
{"id": 302, "name": "NotificationInfo", "stacktrace": false, "color": 36},
{"id": 303, "name": "NotificationDebug", "stacktrace": false},
{"id": 304, "name": "NotificationTrace", "stacktrace": false}
],
"options": {
"out": "stdout"
},
"maxqueuesize": 1000
},
"file1": {
"type": "file",
"format": "json",
"levels": [
{"id": 5, "name": "debug", "stacktrace": false},
{"id": 4, "name": "info", "stacktrace": false},
{"id": 3, "name": "warn", "stacktrace": false},
{"id": 2, "name": "error", "stacktrace": true},
{"id": 1, "name": "fatal", "stacktrace": true},
{"id": 0, "name": "panic", "stacktrace": true},
{"id": 300, "name": "NotificationError", "stacktrace": true},
{"id": 301, "name": "NotificationWarn", "stacktrace": false},
{"id": 302, "name": "NotificationInfo", "stacktrace": false},
{"id": 303, "name": "NotificationDebug", "stacktrace": false},
{"id": 304, "name": "NotificationTrace", "stacktrace": false}
],
"options": {
"filename": "mattermost_logging.log",
"max_size": 100,
"max_age": 1,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
},
"file2": {
"type": "file",
"format": "json",
"levels": [
{"id": 2, "name": "error", "stacktrace": true},
{"id": 1, "name": "fatal", "stacktrace": true},
{"id": 0, "name": "panic", "stacktrace": true},
{"id": 300, "name": "NotificationError", "stacktrace": true},
{"id": 301, "name": "NotificationWarn", "stacktrace": false}
],
"options": {
"filename": "mattermost_logging_errors.log",
"max_size": 100,
"max_age": 30,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
}
}
```
</TabItem>
<TabItem value="v10-12-or-earlier" label="v10.12 or earlier">
``` JSON
{
"console1": {
"type": "console",
"format": "json",
"levels": [
{"id": 5, "name": "debug", "stacktrace": false},
{"id": 4, "name": "info", "stacktrace": false, "color": 36},
{"id": 3, "name": "warn", "stacktrace": false},
{"id": 2, "name": "error", "stacktrace": true, "color": 31},
{"id": 1, "name": "fatal", "stacktrace": true, "color": 31},
{"id": 0, "name": "panic", "stacktrace": true, "color": 31},
{"id": 10, "name": "stdlog", "stacktrace": false}
],
"options": {
"out": "stdout"
},
"maxqueuesize": 1000
},
"file1": {
"type": "file",
"format": "json",
"levels": [
{"id": 5, "name": "debug", "stacktrace": false},
{"id": 4, "name": "info", "stacktrace": false},
{"id": 3, "name": "warn", "stacktrace": false},
{"id": 2, "name": "error", "stacktrace": true},
{"id": 1, "name": "fatal", "stacktrace": true},
{"id": 0, "name": "panic", "stacktrace": true}
],
"options": {
"filename": "mattermost_logging.log",
"max_size": 100,
"max_age": 1,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
},
"file2": {
"type": "file",
"format": "json",
"levels": [
{"id": 2, "name": "error", "stacktrace": true},
{"id": 1, "name": "fatal", "stacktrace": true},
{"id": 0, "name": "panic", "stacktrace": true}
],
"options": {
"filename": "mattermost_logging_errors.log",
"max_size": 100,
"max_age": 30,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
}
}
```
</TabItem>
</Tabs>
------------------------------------------------------------------------------------------------------------------------
## Audit logging
<PlanAvailability slug="ent-plus" />
By default, Mattermost doesn't write audit logs locally to a file on the server, and the ability to enable audit logging in Mattermost is currently in [Beta](/administration-guide/manage/feature-labels#beta).
You can enable and customize advanced audit logging in Mattermost to record activities and events performed within Mattermost, such as user access to the Mattermost REST API or mmctl. Audit logs are recorded asynchronously to reduce latency to the caller, and are stored separately from general logging. During short spans of inability to write to targets, the audit records buffer in memory with a configurable maximum record cap. Based on typical audit record volumes, it could take many minutes to fill the buffer. After that, the records are dropped, and the record drop event is logged.
<Note>
From Mattermost v7.2, audit logging is a breaking change from previous releases in cases where customers looking to parse previous audit logs with the new format. The format and content of an audit log record has changed to become standardized for all events using a [standard JSON schema](/administration-guide/comply/embedded-json-audit-log-schema). Existing tools which ingest or parse audit log records may need to be modified.
From Mattermost v9.3, you can enable and customize advanced logging for AD/LDAP events separately from other logging.
</Note>
<Tabs>
<TabItem value="self-hosted-deployments" label="Self-Hosted Deployments">
Go to **System Console \> Compliance \> Audit Logging** to customize audit logging. You can use the sample JSON below as a starting point.
You can customize console logs for [general](/administration-guide/configure/environment-configuration-settings#log-settings) activities. From v11.0, [notification](/administration-guide/configure/environment-configuration-settings#notification-logging) logs are automatically included unless advanced logging is configured.
Additionally, you can also output audit log records to any combination of [console](#console-target-configuration-options), [local file](#file-target-configuration-options), [syslog](#syslog-target-configuration-options), and [TCP socket](#tcp-target-configuration-options) targets, each featuring additional customization. See [Advanced Logging](#advanced-logging) below for details.
</TabItem>
<TabItem value="cloud-deployments" label="Cloud Deployments">
From Mattermost v10.11, go to **System Console \> Compliance \> Audit Logging** to customize audit logging. You can use the sample JSON below as a starting point. Prior to Mattermost v10.11, customize audit logging by going to **System Console \> Experimental \> Features \> Audit Logging**.
<Note>
- From Mattermost v10.11, Cloud deployments include certificate-based audit logging capabilities not available within self-hosted deployments.
- Cloud-based deployments use the following self-hosted audit logging default values: `FileEnabled: false`
- Cloud deployments can't configure local file-based audit logging, and all file-related settings are hidden.
</Note>
</TabItem>
</Tabs>
------------------------------------------------------------------------------------------------------------------------
## Log path restrictions
From Mattermost v11.4, log file paths are validated to ensure they remain within a designated logging root directory. This security enhancement prevents log files from being written to or read from unauthorized locations on the file system.
### Configure the logging root directory
From Mattermost v11.4, use the `MM_LOG_PATH` environment variable to define the root directory for all log files:
``` sh
export MM_LOG_PATH=/var/log/mattermost
```
If `MM_LOG_PATH` isn't set, Mattermost uses the default `logs` directory relative to the Mattermost binary location.
### Log path validation
All log file paths configured via the following settings are validated:
- `LogSettings.FileLocation` - main [server log file location](/administration-guide/configure/environment-configuration-settings#file-log-directory)
- `LogSettings.AdvancedLoggingJSON` - all [file targets in advanced logging configuration](/administration-guide/configure/environment-configuration-settings#output-logs-to-multiple-targets)
- `ExperimentalAuditSettings.AdvancedLoggingJSON` - all [file targets in audit logging configuration](/administration-guide/configure/environment-configuration-settings#output-audit-logs-to-multiple-targets)
**How validation works**:
1. Paths are resolved to absolute paths
2. Symlinks are resolved to their actual locations
3. The resolved path is validated against the logging root directory
4. Paths outside the root directory generate error logs and are excluded from downloads
**Validation occurs**:
- When accessing log files via **System Console \> Reporting \> Server Logs**
- When generating support packets
- During configuration changes (warnings are logged for invalid paths)
When a log file path is outside the allowed root directory, Mattermost logs an error and excludes the file from support packet downloads: `"Blocked attempt to read log file outside allowed root"`. The error message includes the file path, configuration section, and validation failure details.
### Configuration requirements
**If using default logging**: No action required. Logs are stored in the default `logs` directory.
**If using custom log paths**: Ensure all log file paths in `AdvancedLoggingJSON` point to locations within your logging root directory.
**Valid configuration example**:
``` sh
export MM_LOG_PATH=/var/log/mattermost
```
In `config.json`:
``` JSON
{
"file1": {
"type": "file",
"format": "json",
"levels": [
{"id": 2, "name": "error", "stacktrace": true}
],
"options": {
"filename": "/var/log/mattermost/errors.log",
"max_size": 100,
"max_age": 7,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
}
}
```
This configuration is valid because `/var/log/mattermost/errors.log` is within the `/var/log/mattermost` root.
**Invalid configuration example**:
If `MM_LOG_PATH=/var/log/mattermost`, this configuration fails:
``` JSON
{
"file1": {
"type": "file",
"options": {
"filename": "/tmp/logs/app.log"
}
}
}
```
This fails because `/tmp/logs/app.log` is outside the `/var/log/mattermost` root directory.
See the [troubleshooting](/deployment-guide/server/troubleshooting#log-files-not-accessible) documentation for troubleshooting steps if logs aren't appearing in the System Console or support packets due to log path issues.
------------------------------------------------------------------------------------------------------------------------
## Advanced logging
<PlanAvailability slug="entry-ent" />
System admins can output log and audit records for general and audit activities to any combination of [console](#console-target-configuration-options), [local file](#file-target-configuration-options), [syslog](#syslog-target-configuration-options), and [TCP socket](#tcp-target-configuration-options) targets.
<Tip>
- From Mattermost v11.0, notification activities are included in general logs by default, but can be separated using the new discrete notification log levels. Each output target features additional configuration options you can customize for your Mattermost deployment.
- From Mattermost v9.11, system admins can configure advanced logging JSON options using the `mmctl config set` command. See the [mmctl config set](/administration-guide/manage/mmctl-command-line-tool#mmctl-config-set) documentation for an example slash command.
- From Mattermost v9.3, system admins can configure advanced logging options in the System Console using multi-line JSON by going to **Environment \> Logging**.
- Alternatively, admins can configure advanced logging within the `AdvancedLoggingJSON` section of the `config.json` file using multi-line JSON or escaped JSON as a string.
- Mattermost Team Edition customers can output audit log records to the console or a file.
</Tip>
Advanced logging options can be configured to:
- Enable trace logging for AD/LDAP to troubleshoot authentication issues.
- Capture errors and panics in a separate, monitored file that triggers alerts.
- Capture production debug and error logs in a separate file with log file rotation to reproduce issues, while enforcing a cap on the amount of disk space the debug logs are allowed to use.
- Audit every API endpoint accessed during a user workflow.
### Define advanced log output
<Tabs>
<TabItem value="multi-line-json" label="Multi-line JSON">
In the example below, file output is written to `./logs/audit.log` in plain text and includes all audit log levels & events. Older logs are kept for 1 day, and up to a total of 10 backup log files are kept at a time. Logs are rotated using gzip when the maximum size of the log file reaches 500 MB. A maximum of 1000 audit records can be queued/buffered while writing to the file.
#### v11 or later
``` JSON
"AdvancedLoggingJSON": {
"file_1": {
"type": "file",
"format": "plain",
"format_options": {
"disable_level": false
},
"levels": [
{ "id": 100, "name": "audit-api" },
{ "id": 101, "name": "audit-content" },
{ "id": 102, "name": "audit-permissions" },
{ "id": 103, "name": "audit-cli" }
],
"options": {
"compress": true,
"filename": "./logs/audit.log",
"max_age": 1,
"max_backups": 10,
"max_size": 500
},
"maxqueuesize": 1000
},
"file_notifications": {
"type": "file",
"format": "json",
"levels": [
{ "id": 300, "name": "NotificationError" },
{ "id": 301, "name": "NotificationWarn" },
{ "id": 302, "name": "NotificationInfo" },
{ "id": 303, "name": "NotificationDebug" },
{ "id": 304, "name": "NotificationTrace" }
],
"options": {
"compress": true,
"filename": "./logs/notifications.log",
"max_age": 7,
"max_backups": 5,
"max_size": 200
},
"maxqueuesize": 1000
}
}
```
#### v10.12 or earlier
``` JSON
"AdvancedLoggingJSON": {
"file_1": {
"type": "file",
"format": "plain",
"format_options": {
"disable_level": false
},
"levels": [
{ "id": 100, "name": "audit-api" },
{ "id": 101, "name": "audit-content" },
{ "id": 102, "name": "audit-permissions" },
{ "id": 103, "name": "audit-cli" }
],
"options": {
"compress": true,
"filename": "./logs/audit.log",
"max_age": 1,
"max_backups": 10,
"max_size": 500
},
"maxqueuesize": 1000
}
}
```
</TabItem>
<TabItem value="filespec" label="Filespec">
Advanced logging configuration can be pointed to a filespec to another configuration file, rather than multi-line JSON, to keep the config.json file tidy:
``` JSON
"AdvancedLoggingJSON": "/path/to/audit_log_config.json"
```
The separate configuration file includes the multi-line JSON instead.
In the example below, the first output is written to the console in plain text and includes all audit log levels, events, and command outputs. A pipe `|` delimiter is placed between fields.
A second output is written to `./logs/audit.log` in plain text in a machine-readable JSON format and includes all audit log levels, events, and command outputs. Older logs are kept for 1 day, and up to a total of 10 backup log files are kept at a time. Logs are rotated using GZIP when the maximum size of the log file reaches 500 MB. A maximum of 1000 audit records can be queued/buffered while writing to the file.
Contents of `audit_log_config.json` file:
#### v11 or later
``` JSON
{
"sample-console": {
"type": "console",
"format": "plain",
"format_options": {
"delim": " | "
},
"levels": [
{"id": 100, "name": "audit-api"},
{"id": 101, "name": "audit-content"},
{"id": 102, "name": "audit-permissions"},
{"id": 103, "name": "audit-cli"}
],
"options": {
"out": "stdout"
},
"maxqueuesize": 1000
},
"sample-file": {
"type": "file",
"format": "json",
"levels": [
{"id": 100, "name": "audit-api"},
{"id": 101, "name": "audit-content"},
{"id": 102, "name": "audit-permissions"},
{"id": 103, "name": "audit-cli"}
],
"options": {
"filename": "./logs/audit.log",
"max_size": 500,
"max_age": 1,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
},
"notifications-file": {
"type": "file",
"format": "json",
"levels": [
{"id": 300, "name": "NotificationError"},
{"id": 301, "name": "NotificationWarn"},
{"id": 302, "name": "NotificationInfo"},
{"id": 303, "name": "NotificationDebug"},
{"id": 304, "name": "NotificationTrace"}
],
"options": {
"filename": "./logs/notifications.log",
"max_size": 200,
"max_age": 7,
"max_backups": 5,
"compress": true
},
"maxqueuesize": 1000
}
}
```
#### v10.12 or earlier
``` JSON
{
"sample-console": {
"type": "console",
"format": "plain",
"format_options": {
"delim": " | "
},
"levels": [
{"id": 100, "name": "audit-api"},
{"id": 101, "name": "audit-content"},
{"id": 102, "name": "audit-permissions"},
{"id": 103, "name": "audit-cli"}
],
"options": {
"out": "stdout"
},
"maxqueuesize": 1000
},
"sample-file": {
"type": "file",
"format": "json",
"levels": [
{"id": 100, "name": "audit-api"},
{"id": 101, "name": "audit-content"},
{"id": 102, "name": "audit-permissions"},
{"id": 103, "name": "audit-cli"}
],
"options": {
"filename": "./logs/audit.log",
"max_size": 500,
"max_age": 1,
"max_backups": 10,
"compress": true
},
"maxqueuesize": 1000
}
}
```
</TabItem>
</Tabs>
### Specify destination targets
Log records can be sent to any combination of [console](#console-target-configuration-options), [local file](#file-target-configuration-options), [syslog](#syslog-target-configuration-options), and [TCP socket](#tcp-target-configuration-options) targets. Log targets have been chosen based on support for the vast majority of log aggregators and other log analysis tools, without needing additional software installed.
System admins can define multiple log targets to:
- Mirror log output to files and log aggregators for redundancy.
- Log certain entries to specific destinations. For example, all `audit-content` records can be routed to a different destination than the other levels.
- Admins can also temporarily disable log targets by setting its `type` to `none`.
### Configure format preferences
System admins can control log formatting per target by configuring the `format_options` section.
#### Plain log format configuration options
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '13%'}} />
<col style={{width: '6%'}} />
<col style={{width: '78%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>disable_timestamp</td>
<td>bool</td>
<td>Disables output of the timestamp. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_level</td>
<td>bool</td>
<td>Disables output of the level name. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_msg</td>
<td>bool</td>
<td>Disables output of the message text. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_fields</td>
<td>bool</td>
<td>Disables output of all fields. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disables_stacktrace</td>
<td>bool</td>
<td>Disables output of stack traces. Default is <code>false</code>.</td>
</tr>
<tr>
<td>enable_caller</td>
<td>string</td>
<td>Enables output of the file and line number that emitted a log record. Default is <code>false</code>.</td>
</tr>
<tr>
<td>delim</td>
<td>string</td>
<td>Delimiter placed between fields. Default is single space.</td>
</tr>
<tr>
<td>min_level_len</td>
<td>number</td>
<td>Minimum level name length. When level names are less than the minimum, level names are padded with spaces. Default is <code>0</code>.</td>
</tr>
<tr>
<td>min_msg_len</td>
<td>number</td>
<td>Minimum message length. When message text is less than the minimum, message text is padded with spaces. Default is <code>0</code>.</td>
</tr>
<tr>
<td>timestamp_format</td>
<td>string</td>
<td>Format for timestamps. Default is <a href="https://www.rfc-editor.org/rfc/rfc3339">RFC3339</a>.</td>
</tr>
<tr>
<td>line_end</td>
<td>string</td>
<td>Alternative end of line character(s). Default is <code>n</code>.</td>
</tr>
<tr>
<td>enable_color</td>
<td>bool</td>
<td>Enables color for targets that support color output. Default is <code>false</code>.</td>
</tr>
</tbody>
</table>
#### JSON log format configuration options
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '17%'}} />
<col style={{width: '8%'}} />
<col style={{width: '73%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>disable_timestamp</td>
<td>bool</td>
<td>Disables output of the timestamp. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_level</td>
<td>bool</td>
<td>Disables output of the log level display name. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_msg</td>
<td>bool</td>
<td>Disables output of the message text. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disable_fields</td>
<td>bool</td>
<td>Disables output of all fields. Default is <code>false</code>.</td>
</tr>
<tr>
<td>disables_stacktrace</td>
<td>bool</td>
<td>Disables output of stack traces. Default is <code>false</code>.</td>
</tr>
<tr>
<td>enable_caller</td>
<td>string</td>
<td>Enables output of the file and line number that emitted a log record. Default is <code>false</code>.</td>
</tr>
<tr>
<td>timestamp_format</td>
<td>string</td>
<td>Format for timestamps. Default is <a href="https://www.rfc-editor.org/rfc/rfc3339">RFC3339</a>.</td>
</tr>
</tbody>
</table>
#### GELF log format configuration options
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '13%'}} />
<col style={{width: '8%'}} />
<col style={{width: '77%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>hostname</td>
<td>string</td>
<td>Outputs a custom hostname in log records. If omitted, hostname is taken from the operating system.</td>
</tr>
<tr>
<td>enable_caller</td>
<td>string</td>
<td>Enables output of the file and line number that emitted a log record. Default is <code>false</code>.</td>
</tr>
</tbody>
</table>
### Configure log levels and events
<table style={{width: '72%'}}>
<colgroup>
<col style={{width: '10%'}} />
<col style={{width: '9%'}} />
<col style={{width: '51%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>id</td>
<td>number</td>
<td>Unique identifier of the log level.</td>
</tr>
<tr>
<td>name</td>
<td>string</td>
<td>Name of the log level.</td>
</tr>
<tr>
<td>stacktrace</td>
<td>bool</td>
<td>Outputs a stack trace. Default is <code>false</code>.</td>
</tr>
<tr>
<td>color</td>
<td>number</td>
<td><p>The ANSI color code used to output parts of the log record. Supported values include:</p><ul><li>Black: <code>30</code></li><li>Red: <code>31</code></li><li>Green: <code>32</code></li><li>Yellow: <code>33</code></li><li>Blue: <code>34</code></li><li>Magenta: <code>35</code></li><li>Cyan: <code>36</code></li><li>White: <code>37</code></li></ul></td>
</tr>
</tbody>
</table>
#### Log levels
The following log levels support audit logs:
<table style={{width: '88%'}}>
<colgroup>
<col style={{width: '7%'}} />
<col style={{width: '20%'}} />
<col style={{width: '60%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>ID</strong></td>
<td><strong>Name</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>100</td>
<td><code>audit-api</code></td>
<td>API events</td>
</tr>
<tr>
<td>101</td>
<td><code>audit-content</code></td>
<td>Content changes. This log level can generate considerably more records than the other audit log levels.</td>
</tr>
<tr>
<td>102</td>
<td><code>audit-permissions</code></td>
<td>Permission changes</td>
</tr>
<tr>
<td>103</td>
<td><code>audit-cli</code></td>
<td>CLI operations</td>
</tr>
</tbody>
</table>
The following log levels support application logs:
<table style={{width: '88%'}}>
<colgroup>
<col style={{width: '7%'}} />
<col style={{width: '20%'}} />
<col style={{width: '60%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>ID</strong></td>
<td><strong>Name</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>140</td>
<td><code>LDAPError</code></td>
<td>AD/LDAP authentication errors</td>
</tr>
<tr>
<td>141</td>
<td><code>LDAPWarn</code></td>
<td>AD/LDAP authentication warnings</td>
</tr>
<tr>
<td>142</td>
<td><code>LDAPInfo</code></td>
<td>AD/LDAP authentication information logs</td>
</tr>
<tr>
<td>143</td>
<td><code>LDAPDebug</code></td>
<td>AD/LDAP authentication debug logs</td>
</tr>
<tr>
<td>144</td>
<td><code>LDAPTrace</code></td>
<td>AD/LDAP authentication trace logs. Replaces <code>LdapSetings.trace</code> from Mattermost v9.3.</td>
</tr>
</tbody>
</table>
### Configure target-specific settings
#### Console target configuration options
Console targets can be either `stdout` or `stderr`.
- The standard output stream, `stout`, is typically used for command output that prints the results of a command to the user.
- The standard error stream, `sterr`, is typically used to print any errors that occur when a program is running.
#### File target configuration options
File targets support rotation and compression triggered by size and/or duration.
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '9%'}} />
<col style={{width: '7%'}} />
<col style={{width: '81%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>filename</td>
<td>string</td>
<td>Full path to the output file. From v11.4, should be within the directory specified by <code>MM_LOG_PATH</code>.</td>
</tr>
<tr>
<td>max_size</td>
<td>number</td>
<td>Maximum size, in megabytes (MB), the log file can grow before it gets rotated. Default is <code>100</code> MB.</td>
</tr>
<tr>
<td>max_age</td>
<td>number</td>
<td>Maximum number of days to retain old log files based on the timestamp encoded in the filename. Default is <code>0</code> which disables the removal of old log files.</td>
</tr>
<tr>
<td>max_backups</td>
<td>number</td>
<td>Maximum number of old log files to retain. Default is <code>0</code> which retains all old log files. <strong>Note</strong>: Configuring <code>max_age</code> can result in old log files being deleted regardless of this configuration value.</td>
</tr>
<tr>
<td>compress</td>
<td>bool</td>
<td>Compress rotated log files using <a href="https://www.gnu.org/software/gzip/">gzip</a>. Default is <code>false</code>.</td>
</tr>
</tbody>
</table>
#### Syslog target configuration options
<PlanAvailability slug="entry-ent" />
Syslog targets support local and remote syslog servers, with or without TLS transport. Syslog target support requires Mattermost Enterprise.
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '7%'}} />
<col style={{width: '7%'}} />
<col style={{width: '84%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>host</td>
<td>string</td>
<td>IP or domain name of the server receiving the log records.</td>
</tr>
<tr>
<td>port</td>
<td>number</td>
<td>Port number for the server receiving the log records.</td>
</tr>
<tr>
<td>tls</td>
<td>bool</td>
<td>Create a TLS connection to the server receiving the log records. Default is <code>false</code>.</td>
</tr>
<tr>
<td>cert</td>
<td>string</td>
<td>Path to a cert file (.pem) to be used when establishing a TLS connection to the server.</td>
</tr>
<tr>
<td>insecure</td>
<td>bool</td>
<td>Mattermost accepts any certificate presented by the server, and any host name in that certificate. Default is <code>false</code>. <strong>Note</strong>: Should only be used in testing environments, and shouldn’t be used in production environments.</td>
</tr>
<tr>
<td>tag</td>
<td>string</td>
<td>Syslog tag field.</td>
</tr>
</tbody>
</table>
#### TCP target configuration options
<PlanAvailability slug="entry-ent" />
The TCP socket targets can be configured with an IP address or domain name, port, and optional TLS certificate. TCP socket target support requires Mattermost Enterprise. You can [download a sample JSON file](https://github.com/mattermost/docs/blob/master/source/samples/sample-logger-config.json) of the configuration to use as a starting point.
<table style={{width: '99%'}}>
<colgroup>
<col style={{width: '7%'}} />
<col style={{width: '7%'}} />
<col style={{width: '84%'}} />
</colgroup>
<tbody>
<tr>
<td><strong>Key</strong></td>
<td><strong>Type</strong></td>
<td><strong>Description</strong></td>
</tr>
<tr>
<td>host</td>
<td>string</td>
<td>IP or domain name of the server receiving the log records.</td>
</tr>
<tr>
<td>port</td>
<td>number</td>
<td>Port number for the server receiving the log records.</td>
</tr>
<tr>
<td>tls</td>
<td>bool</td>
<td>Create a TLS connection to the server receiving the log records. Default is <code>false</code>.</td>
</tr>
<tr>
<td>cert</td>
<td>string</td>
<td>Path to a cert file (.pem) to be used when establishing a TLS connection to the server.</td>
</tr>
<tr>
<td>insecure</td>
<td>bool</td>
<td>Mattermost accepts any certificate presented by the server, and any host name in that certificate. Default is <code>false</code>. <strong>Note</strong>: Should only be used in testing environments, and shouldn’t be used in production environments.</td>
</tr>
</tbody>
</table>
------------------------------------------------------------------------------------------------------------------------
## Cluster job execution debug messages
<PlanAvailability slug="ent-plus" />
From Mattermost v11.4, debug-level log messages are available to help system admins understand cluster job execution behavior in [high availability](/administration-guide/scale/high-availability-cluster-based-deployment) deployments for specific Recurring Tasks.
These debug messages apply only to the following Recurring Tasks:
- Scheduled Posts
- Post Reminders
- DND Status Reset
<Important>
- These debug messages aren't available for other job types such as Elasticsearch indexing, SAML sync, LDAP sync, data retention, or compliance exports. The absence of these debug messages for other job types doesn't indicate a problem with job execution.
- These messages are only visible when debug logging is enabled. See [How do I enable debug logging?](#how-do-i-enable-debug-logging) for details.
- In a cluster deployment, only the leader node executes these Recurring Tasks. Non-leader nodes skip these specific jobs and log debug messages to indicate this is expected behavior.
</Important>
### Debug messages for job startup
When a non-leader node starts up, it skips initialization for these specific Recurring Tasks and logs one of the following debug messages:
- `Skipping scheduled posts job startup since this is not the leader node`
- `Skipping unset DND status job startup since this is not the leader node`
- `Skipping post reminder job startup since this is not the leader node`
These messages indicate normal operation. In a high availability cluster, these Recurring Tasks should only run on the leader node to prevent duplicate execution.
### Debug messages for leadership changes
When a node loses leadership status while these Recurring Tasks are running, it cancels the running tasks and logs:
- `This is no longer leader node. Cancelling the [job name] task`
Where `[job name]` is one of: `scheduled posts`, `DND status reset`, or `post reminder`.
This indicates the cluster is performing leader election correctly. The new leader node will take over execution of these Recurring Tasks.
### Troubleshooting with cluster job debug messages
If you're investigating why Recurring Tasks (Scheduled Posts, Post Reminders, or DND Status Reset) aren't running on a specific node:
1. Enable debug logging if not already enabled.
2. Check the logs for the debug messages listed above.
3. If you see these messages, the node is correctly identified as a non-leader and is behaving as expected.
4. These Recurring Tasks will be running on the leader node instead.
<Note>
These debug messages only apply to Recurring Tasks. For other job types (Elasticsearch indexing, SAML sync, LDAP sync, data retention, compliance exports), different logging mechanisms apply. The absence of these specific debug messages for other job types is expected and does not indicate a problem.
</Note>
For more information about leader election and cluster configuration, see [High Availability cluster-based deployment](/administration-guide/scale/high-availability-cluster-based-deployment).
------------------------------------------------------------------------------------------------------------------------
## Frequently asked questions
### How do I enable debug logging?
As a Mattermost system admin, go to **System Console \> Environment \> Logging \> File Log Level**, and set it to **DEBUG**. Then save your changes.
![A screenshot of the System Console page where Mattermost system admins can enable and disable debug logging.](/images/enable-debug-logging.png)
Debug logging can cause log files to expand substantially, and may adversely impact server performance. Keep an eye on your server logs, or only enable it temporarily, or in development environments, but not production enviornments.
### Does Mattermost have an audit log besides the system `auditd`?
Yes. See the [audit logging](#audit-logging) documentation for details.
### When syslog is configured as the target, does it contain the IP address of the emitter of the data (i.e., the Mattermost app node)?
Yes. That is a function of the syslog daemon (receiver). Typically, all the log lines are prefixed with a timestamp and the hostname of the node sending the data. For example, a log line starts with: `Nov 28 10:56:59 tower kernel: [1072437.431123] ....`, where `tower` is the hostname of the server that generated the log line.
### Once audit logs are enabled, can audit logging track events where an admin disables or modifies the audit log settings?
Yes, though it depends on how audit logs are configured. Audit log config can be specified via REST API, mmctl, System Console, file on disk, and using environment variables. When changes are made via the REST API or System Console, there is an audit record. However, the Mattermost server can't capture changes to a configuration file or via environment variables.
### Do server logs show any information about a service or similar stopping/updating when auditing would be disabled?
Yes. When updating the audit log configuration via REST API, mmctl, or System Console, the last event of the audit log should be about the admin user updating the config of the server, which helps your security team identify which actions took place in the system and by whom.
### How do I omit incoming webhook details from the logs?
See [enable-webhook-debugging](/administration-guide/configure/environment-configuration-settings#enable-webhook-debugging)
### How do I adjust the maximum log field size?
See [maximum-field-size](/administration-guide/configure/environment-configuration-settings#maximum-field-size)
### How can I configure Advanced logging via environment variables?
The `MM_LOGSETTINGS_ADVANCEDLOGGINGJSON` environment variable is used to configure Advanced logging . You can use `jq` to generate the JSON payload, e.g.
``` sh
export MM_LOGSETTINGS_ADVANCEDLOGGINGJSON=$(jq -n -c '{
"console1": {
"Type": "console",
"Format": "json",
"Levels": [
{"ID": 5, "Name": "debug", "Stacktrace": false},
{"ID": 4, "Name": "info", "Stacktrace": false, "color": 36},
{"ID": 3, "Name": "warn", "Stacktrace": false},
{"ID": 2, "Name": "error", "Stacktrace": true, "color": 31},
{"ID": 1, "Name": "fatal", "Stacktrace": true, "color": 31},
{"ID": 0, "Name": "panic", "Stacktrace": true, "color": 31},
{"ID": 10, "Name": "stdlog", "Stacktrace": false}
],
"Options": {
"Out": "stdout"
},
"MaxQueueSize": 1000
}
}')
```
### How can I turn on trace logging for LDAP?
Please use the following JSON configuration as as starting point to enable trace logging for LDAP:
``` JSON
{
"ldap-file": {
"type": "file",
"format": "plain",
"levels": [
{
"id": 144,
"name": "LDAPTrace"
},
{
"id": 143,
"name": "LDAPDebug"
},
{
"id": 142,
"name": "LDAPInfo"
},
{
"id": 141,
"name": "LDAPWarn",
"stacktrace": true
},
{
"id": 140,
"name": "LDAPError",
"stacktrace": true
}
],
"options": {
"filename": "./logs/ldap.log",
"max_size": 100,
"max_age": 14,
"max_backups": 3,
"compress": false
},
"maxqueuesize": 1000
}
}
```
### How do I configure centralized logging for Mattermost?
Mattermost recommends centralized logging using Grafana Loki and the OpenTelemetry Collector. See the [Deploy Grafana Loki for centralized logging](/administration-guide/scale/deploy-grafana-loki-for-centralized-logging) guide for details on how to set up log aggregation, useful LogQL queries, and Grafana dashboards.