mirror of
https://github.com/hanzoai/team.git
synced 2026-09-30 01:57:35 +00:00
b5f007d35e
* docs: resolve cross-file anchor mismatches from RST-to-MDX migration
The Sphinx-to-Docusaurus migration converted `:ref:`/`{ref}` links using
naive text-slugification, so anchors that pointed to headings in a
different file (e.g. release-summary tables linking into their
respective changelog pages) were emitted as same-page fragments and
silently resolved to nothing.
Built a label registry from the original Sphinx `.. _label:` / `(label)=`
targets to recover the true source file for each cross-reference, then
rewrote the affected links to point at the correct file + heading
anchor. Fixes 282 broken cross-file anchors, primarily in the mobile/
desktop/server release-summary pages, plus assorted same-page anchor
mismatches caught by re-auditing after the prior admonition/anchor fix.
* docs: manually resolve remaining anchor mismatches after cross-file audit
Hand-verified fixes for anchors that fuzzy-matching alone couldn't
safely resolve (confirmed against actual heading text/context, and in a
few cases against the original RST source, before applying):
- Typos and dropped words carried over from the original Sphinx docs
(e.g. "bot-acocunts", "auto-follow-all-new-channel-threads" ->
"follow-all-new-channel-threads").
- Sphinx auto-generated duplicate-heading IDs (#id1/#id3/#id4) resolved
to their real target headings.
- Numbered-step TOC links in enterprise-roll-out-checklist.mdx that
dropped the leading "N. " used in the actual heading slugs.
- Push-notification docs pointing at the wrong file entirely
(environment-configuration-settings instead of
push-notification-server-configuration-settings).
- SAML "saml-enterprise" label resolved to its true cross-file target
(authentication-configuration-settings#saml-2-0), recovered from the
original RST label definition that the automated registry missed
because it sat after an RST transition marker.
- sso-saml-technical references converted from broken same-page/
malformed-path anchors to correct page links.
* docs: fix SAML 2.0 anchor slug (saml-2-0 -> saml-20)
github-slugger strips periods rather than hyphenating them, so "SAML
2.0" slugifies to saml-20, not saml-2-0 as previously fixed.
* docs: fix broken path links found during docs/main migration cleanup
* docs: convert div.tab blocks to Docusaurus Tabs/TabItem components
* docs: restore missing Kubernetes deployment architecture diagrams
The RST-to-MDX migration never copied over the AWS/Azure/Oracle
architecture diagrams referenced by the deployment-architecture-at-scale
page, leaving broken images in each Tabs/TabItem.
* docs: fix 7 anchor mismatches from the fragment/heading audit
Covers three root causes from the anchor mismatch breakdown:
- Fragment repeated the target page's own title instead of a real
heading, so the link is simplified to point at the bare page.
- Heading exists verbatim but lives on a sibling settings page
(experimental-configuration-settings, push-notification-server-
configuration-settings) rather than environment-configuration-settings.
- A page path was mistakenly pasted into a same-page #fragment instead
of being used as a real cross-page link.
* docs: fix 8 anchor slugs that were close but didn't match the real heading
Repoints links whose fragment was a plausible guess (typo, reworded
heading, or wrong section) at the actual auto-generated slug of the
target heading, confirmed by inspecting each target page directly.
* docs: merge split Tabs blocks so tabs render horizontally
The div.tab conversion script split some sibling tab groups into
separate single-item Tabs blocks whenever a stray blockquote marker
sat between them, causing them to render stacked instead of as one
horizontal tab bar. Merges the MySQL/PostgreSQL and Team/Enterprise
Edition tab groups back into single Tabs blocks.
* docs: remove Azure tab and restore Kubernetes deploy content
The Mattermost Operator, Azure, and Oracle platform tabs on the
Kubernetes deployment guide were split across two separate Tabs
blocks (rendering stacked instead of as one tab bar) and all three
tab bodies were empty because the migration never flattened the
original RST include directives. Removes the unsupported Azure tab
and restores the Mattermost Operator and Oracle content from the
pre-migration RST sources as a single merged Tabs block.
* addresscoderabbitreview
* docs: unify Linux/Kubernetes/Containers deploy pages and fix FIPS/STIG placement
Kubernetes and Containers duplicated full install content across a top-level
page (Tabs) and orphaned sidebar subpages that nothing linked to. Reworked
both to follow the Linux page's pattern: shared content on the landing page,
platform-specific instructions live once in their own subpage.
Also removed configure-fips-at-install-time.mdx, which claimed FIPS is
available for native Linux installs — FIPS/STIG in Mattermost is a container
image property (Chainguard-based build), not available outside Docker/
Kubernetes. Consolidated all FIPS/STIG references onto
containers/fips-stig.mdx and pointed the Mattermost Operator subpage (not
Oracle OKE, which doesn't expose image selection) at it.
* docs: fix review findings — image alt text, Important callout markup, typo
- Replace generic alt="image" on scaling diagrams with descriptive text
identifying cloud provider and user scale.
- Convert raw div.important markup in deploy-containers.mdx to the
repo's <Important> component, matching usage elsewhere in the file.
- Fix "Matermost" typo in manage-your-display-options.mdx.
---------
Co-authored-by: Hanzo Dev <[email protected]>
225 lines
15 KiB
Plaintext
225 lines
15 KiB
Plaintext
---
|
|
title: "Compliance export"
|
|
---
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
|
|
<PlanAvailability slug="ent-plus" />
|
|
|
|
Mattermost Enterprise customers can archive history or transfer message data to third-party systems for auditing and compliance purposes with compliance exports. Supported integrations include [Smarsh (Actiance) Vantage](#actiance-xml), [Global Relay](#global-relay-eml), and [Proofpoint](#proofpoint).
|
|
|
|
From Mattermost v10.5, compliance exports include performance improvements for large daily data sets with changes affecting output formats, system performance, and logic. Compliance exports provide compliance teams complete information to reconstruct the state of a channel, and to determine who had visibility on an initial message, or when the message was edited or deleted. Compliance teams can track a message by its MessageId as it is edited or deleted, and across batches and exports periods.
|
|
|
|
## Overview
|
|
|
|
Compliance exports are produced from the System Console, and contain all messages including:
|
|
|
|
- Messages sent in direct message channels
|
|
- File uploads
|
|
- Posts from plugins
|
|
- Posts from bots/webhooks
|
|
|
|
Exports include information on channel member history at the time the message was posted.
|
|
|
|
## Set up guide
|
|
|
|
Use the following guides to configure exports for [CSV](#csv), [Smarsh / Actiance XML](#actiance-xml), [Global Relay EML](#global-relay-eml), or [Proofpoint](#proofpoint).
|
|
|
|
<Note>
|
|
|
|
- For self-hosted deployments, compliance exports are written to the `exports` subdirectory of the configured filestore in the chosen format. This will either be in the [Local Storage directory](/administration-guide/configure/environment-configuration-settings#file-storage) or the Mattermost S3 bucket if S3 storage is configured.
|
|
- Alternatively, you can specify an alternate filestore target and generate an S3 presigned URL for compliance exports. See the [dedicated export filestore target](/administration-guide/configure/environment-configuration-settings#enable-dedicated-export-filestore-target) configuration settings documentation for details.
|
|
- Compliance exports don't contain posts sent before the feature was enabled. For self-hosted deployments, you can export past history via the `export` [command line tool](/administration-guide/manage/mmctl-command-line-tool).
|
|
|
|
</Note>
|
|
|
|
### CSV
|
|
|
|
1. Go to **System Console \> Compliance \> Compliance Export**.
|
|
2. Set **Enable Compliance Exports** to **true**.
|
|
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form `HH:MM`. Choose a time when fewer people are using your system.
|
|
4. Set the export file format to **CSV**.
|
|
5. Select **Save**.
|
|
|
|
<Tabs>
|
|
<TabItem value="from-mattermost-v10-5" label="From Mattermost v10.5">
|
|
|
|
You can review export job status in the System Console.
|
|
|
|
When the daily compliance export job is finished, a parent directory is created named based on when the export was started and the `startTimestamp` and `endTimestamp` of the export, e.g, `administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075`. That parent directory contains 1 zip file for each batch, named based on the batch number and the start and end timestamps of the messages in that batch, e.g, `batch001-1723105062492-1723106622163.zip`. Each zip file contains the same information available in previous Mattermost server releases.
|
|
|
|
Working from the same example above, the directory would look like this:
|
|
|
|
``` bash
|
|
administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075
|
|
├── batch001-1723105062492-1723106622163.zip
|
|
├── batch002-1723106622163-1723108196005.zip
|
|
└── batch003-1723108196005-1723109100075.zip
|
|
```
|
|
|
|
And each batch would look like this:
|
|
|
|
``` bash
|
|
batch001-1723105062492-1723106622163.zip
|
|
├── files
|
|
├── metadata.json
|
|
└── actiance_export.xml
|
|
```
|
|
|
|
## Updated CSV export fields
|
|
|
|
**Post Creation Time** is always the `CreateAt` for messages and attachments, or `JoinTime` and `LeaveTime` for participant join and leave events, respectively.
|
|
|
|
- **Update Time** indicates that the message has been updated, and this is the `updateAt` time.
|
|
- **Updated Type** helps differentiate what kind of update it was as one of the following:
|
|
- **EditedNewMsg** indicates that the message has been edited, and this is the new message (post-edit) content.
|
|
- **EditedOriginalMsg** indicates that the message has been edited, and this the original message (pre-edit) content. This message will have another field `EditedNewMsgId`, which is the Id of the message which holds the post-edited message contents.
|
|
- **UpdatedNoMsgChange** indicates that message's content hasn't changed, but the post was updated for some reason, such as reaction, replied-to, a reply was edited, or a reply was deleted.
|
|
- **Deleted** indicates that this message was deleted.
|
|
- **FileDeleted** indicates that this message is recording that a file was deleted.
|
|
|
|
</TabItem>
|
|
<TabItem value="prior-to-mattermost-v10-5" label="Prior to Mattermost v10.5">
|
|
|
|
The daily compliance export job creates a `.zip` file with a unique job identifier of all messages posted in the last 24 hours. You can unzip the file to easily transform the default `.csv` format into a desired format for your third-party archive system.
|
|
|
|
For a sample CSV output, [download a CSV export file here](https://github.com/mattermost/docs/blob/master/source/samples/csv_export.zip).
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
|
|
### Actiance XML
|
|
|
|
Actiance XML is the supported format for the 'Smarsh Vantage product \[https://central.smarsh.com/s/product/vantage\](https://central.smarsh.com/s/product/vantage\)\`\_.
|
|
|
|
1. Go to **System Console \> Compliance \> Compliance Export**.
|
|
2. Set **Enable Compliance Exports** to **true**.
|
|
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
|
4. Set the export file format to **Actiance XML**.
|
|
5. Select **Save**.
|
|
|
|
<Note>
|
|
|
|
In Actiance XML exports, channel type is prepended to the channel names.
|
|
|
|
</Note>
|
|
|
|
<Tabs>
|
|
<TabItem value="from-mattermost-v10-5" label="From Mattermost v10.5">
|
|
|
|
You can review export job status in the System Console. Once you've selected Actiance XML as your file format, you can set up an integration with Actiance Vantage archive system.
|
|
|
|
When the daily compliance export job is finished, a parent directory is created named based on when the export was started and the `startTimestamp` and `endTimestamp` of the export, e.g, `administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075`. That parent directory contains 1 zip file for each batch, named based on the batch number and the start and end timestamps of the messages in that batch, e.g, `batch001-1723105062492-1723106622163.zip`. Each zip file contains the same information available in previous Mattermost server releases.
|
|
|
|
Working from the same example above, the directory would look like this:
|
|
|
|
``` bash
|
|
administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075
|
|
├── batch001-1723105062492-1723106622163.zip
|
|
├── batch002-1723106622163-1723108196005.zip
|
|
└── batch003-1723108196005-1723109100075.zip
|
|
```
|
|
|
|
And each batch would look like this:
|
|
|
|
``` bash
|
|
batch001-1723105062492-1723106622163.zip
|
|
├── 20240808
|
|
└── actiance_export.xml
|
|
```
|
|
|
|
## Updated Actiance XML export fields
|
|
|
|
If an XML field is empty, it won't be exported. This is a change from previous Mattermost releases, where empty XML nodes were exported.
|
|
|
|
- `MessageId` is the unique `messageId`.
|
|
- `DateTimeUTC` is always the post's `CreateAt` time.
|
|
- `UpdatedDateTimeUTC` indicates that the message has been updated, and this is the `updateAt` time.
|
|
- `UpdatedType` helps differentiate what kind of update it was, including:
|
|
- `EditedNewMsg` indicates that this message has been edited, and this is the new message (post-edit) content.
|
|
- `EditedOriginalMsg` indicates that this message has been edited, and this the original message (pre-edit) content. This message will have another field `EditedNewMsgId`, which is the Id of the message which holds the post-edited message contents.
|
|
- `UpdatedNoMsgChange` indicates that this message's content hasn't changed, but the post was updated for some reason, such as a reaction, replied-to, a reply was edited, or a reply was deleted.
|
|
- `Deleted` indicates that the message was deleted.
|
|
- `FileDeleted` indicates that the message is recording that a file was deleted.
|
|
|
|
</TabItem>
|
|
<TabItem value="prior-to-mattermost-v10-5" label="Prior to Mattermost v10.5">
|
|
|
|
The daily compliance export job creates a `.zip` file with a unique job identifier of all messages posted in the last 24 hours. Once you've selected Actiance XML as your file format, you can set up an integration with Actiance Vantage archive system. For a sample Actiance output, [download an Actiance XML export file here](https://github.com/mattermost/docs/blob/master/source/samples/actiance_export.xml).
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
|
|
### Global Relay EML
|
|
|
|
For more information on Global Relay archive system, visit [their website](https://www.globalrelay.com/).
|
|
|
|
1. Go to **System Console \> Compliance \> Compliance Export**.
|
|
2. Set **Enable Compliance Export** to **true**.
|
|
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
|
4. Set the **Export Format** to **GlobalRelay EML**.
|
|
5. Select **A9/Type 9**, **A10/Type 10**, or **Custom** for the **Global Relay Customer Account**. This is the type of Global Relay customer account your organization has.
|
|
- For **A9/Type 9** and **A10/Type 10** types, set the **Global Relay SMTP username**, **Global Relay SMTP password**, and **Global Relay SMTP email address**, provided by Global Relay.
|
|
- For a **Custom** type, set the **Global Relay SMTP username**, **Global Relay SMTP password**, **Global Relay SMTP email address**, **SMTP Server Name**, and the **SMTP Server Port**, provided by Global Relay. **Custom** type can be used to integrate with Proofpoint.
|
|
6. Select **Save**.
|
|
|
|
<Note>
|
|
|
|
Messages larger than 250 MB will have their attachments removed because they are too large to send to Global Relay. An error is added to the server logs with id `global_relay_attachments_removed`. It includes the post ID the attachments were removed from, as well as the attachment IDs. A [ticket is queued to better handle large messages](https://mattermost.atlassian.net/browse/MM-10038).
|
|
|
|
</Note>
|
|
|
|
Once you've selected Global Relay EML as your file format, you can set up an integration with Global Relay archive system. For more information, see [Global Relay Archive](https://www.globalrelay.com/products/archive-data-compliance/).
|
|
|
|
### Proofpoint
|
|
|
|
1. Go to **System Console \> Compliance \> Compliance Export**.
|
|
2. Set **Enable Compliance Export** to **true**.
|
|
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
|
4. Set the **Export Format** to **GlobalRelay EML**.
|
|
5. Select **Custom** for the **Global Relay Customer Account** to integrate with Proofpoint.
|
|
6. Set the **SMTP username**, **SMTP password**, **SMTP email address**, **SMTP Server Name**, and the **SMTP Server Port**, provided by Proofpoint.
|
|
7. Select **Save**.
|
|
|
|
See the [Global Relay](#global-relay-eml) section for details on updated Global Relay export fields. Now you can set up an integration with the Proofpoint archive system. For more information, visit the [Proofpoint Archive webiste](https://www.proofpoint.com/us/products/archiving-and-compliance/archive).
|
|
|
|
## Frequently Asked Questions (FAQ)
|
|
|
|
### Are Playbooks and Boards data included in compliance exports?
|
|
|
|
No. Compliance exports only include channel messages, direct messages, file uploads, and posts from plugins/bots/webhooks. Data from Mattermost Playbooks (including run activities, status updates, and retrospectives) and Mattermost Boards (including cards, comments, and board activities) are not included in the compliance export functionality. Organizations requiring compliance archiving of Playbooks and Boards data should consider separate data retention strategies for these features.
|
|
|
|
### How do I export past history?
|
|
|
|
Run the `export` [command line tool](/administration-guide/manage/mmctl-command-line-tool). You can specify an `exportFrom` option to export data from a specified timestamp. All posts that were made after this timestamp will be exported.
|
|
|
|
### How do I download compliance export jobs using mmctl?
|
|
|
|
From Mattermost Server v10.11, system administrators can download compliance export jobs using the [mmctl compliance_export download](/administration-guide/manage/mmctl-command-line-tool#mmctl-compliance-export-download) command. This provides a command-line interface for retrieving completed compliance export jobs by job ID.
|
|
|
|
### What happens if I export data manually?
|
|
|
|
If the compliance export job is run automatically, manually via the System Console, or manually via the CLI (without the `--exportFrom` option), it exports all posts that were made since the last post that the previous execution of the job exported. If this is the first time that the job has ever run, all posts that were made since the feature was enabled will be exported.
|
|
|
|
If the `--exportFrom` option is specified with the CLI command, all posts that have been made since the supplied timestamp will be exported.
|
|
|
|
When run manually via the System Console, `.csv` and Actiance XML files are written to the `exports` subdirectory of the configured [Local Storage Directory](/administration-guide/configure/environment-configuration-settings#local-storage-directory). Files will be written to a folder with names based on an epoch time range. Global Relay EML export format files will be mailed to the configured email address when run manually.
|
|
|
|
### Is there a maximum row limit for CSV files?
|
|
|
|
No. There's no limit to the number of rows within Compliance Monitoring CSV files.
|
|
|
|
### How do I know if a compliance export job fails?
|
|
|
|
Mattermost provides the status of each compliance export job in **System Console \> Compliance \> Compliance Export**. Here, you can see if the job succeeded or failed, including the number of messages and files exported.
|
|
|
|
In addition, any failures are returned in the server logs for self-hosted deployments. The error log begins with the string `Failed job` and includes a `job_id key/value` pair. Compliance export job failures are identified with worker name `MessageExportWorker`. You can optionally create a script that programmatically queries for such failures and notifies the appropriate system.
|
|
|
|
<Note>
|
|
|
|
This compliance export feature replaces legacy Compliance Reporting Oversight functionality. We recommend Enterprise customers migrate to the new system. For a sample CSV output of the new compliance export system, [download a CSV export file here](https://github.com/mattermost/docs/blob/master/source/samples/csv_export.zip).
|
|
|
|
</Note>
|