snapshot of https://github.com/mattermost/mattermost.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
This commit is contained in:
224
docs/main/administration-guide/comply/compliance-export.mdx
Normal file
224
docs/main/administration-guide/comply/compliance-export.mdx
Normal file
@@ -0,0 +1,224 @@
|
||||
---
|
||||
title: "Compliance export"
|
||||
---
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
Mattermost Enterprise customers can archive history or transfer message data to third-party systems for auditing and compliance purposes with compliance exports. Supported integrations include [Smarsh (Actiance) Vantage](#actiance-xml), [Global Relay](#global-relay-eml), and [Proofpoint](#proofpoint).
|
||||
|
||||
From Mattermost v10.5, compliance exports include performance improvements for large daily data sets with changes affecting output formats, system performance, and logic. Compliance exports provide compliance teams complete information to reconstruct the state of a channel, and to determine who had visibility on an initial message, or when the message was edited or deleted. Compliance teams can track a message by its MessageId as it is edited or deleted, and across batches and exports periods.
|
||||
|
||||
## Overview
|
||||
|
||||
Compliance exports are produced from the System Console, and contain all messages including:
|
||||
|
||||
- Messages sent in direct message channels
|
||||
- File uploads
|
||||
- Posts from plugins
|
||||
- Posts from bots/webhooks
|
||||
|
||||
Exports include information on channel member history at the time the message was posted.
|
||||
|
||||
## Set up guide
|
||||
|
||||
Use the following guides to configure exports for [CSV](#csv), [Smarsh / Actiance XML](#actiance-xml), [Global Relay EML](#global-relay-eml), or [Proofpoint](#proofpoint).
|
||||
|
||||
<Note>
|
||||
|
||||
- For self-hosted deployments, compliance exports are written to the `exports` subdirectory of the configured filestore in the chosen format. This will either be in the [Local Storage directory](/administration-guide/configure/environment-configuration-settings#file-storage) or the Mattermost S3 bucket if S3 storage is configured.
|
||||
- Alternatively, you can specify an alternate filestore target and generate an S3 presigned URL for compliance exports. See the [dedicated export filestore target](/administration-guide/configure/environment-configuration-settings#enable-dedicated-export-filestore-target) configuration settings documentation for details.
|
||||
- Compliance exports don't contain posts sent before the feature was enabled. For self-hosted deployments, you can export past history via the `export` [command line tool](/administration-guide/manage/mmctl-command-line-tool).
|
||||
|
||||
</Note>
|
||||
|
||||
### CSV
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Export**.
|
||||
2. Set **Enable Compliance Exports** to **true**.
|
||||
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form `HH:MM`. Choose a time when fewer people are using your system.
|
||||
4. Set the export file format to **CSV**.
|
||||
5. Select **Save**.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="from-mattermost-v10-5" label="From Mattermost v10.5">
|
||||
|
||||
You can review export job status in the System Console.
|
||||
|
||||
When the daily compliance export job is finished, a parent directory is created named based on when the export was started and the `startTimestamp` and `endTimestamp` of the export, e.g, `administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075`. That parent directory contains 1 zip file for each batch, named based on the batch number and the start and end timestamps of the messages in that batch, e.g, `batch001-1723105062492-1723106622163.zip`. Each zip file contains the same information available in previous Mattermost server releases.
|
||||
|
||||
Working from the same example above, the directory would look like this:
|
||||
|
||||
``` bash
|
||||
administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075
|
||||
├── batch001-1723105062492-1723106622163.zip
|
||||
├── batch002-1723106622163-1723108196005.zip
|
||||
└── batch003-1723108196005-1723109100075.zip
|
||||
```
|
||||
|
||||
And each batch would look like this:
|
||||
|
||||
``` bash
|
||||
batch001-1723105062492-1723106622163.zip
|
||||
├── files
|
||||
├── metadata.json
|
||||
└── actiance_export.xml
|
||||
```
|
||||
|
||||
## Updated CSV export fields
|
||||
|
||||
**Post Creation Time** is always the `CreateAt` for messages and attachments, or `JoinTime` and `LeaveTime` for participant join and leave events, respectively.
|
||||
|
||||
- **Update Time** indicates that the message has been updated, and this is the `updateAt` time.
|
||||
- **Updated Type** helps differentiate what kind of update it was as one of the following:
|
||||
- **EditedNewMsg** indicates that the message has been edited, and this is the new message (post-edit) content.
|
||||
- **EditedOriginalMsg** indicates that the message has been edited, and this the original message (pre-edit) content. This message will have another field `EditedNewMsgId`, which is the Id of the message which holds the post-edited message contents.
|
||||
- **UpdatedNoMsgChange** indicates that message's content hasn't changed, but the post was updated for some reason, such as reaction, replied-to, a reply was edited, or a reply was deleted.
|
||||
- **Deleted** indicates that this message was deleted.
|
||||
- **FileDeleted** indicates that this message is recording that a file was deleted.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="prior-to-mattermost-v10-5" label="Prior to Mattermost v10.5">
|
||||
|
||||
The daily compliance export job creates a `.zip` file with a unique job identifier of all messages posted in the last 24 hours. You can unzip the file to easily transform the default `.csv` format into a desired format for your third-party archive system.
|
||||
|
||||
For a sample CSV output, [download a CSV export file here](/files/samples/csv_export.zip).
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
### Actiance XML
|
||||
|
||||
Actiance XML is the supported format for the 'Smarsh Vantage product \[https://central.smarsh.com/s/product/vantage\](https://central.smarsh.com/s/product/vantage\)\`\_.
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Export**.
|
||||
2. Set **Enable Compliance Exports** to **true**.
|
||||
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
||||
4. Set the export file format to **Actiance XML**.
|
||||
5. Select **Save**.
|
||||
|
||||
<Note>
|
||||
|
||||
In Actiance XML exports, channel type is prepended to the channel names.
|
||||
|
||||
</Note>
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="from-mattermost-v10-5" label="From Mattermost v10.5">
|
||||
|
||||
You can review export job status in the System Console. Once you've selected Actiance XML as your file format, you can set up an integration with Actiance Vantage archive system.
|
||||
|
||||
When the daily compliance export job is finished, a parent directory is created named based on when the export was started and the `startTimestamp` and `endTimestamp` of the export, e.g, `administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075`. That parent directory contains 1 zip file for each batch, named based on the batch number and the start and end timestamps of the messages in that batch, e.g, `batch001-1723105062492-1723106622163.zip`. Each zip file contains the same information available in previous Mattermost server releases.
|
||||
|
||||
Working from the same example above, the directory would look like this:
|
||||
|
||||
``` bash
|
||||
administration-guide/comply/compliance-export-2024-08-13-05h08m-1723105062492-1723109100075
|
||||
├── batch001-1723105062492-1723106622163.zip
|
||||
├── batch002-1723106622163-1723108196005.zip
|
||||
└── batch003-1723108196005-1723109100075.zip
|
||||
```
|
||||
|
||||
And each batch would look like this:
|
||||
|
||||
``` bash
|
||||
batch001-1723105062492-1723106622163.zip
|
||||
├── 20240808
|
||||
└── actiance_export.xml
|
||||
```
|
||||
|
||||
## Updated Actiance XML export fields
|
||||
|
||||
If an XML field is empty, it won't be exported. This is a change from previous Mattermost releases, where empty XML nodes were exported.
|
||||
|
||||
- `MessageId` is the unique `messageId`.
|
||||
- `DateTimeUTC` is always the post's `CreateAt` time.
|
||||
- `UpdatedDateTimeUTC` indicates that the message has been updated, and this is the `updateAt` time.
|
||||
- `UpdatedType` helps differentiate what kind of update it was, including:
|
||||
- `EditedNewMsg` indicates that this message has been edited, and this is the new message (post-edit) content.
|
||||
- `EditedOriginalMsg` indicates that this message has been edited, and this the original message (pre-edit) content. This message will have another field `EditedNewMsgId`, which is the Id of the message which holds the post-edited message contents.
|
||||
- `UpdatedNoMsgChange` indicates that this message's content hasn't changed, but the post was updated for some reason, such as a reaction, replied-to, a reply was edited, or a reply was deleted.
|
||||
- `Deleted` indicates that the message was deleted.
|
||||
- `FileDeleted` indicates that the message is recording that a file was deleted.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="prior-to-mattermost-v10-5" label="Prior to Mattermost v10.5">
|
||||
|
||||
The daily compliance export job creates a `.zip` file with a unique job identifier of all messages posted in the last 24 hours. Once you've selected Actiance XML as your file format, you can set up an integration with Actiance Vantage archive system. For a sample Actiance output, [download an Actiance XML export file here](/files/samples/actiance_export.xml).
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
### Global Relay EML
|
||||
|
||||
For more information on Global Relay archive system, visit [their website](https://www.globalrelay.com/).
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Export**.
|
||||
2. Set **Enable Compliance Export** to **true**.
|
||||
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
||||
4. Set the **Export Format** to **GlobalRelay EML**.
|
||||
5. Select **A9/Type 9**, **A10/Type 10**, or **Custom** for the **Global Relay Customer Account**. This is the type of Global Relay customer account your organization has.
|
||||
- For **A9/Type 9** and **A10/Type 10** types, set the **Global Relay SMTP username**, **Global Relay SMTP password**, and **Global Relay SMTP email address**, provided by Global Relay.
|
||||
- For a **Custom** type, set the **Global Relay SMTP username**, **Global Relay SMTP password**, **Global Relay SMTP email address**, **SMTP Server Name**, and the **SMTP Server Port**, provided by Global Relay. **Custom** type can be used to integrate with Proofpoint.
|
||||
6. Select **Save**.
|
||||
|
||||
<Note>
|
||||
|
||||
Messages larger than 250 MB will have their attachments removed because they are too large to send to Global Relay. An error is added to the server logs with id `global_relay_attachments_removed`. It includes the post ID the attachments were removed from, as well as the attachment IDs. A [ticket is queued to better handle large messages](https://mattermost.atlassian.net/browse/MM-10038).
|
||||
|
||||
</Note>
|
||||
|
||||
Once you've selected Global Relay EML as your file format, you can set up an integration with Global Relay archive system. For more information, see [Global Relay Archive](https://www.globalrelay.com/products/archive-data-compliance/).
|
||||
|
||||
### Proofpoint
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Export**.
|
||||
2. Set **Enable Compliance Export** to **true**.
|
||||
3. Set the **Compliance Export time**. This is the start time of the daily scheduled compliance export job and must be a 24-hour time stamp in the form HH:MM. Choose a time when fewer people are using your system.
|
||||
4. Set the **Export Format** to **GlobalRelay EML**.
|
||||
5. Select **Custom** for the **Global Relay Customer Account** to integrate with Proofpoint.
|
||||
6. Set the **SMTP username**, **SMTP password**, **SMTP email address**, **SMTP Server Name**, and the **SMTP Server Port**, provided by Proofpoint.
|
||||
7. Select **Save**.
|
||||
|
||||
See the [Global Relay](#global-relay-eml) section for details on updated Global Relay export fields. Now you can set up an integration with the Proofpoint archive system. For more information, visit the [Proofpoint Archive webiste](https://www.proofpoint.com/us/products/archiving-and-compliance/archive).
|
||||
|
||||
## Frequently Asked Questions (FAQ)
|
||||
|
||||
### Are Playbooks and Boards data included in compliance exports?
|
||||
|
||||
No. Compliance exports only include channel messages, direct messages, file uploads, and posts from plugins/bots/webhooks. Data from Mattermost Playbooks (including run activities, status updates, and retrospectives) and Mattermost Boards (including cards, comments, and board activities) are not included in the compliance export functionality. Organizations requiring compliance archiving of Playbooks and Boards data should consider separate data retention strategies for these features.
|
||||
|
||||
### How do I export past history?
|
||||
|
||||
Run the `export` [command line tool](/administration-guide/manage/mmctl-command-line-tool). You can specify an `exportFrom` option to export data from a specified timestamp. All posts that were made after this timestamp will be exported.
|
||||
|
||||
### How do I download compliance export jobs using mmctl?
|
||||
|
||||
From Mattermost Server v10.11, system administrators can download compliance export jobs using the [mmctl compliance_export download](/administration-guide/manage/mmctl-command-line-tool#mmctl-compliance-export-download) command. This provides a command-line interface for retrieving completed compliance export jobs by job ID.
|
||||
|
||||
### What happens if I export data manually?
|
||||
|
||||
If the compliance export job is run automatically, manually via the System Console, or manually via the CLI (without the `--exportFrom` option), it exports all posts that were made since the last post that the previous execution of the job exported. If this is the first time that the job has ever run, all posts that were made since the feature was enabled will be exported.
|
||||
|
||||
If the `--exportFrom` option is specified with the CLI command, all posts that have been made since the supplied timestamp will be exported.
|
||||
|
||||
When run manually via the System Console, `.csv` and Actiance XML files are written to the `exports` subdirectory of the configured [Local Storage Directory](/administration-guide/configure/environment-configuration-settings#local-storage-directory). Files will be written to a folder with names based on an epoch time range. Global Relay EML export format files will be mailed to the configured email address when run manually.
|
||||
|
||||
### Is there a maximum row limit for CSV files?
|
||||
|
||||
No. There's no limit to the number of rows within Compliance Monitoring CSV files.
|
||||
|
||||
### How do I know if a compliance export job fails?
|
||||
|
||||
Mattermost provides the status of each compliance export job in **System Console \> Compliance \> Compliance Export**. Here, you can see if the job succeeded or failed, including the number of messages and files exported.
|
||||
|
||||
In addition, any failures are returned in the server logs for self-hosted deployments. The error log begins with the string `Failed job` and includes a `job_id key/value` pair. Compliance export job failures are identified with worker name `MessageExportWorker`. You can optionally create a script that programmatically queries for such failures and notifies the appropriate system.
|
||||
|
||||
<Note>
|
||||
|
||||
This compliance export feature replaces legacy Compliance Reporting Oversight functionality. We recommend Enterprise customers migrate to the new system. For a sample CSV output of the new compliance export system, [download a CSV export file here](/files/samples/csv_export.zip).
|
||||
|
||||
</Note>
|
||||
243
docs/main/administration-guide/comply/compliance-monitoring.mdx
Normal file
243
docs/main/administration-guide/comply/compliance-monitoring.mdx
Normal file
@@ -0,0 +1,243 @@
|
||||
---
|
||||
title: "Compliance monitoring"
|
||||
---
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
This feature enables compliance exports to be produced from the System Console, with all query and download actions logged in an audit history to enable oversight and prevent unauthorized queries.
|
||||
|
||||
Compliance exports can be filtered to date range, user account, and keyword list. Requests from queries can be downloaded from the user interface in `.csv` format, with a `.json` metafile documenting the query, as well as placed in a directory set by the system admin.
|
||||
|
||||
Daily compliance reports may also be generated, supporting integration with compliance solutions like [Global Relay](#global-relay-support).
|
||||
|
||||
By default, all Mattermost deployments retain all messages, including edits and deletes, along with all files uploaded.
|
||||
|
||||
## Enable compliance reporting
|
||||
|
||||
To enable the option to generate daily compliance reports:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Monitoring** and set the **Enable Compliance Reporting** value to **true**.
|
||||
2. (Optional) In **Compliance Report Directory** specify the directory in which to place completed compliance reports. Defaults to `./data/` if left blank.
|
||||
3. Select **Save**.
|
||||
|
||||
## Turn on daily compliance reports
|
||||
|
||||
After enabling compliance reporting:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Monitoring** and set the **Enable Daily Report** value to **true**.
|
||||
2. Select **Save**.
|
||||
|
||||
Your system will now export all new messages posted within a 24-hour period as a `.csv` file to the location specified in **Compliance Report Directory**. This feature can be used in conjunction with centralized compliance reporting systems that move.
|
||||
|
||||
## Run compliance reports
|
||||
|
||||
Compliance reports are exports of all messages in Mattermost that match the report criteria. To run a report:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Compliance Monitoring**.
|
||||
|
||||
2. Fill in the following:
|
||||
|
||||
> - **Job Name:** Name the compliance report you are about to run (e.g. "HR Audit 455").
|
||||
> - **From:** Start date for search in YYYY-MM-DD format (e.g. "2016-03-11").
|
||||
> - **To:** End date of search in YYYY-MM-DD format (e.g. "2016-05-11").
|
||||
> - **Emails:** Comma-separated list of email addresses of users whose posted messages you want to search (e.g. `bill@example.com, bob@example.com`).
|
||||
> - **Keywords:** Indicate the words that would be contained in a message for it to be included in the compliance report results.
|
||||
|
||||
3. Select **Run Compliance Report**.
|
||||
|
||||
The report will be queued in the display below the fields described above. The properties of each compliance report run is explained as follows:
|
||||
|
||||
- **Timestamp:** Time at which the report was requested.
|
||||
- **Status:** `running` indicates the report is being run; `finished` indicates the report is complete and ready for download.
|
||||
- **Records:** Shows the number of search results.
|
||||
- **Type:** `adhoc` indicates the report was requested by completing query fields; `daily` indicates the report is a daily export.
|
||||
- **Description:** Job Name indicated in request.
|
||||
- **Requested by:** Email of person requesting the report.
|
||||
- **Params:** Parameters of the compliance report request.
|
||||
|
||||
Each compliance report includes a **Download** link which downloads a compressed file named `adhoc-[UNIQUE_ID].zip`. Inside the file is `meta.json`, which includes the parameters of the search executed and `posts.csv` which includes the contents of messages found by the request.
|
||||
|
||||
### Compliance query definition stored in `meta.json`
|
||||
|
||||
The `meta.json` file contains the following information about the compliance query:
|
||||
|
||||
<table style={{width: '99%'}}>
|
||||
<colgroup>
|
||||
<col style={{width: '17%'}} />
|
||||
<col style={{width: '52%'}} />
|
||||
<col style={{width: '29%'}} />
|
||||
</colgroup>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Field</th>
|
||||
<th>Description</th>
|
||||
<th>Example</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>id</td>
|
||||
<td>Unique identifier for compliance query</td>
|
||||
<td>ja8z8egap7nq9kqetz3rt98khe</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>create_at</td>
|
||||
<td>Timestamp at which compliance query was executed</td>
|
||||
<td>1463637842478</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>user_id</td>
|
||||
<td>Mattermost User ID for person creating query</td>
|
||||
<td>3bq1shta93yztg3i6aiu1tzi5h</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>status</td>
|
||||
<td>Status of query: <em>finished</em> or <em>failed</em></td>
|
||||
<td>"finished"</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>count</td>
|
||||
<td>Count of messages found matching keyword</td>
|
||||
<td>36</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>desc</td>
|
||||
<td>User entered description of compliance query</td>
|
||||
<td>Example Compliance Report</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>type</td>
|
||||
<td>Type of compliance query: <em>adhoc</em> or <em>daily</em></td>
|
||||
<td>"adhoc"</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>start_at</td>
|
||||
<td>Timestamp at which query began to run</td>
|
||||
<td>1451606400000</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>end_at</td>
|
||||
<td>Timestamp at which query ended</td>
|
||||
<td>1463529600000</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>keywords</td>
|
||||
<td>Comma-separated, case insensitive keywords to match in query</td>
|
||||
<td>"drinking"</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>emails</td>
|
||||
<td>Comma-separated emails of users to search. Blank returns all</td>
|
||||
<td><code>frank.yu@ha.ca, mary.li@hi.co</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
### Compliance query results stored in `posts.csv` file
|
||||
|
||||
`posts.csv` contains the following information about the compliance query results, one search result per row:
|
||||
|
||||
<table style={{width: '98%'}}>
|
||||
<colgroup>
|
||||
<col style={{width: '18%'}} />
|
||||
<col style={{width: '53%'}} />
|
||||
<col style={{width: '26%'}} />
|
||||
</colgroup>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Field</th>
|
||||
<th>Description</th>
|
||||
<th>Example</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>TeamName</td>
|
||||
<td>URL name of team</td>
|
||||
<td>contosi</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>TeamDisplayName</td>
|
||||
<td>Display name of team</td>
|
||||
<td>Contosi Corporation</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ChannelDisplayName</td>
|
||||
<td>Display name of channel where keyword was found</td>
|
||||
<td>Community Heartbeat</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ChannelName</td>
|
||||
<td>URL name of channel</td>
|
||||
<td>community-heartbeat</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UserUsername</td>
|
||||
<td>Username of user posting the message containing keyword</td>
|
||||
<td>frank.yu</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UserEmail</td>
|
||||
<td>Email of user posting the message containing keyword</td>
|
||||
<td>"frank.yu@contosi.com"</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UserNickname</td>
|
||||
<td>Nickname of user posting the message containing keyword</td>
|
||||
<td>fan du</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UserType</td>
|
||||
<td>Type of user posting the message ("user" or "bot")</td>
|
||||
<td>user</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostId</td>
|
||||
<td>Unique ID of message post containing keyword</td>
|
||||
<td>xt9anyx6x3fx9y84aehgakdpze</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostCreateAt</td>
|
||||
<td>Timestamp at which post was created</td>
|
||||
<td>2016-03-02T16:01:59Z</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostDeletedAt</td>
|
||||
<td>Timestamp at which post was deleted (if applicable)</td>
|
||||
<td>2016-03-02T16:01:59Z</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostUpdatedAt</td>
|
||||
<td>Timestamp at which post was last edited (if applicable)</td>
|
||||
<td>2016-03-02T16:01:59Z</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostParentId</td>
|
||||
<td>Unique ID of parent post if post is a comment</td>
|
||||
<td>xt9anyx6x3fx9y84aehgakdpze</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostOriginalId</td>
|
||||
<td>Unique ID of post if deleted or edited</td>
|
||||
<td>xt9anyx6x3fx9y84aehgakdpze</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostMessage</td>
|
||||
<td>Message containing keyword</td>
|
||||
<td>Drinking from the fire hose</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PostFilenames</td>
|
||||
<td>Comma separated list of filesnames attached to post</td>
|
||||
<td>["/f../ho.png","/f../hi.png"]</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## Global Relay support
|
||||
|
||||
Mattermost daily compliance reports are compatible with Global Relay compliance solutions through the conversion of Mattermost `.CSV` exports into Global Relay `EML` files.
|
||||
|
||||
- This conversion can be done by in-house developers who have previously written scripts to convert other communication systems into Global Relay format based on your organization's specific needs.
|
||||
- You can also contact your Global Relay account manager about a services project to establish this conversion.
|
||||
|
||||
We recommend using the new [Compliance Export feature](/administration-guide/comply/compliance-export) for Global Relay exports.
|
||||
14
docs/main/administration-guide/comply/comply-index.mdx
Normal file
14
docs/main/administration-guide/comply/comply-index.mdx
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: "Compliance with Mattermost"
|
||||
sidebar_label: "Comply"
|
||||
---
|
||||
Mattermost is purpose-built to help enterprises keep sensitive data safe and compliant in the strictest, most complex environments. Mattermost Enterprise Edition includes features designed to make compliance with all relevant regulations and internal policies easy to achieve and maintain.
|
||||
|
||||
- [Compliance exports](/administration-guide/comply/compliance-export) - Export compliance reports to third-party systems to archive history.
|
||||
- [Compliance monitoring](/administration-guide/comply/compliance-monitoring) - Enable oversight and prevent unauthorized queries with compliance exports.
|
||||
- [Electronic discovery](/administration-guide/comply/electronic-discovery) - Extract data from Mattermost for eDiscovery.
|
||||
- [Data retention](/administration-guide/comply/data-retention-policy) - Control how long data is stored in Mattermost with global and custom retention policies to meet data retention compliance requirements.
|
||||
- [Export channel data](/administration-guide/comply/export-mattermost-channel-data) - Migrate data between systems and back data up for operational continuity.
|
||||
- [Legal hold](/administration-guide/comply/legal-hold) - Preserve relevant Mattermost information when litigation is anticipated.
|
||||
- [Custom terms of service](/administration-guide/comply/custom-terms-of-service) - Require users to accept your own terms before they can access Mattermost.
|
||||
- [Audit log JSON schema](/administration-guide/comply/embedded-json-audit-log-schema) - Learn how to configure Mattermost audit logging using a JSON object.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
title: "Custom terms of service"
|
||||
---
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
Increase clarity on legal Mattermost expectations for internal employees and guests with the ability to set custom Terms of Service (“ToS”) agreements and re-acceptable periods.
|
||||
|
||||
## Configuring terms of service
|
||||
|
||||
To enable custom terms of service:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Custom Terms of Service**.
|
||||
2. Set **Enable Custom Terms of Service** to **true**.
|
||||
3. Set **Custom Terms of Service Text** which contains your terms. Markdown formatting is supported, including lists, headings, and bolding.
|
||||
4. Set **Re-Acceptance Period** to configure the number of days before Terms of Service acceptance expires, and when the terms must be re-accepted. Set to 0 if you don't want your terms to expire.
|
||||
5. Select **Save**.
|
||||
|
||||
Once saved, all users must accept the terms of service by clicking **I Agree** next time they log in, or on the next page refresh. If they do not accept, they will be logged out.
|
||||
|
||||
<Note>
|
||||
|
||||
If you make an update to your Terms of Service, make sure to update your terms of service link at **System Console \> Site Configuration \> Customization \> Terms of Service link**. This link is presented to all users when they log in, and it's easily accessible to end users after accepting the terms.
|
||||
|
||||
</Note>
|
||||
|
||||
## Frequently asked questions
|
||||
|
||||
### What happens if I update my terms of service text?
|
||||
|
||||
There will be no impact to your end users. Users will simply be required to accept the new terms on next login or page refresh.
|
||||
|
||||
### What happens if a user doesn't accept the terms of service?
|
||||
|
||||
Users won't be able to log in if they don't accept the Terms of Service.
|
||||
|
||||
### How do I know if my users accepted the terms of service?
|
||||
|
||||
Each time a user agrees to the terms of service, the agreement is recorded in the database:
|
||||
|
||||
- A new field `AcceptedServiceTermsId` in the **Users** table indicates the most recent terms of service the user has accepted.
|
||||
- Each time a user accepts the terms, the action is recorded in the **Audits** table, containing the username, timestamp of acceptance, and other relevant information.
|
||||
|
||||
### Why isn't this feature in Team Edition for GDPR compliance?
|
||||
|
||||
Terms of service is presented to users on login and account creation, and available to users at all times in the link available by going to **System Console \> Site Configuration \> Customization**.
|
||||
|
||||
This feature is intended to meet compliance requirements for large Enterprise companies.
|
||||
120
docs/main/administration-guide/comply/data-retention-policy.mdx
Normal file
120
docs/main/administration-guide/comply/data-retention-policy.mdx
Normal file
@@ -0,0 +1,120 @@
|
||||
---
|
||||
title: "Data retention policy"
|
||||
---
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
Mattermost stores all message history providing an unlimited search history to system admins and end users.
|
||||
|
||||
Mattermost Enterprise customers can set a global retention policy as well as custom retention policies to manage how long messages and file uploads are kept in Mattermost channels and direct messages in order to meet data retention compliance requirements.
|
||||
|
||||
<Warning>
|
||||
|
||||
- By default, Mattermost uses a “soft delete” system where messages and files deleted based on user actions are removed from the user interface, but persist in the Mattermost database.
|
||||
- Once a message or a file is permanently deleted, the action is irreversible. We recommend using caution when setting up global or custom data retention policies to avoid data loss.
|
||||
|
||||
</Warning>
|
||||
|
||||
## Configure a global data retention policy
|
||||
|
||||
To set a global data retention policy:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Data Retention Policies**.
|
||||
2. Select **Edit** from the menu located to the right of the **Global retention policy** table.
|
||||
3. Specify a global retention policy for channel messages and direct messages by selecting a **Channel & direct message retention** option from the dropdown, then set how long to keep those messages in hours, days, or years. When a time is set, messages and file attachments older than the duration you set will be deleted. The minimum retention period is one hour.
|
||||
4. Select a **File retention** option from the dropdown. Set the number of hours, days, or years to keep files. When a time is set, uploaded files which are older than the duration you set will be deleted from your file storage system (either from your local disk or your Amazon S3 service as specified in **System Console \> Environment \> File Storage**). The minimum retention period is one hour. The global file policy deletes all files regardless of whether they're in a direct message, private, or public channel.
|
||||
5. From Mattermost v10.10, you can optionally preserve pinned posts when data retention policies delete messages by enabling [Preserve pinned posts](/administration-guide/configure/compliance-configuration-settings#preserve-pinned-posts). Once enabled, pinned posts won't be deleted even if they exceed the configured retention period.
|
||||
6. Under the **Policy log** section, select **Edit** to specify the start time of the daily scheduled data retention job. Choose a time when fewer people are using your system.
|
||||
|
||||
Select **Save**. Messages and files older than the duration you set will be deleted at the specified server time, as applicable.
|
||||
|
||||
## Configure a custom data retention policy
|
||||
|
||||
To set a custom data retention policy:
|
||||
|
||||
1. Go to **System Console \> Compliance \> Data Retention Policies**.
|
||||
2. Select **Add policy** to the right of the **Custom retention policies** table.
|
||||
3. Specify a name for your policy.
|
||||
4. Specify a custom retention policy for channel and direct messages by selecting a **Channel & direct message retention** option from the dropdown, then set how long to keep uploaded files in days or years. When a time is set, messages and file attachments older than the duration you set will be deleted. The minimum retention period is one day.
|
||||
5. Assign teams and channels to this policy by selecting **Add teams** and searching for a specific team, or by selecting **Add channels** and searching for a specific channel. If only teams are specified, all channels for selected teams will be included in the a policy.
|
||||
6. Under the **Policy log** section, select **Edit** to specify the start time of the daily scheduled data retention job. Choose a time when fewer people are using your system. If a time is already set for a global retention policy, then the same time applies to custom data retention policies.
|
||||
|
||||
Save the settings. Messages and files older than the duration you set will be deleted at the specified server time, as applicable.
|
||||
|
||||
## Run a deletion job manually
|
||||
|
||||
You can also run the deletion job manually at any time by selecting **Run Deletion Job Now** in **System Console \> Compliance \> Data Retention Policy**.
|
||||
|
||||
<Note>
|
||||
|
||||
If using data retention with [ElasticSearch](/deployment-guide/scale/elasticsearch-setup), ensure the [ElasticSearch aggregate search indexes](/administration-guide/configure/environment-configuration-settings#aggregate-search-indexes) setting is set to a value that is greater than your data retention policy in days.
|
||||
|
||||
</Note>
|
||||
|
||||
## Frequently Asked Questions (FAQs)
|
||||
|
||||
### What happens when a message is deleted?
|
||||
|
||||
The message is removed from the Mattermost user interface and deleted from the `Posts` table. The message is no longer searchable and cannot be retrieved in saved posts lists.
|
||||
|
||||
Replies that did not exceed the message duration are still displayed in the user interface. However, further replies are no longer possible.
|
||||
|
||||
If there was a file attached to the message, it will be removed from the user interface only.
|
||||
|
||||
### Why can I still see messages that were supposedly deleted?
|
||||
|
||||
The web and desktop app cache posts. Posts deleted by a data retention job will be visible to end users until they clear their cache and refresh.
|
||||
|
||||
### What happens when a file is deleted by the file retention policy?
|
||||
|
||||
The file attachment is removed from the Mattermost user interface, deleted from the `FileInfo` table, and from your local disk or Amazon S3 service as specified in **System Console \> Environment \> File Storage**.
|
||||
|
||||
### Why do I see `Pending` in the deletion job table with no details?
|
||||
|
||||
This usually means another data retention job is in progress. You can verify this in the deletion job table in **System Console \> Compliance \> Data Retention Policy**.
|
||||
|
||||
### How is data retention handled in the mobile apps?
|
||||
|
||||
When messages or files are deleted, they are no longer searchable in the Mattermost mobile apps.
|
||||
|
||||
In v1.5 and later of the iOS and Android apps, messages and files are deleted from local storage in the following cases, if they exceed the retention policy duration:
|
||||
|
||||
1. When the user opens the app.
|
||||
2. When the user puts the app into the background.
|
||||
|
||||
In v1.4 and earlier of the mobile apps, messages and files are not cleared from local storage when the data retention policy is enabled.
|
||||
|
||||
### How do I know if a data retention job fails?
|
||||
|
||||
Mattermost provides the status of each data retention job in **System Console** \> **Compliance** \> **Data Retention Policy**. Here, you can see if the job succeeded or failed, including the details of the error.
|
||||
|
||||
Additionally, any failures are returned in the server logs. The error log begins with the string `Failed job` and includes a job_id key/value pair. Data retention job failures are identified with worker name `EnterpriseDataRetention`. You can optionally create a script that programmatically queries for such failures and notifies the appropriate system.
|
||||
|
||||
### What happens when the data retention period is changed?
|
||||
|
||||
Data retention runs once a day at the time specified in the `config.json` file. Changing the retention period does not automatically schedule any additional run of the data retention job - it only updates how long data is kept in Mattermost.
|
||||
|
||||
### Does the system administrator get any notification when the data retention period is changed?
|
||||
|
||||
No, the new config is updated, but the system admin does not receive any feedback on what the effects will be (e.g. reporting of how many messages are to be deleted).
|
||||
|
||||
### Does the data retention job affect the audits table?
|
||||
|
||||
Prior to v5.20, data retention would delete all user activity corresponding to the data retention time configuration. From v5.20, the audit table will retain the user activity corresponding to the data retention time configuration.
|
||||
|
||||
### Does the data retention job include archived channels?
|
||||
|
||||
Posts and attachments in archived channels are affected by the data retention job. If a post exceeds the age configured for the data retention job it will be deleted from the database.
|
||||
|
||||
### How long does it take to run a deletion query and does it affect server performance?
|
||||
|
||||
Data retention runs the actual deletion query in batches, deleting data in blocks of 1000 records per query. This is so the database won’t be locked up for extended periods of time with long-running queries. Keeping to this limit keeps the query down to a few milliseconds' execution time on the database itself.
|
||||
|
||||
Each batch of data is deleted based on indexes - making the queries quick to execute on small batches. This helps the server remain fully responsive while the process is running.
|
||||
|
||||
### How do I know whether the data retention job is running/scheduled?
|
||||
|
||||
The job scheduler runs the data retention job based on the time specified in the configuration settings. At this time a `DEBUG`-level log line is printed: `Scheduling data retention job`.
|
||||
|
||||
When a job server picks up that scheduled job for execution, a `DEBUG`-level log line is generated: `Worker EnterpriseDataRetention: Received a new candidate job`.
|
||||
|
||||
When the job is complete, an `INFO`-level log line is generated: `Worker EnterpriseDataRetention: Job is complete`.
|
||||
113
docs/main/administration-guide/comply/electronic-discovery.mdx
Normal file
113
docs/main/administration-guide/comply/electronic-discovery.mdx
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
title: "Electronic discovery"
|
||||
---
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
Electronic discovery (also known as eDiscovery) refers to a process where electronic data is searched with the intent to use it as evidence in a legal case.
|
||||
|
||||
This page describes how to extract data from Mattermost for eDiscovery. There are three primary methods that can be used to accomplish the goal of extracting user post data from Mattermost:
|
||||
|
||||
- [Mattermost Compliance Exports](/administration-guide/comply/compliance-export)
|
||||
- [Mattermost RESTful API](/administration-guide/comply/electronic-discovery#mattermost-restful-api)
|
||||
- [Mattermost database using standard SQL queries](/administration-guide/comply/electronic-discovery#mattermost-database)
|
||||
|
||||
Each of the options is discussed in detail below.
|
||||
|
||||
<Note>
|
||||
|
||||
Litigation hold (also known as legal hold) is an extension of eDiscovery where in addition to searching for records, no electronically stored information nor paper documents may be discarded if they may be deemed relevant for a present or future legal case.
|
||||
|
||||
</Note>
|
||||
|
||||
## Mattermost compliance exports
|
||||
|
||||
Mattermost Enterprise has compliance report export capabilities.
|
||||
|
||||
Mattermost can export compliance related data, including the content of messages and who might have seen those messages, in three formats: Actiance XML, Global Relay EML, and generic CSV. Reports can be configured to run on a delay basis and stored in a shared location.
|
||||
|
||||
For more information about the exports feature and how to set up reporting, see [our documentation](/administration-guide/comply/compliance-export).
|
||||
|
||||
## Mattermost RESTful API
|
||||
|
||||
The Mattermost API can be used to export a user's posts in CSV compliance format that is part of Mattermost Enterprise. The following section outlines how to use the API to create and retrieve a report for a specific user via the API. Please note that full documentation for the Mattermost API can be found at [https://api.mattermost.com](https://api.mattermost.com).
|
||||
|
||||
To use the API, you must first authenticate [as described here](https://api.mattermost.com/#tag/authentication). The account you are authenticating with must have `manage_system` permissions. If you are using curl, you can authenticate using the following command:
|
||||
|
||||
``` sh
|
||||
curl -i -d '{"login_id": "username", "password": "password"}' https://yourmattermosturl/api/v4/users/login
|
||||
```
|
||||
|
||||
Mattermost will return a response that looks like this:
|
||||
|
||||
``` text
|
||||
HTTP/2 200
|
||||
server: nginx/1.10.3 (Ubuntu)
|
||||
date: Thu, 12 Jul 2018 14:43:45 GMT
|
||||
content-type: application/json
|
||||
content-length: 679
|
||||
set-cookie: MMAUTHTOKEN=yi94pwci6ibjfc9phbikhqutbe; Path=/; Expires=Sat, 11 Aug 2018 14:43:45 GMT; Max-Age=2592000; HttpOnly; Secure
|
||||
set-cookie: MMUSERID=qfjzamfg47bu9gsyyfbqjk4s6a; Path=/; Expires=Sat, 11 Aug 2018 14:43:45 GMT; Max-Age=2592000; Secure
|
||||
token: yi94pwci6ibjfc9phbikhqutbe
|
||||
x-request-id: 5y45pxyfxpb8updtge1zts39he
|
||||
x-version-id: 5.0.0.5.0.1.18c0fbd759664a85fe95132bb7fc04cf.true
|
||||
```
|
||||
|
||||
Include the `token` value sent in the response as part of the Authorization header on your future API requests with the Bearer method, e.g.:
|
||||
|
||||
``` sh
|
||||
curl -i -H 'Authorization: Bearer yi94pwci6ibjfc9phbikhqutbe' https://yourmattermosturl/api/v4/users/me
|
||||
```
|
||||
|
||||
Once you're authenticated into Mattermost, you can use the [Compliance API to create a new compliance report](https://api.mattermost.com/#tag/compliance%2Fpaths%2F~1compliance~1reports%2Fpost). The curl based example below demonstrates how to send a request that bases the authentication token and asks Mattermost to create a report that spans posts from Dec 31, 2017 - 8:15 PM to Dec 31, 2018 - 8:15 PM for a user with the email address [craig@mattermost.com](mailto:craig@mattermost.com):
|
||||
|
||||
<Note>
|
||||
|
||||
The data in the JSON payload must be formatted as Unix Epoch. A tool like [https://www.epochconverter.com/](https://www.epochconverter.com/) can be useful when converting to and from the required format.
|
||||
|
||||
</Note>
|
||||
|
||||
``` sh
|
||||
curl --header "Content-Type: application/json" \
|
||||
--request POST \
|
||||
-H 'Authorization: Bearer yi94pwci6ibjfc9phbikhqutbe' \
|
||||
--data '{"id":"","create_at":0,"user_id":"craig","status":"","count":0,"desc":" ","type":"","start_at":1514769359000,"end_at": 1546305359000,"keywords":"","emails":"craig@mattermost.com"}' \
|
||||
https://yourmattermosturl/api/v4/compliance/reports
|
||||
```
|
||||
|
||||
If the post is successful, Mattermost will return a message that looks like the following, indicating that the server is running the compliance export process:
|
||||
|
||||
``` json
|
||||
{"id":"du6kektczifqxexeroywpz3nbc"," create_at":1531444617901, "user_id":"qfjzamfg47bu9gsyyfbqjk4s6a", "status":"running", "count":0, "desc":" ", "type":"adhoc", "start_at":1514769359000, "end_at":1546305359000, "keywords":"", "emails":"craig@mattermost.com"}
|
||||
```
|
||||
|
||||
When the export process is complete (the execution time is based on the number of records to return and the current server load), you will need to send another HTTP Post request to Mattermost to retrieve and download a zip file containing the report that looks like the following curl request:
|
||||
|
||||
``` sh
|
||||
curl --request GET \
|
||||
-H 'Authorization: Bearer p9o1qx457fbc9gdrn39z9ah59o' \
|
||||
--data '{"status_code":0,"id":"du6kektczifqxexeroywpz3nbc","message":"","requestion_id":""}' \
|
||||
--output report-zip.zip \
|
||||
https://yourmattermosturl/api/v4/compliance/reports/du6kektczifqxexeroywpz3nbc/download
|
||||
```
|
||||
|
||||
When sending the request, you need to get the report ID from the response returned by Mattermost when the report was created. You also need to supply a name to save that file as. In the example above, the file will be saved as `report-zip.zip`.
|
||||
|
||||
## Mattermost database
|
||||
|
||||
Selecting messages from the Mattermost database using standard SQL is quite easy. If you know the username, the following query can be used to select all messages for the specified user:
|
||||
|
||||
``` sql
|
||||
SELECT * FROM mattermost.Posts WHERE UserId = (SELECT Id FROM mattermost.Users WHERE Username = 'username');
|
||||
```
|
||||
|
||||
If you want to limit the results of the query based on the date and time that the messages were posted, you can modify the above query to:
|
||||
|
||||
``` sql
|
||||
SELECT * FROM mattermost.Posts WHERE UserId = (SELECT Id FROM mattermost.Users WHERE Username = 'username' AND CreateAt > 1530405832000 AND CreateAt < 1532997832000);
|
||||
```
|
||||
|
||||
<Note>
|
||||
|
||||
The Mattermost database stores date and time stamps in the [Unix Epoch](https://en.wikipedia.org/wiki/Unix_time) format. A tool like the [Epoch Converter](https://www.epochconverter.com/) can be useful in converting to and from the required format.
|
||||
|
||||
</Note>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "Export channel data"
|
||||
---
|
||||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
Ensure important data isn't trapped in silos by migrating data between systems or backing data up for operational continuity. Once exported, channel data can be analyzed using various tools for insights into team dynamics, productivity, and communication patterns, and to fulfill reporting and auditability requirements.
|
||||
|
||||
## Enable
|
||||
|
||||
<Note>
|
||||
|
||||
For Mattermost Cloud deployments, no setup is required. See the [usage](#usage) section below for details on exporting channel data.
|
||||
|
||||
</Note>
|
||||
|
||||
For self-hosted deployments, a Mattermost system admin must install the Channel Export integration from the in-product App Marketplace:
|
||||
|
||||
1. In Mattermost, from the Product menu <img src={useBaseUrl('/img/ui/products_E82F.svg')} alt="Navigate between Channels, collaborative playbooks, and boards using the product menu icon." className="theme-icon" />, select **App Marketplace**.
|
||||
2. Search for or scroll to **Channel Export**, and select **Install**.
|
||||
3. Once installed, select **Configure**. You're taken to the System Console.
|
||||
4. On the **Channel Export** configuration page, enable and configure the plugin, and then select **Save**. You can restrict the ability to export channels to system, team, and channel admins only, and you can configure a maximum file size for channel export files.
|
||||
|
||||
## Upgrade
|
||||
|
||||
We recommend upgrading this feature as new versions are released. Generally, updates are seamless and don't interrupt the user experience in Mattermost. Visit the [Releases page](https://github.com/mattermost/mattermost-plugin-channel-export/releases) for the latest release, available releases, and compatibiilty considerations.
|
||||
|
||||
## Usage
|
||||
|
||||
You need to be a Mattermost system admin to export channel data.
|
||||
|
||||
Use the `/export` slash command in a channel to export the current channel's message data into a CSV-formatted file.
|
||||
|
||||
## Get help
|
||||
|
||||
Mattermost customers can open a [Mattermost support case](https://support.mattermost.com/hc/en-us/requests/new). To report a bug, please open a GitHub issue against the [Mattermost Channel Export plugin GitHub repository](https://github.com/mattermost/mattermost-plugin-channel-export).
|
||||
|
||||
For questions, feedback, and assistance, join our pubic [Integrations and Apps channel](https://community.mattermost.com/core/channels/integrations) on the [Mattermost Community Server](https://community.mattermost.com/) for assistance.
|
||||
|
||||
## Customize this integration
|
||||
|
||||
This plugin contains both a server and web app portion. See the [Channel Export plugin README documentation](https://github.com/mattermost/mattermost-plugin-channel-export?tab=readme-ov-file#development) on GitHub for details. Visit the [Mattermost Developer Workflow](https://developers.mattermost.com/extend/plugins/developer-workflow/) and [Mattermost Developer environment setup](https://developers.mattermost.com/extend/plugins/developer-setup/) for information about developing, customizing, and extending Mattermost functionality.
|
||||
200
docs/main/administration-guide/comply/legal-hold.mdx
Normal file
200
docs/main/administration-guide/comply/legal-hold.mdx
Normal file
@@ -0,0 +1,200 @@
|
||||
---
|
||||
title: "Legal hold"
|
||||
---
|
||||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
|
||||
<PlanAvailability slug="ent-plus" />
|
||||
|
||||
A Legal Hold, also known as a litigation hold, is a process that an organization uses to preserve all forms of relevant information when litigation is reasonably anticipated. It's a requirement established by the Federal Rules of Civil Procedure (FRCP) in the United States and similar laws in other jurisdictions.
|
||||
|
||||
Primary use cases include:
|
||||
|
||||
1. **Litigation**: In anticipation or in the event of a lawsuit, organizations need to preserve all relevant documents and electronic data to ensure they can adequately defend their position. A failure to do so could result in court penalties.
|
||||
2. **Regulatory investigation**: If an organization is being investigated by a regulatory body, it may be required to preserve and produce certain documents or data.
|
||||
3. **Audits**: During an audit, whether internal or external, an organization might need to put a hold on certain data that is relevant to the audit.
|
||||
4. **Records management**: In some cases, organizations might use a Legal Hold to temporarily suspend the deletion of data that would otherwise be purged as part of its records management policy.
|
||||
|
||||
Mattermost is used as a secure collaboration hub by technical and operational teams, with critical documents and data shared on a daily basis. Thus, Legal Hold is a key requirement for Enterprises and public sector organizations who have deployed Mattermost for their teams, to meet compliance & auditory requirements while minimizing risk.
|
||||
|
||||
Mattermost Legal Hold can be combined with [eDiscovery](/administration-guide/comply/electronic-discovery) integration and [data retention policies](/administration-guide/comply/data-retention-policy) to customize the data retained and deleted to comply with compliance requirements.
|
||||
|
||||
## Legal Hold demo (Sneak Peek)
|
||||
|
||||
Check out this [YouTube sneak peek demo](https://www.youtube.com/watch?v=86c8NoOxlQw&feature=youtu.be) to learn about Mattermost's Legal Hold workflow.
|
||||
|
||||
<iframe width="560" height="315" src="https://www.youtube.com/embed/86c8NoOxlQw" alt="Mattermost Legal Hold workflow" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
|
||||
|
||||
Below are step-by-step instructions on how to carry out a Legal Hold for Mattermost using the Mattermost Legal Hold plugin.
|
||||
|
||||
## How to carry out a Legal Hold
|
||||
|
||||
### Step 1: Upgrade to Mattermost Enterprise
|
||||
|
||||
Legal Hold is available in [Mattermost Enterprise](/product-overview/editions-and-offerings#mattermost-enterprise-edition). Learn more about the Enterprise plan & request a quote online at [https://mattermost.com/pricing/](https://mattermost.com/pricing/)
|
||||
|
||||
### Step 2: Establish a Legal Hold policy
|
||||
|
||||
Establish a policy for when to implement a Legal Hold. This should be developed in consultation with your legal counsel and should include procedures for identifying relevant users (those who have potentially relevant information).
|
||||
|
||||
Establishing a Legal Hold policy first enables you to configure the Mattermost system correctly to meet your compliance & auditory requirements, minimizing associated risk.
|
||||
|
||||
### Step 3: Set up the Mattermost Legal Hold plugin
|
||||
|
||||
#### Install the plugin
|
||||
|
||||
1. Log in to your Mattermost [workspace](/end-user-guide/end-user-guide-index) as a system administrator.
|
||||
2. Download the latest version of the [plugin binary release](https://github.com/mattermost/mattermost-plugin-legal-hold/releases/), compatible with Mattermost v8.0.1 and later. If you are using an earlier version of Mattermost, [follow our documentation](/administration-guide/upgrade/upgrading-mattermost-server) to upgrade to Mattermost v8.0.1 or later.
|
||||
3. Go to **System Console \> Plugins \> Plugin Management \> Upload Plugin**, and upload the plugin binary you downloaded in the previous step.
|
||||
4. In the **Installed Plugins** section, scroll to **Legal Hold Plugin**, and select **Enable**.
|
||||
|
||||
#### Configure the plugin
|
||||
|
||||
When the Legal Hold integration is enabled, you can configure when it runs using the format `HH:MM ±HHMM` and `+0000` for UTC.
|
||||
|
||||
You can configure a custom Amazon S3 bucket for Legal Holds by specifying Amazon S3 configuration settings. If no S3 configuration is specified, the Mattermost server file store used. Learn more about file storage configuration options in our [product documentation](/administration-guide/configure/environment-configuration-settings#file-storage).
|
||||
|
||||
#### (Optional) Configure a data retention policy
|
||||
|
||||
You can optionally configure a [data retention policy](/administration-guide/comply/data-retention-policy) to control how long data and file attachments are retained in the Mattermost database.
|
||||
|
||||
### Step 4: Create a Legal Hold
|
||||
|
||||
In Mattermost, create a Legal Hold by completing the following steps:
|
||||
|
||||
1. Go to **System Console \> Plugins \> Legal Hold Plugin**, and select **Create new**.
|
||||
2. Enter a name for the Legal Hold.
|
||||
3. Specify the user names or user groups of users you want to place on Legal Hold.
|
||||
4. (Optional) Public channels are excluded by default. You can choose to include public channels that the specified users or user groups are members of, if preferred; however, doing so will significantly increase the amount of data held based on the number public channels available.
|
||||
5. Specify the number of days that users are placed in Legal Hold with a start date. An end date is optional.
|
||||
6. Select **Create Legal Hold**. [Downloadable data](#download-legal-hold-data) won't be available until the next scheduled job runs.
|
||||
|
||||
#### Manage Legal Holds
|
||||
|
||||
While a Legal Hold is in place, you can edit details of the Legal Hold, access the Legal Hold Secret, as well as download a copy of the preserved data to your local machine.
|
||||
|
||||

|
||||
|
||||
##### Edit a Legal Hold
|
||||
|
||||
Select the **Edit** <img src={useBaseUrl('/img/ui/edit-on-github.png')} alt="Contribute to Mattermost documentation by selecting the Edit option located in the top right corner of any documentation page." className="theme-icon" /> icon to change the name of the Legal Hold, add or remove users, change the end date, as well as include or exclude public channels.
|
||||
|
||||
##### Access a Legal Hold secret
|
||||
|
||||
A Legal Hold secret enables you to verify the authenticity of the data for a Legal Hold in Mattermost.
|
||||
|
||||
Select the **Show** <img src={useBaseUrl('/img/ui/eye-outline_F06D0.svg')} alt="Review your message text formatting using the Show/Hide preview icon in the message formatting toolbar." className="theme-icon" /> icon to display the Legal Hold secret key. Keep a copy of this key in a secure location.
|
||||
|
||||

|
||||
|
||||
To verfiy the contents of the files in this Legal Hold, you must append the processor command with the following flag: `--legal-hold-secret <KEY>`. The output verifies the file and returns the authenticity state of files along with the rest of the output for the processor, as follows:
|
||||
|
||||
Success:
|
||||
|
||||
``` text
|
||||
Secret key was provided, verifying legal holds...
|
||||
- Verifying Legal Hold *processor9*: Verified
|
||||
```
|
||||
|
||||
Error:
|
||||
|
||||
``` text
|
||||
...
|
||||
Secret key was provided, verifying legal holds...
|
||||
- Verifying Legal Hold *processor9*: [Error] hash mismatch for file: legal_hold/processor9_i7k1dbkipiyojeess6ozi4agyr/index.json
|
||||
...
|
||||
```
|
||||
|
||||
##### Download Legal Hold data
|
||||
|
||||
Select the **Download** <img src={useBaseUrl('/img/ui/download-outline_F0B8F.svg')} alt="Use the Download icon to download an attached file to your local system." className="theme-icon" /> icon to download a copy of the preserved data to a location on your local machine. Note, no data will be available to download until at least one scheduled job is completed. This may take up to 24 hours.
|
||||
|
||||
### Step 5: Release a Legal Hold
|
||||
|
||||
Once the Legal Hold has completed, release it to take users off of the Legal Hold by selecting the **Release** option to the right of the Legal Hold task.
|
||||
|
||||
<Important>
|
||||
|
||||
Once a Legal Hold is released, all data is irretrievably deleted from Mattermost and can't be recovered.
|
||||
|
||||
</Important>
|
||||
|
||||
## Frequently asked questions
|
||||
|
||||
### Who can implement Legal Hold?
|
||||
|
||||
Only Mattermost system admins can implement a Legal Hold.
|
||||
|
||||
### Does a user know if they're placed under a Legal Hold in Mattermost?
|
||||
|
||||
No, users won't be notified if they're placed under a Legal Hold, and no reference to Legal Holds will be visible in their Mattermost client or accessible via the Mattermost API. This allows for investigations to be conducted without influencing user behavior and without conflicts of interest.
|
||||
|
||||
### What types of content does Legal Hold cover?
|
||||
|
||||
The Legal Hold covers all messages and file uploads shared in conversations where the Legal Hold is active, including messages posted by plugins, bots or webhooks. This includes messages or files shared in public channels, private channels, direct messages and group messages.
|
||||
|
||||
However, Legal Hold does not apply to reactions, collaborative playbooks, or audio calls.
|
||||
|
||||
### Can users delete their messages while on a Legal Hold?
|
||||
|
||||
Yes, users can delete messages, but they are retained for the purposes of Legal Hold when implemented with the aforementioned steps.
|
||||
|
||||
### Can a Legal Hold be applied retroactively to collect past data?
|
||||
|
||||
Yes, but this is only guaranteed for existing and future messages/files once Legal Hold is activated. It won't recover messages or files that were deleted before the Legal Hold was activated.
|
||||
|
||||
### Is Legal Hold the same as e-discovery?
|
||||
|
||||
No. While they serve a related use case, they are not the same.
|
||||
|
||||
Legal Hold is an initial step to ensure relevant electronically stored information (ESI) is preserved. On the other hand, e-discovery is a multi-step process that uses this preserved data to identify, collect, preserve, process, review, and produce ESI in the context of a legal or investigative process.
|
||||
|
||||
### How do I enable e-discovery for Mattermost?
|
||||
|
||||
Learn more about extracting data for e-discovery in our [product documentation](/administration-guide/comply/electronic-discovery).
|
||||
|
||||
### How do I manage storage costs and version retention in S3?
|
||||
|
||||
If you plan to use an existing S3 bucket for Legal Hold data storage, and your existing S3 bucket has versioning enabled, we strongly recommend using a dedicated S3 bucket with versioning disabled.
|
||||
|
||||
The Legal Hold plugin frequently modifies files in the `legalhold` directory, and when S3 bucket versioning is enabled, each modification creates a new version. This can result in a rapid accumulation of object versions, increased storage costs, potential performance impact, higher S3 API usage, and complicating version management over time. See the [S3 Lifecycle Rules](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html) documentation for additional details.
|
||||
|
||||
### How to view the downloaded Legal Hold zip file
|
||||
|
||||
To make use of the Legal Hold data, you use the [processor tool](https://github.com/mattermost/mattermost-plugin-legal-hold/releases#:~:text=processor%2Dv1.0.2%2Ddarwin%2Damd64) available on the Mattermost GitHub repository. This will output a self-contained HTML site you can open with your browser.
|
||||
|
||||
**Download and setup the processor tool:**
|
||||
|
||||
1. Download the appropriate processor binary for your platform from the [GitHub releases page](https://github.com/mattermost/mattermost-plugin-legal-hold/releases).
|
||||
2. On macOS and Linux, make the processor executable:
|
||||
|
||||
``` bash
|
||||
$ chmod +x processor-vX.X.X-<platform> # Replace with the actual filename you downloaded
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
|
||||
``` bash
|
||||
$ ./processor --legal-hold-data ./legalholddata.zip --output-path ./path/to/where/you/want/the/html/output --legal-hold-secret "your secret"
|
||||
```
|
||||
|
||||
**Arguments:**
|
||||
|
||||
- `--legal-hold-data`: Path to the Legal Hold zip file downloaded from Mattermost
|
||||
- `--output-path`: Directory where the HTML output will be generated
|
||||
- `--legal-hold-secret`: (Optional) Used as a security measure for an operator to ensure the authenticity of a downloaded zip file. The operator can copy the key corresponding to a particular "hold" from the Legal Hold Plugin settings page in the System Console by selecting the **Show** <img src={useBaseUrl('/img/ui/eye-outline_F06D0.svg')} alt="Review your message text formatting using the Show/Hide preview icon in the message formatting toolbar." className="theme-icon" /> icon next to the Legal Hold entry.
|
||||
|
||||
### Is data gathered when a channel has been archived?
|
||||
|
||||
Yes. Filtering isn't done based on archive or deleted status. However, archived channels can be permanently deleted based on Data Retention settings, at which point they will no longer appear in legal hold reports.
|
||||
|
||||
### Is data gathered when a user was a member of a channel for a period of time but has since left the channel?
|
||||
|
||||
Yes. Channels are included when a target user was a member at any time during the legal hold start and end dates.
|
||||
|
||||
### Is data gathered from a direct message or group message with a user who has since been deactivated?
|
||||
|
||||
Yes. Filtering isn't done based on user deactivation state.
|
||||
|
||||
### Is data collected from group messages?
|
||||
|
||||
Yes. Data is collected from group messages like any public or private channel or direct message.
|
||||
Reference in New Issue
Block a user