snapshot of https://github.com/mattermost/mattermost.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
This commit is contained in:
122
docs/develop/contribute/more-info/server/tests.md
Normal file
122
docs/develop/contribute/more-info/server/tests.md
Normal file
@@ -0,0 +1,122 @@
|
||||
---
|
||||
title: "Tests"
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
## Handling Flaky Tests
|
||||
|
||||
A flaky test is one that exhibits both passing and failing results when run multiple times without any code changes. When our automation detects a flaky test on your PR:
|
||||
|
||||
1. **Check if the Test is Newly Introduced**
|
||||
- Review your PR changes to determine if the flaky test was introduced by your changes
|
||||
- If the test is new, fix the flakiness in your PR before merging
|
||||
|
||||
2. **For Existing Flaky Tests**
|
||||
- Create a JIRA ticket titled "Flaky Test: \{TestName\}", e.g. "Flaky Test: TestGetMattermostLog"
|
||||
- Copy the test failure message into the JIRA ticket description
|
||||
- Add the `flaky-test` and `triage-global` labels
|
||||
- Create a PR to skip the test by adding:
|
||||
|
||||
```go
|
||||
t.Skip("https://mattermost.atlassian.net/browse/MM-XXXXX")
|
||||
```
|
||||
|
||||
where MM-XXXXX is your JIRA ticket number
|
||||
- Link the JIRA ticket in the skip message for tracking
|
||||
|
||||
This process helps us track and systematically address flaky tests while preventing them from blocking development work.
|
||||
|
||||
## Writing Parallel Tests
|
||||
|
||||
Leveraging parallel tests can drastically reduce execution time for entire test packages, such as [`api4`](https://github.com/mattermost/mattermost/tree/master/server/channels/api4) and [`app`](https://github.com/mattermost/mattermost/tree/master/server/channels/app), which are notably heavy with hundreds of tests. However, careful implementation is essential to ensure reliability and prevent flakiness. Follow these guidelines when writing parallel tests:
|
||||
|
||||
### Enabling Parallel Tests
|
||||
|
||||
In [`api4`](https://github.com/mattermost/mattermost/tree/master/server/channels/api4), [`app`](https://github.com/mattermost/mattermost/tree/master/server/channels/app), [`platform`](https://github.com/mattermost/mattermost/tree/master/server/channels/app/platform), [`email`](https://github.com/mattermost/mattermost/tree/master/server/channels/app/email), [`jobs`](https://github.com/mattermost/mattermost/tree/master/server/channels/jobs) packages:
|
||||
|
||||
```go
|
||||
func TestExample(t *testing.T) {
|
||||
mainHelper.Parallel(t)
|
||||
|
||||
...
|
||||
}
|
||||
|
||||
// OR
|
||||
|
||||
func TestExample(t *testing.T) {
|
||||
th := Setup(t)
|
||||
th.Parallel(t)
|
||||
|
||||
...
|
||||
}
|
||||
|
||||
// OR
|
||||
|
||||
func TestExample(t *testing.T) {
|
||||
if mainHelper.Options.RunParallel {
|
||||
t.Parallel()
|
||||
}
|
||||
|
||||
...
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
If [`sqlstore`](https://github.com/mattermost/mattermost/tree/master/server/channels/store/sqlstore) package:
|
||||
|
||||
```go
|
||||
func TestExample(t *testing.T) {
|
||||
if enableFullyParallelTests {
|
||||
t.Parallel()
|
||||
}
|
||||
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
To enable parallel execution, you should set the `ENABLE_FULLY_PARALLEL_TESTS` environment variable. Example:
|
||||
|
||||
```bash
|
||||
ENABLE_FULLY_PARALLEL_TESTS=true go test -v ./api4/...
|
||||
```
|
||||
|
||||
### When to Use Parallel Tests
|
||||
|
||||
- **Generally Safe**: Tests with dedicated setup functions that ensure independence from other tests.
|
||||
- **Subtests**: Only safe if each subtest features its own setup function, ensuring they are decoupled and independent of execution order.
|
||||
- **Unsafe**: When a subtest depends on state changes made by another subtest, thus coupling their execution order.
|
||||
|
||||
### Common Issues That Break Parallel Safety
|
||||
|
||||
#### Global State
|
||||
|
||||
Avoid reliance on global variables and registrations such as:
|
||||
|
||||
- `LicenseValidator`
|
||||
- `platform.RegisterMetricsInterface`
|
||||
- `platform.PurgeLinkCache`
|
||||
- `model.BuildEnterpriseReady`
|
||||
- `jobs.DefaultWatcherPollingInterval`
|
||||
|
||||
#### Filesystem Operations
|
||||
|
||||
Avoid using `os.Chdir` (or `t.Chdir`) and relative paths tied to the test executable, as they may introduce inconsistencies when tests run in parallel. When possible, rely on temporary directories such as `th.tempWorkspace` which are dedicated to the test.
|
||||
|
||||
#### Environment Variables
|
||||
|
||||
Using `os.Setenv` for feature flags and other settings can cause interference between parallel tests. Instead, use the configuration API:
|
||||
|
||||
```go
|
||||
// UNSAFE for parallel tests:
|
||||
os.Setenv("MM_FEATUREFLAGS_CUSTOMFEATURE", "true")
|
||||
defer os.Unsetenv("MM_FEATUREFLAGS_CUSTOMFEATURE")
|
||||
|
||||
// SAFE for parallel tests:
|
||||
th.App.UpdateConfig(func(cfg *model.Config) {
|
||||
cfg.FeatureFlags.CustomFeature = true
|
||||
})
|
||||
```
|
||||
|
||||
#### Process-Level Methods
|
||||
|
||||
Be cautious with methods affecting the entire process, such as `pprof.StartCPUProfile`, which can introduce contention between tests.
|
||||
Reference in New Issue
Block a user