snapshot of https://github.com/mattermost/mattermost.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
This commit is contained in:
187
server/cmd/mattermost/commands/export.go
Normal file
187
server/cmd/mattermost/commands/export.go
Normal file
@@ -0,0 +1,187 @@
|
||||
// Copyright (c) 2015-present Mattermost, Inc. All Rights Reserved.
|
||||
// See LICENSE.txt for license information.
|
||||
|
||||
package commands
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/mattermost/mattermost/server/public/model"
|
||||
"github.com/mattermost/mattermost/server/public/shared/request"
|
||||
"github.com/mattermost/mattermost/server/v8/channels/app"
|
||||
|
||||
"github.com/pkg/errors"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
var ExportCmd = &cobra.Command{
|
||||
Use: "export",
|
||||
Short: "Export data from Mattermost",
|
||||
Long: "Export data from Mattermost in a format suitable for import into a third-party application or another Mattermost instance",
|
||||
}
|
||||
|
||||
var ScheduleExportCmd = &cobra.Command{
|
||||
Use: "schedule",
|
||||
Short: "Schedule an export data job in Mattermost",
|
||||
Long: "Schedule an export data job in Mattermost (this will run asynchronously via a background worker)",
|
||||
Example: "export schedule --format=actiance --exportFrom=12345 --timeoutSeconds=12345",
|
||||
RunE: scheduleExportCmdF,
|
||||
}
|
||||
|
||||
var BulkExportCmd = &cobra.Command{
|
||||
Use: "bulk [file]",
|
||||
Short: "Export bulk data.",
|
||||
Long: "Export data to a file compatible with the Mattermost Bulk Import format.",
|
||||
Example: "export bulk bulk_data.json",
|
||||
RunE: bulkExportCmdF,
|
||||
Args: cobra.ExactArgs(1),
|
||||
}
|
||||
|
||||
func init() {
|
||||
ScheduleExportCmd.Flags().String("format", "actiance", "The format to export data")
|
||||
ScheduleExportCmd.Flags().Int64("exportFrom", -1, "The timestamp of the earliest post to export, expressed in seconds since the unix epoch.")
|
||||
ScheduleExportCmd.Flags().Int("timeoutSeconds", -1, "The maximum number of seconds to wait for the job to complete before timing out.")
|
||||
|
||||
BulkExportCmd.Flags().Bool("all-teams", true, "Export all teams from the server.")
|
||||
BulkExportCmd.Flags().Bool("with-archived-channels", false, "Also exports archived channels.")
|
||||
BulkExportCmd.Flags().Bool("with-profile-pictures", false, "Also exports profile pictures.")
|
||||
BulkExportCmd.Flags().Bool("attachments", false, "Also export file attachments.")
|
||||
BulkExportCmd.Flags().Bool("archive", false, "Outputs a single archive file.")
|
||||
|
||||
ExportCmd.AddCommand(ScheduleExportCmd)
|
||||
ExportCmd.AddCommand(BulkExportCmd)
|
||||
|
||||
RootCmd.AddCommand(ExportCmd)
|
||||
}
|
||||
|
||||
func scheduleExportCmdF(command *cobra.Command, args []string) error {
|
||||
a, err := InitDBCommandContextCobra(command, app.SkipPostInitialization())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer a.Srv().Shutdown()
|
||||
|
||||
if !*a.Config().MessageExportSettings.EnableExport {
|
||||
return errors.New("ERROR: The message export feature is not enabled")
|
||||
}
|
||||
|
||||
var rctx request.CTX = request.EmptyContext(a.Log())
|
||||
|
||||
// for now, format is hard-coded to actiance. In time, we'll have to support other formats and inject them into job data
|
||||
format, err := command.Flags().GetString("format")
|
||||
if err != nil {
|
||||
return errors.New("format flag error")
|
||||
}
|
||||
if format != "actiance" {
|
||||
return errors.New("unsupported export format")
|
||||
}
|
||||
|
||||
startTime, err := command.Flags().GetInt64("exportFrom")
|
||||
if err != nil {
|
||||
return errors.New("exportFrom flag error")
|
||||
}
|
||||
if startTime < 0 {
|
||||
return errors.New("exportFrom must be a positive integer")
|
||||
}
|
||||
|
||||
timeoutSeconds, err := command.Flags().GetInt("timeoutSeconds")
|
||||
if err != nil {
|
||||
return errors.New("timeoutSeconds error")
|
||||
}
|
||||
if timeoutSeconds < 0 {
|
||||
return errors.New("timeoutSeconds must be a positive integer")
|
||||
}
|
||||
|
||||
if messageExportI := a.MessageExport(); messageExportI != nil {
|
||||
ctx := context.Background()
|
||||
if timeoutSeconds > 0 {
|
||||
var cancel context.CancelFunc
|
||||
ctx, cancel = context.WithTimeout(ctx, time.Second*time.Duration(timeoutSeconds))
|
||||
defer cancel()
|
||||
}
|
||||
|
||||
rctx = rctx.WithContext(ctx)
|
||||
|
||||
job, err := messageExportI.StartSynchronizeJob(rctx, startTime)
|
||||
if err != nil || job.Status == model.JobStatusError || job.Status == model.JobStatusCanceled {
|
||||
CommandPrintErrorln("ERROR: Message export job failed. Please check the server logs")
|
||||
} else {
|
||||
CommandPrettyPrintln("SUCCESS: Message export job complete")
|
||||
|
||||
auditRec := a.MakeAuditRecord(rctx, model.AuditEventScheduleExport, model.AuditStatusSuccess)
|
||||
auditRec.AddMeta("format", format)
|
||||
auditRec.AddMeta("start", startTime)
|
||||
a.LogAuditRec(rctx, auditRec, nil)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func bulkExportCmdF(command *cobra.Command, args []string) error {
|
||||
a, err := InitDBCommandContextCobra(command, app.SkipPostInitialization())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer a.Srv().Shutdown()
|
||||
|
||||
rctx := request.EmptyContext(a.Log())
|
||||
|
||||
allTeams, err := command.Flags().GetBool("all-teams")
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "all-teams flag error")
|
||||
}
|
||||
if !allTeams {
|
||||
return errors.New("Nothing to export. Please specify the --all-teams flag to export all teams.")
|
||||
}
|
||||
|
||||
attachments, err := command.Flags().GetBool("attachments")
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "attachments flag error")
|
||||
}
|
||||
|
||||
archive, err := command.Flags().GetBool("archive")
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "archive flag error")
|
||||
}
|
||||
|
||||
withArchivedChannels, err := command.Flags().GetBool("with-archived-channels")
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "with-archived-channels flag error")
|
||||
}
|
||||
|
||||
includeProfilePictures, err := command.Flags().GetBool("with-profile-pictures")
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "with-profile-pictures flag error")
|
||||
}
|
||||
|
||||
fileWriter, err := os.Create(args[0])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer fileWriter.Close()
|
||||
|
||||
outPath, err := filepath.Abs(args[0])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var opts model.BulkExportOpts
|
||||
opts.IncludeAttachments = attachments
|
||||
opts.CreateArchive = archive
|
||||
opts.IncludeArchivedChannels = withArchivedChannels
|
||||
opts.IncludeProfilePictures = includeProfilePictures
|
||||
if err := a.BulkExport(rctx, fileWriter, filepath.Dir(outPath), nil /* nil job since it's spawned from CLI */, opts); err != nil {
|
||||
CommandPrintErrorln(err.Error())
|
||||
return err
|
||||
}
|
||||
|
||||
auditRec := a.MakeAuditRecord(rctx, model.AuditEventBulkExport, model.AuditStatusSuccess)
|
||||
auditRec.AddMeta("all_teams", allTeams)
|
||||
auditRec.AddMeta("file", args[0])
|
||||
a.LogAuditRec(rctx, auditRec, nil)
|
||||
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user