Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s
113 lines
5.9 KiB
Plaintext
113 lines
5.9 KiB
Plaintext
---
|
||
title: "Deploy for Out-of-Band (OOB) Communications"
|
||
---
|
||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||
|
||
When a critical incident disrupts an organization’s primary infrastructure—due to a cyberattack, cloud outage, or internal failure—a secure, independent communication channel becomes essential. Mattermost can serve as that channel by operating in an Out-of-Band (OOB) environment: isolated, pre-provisioned, and ready to support incident response and business continuity teams.
|
||
|
||
This document outlines architectural guidance for deploying Mattermost in an OOB scenario across [generic](#generic-oob-architecture), [AWS](#aws-based-oob-deployment), and [Azure](#azure-based-oob-deployment) environments.
|
||
|
||
## Generic OOB architecture
|
||
|
||
This platform-agnostic design establishes a fallback communication system, decoupled from primary networks and identity providers.
|
||
|
||
<img src={useBaseUrl('/images/oob-generic-architecture.png')} class="align-center" alt="Generic OOB Architecture" />
|
||
|
||
### Architecture components
|
||
|
||
- **Users (Cybersecurity, IR Teams)**: Pre-authorized personnel trained to access and use the OOB platform during emergencies. Credentials and access methods should be provisioned in advance.
|
||
- **Secure Access Layer**: A firewall or access gateway protecting entry into the OOB environment. May include network policies, IP allowlists, or WAFs.
|
||
- **Secondary IdP / Emergency Auth**: A dedicated identity provider not dependent on the organization’s primary SSO or IdP (e.g., a minimal Azure AD tenant, Okta org, or locally stored credentials with MFA).
|
||
- **VPN / Bastion Host**: A hardened jumpbox or independent VPN granting access to internal OOB components and Kubernetes control plane.
|
||
- **Failover DNS Manager**: Services like Cloudflare Load Balancer, Azure Traffic Manager, or Route 53 to monitor primary system health and reroute DNS to the OOB ingress.
|
||
- **Ingress Controller**: e.g., ingress-nginx to handle HTTP(S) routing within the cluster, directing traffic to Mattermost Web, API, and WebSocket endpoints.
|
||
- **Mattermost Operator & App**: Automates lifecycle of Mattermost instances in Kubernetes; ensures updates, scaling, and configuration consistency.
|
||
- **PostgreSQL Database**: Persistent storage for channels, messages, users, and configurations; can be self-hosted or managed externally with hardened access.
|
||
- **Backups**:
|
||
- **Velero**: Backs up Kubernetes resources to external object store (S3, Blob).
|
||
- **pg_dump / Cloud-native backup**: Regular logical exports or managed backups.
|
||
- **Object Storage**: Durable, versioned storage (e.g., S3, Azure Blob) for snapshots and dumps.
|
||
|
||
## AWS-based OOB deployment
|
||
|
||
Deploying Mattermost in AWS leverages managed services and strong isolation capabilities.
|
||
|
||
<img src={useBaseUrl('/images/oob-aws-architecture.png')} class="align-center" alt="AWS OOB Deployment diagram" />
|
||
|
||
### AWS components
|
||
|
||
- **Amazon EKS**: Managed Kubernetes for Mattermost Operator and application, with HA and auto-scaling.
|
||
- **Route 53**: DNS service for health-checked failover to the OOB environment.
|
||
- **Amazon Aurora PostgreSQL**: High-performance managed PostgreSQL with automated failovers.
|
||
- **Amazon S3**: Object storage for Velero snapshots, configurations, and database backups (with versioning and cross-region replication).
|
||
- **IAM Roles**: Scoped roles for EKS nodes and Velero using IRSA, enforcing least-privilege.
|
||
|
||
<Tip>
|
||
|
||
- Use separate AWS accounts or VPCs for full isolation.
|
||
- Configure Route 53 health checks against public or synthetic endpoints.
|
||
- Encrypt secrets and enforce least-privilege IAM policies.
|
||
|
||
</Tip>
|
||
|
||
## Azure-Based OOB deployment
|
||
|
||
For Azure customers, AKS and related services provide a robust OOB platform for Mattermost.
|
||
|
||
<img src={useBaseUrl('/images/oob-azure-architecture.png')} class="align-center" alt="Azure OOB Deployment diagram" />
|
||
|
||
### Azure components
|
||
|
||
- **Azure Kubernetes Service (AKS)**: Runs Mattermost workloads with native autoscaling and Azure AD integration.
|
||
- **Azure DNS + Traffic Manager**: Global traffic routing and DNS failover based on endpoint health.
|
||
- **Azure Database for PostgreSQL**: Managed DB service with automated backups and patching.
|
||
- **Azure Blob Storage**: Destination for Velero snapshots and DB dumps; supports lifecycle policies and secure access.
|
||
- **Azure AD (Secondary Tenant)**: Acts as the IdP for OOB; deployed in a separate directory with scoped permissions and MFA.
|
||
|
||
<Tip>
|
||
|
||
- Deploy in a dedicated subscription for security and billing separation.
|
||
- Use Azure Monitor for health checks and Traffic Manager probes.
|
||
- Harden AKS node pools and isolate workloads with namespaces and network policies.
|
||
|
||
</Tip>
|
||
|
||
## Operational guidelines
|
||
|
||
<table>
|
||
<colgroup>
|
||
<col style={{width: '20%'}} />
|
||
<col style={{width: '80%'}} />
|
||
</colgroup>
|
||
<thead>
|
||
<tr>
|
||
<th><strong>Domain</strong></th>
|
||
<th><strong>Recommendation</strong></th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td>Access Control</td>
|
||
<td>Isolate authentication from primary infrastructure. Use MFA, secure vaults, and emergency tokens. Preload user accounts in the secondary IdP.</td>
|
||
</tr>
|
||
<tr>
|
||
<td>DNS Failover</td>
|
||
<td>Employ TTL-aware health checks via Route 53 or Traffic Manager. Monitor critical HTTP(S) or TCP endpoints.</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Backup Strategy</td>
|
||
<td>Schedule frequent Velero and database backups. Store in versioned, encrypted object storage.</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Disaster Recovery Drills</td>
|
||
<td>Conduct quarterly failover exercises: DNS switching, Mattermost access, and data restores.</td>
|
||
</tr>
|
||
<tr>
|
||
<td>Security</td>
|
||
<td>Treat OOB as Tier 0. Use hardened OS images, audit logging, strict RBAC, and centralized monitoring. Rotate secrets regularly.</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
|
||
Mattermost can be an effective OOB communication platform when deployed with isolation, automation, and operational readiness. Whether on AWS, Azure, or a generic environment, prioritize independence, simplicity, and resilience. Pair these designs with standard IR playbooks and routine failover testing to build a confident, secure response capability.
|