snapshot of https://github.com/mattermost/mattermost.git (master, 2026-09-16; blobs > 131072B omitted)
Some checks failed
Server CI / Postgres (shard 0) (push) Has been skipped
Server CI / Postgres (shard 3) (push) Has been skipped
BuildEnv Docker Image / build-image (push) Failing after 4s
BuildEnv Docker Image / build-image-fips (push) Failing after 4s
Migration Review & Release Notes / Generate migration review & release notes (push) Failing after 3s
Server CI / Build Environment (push) Failing after 3s
Server CI / Postgres (shard 1) (push) Has been skipped
Tools CI / check-style (mattermost-govet) (push) Failing after 4s
Server CI / Postgres (shard 2) (push) Has been skipped
Tools CI / check-style (mmgotool) (push) Failing after 3s
Server CI / Elasticsearch v8 Compatibility (push) Has been skipped
Server CI / OpenSearch v2 Compatibility (push) Has been skipped
Server CI / Postgres FIPS (shard 0) (push) Has been skipped
Tools CI / Test (mattermost-govet) (push) Failing after 3s
Server CI / Postgres FIPS (shard 1) (push) Has been skipped
Server CI / Postgres FIPS (shard 2) (push) Has been skipped
Tools CI / Test (mmgotool) (push) Failing after 3s
Server CI / Postgres FIPS (shard 3) (push) Has been skipped
Server CI / Run mmctl tests (push) Has been skipped
Server CI / Run mmctl tests (FIPS) (push) Has been skipped
Server CI / Merge Postgres Test Results (push) Has been skipped
Server CI / Merge Postgres FIPS Test Results (push) Has been skipped
Server CI / Check backport migrations (push) Has been skipped
Server CI / Check generated files (push) Has been skipped
Server CI / check-style (push) Has been skipped
Server CI / Vet API (push) Has been skipped
Server CI / Build mattermost server app (push) Has been skipped
Server CI / Server CI Complete (push) Failing after 2s
Server CI / Server CI Artifacts (push) Has been skipped
Server CI / Sentry Upload (push) Has been skipped
Server CI / Server CI Report (push) Failing after 5m1s
Server CI Nightly Race / Race Detector (push) Has been skipped
CodeQL / Analyze (go) (push) Has been cancelled
CodeQL / Analyze (javascript) (push) Has been cancelled
Webapp CI Cache Warm / Warm Webapp node_modules Cache (push) Has been cancelled
E2E CI Cache Warm / Warm E2E npm Registry Cache (push) Has been cancelled
API / build (push) Has been cancelled
Web App CI / check-types (push) Has been cancelled
Web App CI / check-lint (push) Has been cancelled
Web App CI / check-i18n (push) Has been cancelled
Web App CI / check-external-links (push) Has been cancelled
Web App CI / test (platform) (push) Has been cancelled
Web App CI / test (mattermost-redux) (push) Has been cancelled
Web App CI / test (channels shard 1/4) (push) Has been cancelled
Web App CI / test (channels shard 2/4) (push) Has been cancelled
Web App CI / test (channels shard 3/4) (push) Has been cancelled
Web App CI / test (channels shard 4/4) (push) Has been cancelled
Web App CI / upload-coverage (push) Has been cancelled
Web App CI / build (push) Has been cancelled
YAML Lint / yamllint (push) Has been cancelled
Push mirrored docker images / cd/Push mirrored docker images (push) Has been cancelled
docs-cd / Build and deploy docs (push) Has been cancelled
Docs CI / build (push) Has been cancelled
E2E Cache Warm: Cypress deps / Warm Cypress deps cache (push) Has been cancelled
Scorecards supply-chain security / Scorecard analysis (push) Has been cancelled
Server CI Weekly / Postgres FIPS (push) Has been skipped
Server CI Weekly / Run mmctl tests (FIPS) (push) Has been skipped
Server CI Weekly / Postgres with binary parameters (push) Has been skipped
Server CI Weekly / Build Environment (push) Failing after 1s
Server CI Cache Warm / Warm Go Module and Build Cache (push) Failing after 2s
Server CI Nightly Race / Build Environment (push) Failing after 1s

This commit is contained in:
upstream-mirror
2026-09-16 00:29:55 +05:00
commit 240db4e2a7
15309 changed files with 2173406 additions and 0 deletions

View File

@@ -0,0 +1,613 @@
#!/usr/bin/env python3
"""
.github/scripts/check_config_changes_ci.py
CI script that detects notable changes across several Mattermost source files
and appends structured release-note entries to the PR description.
Checkers
────────
1. config.go — exported struct field additions/removals
2. api4/ — API endpoint additions/removals (Handle() calls)
3. audit_events.go — AuditEvent* constant additions/removals
4. Dockerfile.buildenv — Go (base-image) version changes
All inputs come from environment variables set by the GitHub Actions workflow:
GITHUB_TOKEN — built-in Actions token (pull-requests: write scope)
PR_NUMBER — pull request number
BASE_SHA — base commit SHA
HEAD_SHA — head commit SHA
REPO — owner/repo (e.g. mattermost/mattermost)
"""
import os
import re
import sys
import subprocess
import requests
from dataclasses import dataclass, field
from typing import Optional
# ── Environment ────────────────────────────────────────────────────────────────
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
PR_NUMBER = int(os.environ["PR_NUMBER"])
BASE_SHA = os.environ["BASE_SHA"]
HEAD_SHA = os.environ["HEAD_SHA"]
REPO = os.environ.get("REPO", "mattermost/mattermost")
BASE_URL = "https://api.github.com"
HEADERS = {
"Authorization": f"token {GITHUB_TOKEN}",
"Accept": "application/vnd.github.v3+json",
}
# Timeout for all GitHub API requests: (connect seconds, read seconds).
# Prevents the workflow from hanging indefinitely on a slow/unresponsive API.
_TIMEOUT = (5, 30)
# Paths watched by this script (must align with `paths:` in the workflow YAML)
WATCHED_PATHS = [
"server/public/model/config.go",
"server/channels/api4/",
"server/public/model/audit_events.go",
"server/build/Dockerfile.buildenv",
]
# ── Data types ─────────────────────────────────────────────────────────────────
@dataclass
class CheckResult:
"""Holds the findings from one checker."""
label: str # Section heading, e.g. "`config.json` Changes"
additions: list = field(default_factory=list)
removals: list = field(default_factory=list)
changes: list = field(default_factory=list) # for free-form entries (version bumps)
def has_findings(self) -> bool:
return bool(self.additions or self.removals or self.changes)
def to_markdown(self) -> str:
lines = [f"### {self.label}"]
if self.additions:
lines.append("**Added:** " + ", ".join(self.additions))
if self.removals:
lines.append("**Removed:** " + ", ".join(self.removals))
for change in self.changes:
lines.append(change)
return "\n".join(lines)
# ── Diff helpers ───────────────────────────────────────────────────────────────
def get_full_patch() -> str:
"""Return unified diff for all watched paths between base and head."""
result = subprocess.run(
["git", "diff", f"{BASE_SHA}...{HEAD_SHA}", "--"] + WATCHED_PATHS,
capture_output=True,
text=True,
check=True,
)
return result.stdout
def split_patch_by_file(full_patch: str) -> dict[str, str]:
"""
Split a multi-file unified diff into {filename: patch} mapping.
Filenames are the b-side (new) path, stripped of the 'b/' prefix.
"""
patches: dict[str, str] = {}
current_file: Optional[str] = None
current_lines: list[str] = []
for line in full_patch.splitlines(keepends=True):
if line.startswith("diff --git "):
if current_file:
patches[current_file] = "".join(current_lines)
current_lines = [line]
# Extract filename from "diff --git a/foo b/foo"
m = re.search(r" b/(.+)$", line.rstrip())
current_file = m.group(1) if m else None
else:
current_lines.append(line)
if current_file:
patches[current_file] = "".join(current_lines)
return patches
def file_at(ref: str, path: str) -> str:
"""Return the full contents of `path` at git ref `ref`, or '' if absent."""
try:
return subprocess.run(
["git", "show", f"{ref}:{path}"],
capture_output=True, text=True, check=True,
).stdout
except subprocess.CalledProcessError:
return ""
def _compute_merge_base() -> str:
"""Resolve the merge-base of BASE_SHA and HEAD_SHA.
Per-checker comparisons must use this rather than BASE_SHA. BASE_SHA is the
tip of the target branch at PR-event time; if that branch advances on a
watched file after the PR diverges, comparing branch-tip vs target-tip
would attribute those upstream edits to this PR (false add/remove).
`git diff A...B` already does this implicitly; the per-file snapshots must
match.
"""
return subprocess.run(
["git", "merge-base", BASE_SHA, HEAD_SHA],
capture_output=True, text=True, check=True,
).stdout.strip()
MERGE_BASE = _compute_merge_base()
# ── Checker 1 — config.go ──────────────────────────────────────────────────────
_CONFIG_PATH = "server/public/model/config.go"
_STRUCT_DECL_RE = re.compile(r"^type\s+(\w+)\s+struct\s*\{")
_FIELD_LINE_RE = re.compile(r"^\t([A-Z][A-Za-z0-9_]*)\s+\S")
def _scan_struct_fields(src: str) -> set[tuple[str, str]]:
"""
Walk Go source and return {(StructName, FieldName)} for every exported
field in every struct.
Uses a brace-depth stack so nested anonymous structs, interface bodies,
and function literals don't corrupt the enclosing struct context.
Named type declarations cannot be nested in Go, so the struct_stack
never grows beyond one entry for named structs.
"""
fields: set[tuple[str, str]] = set()
# Each entry: (struct_name, brace_depth_when_opened)
struct_stack: list[tuple[str, int]] = []
depth = 0
for line in src.splitlines():
sm = _STRUCT_DECL_RE.match(line)
if sm:
# Record depth *before* counting this line's braces
struct_stack.append((sm.group(1), depth))
depth += line.count("{") - line.count("}")
# Pop any structs whose closing brace has been passed
while struct_stack and depth <= struct_stack[-1][1]:
struct_stack.pop()
# Record fields only when we're directly inside the named struct body.
# The depth check (depth == struct_stack[0][1] + 1) ensures that fields
# inside nested anonymous struct { ... } blocks are not incorrectly
# attributed to the outer named struct.
if len(struct_stack) == 1 and depth == struct_stack[0][1] + 1:
fm = _FIELD_LINE_RE.match(line)
if fm:
fields.add((struct_stack[0][0], fm.group(1)))
return fields
def check_config(patches: dict[str, str]) -> CheckResult:
"""
Detect exported Go struct field additions/removals in config.go.
Compares full-file snapshots at MERGE_BASE and HEAD_SHA so that fields
are always attributed to the correct struct regardless of which diff
hunks are present.
"""
result = CheckResult(label="`config.json` Field Changes")
if _CONFIG_PATH not in patches:
return result
base_fields = _scan_struct_fields(file_at(MERGE_BASE, _CONFIG_PATH))
head_fields = _scan_struct_fields(file_at(HEAD_SHA, _CONFIG_PATH))
added = head_fields - base_fields
removed = base_fields - head_fields
result.additions = sorted(f"``{s}.{f}``" for s, f in added)
result.removals = sorted(f"``{s}.{f}``" for s, f in removed)
return result
# ── Checker 2 — api4/ ─────────────────────────────────────────────────────────
# Matches Handle() route registrations after whitespace-collapsing the source.
# Whitespace collapse makes multi-line declarations single-searchable.
# Group 1: path Group 2: handler func Group 3: raw Methods(...) content
#
# The wrapper pattern uses [^)]* so it tolerates any middleware arguments
# (e.g. r.APIHandler(...), r.ApiSessionRequired(..., isLocal=true), etc.)
# without having to enumerate every possible wrapper signature.
_HANDLE_RE = re.compile(
r'\.Handle\("([^"]*)"' # path
r',\s*[^)]*\((\w+)\)\)' # wrapper(...handlerFunc))
r'\.Methods\(([^)]+)\)', # .Methods(one or more methods)
)
_METHOD_RE = re.compile(r'(?:http\.Method)?(\w+)')
_HTTP_METHODS = {"GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"}
def _parse_methods(raw: str) -> list[str]:
"""Split raw Methods(...) content into individual uppercase HTTP verbs.
Filters against the set of known HTTP methods so that incidental
identifiers (handler names, constants, etc.) that happen to appear
inside Methods(...) don't produce spurious results.
"""
return [
verb
for token in raw.split(",")
if (m := _METHOD_RE.search(token.strip()))
and (verb := m.group(1).upper()) in _HTTP_METHODS
]
def _format_endpoint(path: str, handler: str, method: str) -> str:
return f"`{method.upper()} {path or '/'}` (`{handler}`)"
def _parse_endpoints(src: str) -> set[tuple[str, str, str]]:
"""
Parse Handle() registrations from a Go source file.
Whitespace-collapses the entire file first so multi-line declarations
(e.g. the 18 in group.go) are matched as a single token sequence.
Returns {(path, handler, method)} tuples.
"""
blob = " ".join(src.split())
endpoints: set[tuple[str, str, str]] = set()
for m in _HANDLE_RE.finditer(blob):
path, handler, methods_raw = m.group(1), m.group(2), m.group(3)
for method in _parse_methods(methods_raw):
endpoints.add((path or "/", handler, method))
return endpoints
def check_api(patches: dict[str, str]) -> CheckResult:
"""
Detect API endpoint additions/removals in the api4/ directory.
Compares full-file snapshots at MERGE_BASE and HEAD_SHA via set arithmetic,
so multi-line and multi-method registrations are handled correctly.
"""
result = CheckResult(label="API Changes (`api4`)")
api4_patches = {
fname: patch
for fname, patch in patches.items()
if fname.startswith("server/channels/api4/") and fname.endswith(".go")
}
if not api4_patches:
return result
added_eps: set[tuple[str, str, str]] = set()
removed_eps: set[tuple[str, str, str]] = set()
for fname, patch in api4_patches.items():
base_eps = _parse_endpoints(file_at(MERGE_BASE, fname))
head_eps = _parse_endpoints(file_at(HEAD_SHA, fname))
added_eps |= head_eps - base_eps
removed_eps |= base_eps - head_eps
# Anchor the check to avoid false positives from unrelated source text
if re.search(r"^new file mode \d+", patch, re.MULTILINE):
result.changes.append(f"🆕 New API file: `{fname.split('/')[-1]}`")
if re.search(r"^deleted file mode \d+", patch, re.MULTILINE):
result.changes.append(f"🗑️ Removed API file: `{fname.split('/')[-1]}`")
result.additions = sorted(_format_endpoint(p, h, m) for p, h, m in added_eps)
result.removals = sorted(_format_endpoint(p, h, m) for p, h, m in removed_eps)
return result
# ── Checker 3 — audit_events.go ───────────────────────────────────────────────
_AUDIT_EVENT_PATH = "server/public/model/audit_events.go"
_AUDIT_CONST_RE = re.compile(r"^\t(AuditEvent\w+)\s*=")
def _parse_audit_events(src: str) -> set[str]:
return {m.group(1) for line in src.splitlines() if (m := _AUDIT_CONST_RE.match(line))}
def check_audit_events(patches: dict[str, str]) -> CheckResult:
"""
Detect AuditEvent* constant additions/removals.
Uses full-file snapshots at MERGE_BASE/HEAD_SHA so reorderings and
cross-constant name collisions don't produce false results.
"""
result = CheckResult(label="Audit Log Event Changes")
if _AUDIT_EVENT_PATH not in patches:
return result
base_events = _parse_audit_events(file_at(MERGE_BASE, _AUDIT_EVENT_PATH))
head_events = _parse_audit_events(file_at(HEAD_SHA, _AUDIT_EVENT_PATH))
result.additions = sorted(f"``{e}``" for e in head_events - base_events)
result.removals = sorted(f"``{e}``" for e in base_events - head_events)
return result
# ── Checker 4 — Dockerfile.buildenv (Go version) ──────────────────────────────
# The Go version lives in the base image tag, e.g.:
# FROM mattermost/golang-bullseye:1.25.8@sha256:...
_DOCKERFILE_PATH = "server/build/Dockerfile.buildenv"
_IMAGE_VER_RE = re.compile(r"^FROM \S+:([0-9]+\.[0-9]+(?:\.[0-9]+)?)")
def _parse_go_version(src: str) -> Optional[str]:
for line in src.splitlines():
m = _IMAGE_VER_RE.match(line.strip())
if m:
return m.group(1)
return None
def check_go_version(patches: dict[str, str]) -> CheckResult:
"""
Detect Go runtime version changes via the base image tag.
Uses full-file snapshots so the version is read from the actual file
state at each ref rather than reconstructed from patch lines.
"""
result = CheckResult(label="Go Runtime Version")
if _DOCKERFILE_PATH not in patches:
return result
old_ver = _parse_go_version(file_at(MERGE_BASE, _DOCKERFILE_PATH))
new_ver = _parse_go_version(file_at(HEAD_SHA, _DOCKERFILE_PATH))
if old_ver and new_ver and old_ver != new_ver:
result.changes.append(f"Go updated: ``{old_ver}`` → ``{new_ver}``")
elif new_ver and not old_ver:
result.additions.append(f"``{new_ver}``")
return result
# ── PR description helpers ─────────────────────────────────────────────────────
# Matches lines that were auto-generated by this script so they can be stripped
# before re-injecting a fresh set on subsequent commits.
# Handles both single-backtick (older runs) and double-backtick (current) format.
_AUTO_LINE_RE = re.compile(
r"^(Added|Removed) `{1,2}[^`]+`{1,2}.*(configuration setting|API endpoint|audit log event)\."
r"|^Go runtime updated from \S+ to \S+\."
r"|^Go runtime set to `{1,2}[^`]+`{1,2}\."
r"|^🆕 New API file:"
r"|^🗑️ Removed API file:"
)
# Matches placeholder content inside a release-note fence that means "nothing
# to report yet" (e.g. NONE, N/A, ---). When detected, we replace the
# placeholder rather than appending alongside it.
_PLACEHOLDER_RE = re.compile(r"^\s*(?:NONE|N/?A|-+)\s*$", re.IGNORECASE)
def _format_lines(result: CheckResult) -> list[str]:
"""Produce natural-language lines for one checker result."""
lines = []
if "`config.json`" in result.label:
for item in result.additions:
lines.append(f"Added {item} configuration setting.")
for item in result.removals:
lines.append(f"Removed {item} configuration setting.")
elif "API Changes" in result.label:
for item in result.additions:
lines.append(f"Added {item} API endpoint.")
for item in result.removals:
lines.append(f"Removed {item} API endpoint.")
lines.extend(result.changes) # new/deleted file entries
elif "Audit" in result.label:
for item in result.additions:
lines.append(f"Added {item} audit log event.")
for item in result.removals:
lines.append(f"Removed {item} audit log event.")
elif "Go Runtime" in result.label:
for item in result.additions:
# item is e.g. "``1.22``"
lines.append(f"Go runtime set to {item}.")
for c in result.changes:
# c arrives as "Go updated: ``1.21`` → ``1.22``" — rewrite it
m = re.search(r"``([^`]+)``\s*→\s*``([^`]+)``", c)
if m:
lines.append(f"Go runtime updated from {m.group(1)} to {m.group(2)}.")
else:
lines.append(c)
return lines
def build_pr_note(results: list[CheckResult]) -> str:
"""Assemble all findings into a clean plain-text block."""
lines = []
for r in results:
if r.has_findings():
lines.extend(_format_lines(r))
return "\n".join(lines)
def strip_old_note(body: str) -> str:
"""
Remove previously auto-generated lines from the PR description.
Primary path — lines inside the ```release-note ... ``` fence.
Fallback path — auto-generated lines that were appended outside any fence
(e.g. via the ## Release Notes section on earlier runs).
Lines are identified by pattern rather than visible markers, so the PR
description stays clean for human readers.
"""
def _clean_fence(m: re.Match) -> str:
open_tag, content, close_tag = m.group(1), m.group(2), m.group(3)
cleaned_lines = [
line for line in content.split("\n")
if not _AUTO_LINE_RE.match(line.strip())
]
return open_tag + "\n".join(cleaned_lines) + close_tag
cleaned = re.sub(
r"(```release-note)(.*?)(```)",
_clean_fence,
body or "",
flags=re.DOTALL | re.IGNORECASE,
)
# Fallback: strip any auto-generated lines that appear outside a fence
# (written by an older version of this script or via the header-inject path).
cleaned_lines = [
line for line in cleaned.splitlines()
if not _AUTO_LINE_RE.match(line.strip())
]
return "\n".join(cleaned_lines).rstrip()
def inject_note(body: str, note: str) -> str:
"""
Insert `note` using this priority order:
1. INSIDE the ```release-note block, before its closing ```
(Mattermost convention — keeps everything in one place for reviewers)
2. After a recognised release-notes section header (## Release Notes, etc.)
3. Fallback: append a new ## Release Notes section at the end
"""
body = strip_old_note(body)
if not note:
return body
# 1. Mattermost-style ```release-note ... ``` block — inject INSIDE the fence.
# If the fence currently contains only a placeholder (NONE / N/A / ---),
# replace the placeholder rather than appending alongside it.
release_note_block = re.search(
r"(```release-note)(.*?)(```)",
body,
flags=re.DOTALL | re.IGNORECASE,
)
if release_note_block:
open_tag = release_note_block.group(1)
content = release_note_block.group(2) # everything between the fences
close_tag = release_note_block.group(3)
block_start = release_note_block.start()
block_end = release_note_block.end()
# Strip leading/trailing newlines inside the fence for comparison
inner = content.strip()
if _PLACEHOLDER_RE.match(inner):
# Replace the entire fence with a fresh one
new_block = f"{open_tag}\n{note}\n{close_tag}"
else:
# Append before the closing fence
new_block = open_tag + content + note + "\n" + close_tag
return body[:block_start] + new_block + body[block_end:]
# 2. Markdown section headers
for header in ["## Release Notes", "## Changelog", "## What Changed", "## What's Changed"]:
if header.lower() in body.lower():
idx = body.lower().index(header.lower()) + len(header)
return body[:idx] + "\n\n" + note + body[idx:]
# 3. Fallback — append
return body + "\n\n## Release Notes\n\n" + note
# ── GitHub API ─────────────────────────────────────────────────────────────────
def get_pr_body() -> str:
r = requests.get(
f"{BASE_URL}/repos/{REPO}/pulls/{PR_NUMBER}",
headers=HEADERS,
timeout=_TIMEOUT,
)
r.raise_for_status()
return r.json().get("body") or ""
def update_pr_body(new_body: str) -> None:
r = requests.patch(
f"{BASE_URL}/repos/{REPO}/pulls/{PR_NUMBER}",
headers=HEADERS,
json={"body": new_body},
timeout=_TIMEOUT,
)
r.raise_for_status()
# ── Main ───────────────────────────────────────────────────────────────────────
def main():
print(f"📋 PR #{PR_NUMBER} | base {BASE_SHA[:8]} → head {HEAD_SHA[:8]}")
print("🔍 Collecting diffs …")
full_patch = get_full_patch()
if not full_patch.strip():
print("ℹ️ No changes in watched paths. Nothing to do.")
return
patches = split_patch_by_file(full_patch)
print(f" {len(patches)} file(s) changed in watched paths.\n")
# Run all checkers
checkers = [
check_config,
check_api,
check_audit_events,
check_go_version,
]
results: list[CheckResult] = [fn(patches) for fn in checkers]
for r in results:
if r.has_findings():
print(f" ✅ {r.label}")
if r.additions:
print(f" Added: {', '.join(r.additions)}")
if r.removals:
print(f" Removed: {', '.join(r.removals)}")
for c in r.changes:
print(f" {c}")
else:
print(f" – {r.label}: no changes")
note = build_pr_note(results)
if not note:
print("\nℹ️ No notable changes found across all checkers.")
return
print("\n🔄 Fetching PR description …")
body = get_pr_body()
new_body = inject_note(body, note)
if new_body == body:
print("ℹ️ PR description already up to date — no changes needed.")
return
update_pr_body(new_body)
print(f"✅ PR #{PR_NUMBER} description updated.")
if __name__ == "__main__":
try:
main()
except subprocess.CalledProcessError as e:
print(f"❌ git diff failed:\n{e.stderr}", file=sys.stderr)
sys.exit(1)
except requests.HTTPError as e:
# Avoid dumping the full response body (can be large / noisy).
# status + reason gives enough context for debugging (e.g. "403 Forbidden").
reason = e.response.reason or "unknown"
print(f"❌ GitHub API error: {e.response.status_code} {reason}", file=sys.stderr)
sys.exit(1)

480
.github/scripts/generate_changelog.py vendored Normal file
View File

@@ -0,0 +1,480 @@
#!/usr/bin/env python3
"""
Generate a changelog from merged PR release notes for a given milestone.
Expects these environment variables:
GITHUB_TOKEN - GitHub personal access token or Actions token
REPOS - Comma-separated list of repositories in "owner/repo" format
(e.g. "mattermost/mattermost,mattermost/enterprise")
MILESTONE - Milestone title (e.g. "v11.7.0")
VERSION - Version label for the changelog entry (e.g. "v11.7.0")
RELEASE_TYPE - (optional) "feature" (default) or "esr" (Extended Support Release).
RELEASE_DATE - (optional) Release day date (e.g. "2026-05-15"). Defaults to today.
GO_VERSION - (optional) Go version used in this release (e.g. "go1.22.5").
If not provided, changelog notes it is unchanged from previous release.
BLOG_POST_URL - (optional) Blog post URL for the Improvements section.
Auto-constructed from VERSION if not provided.
ANTHROPIC_API_KEY - (optional) If set, release notes are polished by Claude
before being written to the changelog.
CHANGELOG_PATH - (optional) Path to the changelog file to update. Defaults to
"CHANGELOG.md". In mattermost/mattermost this should be
"docs/main/product-overview/mattermost-v11-changelog.mdx".
The target changelog is an MDX file: it opens with YAML frontmatter, imports MDX
components, and uses JSX elements such as <Note> and <Important>. Generated content
must therefore be MDX-safe (see the "MDX safety" rule in SYSTEM_PROMPT).
"""
import os
import re
import requests
from datetime import date
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
GITHUB_TOKEN = os.environ["GITHUB_TOKEN"]
REPOS = [r.strip() for r in os.environ["REPOS"].split(",") if r.strip()]
MILESTONE_TITLE = os.environ["MILESTONE"]
VERSION = os.environ["VERSION"]
HEADERS = {
"Authorization": f"token {GITHUB_TOKEN}",
"Accept": "application/vnd.github.v3+json",
}
# Shared timeout for all GitHub API requests (seconds).
# Prevents the workflow from hanging indefinitely if the API stalls.
API_TIMEOUT = 30
# Standard requests retry strategy using urllib3 — handles transient failures
# (rate limits, server errors) with exponential back-off and respects Retry-After.
GITHUB_RETRY = Retry(
total=5,
connect=5,
read=5,
backoff_factor=1,
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=frozenset(["GET"]),
respect_retry_after_header=True,
)
GITHUB_ADAPTER = HTTPAdapter(max_retries=GITHUB_RETRY)
SESSION = requests.Session()
SESSION.headers.update(HEADERS)
SESSION.mount("https://", GITHUB_ADAPTER)
SYSTEM_PROMPT = """You are an expert technical writer and copyeditor for Mattermost software release notes. Your task is to transform raw, unstructured release notes from pull requests into a clean, categorized, and grammatically correct changelog entry that matches Mattermost's established changelog format exactly.
Here are your instructions:
1. **Section structure:** Use `###` for top-level sections and `####` for subsections. Only include sections that have relevant content — do not output empty sections. Do NOT add horizontal rules or line separators between sections. Do NOT add a blank line between a section/subsection heading and its first bullet point.
Top-level sections and their subsections, in this order:
- `### Upgrade Impact` — for changes that affect upgrading, with subsections as applicable:
- `#### Database Schema Changes` — schema migrations such as new tables, new columns, changed columns, or new indexes. Example items: "Added a new ``Watermarks`` table.", "Added a new column ``DeleteAt`` to the ``ChannelMembers`` table."
- `#### config.json` — new or changed configuration settings. Use this exact block format for each plan grouping (no blank line between the `#### config.json` heading and the description paragraph, and no blank line between the description paragraph and the first bullet):
New setting options were added to ``config.json``. Below is a list of the additions and their default values on install. The settings can be modified in ``config.json``, or the System Console when available.
- **Changes to Enterprise Advanced plan:**
- Under ``ExperimentalSettings`` in ``config.json``, added ``EnableWatermark`` configuration setting to add watermarking toggle in the server.
Adapt the plan name (e.g. "Changes to All plans:", "Changes to Enterprise plan:", "Changes to Enterprise Advanced plan:") and list each setting change as a bullet under the appropriate plan heading.
- `#### Compatibility` — minimum version requirement changes for browsers, OS, or clients. Example: "Updated minimum Edge and Chrome versions to 146+."
- `### Improvements` — for new features and enhancements only. Do NOT place items beginning with "Fixed..." here — those belong in Bug Fixes. Begin this section with the line `See BLOG_POST_LINK on the highlights in our latest release.` (use the exact placeholder `BLOG_POST_LINK` — it will be replaced automatically with a Markdown link), followed by a blank line before the first `####` subsection heading. Then add subsections as applicable:
- `#### User Interface` — user interface and UX changes and new visual features. Pre-packaged plugin version updates go at the TOP of this subsection, before other items. Always write "user interface" in full — never abbreviate as "UI".
- `#### Plugins/Integrations` — plugin and integration improvements (use as a separate subsection when there are enough items to warrant it)
- `#### Administration` — System Console features, logging, support packet changes
- `#### mmctl` — mmctl command additions or changes (use as a separate subsection when there are enough items to warrant it)
- `#### Performance` — performance improvements
- `### Bug Fixes` — corrections to defects. All items starting with "Fixed an issue..." go here, even if the raw note appeared in an Improvements context.
- `### API Changes` — API additions, changes, or deprecations
- `### WebSocket Event Changes` — new or changed WebSocket events, if applicable
- `### Audit Log Event Changes` — new or changed audit log events
- `### Go Version` — Always include this section. The Go version content will be injected automatically — output only this heading with no content beneath it.
- `### Open Source Components` — open source component additions or removals. Format each item as: "Added ``<package>`` to <repo_url>." or "Removed ``<package>`` from <repo_url>." Example: "Added ``x/text`` to https://github.com/mattermost/mattermost/." Only include if there are relevant notes.
- `### Security` — security-related fixes not already covered under Bug Fixes
- `### Contributors` — contributor acknowledgements. Only include if the raw notes contain contributor information; otherwise omit this section entirely (it is usually added manually after generation).
2. **Sentence patterns:** Follow these conventions consistently:
- New features and additions: "Added [feature]..." or "Added support for [feature]..."
- Bug fixes: "Fixed an issue where..." or "Fixed an issue with..." — never use "Fixed a bug"; always use "Fixed an issue".
- Improvements to existing things: "Improved [thing]..." or "Updated [thing]..."
- Removals: "Removed [thing]..."
3. **Terminology:** Always write "user interface" in full — never use the abbreviation "UI". Always spell out messaging abbreviations: "DM" → "Direct Message", "GM" → "Group Message", "DM/GM" → "Direct/Group Message".
4. **Code formatting:** Use double backticks for all of the following:
- Configuration settings (e.g., ``ServiceSettings.EnableDynamicClientRegistration``)
- API endpoints (e.g., ``/api/v4/posts``)
- Command names (e.g., ``mmctl license get``)
- Environment variables (e.g., ``MM_LOG_PATH``)
- Database table and column names (e.g., ``channelmembers.autotranslation``)
- File names (e.g., ``config.json``)
- Feature flags (e.g., ``MM_FEATUREFLAGS_CJKSEARCH``)
- Package names in Open Source Components (e.g., ``x/text``)
5. **Markdown formatting:** Indent each bullet point with two spaces (e.g., ` - item`). Ensure correct and clean Markdown syntax throughout. Do not insert horizontal rules (`---`) or any other separators between sections. Do not add a blank line between a section/subsection heading and its first bullet point.
6. **MDX safety:** The changelog is an `.mdx` file, so raw `{`, `}`, and `<` characters are parsed as JSX and will break the docs build. Therefore:
- Never output a bare `{` or `}` in prose. If a release note needs braces, wrap the text in double backticks (e.g. ``{"key": "value"}``) so it becomes inline code.
- Never output a bare `<` followed by a letter (e.g. `<Note>`, `<div>`, `<T>`). Wrap such text in double backticks, or rewrite it (e.g. "less than 5" instead of "<5").
- Do not emit JSX/HTML components yourself. Components like `<Note>` and `<Important>` already exist in the file and are added manually — never generate, duplicate, or close them.
- Use standard Markdown links (`[text](url)`) and plain Markdown bullets only.
7. **License requirements:** When a feature requires a specific Mattermost license, note it inline at the end of the bullet point (e.g., "Requires Enterprise Advanced license" or "Requires Enterprise license").
8. **One section per release note:** Each raw release note must appear in exactly one section — whichever section best fits its primary purpose. Do not split a single release note across multiple sections or subsections, even if it touches more than one area (e.g. an admin feature that also introduces an API endpoint belongs entirely in Administration, not partially in Administration and partially in API Changes).
9. **Proofreading:** Correct any typos, grammatical errors, awkward phrasing, or inconsistencies. Replace any instance of "Fixed a bug" with "Fixed an issue". Aim for clear, concise, and professional language.
10. **Tone:** Maintain a neutral, informative, and professional tone consistent with technical documentation.
11. **Focus:** Output only the section content (headings and bullet points). Do not include the release version header line or any introductory or concluding remarks from yourself."""
def get_milestone_number(repo: str, title: str) -> int | None:
"""Look up the numeric ID for a milestone by its title in the given repo."""
url = f"https://api.github.com/repos/{repo}/milestones"
page = 1
recent_titles = []
while True:
params = {
"state": "all",
"per_page": 100,
"page": page,
"sort": "due_on", # sort by due date
"direction": "desc", # most recently due first, so active milestones are found quickly
}
resp = SESSION.get(url, params=params, timeout=API_TIMEOUT)
resp.raise_for_status()
milestones = resp.json()
if not milestones:
break
for m in milestones:
if m["title"] == title:
return m["number"]
if page == 1:
recent_titles = [m["title"] for m in milestones[:10]]
page += 1
print(f" ⚠️ Milestone '{title}' not found in {repo} — skipping")
if recent_titles:
print(f" Most recently due milestones: {', '.join(recent_titles)}")
return None
def get_merged_prs(repo: str, milestone_number: int) -> list:
"""Fetch all merged PRs belonging to the given milestone in the given repo."""
prs = []
page = 1
while True:
url = f"https://api.github.com/repos/{repo}/issues"
params = {
"milestone": milestone_number,
"state": "closed",
"per_page": 100,
"page": page,
}
resp = SESSION.get(url, params=params, timeout=API_TIMEOUT)
resp.raise_for_status()
items = resp.json()
if not items:
break
for item in items:
if "pull_request" not in item:
continue # plain issue, not a PR
if item["pull_request"].get("merged_at"):
prs.append(item)
else:
# merged_at can be null for very recently merged PRs due to an API
# propagation delay. Verify directly against the Pulls API.
pr_url = f"https://api.github.com/repos/{repo}/pulls/{item['number']}"
pr_resp = SESSION.get(pr_url, timeout=API_TIMEOUT)
pr_resp.raise_for_status()
if pr_resp.json().get("merged"):
prs.append(item)
page += 1
return prs
def extract_release_notes(body: str) -> list[str] | None:
"""
Extract release note text from a PR body.
Looks for a '#### Release Note' section, then pulls the content of any
fenced code blocks within it (plain ``` or ```release-note).
Returns None if the section is missing or all entries are NONE.
"""
if not body:
return None
# Normalize line endings (GitHub API may return \r\n on some PR bodies)
body = body.replace("\r\n", "\n").replace("\r", "\n")
notes = []
# Primary path: look for a '#### Release Note(s)' section heading
section_match = re.search(
r"####\s+Release\s+Notes?\s*\n(.*?)(?=\n####|\Z)",
body,
re.DOTALL | re.IGNORECASE,
)
if section_match:
section = section_match.group(1)
# Strip HTML comments (the instructional block in the template)
section = re.sub(r"<!--.*?-->", "", section, flags=re.DOTALL)
# Extract fenced code blocks (supports both ``` and ```release-note)
code_blocks = re.findall(r"```(?:release-note)?\s*\n(.*?)\n?```", section, re.DOTALL)
for block in code_blocks:
content = block.strip()
if content and content.upper() != "NONE":
notes.append(content)
# Fallback to plain text only when there are no code blocks at all in the section
if not notes and "```" not in section:
plain = section.strip()
if plain and plain.upper() != "NONE":
notes.append(plain)
# Secondary path: scan the entire body for ```release-note blocks.
# Catches PRs that use the block format without a #### Release Note heading.
if not notes:
body_no_comments = re.sub(r"<!--.*?-->", "", body, flags=re.DOTALL)
raw_blocks = re.findall(r"```release-note\s*\n(.*?)\n?```", body_no_comments, re.DOTALL)
for block in raw_blocks:
content = block.strip()
if content and content.upper() != "NONE":
notes.append(content)
return notes if notes else None
def polish_with_ai(raw_notes: list[str]) -> str:
"""
Send raw release notes to Claude for categorization, formatting, and proofreading.
Falls back to a simple bullet list if ANTHROPIC_API_KEY is not set.
"""
api_key = os.environ.get("ANTHROPIC_API_KEY")
if not api_key:
print("ℹ️ ANTHROPIC_API_KEY not set — skipping AI polish, using raw notes")
return "\n".join(f"- {note}" for note in raw_notes)
try:
import anthropic
except ImportError:
print("⚠️ anthropic package not installed — skipping AI polish")
return "\n".join(f"- {note}" for note in raw_notes)
print("✨ Sending notes to Claude for categorization and proofreading...")
client = anthropic.Anthropic(api_key=api_key)
raw_text = "\n\n---\n\n".join(raw_notes)
user_message = f"Here are the raw release notes to process:\n\n{raw_text}"
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=4096,
system=SYSTEM_PROMPT,
messages=[{"role": "user", "content": user_message}],
)
return response.content[0].text.strip()
def normalize_go_version(go_version: str) -> str:
"""Normalize a Go version string to v-prefixed form.
Accepts both "go1.22.5" (Go toolchain convention) and "v1.22.5"
(Mattermost changelog convention) and always returns the latter.
"""
v = go_version.strip()
if v.startswith("go"):
v = v[2:] # strip the "go" prefix
if not v.startswith("v"):
v = "v" + v
return v
def extract_previous_go_version(changelog_path: str) -> str | None:
"""Scan the changelog file for the most recently documented Go version.
Used when GO_VERSION is not supplied (i.e. unchanged from the previous
release) so the generated entry still shows a concrete version number
rather than a generic "unchanged" message.
"""
if not os.path.exists(changelog_path):
return None
with open(changelog_path, "r", encoding="utf-8") as f:
content = f.read()
match = re.search(r"is built with Go\s+``(v?[\d.]+)``", content)
if match:
return normalize_go_version(match.group(1))
return None
def insert_changelog_entry(entry: str, changelog_path: str = "CHANGELOG.md") -> None:
"""Insert a new version entry into the changelog after the static file header block.
The v11 changelog is an MDX file whose header consists of YAML frontmatter, MDX
component imports, and two JSX blocks (<Important> and <Note>). New entries are
inserted immediately after that header so it is preserved intact.
"""
existing = ""
if os.path.exists(changelog_path):
with open(changelog_path, "r", encoding="utf-8") as f:
existing = f.read()
# The header block ends with the platform scope note, which closes a <Note> component.
HEADER_END_MARKER = "may not represent all affected configurations.\n\n</Note>"
if not existing.strip():
# New or empty file: the entry is the whole document.
new_content = entry
elif HEADER_END_MARKER in existing:
idx = existing.index(HEADER_END_MARKER) + len(HEADER_END_MARKER)
new_content = existing[:idx] + "\n\n\n" + entry + existing[idx:]
else:
# Fallback: insert immediately before the first "## Release" heading, which keeps
# frontmatter, imports, and any header components above the newest entry.
first_release = re.search(r"(?m)^## Release\b", existing)
if not first_release:
# Never prepend above the file header: MDX requires frontmatter to be the very
# first thing in the file, so blindly prepending would corrupt the document and
# break the docs build. Fail loudly instead — the header format has changed and
# HEADER_END_MARKER needs updating.
raise RuntimeError(
f"Could not find an insertion point in {changelog_path}: neither the header "
f"marker ({HEADER_END_MARKER!r}) nor a '## Release' heading was found. "
"The changelog header format has likely changed — update HEADER_END_MARKER "
"in this script to match."
)
new_content = (
existing[: first_release.start()].rstrip()
+ "\n\n\n"
+ entry.rstrip()
+ "\n\n"
+ existing[first_release.start():]
)
with open(changelog_path, "w", encoding="utf-8") as f:
f.write(new_content)
def main():
print(f"🔍 Milestone: {MILESTONE_TITLE} | Repos: {', '.join(REPOS)}\n")
all_notes = []
total_prs = 0
no_notes_prs = []
for repo in REPOS:
print(f"── {repo}")
milestone_number = get_milestone_number(repo, MILESTONE_TITLE)
if milestone_number is None:
continue
prs = get_merged_prs(repo, milestone_number)
print(f" 📋 Found {len(prs)} merged PR(s)")
total_prs += len(prs)
for pr in sorted(prs, key=lambda p: p["number"]):
notes = extract_release_notes(pr.get("body") or "")
if notes:
for note in notes:
all_notes.append(note)
print(f" ✅ #{pr['number']}: {pr['title']}")
else:
no_notes_prs.append((repo, pr))
print(f" ⏭️ #{pr['number']}: {pr['title']} (NONE / no notes)")
print()
# Derive short version for heading/anchor: "v11.7.0" → "v11.7", "11.7.0" → "v11.7"
version_short = "v" + re.sub(r"\.0$", "", VERSION.lstrip("v"))
release_type = os.environ.get("RELEASE_TYPE", "feature").strip().lower()
release_date = os.environ.get("RELEASE_DATE", "").strip() or date.today().strftime("%Y-%m-%d")
go_version = os.environ.get("GO_VERSION", "").strip()
changelog_path = os.environ.get("CHANGELOG_PATH", "CHANGELOG.md")
# Anchor slugs replace dots with dashes: "v11.9" → "v11-9"
version_slug_anchor = version_short.replace(".", "-")
if release_type == "esr":
release_label = "Extended Support Release"
anchor_slug = f"release-{version_slug_anchor}-extended-support-release"
else:
release_label = "Feature Release"
anchor_slug = f"release-{version_slug_anchor}-feature-release"
# MDX heading anchor. The opening brace is escaped (\{) because MDX would otherwise
# parse it as a JSX expression.
heading = (
f"## Release {version_short} - "
f"[{release_label}]"
f"(https://docs.mattermost.com/product-overview/release-policy.html#release-types)"
f" \\{{#{anchor_slug}}}"
)
entry = f"{heading}\n\n**Release day: {release_date}**\n\n"
# Build the Go Version section content.
# Mattermost uses a single bullet point in this section.
# When GO_VERSION is not provided (unchanged from previous release), we look up
# the previous version from the changelog so the line still shows a concrete number.
# Note: this section uses a single leading space before the bullet, matching the
# established formatting of the ### Go Version section in the changelog file.
if go_version:
formatted_go = normalize_go_version(go_version)
go_section = f"### Go Version\n - {version_short} is built with Go ``{formatted_go}``."
else:
prev_go = extract_previous_go_version(changelog_path)
if prev_go:
go_section = f"### Go Version\n - {version_short} is built with Go ``{prev_go}``."
else:
go_section = f"### Go Version\n - {version_short} uses the same Go version as the previous release."
if all_notes:
polished = polish_with_ai(all_notes)
blog_url = os.environ.get("BLOG_POST_URL", "").strip()
if not blog_url:
# Auto-construct short URL (no patch suffix): v11.6.0 → mattermost-v11-6-is-now-available
version_slug = re.sub(r"-\d+$", "", VERSION.lstrip("v").replace(".", "-"))
blog_url = f"https://mattermost.com/blog/mattermost-v{version_slug}-is-now-available/"
print(f"ℹ️ No blog post URL provided — using auto-constructed URL: {blog_url}")
# Format as a Markdown link matching existing changelog style: [this blog post](url)
blog_link = f"[this blog post]({blog_url})"
polished = polished.replace("BLOG_POST_LINK", blog_link)
# Inject the Go Version section: replace the placeholder heading the AI outputs,
# or insert it before ### Open Source Components / ### Security to preserve
# section order, or append at the end if neither anchor exists.
if re.search(r"(?m)^### Go Version\b", polished):
polished = re.sub(
r"(?ms)^### Go Version\b.*?(?=^### \S|\Z)",
go_section + "\n\n",
polished,
count=1,
)
else:
anchor = re.search(
r"(?m)^### (?:Open Source Components|Security|Contributors)\b", polished
)
if anchor:
idx = anchor.start()
polished = polished[:idx].rstrip() + "\n\n" + go_section + "\n\n" + polished[idx:]
else:
polished = polished.rstrip() + "\n\n" + go_section + "\n"
entry += polished + "\n"
else:
entry += go_section + "\n"
entry += "\n_No other release notes for this version._\n"
insert_changelog_entry(entry, changelog_path)
prs_with_notes = total_prs - len(no_notes_prs)
print(f"✅ Changelog updated with notes from {prs_with_notes} PR(s) across {len(REPOS)} repo(s)")
if no_notes_prs:
print(f"\n⚠️ {len(no_notes_prs)} PR(s) had no release notes (marked NONE or section missing):")
for repo, pr in no_notes_prs:
print(f" [{repo}] #{pr['number']}: {pr['title']}")
if __name__ == "__main__":
main()

358
.github/scripts/migration_automation.py vendored Normal file
View File

@@ -0,0 +1,358 @@
#!/usr/bin/env python3
"""
migration_automation.py
Triggered by GitHub Actions when a new .up.sql file lands on master.
Each Mattermost migration ships as a pair of files: NNNNNN_slug.up.sql
(applied on upgrade) and NNNNNN_slug.down.sql (applied on rollback). The
workflow filters on .up.sql files only because:
- The .up.sql is the canonical identifier for a new migration.
- Both files are committed together, so detecting the up file is enough
to locate the pair.
- We never want to trigger a separate review run for a down migration
that arrives without a matching up migration.
For each new .up.sql file detected:
1. Reads the .up.sql and its paired .down.sql (if present) from the repo
2. Fetches the review-migration skill from the AI marketplace
3. Calls Claude to produce the schema review report
4. Calls Claude to produce the RST release note draft + changelog summary
5. Appends the combined output to $GITHUB_STEP_SUMMARY so it renders
inline in the Actions run UI — no branch, PR, or extra secrets needed
Required environment variables:
ANTHROPIC_API_KEY — repo secret
GITHUB_STEP_SUMMARY — file path provided automatically by GitHub Actions
"""
import os
import re
import sys
import time
import urllib.error
import urllib.request
from pathlib import Path
import anthropic
# ── Config ────────────────────────────────────────────────────────────────────
MODEL = "claude-sonnet-4-6"
# The marketplace URL intentionally points to /main so the script always uses
# the latest version of Ben Cooke's review-migration skill. Any push to
# mattermost-ai-marketplace/main WILL change the skill used by this workflow —
# that is by design. To pin to a specific revision instead, replace "main"
# with a commit SHA (e.g. "/abc1234/plugins/...").
MARKETPLACE_BASE = (
"https://raw.githubusercontent.com/mattermost/mattermost-ai-marketplace"
"/main/plugins/review-migration/skills/review-migration"
)
MM_GUIDE_URL = (
"https://developers.mattermost.com/contribute/more-info/server/schema-migration-guide/"
)
# Retry configuration
MAX_RETRIES = 3
RETRY_BACKOFF_BASE = 2 # seconds; waits 2, 4, 8 between attempts
# Allowlist pattern for migration file paths accepted by this script.
# Format: server/channels/db/migrations/postgres/NNNNNN_slug.up.sql
MIGRATION_PATH_RE = re.compile(
r"^server/channels/db/migrations/postgres/\d{6}_[\w-]+\.up\.sql$"
)
RELEASE_NOTES_SKILL = """
You are helping a Mattermost release manager write database migration release notes.
When given a migration review report, produce two things:
## 1. Release Note Block
Three parts in this exact order:
### Part A — Description
A clear, concise paragraph (2–5 sentences) describing what changed and why.
Write for database admins and self-hosters who need to understand the change at a glance.
Cover what tables/columns/indexes changed, the purpose, and any performance impact.
Do NOT include upgrade instructions here.
Inline code formatting: use DOUBLE backticks for all table names, column names, and
identifiers in prose (RST/Sphinx style). E.g. ``roles``, ``schemeid``, ``permission_level``.
Never use single backticks in description prose.
### Part B — Fixed compatibility statement (copy verbatim every time):
The migrations are fully backwards-compatible and no database downtime is expected for this upgrade. The SQL queries included are:
### Part C — SQL in RST format (NOT markdown fences):
.. code-block:: sql
<SQL here, indented 4 spaces>
For multiple SQL dialects use a separate labeled ``.. code-block:: sql`` block for each.
## 2. One-Line Changelog Summary
**Changelog summary:** `<one sentence under ~30 words, ending with an impact note>`
"""
# ── Startup validation ────────────────────────────────────────────────────────
def validate_env() -> str:
"""
Validate required env vars and return the GITHUB_STEP_SUMMARY path.
Exits with a clear error message if any required var is missing.
ANTHROPIC_API_KEY is validated here so we fail fast with a readable
message rather than an opaque SDK error, but it is not returned —
the Anthropic SDK reads it directly from the environment.
"""
required = {
"ANTHROPIC_API_KEY": "Anthropic API key (repo secret)",
"GITHUB_STEP_SUMMARY": "Job summary file path (provided automatically by Actions)",
}
missing = [
f" {var} ({desc})"
for var, desc in required.items()
if not os.environ.get(var, "").strip()
]
if missing:
print("ERROR: The following required environment variables are not set:\n")
for m in missing:
print(m)
sys.exit(1)
return os.environ["GITHUB_STEP_SUMMARY"]
# ── Input validation ──────────────────────────────────────────────────────────
def validate_migration_paths(paths: list[str]) -> None:
"""
Server-side validation: reject any path that does not match the canonical
migration file pattern. Defense-in-depth against path traversal or
unexpected input if the workflow filter is ever bypassed.
"""
invalid = [p for p in paths if not MIGRATION_PATH_RE.match(p)]
if invalid:
print(
"ERROR: The following paths do not match the expected migration pattern "
f"({MIGRATION_PATH_RE.pattern}):"
)
for p in invalid:
print(f" {p!r}")
print("Aborting — only files under the canonical migrations directory are accepted.")
sys.exit(1)
# ── Retry helper ──────────────────────────────────────────────────────────────
def _is_retryable_http_error(code: int) -> bool:
return code in (429, 500, 502, 503, 504)
def with_retry(fn, *, label: str, retries: int = MAX_RETRIES):
"""Call fn() up to `retries` times with exponential backoff."""
last_exc = None
for attempt in range(1, retries + 1):
try:
return fn()
except urllib.error.HTTPError as e:
last_exc = e
if not _is_retryable_http_error(e.code):
raise
wait = RETRY_BACKOFF_BASE ** attempt
print(f" [{label}] HTTP {e.code} on attempt {attempt}/{retries}. "
f"Retrying in {wait}s…")
time.sleep(wait)
except (urllib.error.URLError, TimeoutError, OSError) as e:
last_exc = e
wait = RETRY_BACKOFF_BASE ** attempt
print(f" [{label}] Network error on attempt {attempt}/{retries}: {e}. "
f"Retrying in {wait}s…")
time.sleep(wait)
except anthropic.RateLimitError as e:
last_exc = e
wait = RETRY_BACKOFF_BASE ** attempt
print(f" [{label}] Anthropic rate limit on attempt {attempt}/{retries}. "
f"Retrying in {wait}s…")
time.sleep(wait)
except anthropic.APIStatusError as e:
last_exc = e
if e.status_code not in (429, 500, 502, 503, 529):
raise
wait = RETRY_BACKOFF_BASE ** attempt
print(f" [{label}] Anthropic API error {e.status_code} on attempt "
f"{attempt}/{retries}. Retrying in {wait}s…")
time.sleep(wait)
raise last_exc
# ── HTTP helpers ──────────────────────────────────────────────────────────────
def fetch_url(url: str) -> str:
def _do():
req = urllib.request.Request(url, headers={"User-Agent": "migration-bot/1.0"})
with urllib.request.urlopen(req, timeout=30) as r:
return r.read().decode()
return with_retry(_do, label=f"GET {url}")
# ── Claude calls ──────────────────────────────────────────────────────────────
def call_claude(system: str, user: str) -> str:
# No explicit api_key — the SDK reads ANTHROPIC_API_KEY from the environment
# automatically, keeping the key out of the call stack and reducing the
# chance of accidental logging.
client = anthropic.Anthropic()
def _do():
msg = client.messages.create(
model=MODEL,
max_tokens=4096,
system=system,
messages=[{"role": "user", "content": user}],
)
return msg.content[0].text
return with_retry(_do, label="Anthropic API")
def run_review(
up_sql: str,
down_sql: str,
skill_md: str,
reference_md: str,
guide_text: str,
) -> str:
system = f"""You are a Mattermost database migration reviewer.
Your instructions (the review-migration skill):
{skill_md}
Supplementary internal reference:
{reference_md}
Official Mattermost DB Migration Guide (source of truth for rules and lock types):
{guide_text[:12000]}
"""
user = f"""Review this migration and produce the full report in the markdown
template format from your instructions.
## Up migration (.up.sql):
```sql
{up_sql}
```
## Down migration (.down.sql):
```sql
{down_sql}
```
"""
return call_claude(system, user)
def run_release_notes(review: str) -> str:
user = f"""Based on this migration review, produce the formatted release note
block and one-line changelog summary.
## Migration Review:
{review}
"""
return call_claude(RELEASE_NOTES_SKILL, user)
# ── Processing ────────────────────────────────────────────────────────────────
def process(
up_path: Path,
skill_md: str,
reference_md: str,
guide_text: str,
) -> str:
"""Run both skills for one migration. Returns the combined markdown."""
up_sql = up_path.read_text(encoding="utf-8")
down_path = up_path.with_name(up_path.name.replace(".up.sql", ".down.sql"))
down_sql = down_path.read_text(encoding="utf-8") if down_path.exists() else "(no down migration)"
print(" Running review-migration skill…")
review = run_review(up_sql, down_sql, skill_md, reference_md, guide_text)
print(" Running migration-release-notes skill…")
release_notes = run_release_notes(review)
return f"{review}\n\n---\n\n## Release Note Draft\n\n{release_notes}"
# ── Job Summary output ────────────────────────────────────────────────────────
def write_summary(summary_path: str, sections: list[tuple[str, str]]) -> None:
"""Append all review sections to the GitHub Actions job summary file."""
with open(summary_path, "a", encoding="utf-8") as f:
f.write("# 🔍 Migration Review & Release Notes\n\n")
for name, content in sections:
f.write(f"## `{name}`\n\n")
f.write(content)
f.write("\n\n---\n\n")
print(f"\nSummary written to job summary ({len(sections)} migration(s)).")
# ── Main ──────────────────────────────────────────────────────────────────────
def main() -> None:
summary_path = validate_env()
new_files = [f for f in sys.argv[1:] if f.strip()]
if not new_files:
print("No new migration files provided. Exiting.")
return
validate_migration_paths(new_files)
print(f"Processing {len(new_files)} migration(s): {new_files}\n")
# Fetch shared resources once
print("Fetching review-migration skill from AI marketplace…")
try:
skill_md = fetch_url(f"{MARKETPLACE_BASE}/SKILL.md")
reference_md = fetch_url(f"{MARKETPLACE_BASE}/reference.md")
except Exception as e:
print(
f"\nERROR: Could not fetch review-migration skill.\n"
f"URL: {MARKETPLACE_BASE}\nReason: {e}"
)
sys.exit(1)
print("Fetching Mattermost DB Migration Guide…")
try:
guide_text = fetch_url(MM_GUIDE_URL)
except Exception as e:
print(f"WARNING: Could not fetch DB Migration Guide ({e}). Proceeding without it.")
guide_text = ""
print()
# Process each migration
sections: list[tuple[str, str]] = []
for filepath in new_files:
up_path = Path(filepath)
print(f"→ {up_path.name}")
try:
content = process(up_path, skill_md, reference_md, guide_text)
except anthropic.AuthenticationError as e:
print(f"\nERROR: Anthropic authentication failed. "
f"Check that ANTHROPIC_API_KEY is valid.\nDetail: {e}")
sys.exit(1)
except Exception as e:
print(f"\nERROR: Failed to process {up_path.name}: {e}")
sys.exit(1)
sections.append((up_path.name, content))
write_summary(summary_path, sections)
if __name__ == "__main__":
main()