snapshot of https://github.com/RocketChat/Rocket.Chat.git (develop, 2026-09-16; blobs > 131072B omitted)
Some checks failed
Create issues from TODOs / todos (push) Failing after 3s
Close inactive issues / close-issues (push) Has been cancelled
Deploy GitHub Pages / deploy-preview (push) Has been cancelled
CI / ⚙️ Variables Setup (push) Has been cancelled
CI / ⚙️ Test Guard (push) Has been cancelled
CI / 🚀 Notify external services - draft (push) Has been cancelled
CI / 📦 Build Packages (push) Has been cancelled
CI / 📦 Meteor Build (coverage) (push) Has been cancelled
CI / 📦 Meteor Build (production) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [account-service presence-service omnichannel-transcript-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [account-service presence-service omnichannel-transcript-service], fips) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [authorization-service queue-worker-service ddp-streamer-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [authorization-service queue-worker-service ddp-streamer-service], fips) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], coverage) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], fips) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [account-service presence-service omnichannel-transcript-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [authorization-service queue-worker-service ddp-streamer-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [rocketchat], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [rocketchat], coverage) (push) Has been cancelled
CI / 🚢 Publish Docker Images (ghcr.io) (push) Has been cancelled
CI / 📦 Track Image Sizes (push) Has been cancelled
CI / 🔎 Code Check - Action Lint (push) Has been cancelled
CI / 🔎 Code Check (push) Has been cancelled
CI / 🔨 Test Storybook (push) Has been cancelled
CI / 🔨 Test Unit (push) Has been cancelled
CI / 🔨 Test API (CE) (push) Has been cancelled
CI / 🔨 Test API Livechat (CE) (push) Has been cancelled
CI / 🔨 Test UI (CE) (push) Has been cancelled
CI / 🔨 Test API (EE) (push) Has been cancelled
CI / 🔨 Test API Livechat (EE) (push) Has been cancelled
CI / 🔨 Test API Apps (deno-runtime - EE) (push) Has been cancelled
CI / 🔨 Test UI (EE) (push) Has been cancelled
CI / 🔨 Test API (FIPS) (push) Has been cancelled
CI / 🔨 Test API Livechat (FIPS) (push) Has been cancelled
CI / 🔨 Test UI (FIPS) (push) Has been cancelled
CI / 🔨 Test Federation Matrix (push) Has been cancelled
CI / 📊 Report Coverage (push) Has been cancelled
CI / ✅ Tests Done (push) Has been cancelled
CI / 🚀 Publish build assets (push) Has been cancelled
CI / 🚀 Publish Docker Images (DockerHub) (push) Has been cancelled
CI / 🚀 Notify external services (push) Has been cancelled
CI / Update Version Durability (push) Has been cancelled
Release / Release (push) Has been cancelled
Auto-close duplicate issues / auto-close-duplicates (push) Failing after 1s
Code scanning - action / CodeQL-Build (push) Failing after 2s
Some checks failed
Create issues from TODOs / todos (push) Failing after 3s
Close inactive issues / close-issues (push) Has been cancelled
Deploy GitHub Pages / deploy-preview (push) Has been cancelled
CI / ⚙️ Variables Setup (push) Has been cancelled
CI / ⚙️ Test Guard (push) Has been cancelled
CI / 🚀 Notify external services - draft (push) Has been cancelled
CI / 📦 Build Packages (push) Has been cancelled
CI / 📦 Meteor Build (coverage) (push) Has been cancelled
CI / 📦 Meteor Build (production) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [account-service presence-service omnichannel-transcript-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [account-service presence-service omnichannel-transcript-service], fips) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [authorization-service queue-worker-service ddp-streamer-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [authorization-service queue-worker-service ddp-streamer-service], fips) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], coverage) (push) Has been cancelled
CI / 🚢 Build Docker (amd64, [rocketchat], fips) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [account-service presence-service omnichannel-transcript-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [authorization-service queue-worker-service ddp-streamer-service], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [rocketchat], ${{ (github.event_name != 'release' && github.ref != 'refs/heads/develop') && 'coverage' || 'production' }}) (push) Has been cancelled
CI / 🚢 Build Docker (arm64, [rocketchat], coverage) (push) Has been cancelled
CI / 🚢 Publish Docker Images (ghcr.io) (push) Has been cancelled
CI / 📦 Track Image Sizes (push) Has been cancelled
CI / 🔎 Code Check - Action Lint (push) Has been cancelled
CI / 🔎 Code Check (push) Has been cancelled
CI / 🔨 Test Storybook (push) Has been cancelled
CI / 🔨 Test Unit (push) Has been cancelled
CI / 🔨 Test API (CE) (push) Has been cancelled
CI / 🔨 Test API Livechat (CE) (push) Has been cancelled
CI / 🔨 Test UI (CE) (push) Has been cancelled
CI / 🔨 Test API (EE) (push) Has been cancelled
CI / 🔨 Test API Livechat (EE) (push) Has been cancelled
CI / 🔨 Test API Apps (deno-runtime - EE) (push) Has been cancelled
CI / 🔨 Test UI (EE) (push) Has been cancelled
CI / 🔨 Test API (FIPS) (push) Has been cancelled
CI / 🔨 Test API Livechat (FIPS) (push) Has been cancelled
CI / 🔨 Test UI (FIPS) (push) Has been cancelled
CI / 🔨 Test Federation Matrix (push) Has been cancelled
CI / 📊 Report Coverage (push) Has been cancelled
CI / ✅ Tests Done (push) Has been cancelled
CI / 🚀 Publish build assets (push) Has been cancelled
CI / 🚀 Publish Docker Images (DockerHub) (push) Has been cancelled
CI / 🚀 Notify external services (push) Has been cancelled
CI / Update Version Durability (push) Has been cancelled
Release / Release (push) Has been cancelled
Auto-close duplicate issues / auto-close-duplicates (push) Failing after 1s
Code scanning - action / CodeQL-Build (push) Failing after 2s
This commit is contained in:
upstream-mirror
463
docs/adr/0001-app-accessor-logic-in-base-runtime.md
Normal file
463
docs/adr/0001-app-accessor-logic-in-base-runtime.md
Normal file
@@ -0,0 +1,463 @@
|
||||
# ADR 0001 — Accessor logic lives in the base-runtime, not the host
|
||||
|
||||
## Status
|
||||
|
||||
**Accepted — implemented.** All phases landed. Supersedes the `apps-accessor-consolidation` proposal.
|
||||
|
||||
- **Date:** 2026-07
|
||||
- **Scope:** `packages/apps` (host) and `packages/apps/base-runtime` (subprocess runtime)
|
||||
|
||||
## Decision
|
||||
|
||||
1. The base-runtime is the **single source of truth** for accessor behavior.
|
||||
2. The `accessor:*` message category — and `handleAccessorMessage` with it — is **eliminated**;
|
||||
`bridges:*` is the only host-bound RPC category originated by accessors.
|
||||
3. Registration accessors, whose side effects had no bridge `do*`, ride `bridges:*` through one new
|
||||
internal host bridge, `AppResourceBridge`.
|
||||
|
||||
**Drift rule.** Where the runtime and host implementations had diverged: **merge non-conflicting logic;
|
||||
on genuine conflict the runtime supersedes the host.** The runtime implementations are what apps
|
||||
actually experienced in production (host copies were unreachable from the subprocess). Two refinements:
|
||||
|
||||
- **Direction-aware.** For **MOVE** accessors the host was authoritative (the app hit it via the
|
||||
proxy), so merging preserves live behavior. For **RECONCILE** accessors the runtime was
|
||||
authoritative, so folding in *host-only* logic **adds observable behavior** — a deliberate, tested
|
||||
change, never a neutral reconciliation.
|
||||
- **Drift is classified before adoption.** Runtime-is-superset → adopt silently. Runtime *dropped or
|
||||
changed* observable behavior → requires a justification that the drop is intentional plus a test
|
||||
pinning the resulting shape, so the migration cannot immortalize an accidental regression.
|
||||
|
||||
## End state
|
||||
|
||||
`BaseRuntimeSubprocessController` handles exactly one app-originated RPC category — `bridges:*` — with
|
||||
a single dispatcher and a single permission model. The `@rocket.chat/apps` host contains no accessor
|
||||
implementation and no accessor manager (`src/server/accessors/` and `AppAccessorManager` are deleted);
|
||||
the only accessor-shaped host code is the bridge layer (`AppBridges`/`do*`) plus `AppResourceBridge`.
|
||||
Every accessor an app uses is built and resolved inside `packages/apps/base-runtime`.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
Out of scope here; unblocked or motivated by this work.
|
||||
|
||||
1. **Bridge-level input hardening** — move load-bearing caps into the bridge `do*` wrappers (the real
|
||||
trust boundary). Caps like `getMessages` ≤ 100 and `removeUsersFromRoom` ≤ 50 were *already*
|
||||
bypassable before this migration: app code runs via `new Function` in the runtime's own JS realm
|
||||
(globals are shadowed, not a security boundary — see `construct.ts`) and can emit arbitrary
|
||||
`bridges:*` messages. This work introduced **no new exposure**, it only made the advisory-ness
|
||||
visible. Independent security item, non-blocking.
|
||||
2. **Replace the restart registration guard** with an explicit `reinitialize` request so the subprocess
|
||||
knows not to re-send registrations, removing controller-state-dependent message dropping.
|
||||
3. **`messenger.sendRequest` timeout** — the runtime-side TODO is more prominent now that all accessor
|
||||
traffic flows through it.
|
||||
4. **`createProcessorId` suffix check** (`SchedulerModify`) — job-id namespacing uses `includes`
|
||||
(substring) instead of `endsWith` (suffix), so an appId substring in the *middle* of a job id makes
|
||||
it skip the suffix, risking collisions and jobs `cancelJob` can't reach. Flagged in review but not
|
||||
fixed inside the migration: the port was a faithful copy of the host code, and fixing only the
|
||||
runtime copy would have re-introduced the drift this ADR removes. The host copy is gone now, so it
|
||||
can land standalone with a test.
|
||||
5. **Consolidated host↔subprocess protocol/SDK** — one typed manifest of every host-bound method and
|
||||
its accepted params (the APP_ID bucket table is the seed), replacing hand-rolled message strings and
|
||||
per-param normalization judgment. Also where bucket C's nested-identity gap gets closed (host
|
||||
substitutes nested `appId` fields, or `HttpBridge.doCall` ignores the payload `appId` and uses the
|
||||
connection-known id).
|
||||
6. **Deferred review findings** — reviewers flagged the items below on ported accessors; in every case
|
||||
the port was byte-identical to its host original, so the flag is *pre-existing* behavior, not a
|
||||
regression. Deferred for the same reason as 4, and likewise unblocked now.
|
||||
|
||||
*Latent behavior (identical in host):*
|
||||
- `UploadCreator.uploadBuffer` uses `Object.hasOwn(descriptor, 'user')` to decide whether to fetch
|
||||
the app user; with `{ user: null }` (nullable per `IUploadDescriptor`) and no visitor token it
|
||||
treats the user as present and sends `userId: undefined`. A value check (`!descriptor.user`) would
|
||||
fall back to the app user.
|
||||
- `RoomRead.getMessages` accepts `0`/negative `limit` — the guard only rejects `> 100`, unlike the
|
||||
`1–100` checks in `getAllRooms`/`getUnreadByUser`.
|
||||
- `RoomRead.getMessages` mutates the caller's `options` in place (`options.limit ??= 100`,
|
||||
`options.showThreadMessages ??= true`). Fix = copy before defaulting.
|
||||
- `ServerSettingRead.getOneById` casts the bridge result to `ISetting` with no null guard, so a
|
||||
missing setting returns `null`/`undefined` despite the typed return; sibling `getValueById` throws.
|
||||
- `UIController`'s deprecated surface APIs (`openModalView`, `updateModalView`,
|
||||
`openContextualBarView`, `updateContextualBarView`) call the serializers directly, skipping the
|
||||
`UIHelper.assignIds` block-ID scoping that `openSurfaceView`/`updateSurfaceView` apply, so legacy
|
||||
interactions can emit un-scoped block IDs.
|
||||
|
||||
*Type/cosmetic (no runtime change):* `MessageRead.getSenderUser`/`getRoom` cast the bridge result
|
||||
(`as IMessage`) and declare `Promise<IUser>`/`Promise<IRoom>`, so the `undefined` they return for a
|
||||
missing message — and which `IMessageRead` does declare — is absent from their own signatures.
|
||||
|
||||
## Context
|
||||
|
||||
Accessors used to have two implementations. The subprocess got a thin proxy layer that forwarded
|
||||
`accessor:*` JSON-RPC messages to the host, where `handleAccessorMessage` walked the *real* accessor
|
||||
objects built by `AppAccessorManager` from `src/server/accessors/`. That split existed because, when
|
||||
the Deno runtime was introduced, module-resolution conflicts made sharing code between subprocess and
|
||||
host impractical. The constraint no longer held: base-runtime assumes Node module resolution, is
|
||||
compiled by `tsc`, and is consumed by Deno as TypeScript source through an import map.
|
||||
|
||||
Three call paths coexisted:
|
||||
|
||||
| Path | Example | Mechanism |
|
||||
| --- | --- | --- |
|
||||
| **Pure proxy** | `read.getRoomReader().getById(id)` | `proxify()` sends `accessor:getReader:getRoomReader:getById`; host resolves it via `handleAccessorMessage` → `AppAccessorManager` → `RoomBridge.doGetById` |
|
||||
| **Local + direct bridge call** | `modify.getCreator().finish(builder)` | runtime validates locally, then sends `bridges:getMessageBridge:doCreate` to the permission-wrapped bridge method |
|
||||
| **Fully local** | `configurationExtend.http.provideDefaultHeader()` | state lives entirely in the subprocess |
|
||||
|
||||
Paths 2 and 3 were already the target, so the work was migrating path 1 into them and closing the gaps
|
||||
that forced accessors to stay on path 1.
|
||||
|
||||
**Why `bridges:*` is sufficient security-wise.** Permission enforcement never lived in the accessors;
|
||||
it lives in the host bridge `do*` wrappers (`MessageBridge.doCreate` checks
|
||||
`AppPermissionManager.hasPermission(appId, AppPermissions.message.write)`). Three properties make that
|
||||
channel a complete trust boundary: `handleBridgeMessage` only dispatches `do`-prefixed methods; any
|
||||
param equal to the literal `'APP_ID'` is replaced host-side with the app's real id, so a subprocess
|
||||
cannot impersonate another app; and `hasPermission(appId, …)` takes the appId explicitly, with no
|
||||
call-stack or async-context magic that breaks when the caller leaves the host process. So moving
|
||||
accessor logic into the subprocess **loses nothing permission-wise**, provided every side effect still
|
||||
flows through a `do*` method. The corollary: accessor-level validation is *advisory* inside the sandbox
|
||||
— but it already was, since a subprocess can emit arbitrary `bridges:*` messages (follow-up 1).
|
||||
|
||||
**Who consumed the host accessors.** Almost exclusively the RPC channel. The only load-bearing
|
||||
non-RPC use was `AppListenerManager.executePostMessageSent` calling
|
||||
`getReader(appId).getUserReader().getAppUser()` to gate `IPostMessageSentToBot`. `accessors/Http.ts`
|
||||
reached back into `AppAccessorManager` for HTTP pre/post handlers (dead for subprocess apps);
|
||||
`accessors/AppAccessors.ts` was never instantiated; and `AppApiManager`,
|
||||
`AppSlashCommandManager`, `AppVideoConfProviderManager`, `AppOutboundCommunicationProviderManager`
|
||||
threaded `this.accessors` into executors that ignored it (`_accessors`) — legacy in-process-era
|
||||
signatures. Nothing in `apps/meteor` touched `AppAccessorManager`.
|
||||
|
||||
## Architecture
|
||||
|
||||
### `AppResourceBridge`
|
||||
|
||||
Every bridge gap reduces to one capability: *invoke an app-scoped operation on a host manager or the
|
||||
app's own storage item*. One internal bridge lets them ride the existing permission-checked,
|
||||
appId-substituting `bridges:*` channel:
|
||||
|
||||
```text
|
||||
AppResourceBridge (host-side, internal — not part of the app-facing definition surface)
|
||||
├── doProvideSetting(setting, appId) → ProxiedApp.getStorageItem() mutation (SettingsExtend)
|
||||
├── doGetAppSetting(id, appId) → app.getStorageItem().settings[id] (SettingRead)
|
||||
├── doUpdateAppSetting(setting, appId) → AppSettingsManager.updateAppSetting
|
||||
├── doProvideSlashCommand(command, appId) → AppSlashCommandManager.addCommand
|
||||
├── doModifySlashCommand(command, appId) → AppSlashCommandManager.modifyCommand
|
||||
├── doEnableSlashCommand(name, appId) → AppSlashCommandManager.enableCommand
|
||||
├── doDisableSlashCommand(name, appId) → AppSlashCommandManager.disableCommand
|
||||
├── doProvideApi(api, appId) → AppApiManager.addApi
|
||||
├── doListApis(appId) → AppApiManager.listApis
|
||||
├── doRegisterProcessors(processors, appId) → AppSchedulerManager.registerProcessors
|
||||
├── doRegisterActionButton(button, appId) → UIActionButtonManager.registerActionButton
|
||||
├── doRegisterExternalComponent(component, appId) → AppExternalComponentManager.addExternalComponent
|
||||
├── doProvideVideoConfProvider(provider, appId) → AppVideoConfProviderManager.addProvider
|
||||
└── doRegisterOutboundProvider(provider, appId) → AppOutboundCommunicationProviderManager.addProvider
|
||||
```
|
||||
|
||||
- **Placement and ownership.** The managers live in `packages/apps`, so the bridge does too. It is
|
||||
*not* an abstract getter on `AppBridges` — that would force every orchestrator/host embedding to
|
||||
implement it. `handleBridgeMessage` resolves names from `this.bridges` first, then from a dedicated
|
||||
`getAppResourceBridge` lookup. The bridge is **stateless** (every method takes the `appId` and
|
||||
delegates to an `AppManager`-owned manager), so `AppManager` constructs it once
|
||||
(`getAppResourceBridge()`) and every `BaseRuntimeSubprocessController` shares that instance. The
|
||||
`do*` gate and `'APP_ID'` substitution apply identically.
|
||||
- **Permission and error behavior preserved, including throw-vs-silent.** Each method calls the same
|
||||
manager the host accessor used: `AppVideoConfProviderManager`/`AppOutboundCommunicationProviderManager`
|
||||
`addProvider` still **throw** `PermissionDeniedError`, `UIActionButtonManager.registerActionButton`
|
||||
still logs-and-refuses silently, and conflict errors (`CommandAlreadyExistsError`,
|
||||
`CommandHasAlreadyBeenTouchedError`, `PathAlreadyExistsError`, `VideoConfProviderAlreadyExistsError`)
|
||||
still propagate as JSON-RPC errors.
|
||||
- **The restart guard moved with it — via an explicit method set, not a name prefix.**
|
||||
`handleAccessorMessage` used to short-circuit any `getConfigurationExtend` call to `success(null)`
|
||||
while the controller is `restarting` (re-running `app:initialize` must not re-register resources, but
|
||||
the subprocess must still rebuild its local `AppObjectRegistry` entries). That guard keyed on the
|
||||
accessor *origin*, structurally capturing the whole registration surface. The bridge version
|
||||
deliberately does **not** re-derive the boundary from `doProvide*`/`doRegister*` prefixes — a future
|
||||
registration method named otherwise would double-register during restart, and a future
|
||||
non-registration method matching the prefix would be silently dropped. Instead a static
|
||||
`AppResourceBridge.REGISTRATION_METHODS` set enumerates exactly the suppressed names (registrations
|
||||
only — not `doGetAppSetting`/`doUpdateAppSetting`/`doModifySlashCommand`/`doEnable*`/`doDisable*`/
|
||||
`doListApis`), consulted by `handleBridgeMessage` while `state === 'restarting'`. Runtime `_proxy`
|
||||
wrappers still stash live instances into `AppObjectRegistry` *before* the RPC, so local
|
||||
re-registration works. Stopgap; follow-up 2 is the real fix.
|
||||
|
||||
### `bridgeCall`
|
||||
|
||||
Runtime accessors do not hand-roll message strings. `base-runtime/src/lib/bridges/bridgeCall.ts`
|
||||
exposes one helper:
|
||||
|
||||
```ts
|
||||
bridgeCall<T>(senderFn, bridge: BridgeName, method: `do${string}`, ...params: unknown[]): Promise<T>
|
||||
```
|
||||
|
||||
It encodes `bridges:<bridge>:<method>`, unwraps `response.result`, and normalizes rejections through
|
||||
`formatErrorResponse`. `BridgeName` is the union of host bridges reachable from the subprocess
|
||||
(including `getAppResourceBridge`); the `` `do${string}` `` type mirrors the host gate. Accessors get a
|
||||
`(request) => this.senderFn(request)` thunk so instance-level `senderFn` test stubs stay intercepted.
|
||||
(An earlier iteration used a `RemoteBridges` proxy facade with a getter per bridge; `bridgeCall`
|
||||
replaced it with the same guarantees and no `Proxy` layer.)
|
||||
|
||||
**It deliberately does not auto-inject `'APP_ID'`.** Identity is passed explicitly and positionally by
|
||||
each accessor, exactly as the host accessor did, so a faithful copy keeps caller-identity params
|
||||
carrying the sentinel and leaves app-supplied argument-appIds raw *by construction*. Centralizing
|
||||
injection would be cleaner but would make the argument-appId exceptions the dangerous default.
|
||||
|
||||
### `APP_ID` sentinel everywhere
|
||||
|
||||
All bridge params denoting the *calling app* use the `'APP_ID'` literal, substituted host-side. This
|
||||
closed the pre-existing inconsistency (drift 7) and removed the runtime's reliance on
|
||||
`AppObjectRegistry.get('id')` for identity — it stays available for non-identity uses like
|
||||
`UIHelper.assignIds` block-id prefixes and scheduler job-id suffixes.
|
||||
|
||||
This is a per-param judgment that fails in **both** directions, caught by neither the type-checker nor
|
||||
a green suite: over-normalizing a genuine app-supplied argument-appId silently strips the app's ability
|
||||
to pass any appId; under-normalizing a caller-identity param re-opens the impersonation gap. So every
|
||||
`do*` appId param was audited into exactly one of three buckets — only A is normalized:
|
||||
|
||||
| Bucket | Params | Value sent |
|
||||
| --- | --- | --- |
|
||||
| **A** — caller identity, top-level positional | The trailing appId of every `Notifier`, `ModifyCreator`, `ModifyUpdater`, `ModifyExtender`, reader, and modifier bridge call (`doNotifyUser`, `doNotifyRoom`, `doTyping`, `doGetAppUser`, `doCreate*`, `doGetById`, `doUpdate`, …), and everything not listed in B/C | `'APP_ID'` sentinel |
|
||||
| **B** — app-supplied *argument* appId | `ModerationBridge.doReport`, `.doDismissReportsByMessageId`, `.doDismissReportsByUserId` (`ModerationModify` ignores its constructor `_appId` and forwards the caller's arg); `UserBridge.doDeleteUsersCreatedByApp` (the id is the *target* app that created bot users); `UserBridge.doGetAppUser` (legacy param, removing it risks a break) | raw caller value |
|
||||
| **C** — nested identity field | `HttpBridge.doCall`'s single payload param `{ appId, method, url, request }` — `appId` is a *field*, and the host substitution maps only top-level positional params, so the sentinel would travel through unrewritten | resolved `AppObjectRegistry.get('id')` |
|
||||
|
||||
Before this work every site sent the raw resolved id except `ModifyCreator._finishMessage`'s
|
||||
`doGetAppUser` — the lone correct one, and the drift that motivated the normalization. Host-observable
|
||||
behavior is unchanged (the host still ends up with the real id); the wire now carries the sentinel and
|
||||
the impersonation gap is closed for buckets A and B. **Audit rule when adding a `do*` call:** an appId
|
||||
populated from a method parameter rather than the app's own identity belongs in bucket B.
|
||||
|
||||
Bucket C is a pre-existing impersonation surface predating this migration (the sandbox picks the id it
|
||||
sends), so this work introduced no new exposure there; closing it needs a host change — either
|
||||
substituting nested `appId` fields for known payload params, or having `HttpBridge.doCall` ignore the
|
||||
payload `appId` and use the connection-known id (follow-up 5).
|
||||
|
||||
### Module-resolution constraints (the original reason for the proxies)
|
||||
|
||||
The Deno subprocess consumes base-runtime **as TypeScript source** through the import map
|
||||
(`@rocket.chat/apps/base-runtime/` → `../base-runtime/src/`); compiled `dist` CJS is avoided because
|
||||
its `require()` calls bypass the import map. So moved code may import only
|
||||
`@rocket.chat/apps-engine/definition/*` (mapped to source), `node:` builtins, and npm deps already in
|
||||
`deno.jsonc`'s import map or Deno cache — which every moved accessor satisfies.
|
||||
|
||||
`UIHelper` was **duplicated** into base-runtime rather than shared. It is a small pure helper (only
|
||||
`node:crypto` plus two type-only imports) needed by `ModifyCreator`/`ModifyUpdater`/`UIController`, and
|
||||
the runtime used to import it from `@rocket.chat/apps/dist/server/misc/UIHelper` — exactly the fragile
|
||||
dist-CJS path the runtime avoids. Having the host import it *back* from base-runtime was rejected:
|
||||
base-runtime already imports `apps/dist`, so that would create an unlinearizable **build cycle**
|
||||
(flipping the order only moves it). Phase 0 copied it into `base-runtime/src/lib/` and left the host
|
||||
copy for the still-living host accessors; since all three `src/` importers were MOVE accessors, that
|
||||
copy died in teardown — single source of truth *at teardown*, no cross-package import, no build-order
|
||||
change. Other small carry-overs: `createProcessorId` (a local function) and `RoomRead`'s
|
||||
`GetMessagesSortableFields` sort validation (constants from apps-engine definitions).
|
||||
`lodash.clonedeep` was not needed — the runtime extenders already exist without it.
|
||||
|
||||
### Error semantics
|
||||
|
||||
Validation errors formerly thrown host-side inside accessor methods (serialized as JSON-RPC error 1000)
|
||||
now throw *locally* in the subprocess: same message, better stack traces, one less round trip.
|
||||
Bridge-side errors keep the `handleBridgeMessage` shape (code -32000, message preserved, formatted by
|
||||
`formatErrorResponse`). Permission failures on read/write bridges stay *silent-undefined*;
|
||||
manager-level registration failures keep *throwing*. Both are preserved verbatim because both layers
|
||||
are reused untouched.
|
||||
|
||||
### `AppObjectRegistry` and one-app-per-process
|
||||
|
||||
The registry (a process-global singleton keyed by simple strings, including a single `id`) assumes one
|
||||
app per subprocess. The migration keeps that assumption, restated in code comments since consolidating
|
||||
logic into the runtime makes it more load-bearing.
|
||||
|
||||
## Per-accessor disposition
|
||||
|
||||
**MOVE** — implement in base-runtime, side effects via existing bridge `do*`, delete host class.
|
||||
**RECONCILE** — already in base-runtime; make canonical, delete host copy. **BRIDGE-GAP** — needs
|
||||
`AppResourceBridge`. **DELETE** — already dead on the host.
|
||||
|
||||
**Readers (all MOVE — bridge passthroughs plus portable validation).** `Reader` (facade over 15
|
||||
sub-readers); `MessageRead` (`getSenderUser`/`getRoom` are fetch-then-extract compositions); `RoomRead`
|
||||
(`getMessages`/`getAllRooms`/`getUnreadByUser` limit/skip/sort validation); `UserRead`
|
||||
(`getAppUser(appId?)` arg defaulting); `PersistenceRead` (single-assoc → array wrapping); `LivechatRead`
|
||||
(`isOnline` deprecation warning — runtime `console` is piped to app logs); `UploadRead`
|
||||
(`getBufferById` becomes two bridge RPCs); `CloudWorkspaceRead`, `VideoConferenceRead`, `OAuthAppsReader`
|
||||
(`doGetByid` typo is on the bridge, unchanged), `ContactRead`, `ThreadRead`, `RoleRead` (1:1);
|
||||
`ExperimentalRead` (empty shell → empty local class).
|
||||
|
||||
**Environment.** MOVE: `EnvironmentRead`/`EnvironmentWrite` (facades), `ServerSettingRead`
|
||||
(`getValueById` `value → packageValue` fallback; `getAll` keeps throwing "not implemented" locally),
|
||||
`EnvironmentalVariableRead`, `ServerSettingUpdater` (+ `incrementValue` default). BRIDGE-GAP:
|
||||
`SettingRead` (reads `ProxiedApp.getStorageItem().settings`) → `doGetAppSetting`; `SettingUpdater`
|
||||
(`AppSettingsManager.updateAppSetting` persists, fires `doOnAppSettingsChange`, runs the
|
||||
`ON_PRE_SETTING_UPDATE` round-trip back into the app) → `doUpdateAppSetting`.
|
||||
|
||||
**Modify.** RECONCILE: `ModifyCreator`, `ModifyUpdater` (diff-update semantics), `ModifyExtender`,
|
||||
`Notifier` (awaits typing calls). MOVE: `Modify` (facade); `LivechatCreator` (`createToken` already
|
||||
local in the runtime, supersedes; rest 1:1); `UploadCreator` (`uploadBuffer` defaults user via
|
||||
`doGetAppUser` — two RPCs); `EmailCreator`, `ContactCreator`, `MessageUpdater`, `OAuthAppsModify` (1:1);
|
||||
`LivechatUpdater` (+ boolean coercion in `setCustomFields`); `UserUpdater` (partial-object wrapping);
|
||||
`ModifyDeleter` (`removeUsersFromRoom` ≤ 50 validation); `UIController` (UIKit interaction payloads +
|
||||
`UIHelper.assignIds`, then `doNotifyUser`; required relocating `UIHelper`); `SchedulerModify`
|
||||
(`createProcessorId` is pure and idempotent — the subprocess knows its own appId for the suffix and the
|
||||
bridge still permission-checks the real id); `ModerationModify` (appId comes from method args by design).
|
||||
|
||||
**Configuration extend/modify — the registration surface.** These are why `handleAccessorMessage` could
|
||||
not be deleted by bridge calls alone: they write into host-only managers whose in-memory registries the
|
||||
host reads to render UI, route events, and dispatch executions back into the subprocess. MOVE:
|
||||
`ConfigurationExtend`, `ConfigurationModify` (facades), `ServerSettingsModify` (1:1 + increment
|
||||
default). RECONCILE: `HttpExtend` (already fully local). BRIDGE-GAP: `SettingsExtend` →
|
||||
`doProvideSetting`; `SlashCommandsExtend` (registry, touched-command conflict tracking,
|
||||
`CommandAlreadyExistsError`) → `doProvideSlashCommand`; `ApiExtend` (registry, `PathAlreadyExistsError`)
|
||||
→ `doProvideApi` plus `listApis` → `doListApis`, killing the `accessor:api:listApis` special case;
|
||||
`ExternalComponentsExtend` → `doRegisterExternalComponent`; `SchedulerExtend` (wraps processors in host
|
||||
callbacks, namespaces ids, guards disabled apps) → `doRegisterProcessors`; `UIExtend` (button registry
|
||||
read directly by host UI, `ui.registerButtons` permission, `doActionsChanged` side effect) →
|
||||
`doRegisterActionButton`; `VideoConfProviderExtend` (`PermissionDeniedError`, name-collision tracking) →
|
||||
`doProvideVideoConfProvider`; `OutboundMessageProviderExtend` → `doRegisterOutboundProvider`;
|
||||
`SlashCommandsModify` → `doModifySlashCommand`/`doEnableSlashCommand`/`doDisableSlashCommand`.
|
||||
|
||||
*Serialization note:* `ISlashCommand`, `IProcessor`, `IApi`, and the provider objects contain executor
|
||||
functions. The old `accessor:*` messages already stripped them in JSON serialization — the host managers
|
||||
keep metadata and dispatch executions *back into* the subprocess, which looks the live instances up in
|
||||
`AppObjectRegistry`. The new bridge methods receive the same serialized shapes, so **the wire semantics
|
||||
did not change, only the message prefix**.
|
||||
|
||||
**Standalone.** `Http` — RECONCILE (runtime version already merges `HttpExtend` defaults, runs pre/post
|
||||
handlers, calls `doCall`; the host copy and its reach-back into `AppAccessorManager` are deleted).
|
||||
`Persistence` — MOVE (1:1 + arg wrapping/defaults). `AppAccessors` — DELETE (never instantiated on the
|
||||
host; the runtime one in `mod.ts` is live).
|
||||
|
||||
## Drift resolutions
|
||||
|
||||
Only the **bounded subset carrying logic on both sides** — the RECONCILE accessors plus the few MOVE
|
||||
accessors already partially local. Pure-proxy accessors had no runtime counterpart to reconcile; they
|
||||
were faithful-ported and covered by the parity check instead.
|
||||
|
||||
Classes: **superset/equiv** — runtime already covers the host behavior, adopt silently.
|
||||
**conflict→runtime** — genuine divergence; runtime wins, observationally neutral since the runtime was
|
||||
authoritative. **merge (host→runtime)** — host-only logic folded in, adds observable behavior, ships
|
||||
with a test (no row needed one).
|
||||
|
||||
| # | Drift | Class | Decision |
|
||||
| --- | --- | --- | --- |
|
||||
| 1 | **Update payload shape** — host `ModifyUpdater._finishMessage/_finishRoom` sent the *full* builder object to `doUpdate`; runtime sends a *diff* (`{ id, ...builder.getChanges() }`). | conflict→runtime | Diff wins; bridges demonstrably accept it (production sends it). Host path deleted, diff shape pinned by a test. |
|
||||
| 2 | **Editor tracking** — runtime `ModifyUpdater.message()` calls `builder.setEditor(editor)`; host ignored the updater param. | superset/equiv | Runtime wins; editor recorded on update. |
|
||||
| 3 | **`typing()` awaits** — runtime `Notifier` awaits `doTyping`; host fire-and-forget. | superset/equiv | Runtime wins. |
|
||||
| 4 | **`createToken`** — runtime generates `randomBytes(16).toString('hex')` locally; host generated it host-side. | conflict→runtime | Runtime wins; same format. |
|
||||
| 5 | **HTTP method representation** — runtime uses lowercase literals; host used the `RequestMethod` enum. | superset/equiv | Runtime wins; `doCall` payload identical. |
|
||||
| 6 | **`BlockBuilder` appId** — host `getBlockBuilder()` passed appId into the builder; runtime builder takes none (ids assigned later by `UIHelper.assignIds`). | conflict→runtime | Runtime wins; the drop is intentional (ids assigned later) and the block shape is pinned by a test. |
|
||||
| 7 | **`APP_ID` inconsistency** — exactly one runtime call site (`ModifyCreator._finishMessage` → `doGetAppUser`) sent the sentinel; every other sent the raw `AppObjectRegistry.get('id')`. | normalization | Not a pick-a-side drift; all caller-identity params now use `'APP_ID'`, argument appIds stay raw. |
|
||||
| 8 | Cosmetic error wording ("can not" vs "can't"). | conflict→runtime | Runtime wording wins. |
|
||||
|
||||
Where a ported accessor interacts with a reconciled one, the reconciled (runtime) semantics apply.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Move the registries into the subprocess** — non-starter. The host reads
|
||||
`UIActionButtonManager.getAllActionButtons` to render UI, routes slash-command/api/scheduler/provider
|
||||
invocations off these registries, and enforces cross-app conflict rules (two apps claiming the same
|
||||
slash command) that no single subprocess can arbitrate; app settings are host-persisted metadata read
|
||||
by other subsystems. A host-side write surface must remain.
|
||||
- **Keep a residual `accessor:*` channel just for registration** — works, but keeps two RPC categories,
|
||||
two dispatchers, and the reflective accessor-path-walking code alive. The bridge approach reuses an
|
||||
existing dispatcher, its security gates, and its error model, and let `handleAccessorMessage`,
|
||||
`ALLOWED_ACCESSOR_METHODS`, `isValidOrigin`, and `getAccessorForOrigin` be deleted entirely.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive.** One source of truth for accessor behavior; the host accessor layer and
|
||||
`AppAccessorManager` deleted; subprocess controllers lose the whole `accessor:*` family and its
|
||||
dispatcher; working LSP navigation into real implementations instead of proxies; local validation
|
||||
throws with real stack traces and one less round trip.
|
||||
|
||||
**Accepted trade-offs.** Slightly chattier composite reads — `UploadRead.getBufferById` and
|
||||
`UploadCreator.uploadBuffer` became two bridge RPCs where one accessor RPC (wrapping two in-host bridge
|
||||
calls) sufficed; bounded, rare, symmetrical with what `Notifier`/`ModifyCreator` already did, and no
|
||||
pagination-style N+1 patterns exist in this surface. Sandbox validation is advisory (unchanged in
|
||||
practice, now explicit — follow-up 1). Two implementations existed transiently during rollout,
|
||||
mitigated by per-accessor merge units and the parity harness.
|
||||
|
||||
**"Faithful port" was verified, not asserted.** For MOVE accessors the host logic was *live in
|
||||
production*, so a subtly wrong port — a dropped default, an off-by-one cap, a renamed sort field — is an
|
||||
immediate observable regression. Ported unit tests only prove the new class satisfies carried-over
|
||||
assertions; they do not prove equivalence, so a port that drops an uncovered branch passes green. Each
|
||||
host MOVE class deletion was gated on a **mechanical parity check**: either a differential harness
|
||||
driving representative calls through both paths and asserting identical bridge-message output (method
|
||||
string + params) and return shaping, or an enforced branch-coverage audit of the ported tests against
|
||||
the host source. RECONCILE accessors did not need it (their host copy was already unreachable); the
|
||||
drift rules govern them instead. The harness
|
||||
(`base-runtime/src/lib/accessors/tests/helpers/parityHarness.ts`) was planned as throwaway but **kept**
|
||||
— it became a test utility the base-runtime accessor suites depend on.
|
||||
|
||||
## Implementation record
|
||||
|
||||
**A phase was a milestone, not a single PR.** Each phase is independently shippable and keeps
|
||||
`test:node`/`test:deno`/`test:base-runtime` green, but the *unit of merge* was the accessor: Phases 1–2
|
||||
landed as a series of per-accessor (or tight cohesive-group) PRs, so blast radius per merge is one
|
||||
accessor family. This mattered because the migration flips *live production behavior* for MOVE
|
||||
accessors with no rollback but `git revert`, and the unit of revert equals the unit of merge. Host
|
||||
accessor tests were ported to `base-runtime/src/lib/accessors/tests/`.
|
||||
|
||||
**Phase 0 — foundations, no behavior change.** Added the typed bridge entry point (now `bridgeCall.ts`)
|
||||
with tests for message-string generation, the `do*` gate, and error formatting. Produced the APP_ID
|
||||
exception list — the audit surfaced bucket C, since the sentinel substitution rewrites only *top-level
|
||||
positional* params. Refactored the existing runtime accessors (`Http`, `Notifier`, `ModifyCreator`,
|
||||
`ModifyUpdater`, `ModifyExtender`, `roomFactory`) onto it and normalized caller-identity params;
|
||||
accessors still holding `accessor:*` sub-proxies kept `senderFn` for those until Phase 2. Duplicated
|
||||
`UIHelper` into `base-runtime/src/lib/`. Stood up the parity harness. No changeset:
|
||||
`@rocket.chat/apps` is `private` and has never carried one — this ADR and the exception list are the
|
||||
recorded contract.
|
||||
|
||||
**Phase 1 — readers, `Persistence`, server-side environment.** Ported the reader family plus
|
||||
`ServerSettingRead`, `EnvironmentalVariableRead`, `ServerSettingUpdater`, `ServerSettingsModify`,
|
||||
`Persistence`, and the `Reader`/`EnvironmentRead`/`EnvironmentWrite` facades, moving all portable
|
||||
validation/defaulting verbatim; app-settings members stayed proxied until Phase 3. Flipped the
|
||||
corresponding `mod.ts` entries to local. *Host-class deletion was deferred to Phase 4* (deliberate
|
||||
deviation): `AppAccessorManager.getReader()` builds the whole reader family and was still called by
|
||||
`AppListenerManager.executePostMessageSent` and to build not-yet-migrated sub-accessors, so deleting
|
||||
e.g. `RoomRead` then would have pulled the Phase-4 refactor forward mid-phase. *RPC-boundary
|
||||
adaptation:* `ServerSettingRead.getValueById` checked `typeof set === 'undefined'` host-side; across the
|
||||
boundary an absent return arrives as `null`, so the runtime treats both as "not found" — identical for
|
||||
apps, since the bridge only ever returns a setting or nothing.
|
||||
|
||||
**Phase 2 — modify family.** Ported `ModifyDeleter`, `MessageUpdater`, `LivechatUpdater`, `UserUpdater`,
|
||||
`LivechatCreator`, `UploadCreator`, `EmailCreator`, `ContactCreator`, `UIController`, `SchedulerModify`,
|
||||
`OAuthAppsModify`, `ModerationModify`; caller-identity params use the sentinel, the
|
||||
`ModerationModify`/`ModifyDeleter.deleteUsers` argument-appIds stay raw (bucket B), and
|
||||
`UIController`/`SchedulerModify` read the real id from `AppObjectRegistry` for their non-identity uses.
|
||||
Removed the remaining `proxify` entries in `getModifier` and the `accessor:*` sub-creator/sub-updater
|
||||
proxies, after which `getReader`/`getModifier`/`getPersistence`/`getHttp` generated **zero**
|
||||
`accessor:*` traffic. *RPC-boundary note:* sub-accessor methods now flow through `Promise<unknown>` and
|
||||
are cast to their interface return types; `void`-returning methods `await` instead of returning the
|
||||
bridge value.
|
||||
|
||||
**Phase 3 — registration surface.** Added `src/server/bridges/AppResourceBridge.ts` and wired it into
|
||||
`handleBridgeMessage` via the dedicated `getAppResourceBridge` lookup plus the `restarting` guard keyed
|
||||
on `REGISTRATION_METHODS`; no `apps/meteor` orchestrator changes needed. Rewrote
|
||||
`getConfigurationExtend`, `getConfigurationModify`, and the app-settings
|
||||
`SettingRead`/`SettingUpdater`/`SettingsExtend` members onto `getAppResourceBridge().do*`, preserving
|
||||
the `AppObjectRegistry` stash-then-forward; `accessor:api:listApis` became `doListApis`;
|
||||
`registerButton` stays a synchronous `void` per its interface. The dead `proxify` machinery and
|
||||
`WithProxy` type were removed from `mod.ts` — **the runtime stopped emitting `accessor:*` entirely.**
|
||||
*RPC-boundary note:* `SettingRead.getValueById` treats `null` and `undefined` alike, same adaptation as
|
||||
`ServerSettingRead`.
|
||||
|
||||
**Phase 4 — message-path teardown.** Replaced the `executePostMessageSent` accessor call with
|
||||
`this.manager.getBridges().getUserBridge().doGetAppUser(appId)`. Deleted `handleAccessorMessage`,
|
||||
`ALLOWED_ACCESSOR_METHODS`, `isValidOrigin`, `getAccessorForOrigin`, the `accessor:` branch in
|
||||
`handleIncomingMessage`, and the controller's unused `accessors`/`api` fields — **the `accessor:*`
|
||||
category no longer exists**; the controller dispatches only `bridges:*`, `ready`, `log`, and the error
|
||||
notifications. `JSONRPC_METHOD_NOT_FOUND` is kept (imported by `ProxiedApp`, `AppListenerManager`,
|
||||
`AppVideoConfProvider`). Removed the four now-meaningless `handleAccessorMessage` cases from
|
||||
`DenoRuntimeSubprocessController.test.ts`. Physical deletion of the dead classes was folded into
|
||||
Phase 5, since it first required un-threading the ignored `accessors` parameter from the
|
||||
sandbox-execution core — a separate behavior-neutral change that should not ride at the tail of the
|
||||
teardown.
|
||||
|
||||
**Phase 5 — dead host accessor layer removal.** Behavior-neutral; nothing on the subprocess path
|
||||
reached any of it after Phase 4. Un-threaded the vestigial `accessors` parameter from `AppApi`,
|
||||
`AppSlashCommand`, `AppVideoConfProvider`, and `AppOutboundCommunicationProvider` `run*`/`runTheCode`
|
||||
methods (which already ignored it as `_accessors`), updated the four managers' call sites, and dropped
|
||||
their unused `accessors` fields. Deleted `AppAccessorManager`, its `managers/index.ts` export, and
|
||||
`AppManager`'s field, construction, `getAccessorManager()`, and the `purifyApp(...)` call in
|
||||
`removeLocal` (a no-op once no host code called `getReader`/`getModifier`). Deleted
|
||||
`src/server/accessors/` (64 files) and the host `src/server/misc/UIHelper.ts`. Adapted tests: removed
|
||||
`tests/server/accessors/` (35 suites, covered by the base-runtime ones),
|
||||
`AppAccessorManager.test.ts`, and the accessor setup/assertions in the four manager tests,
|
||||
`AppSettingsManager.test.ts`, `AppManager.test.ts`, `AppListenerManager.test.ts`, and the two
|
||||
Deno-runtime integration tests. *Verification:* host `tsc` error-set unchanged vs the
|
||||
environmental-only baseline; `test:node` host suites green (excluding the two `deno`-spawn integration
|
||||
suites needing a `deno` binary unavailable in that environment); base-runtime suite unaffected.
|
||||
137
docs/ajv-instances.md
Normal file
137
docs/ajv-instances.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# AJV instances: `ajv` and `ajvQuery`
|
||||
|
||||
The `@rocket.chat/rest-typings` package uses two [AJV](https://ajv.js.org/) instances for JSON schema validation: **`ajv`** and **`ajvQuery`**. The choice between them depends on the **source of the data** being validated (request body vs query string).
|
||||
|
||||
---
|
||||
|
||||
## Instance definitions
|
||||
|
||||
Both are created in `packages/rest-typings/src/v1/Ajv.ts` with the same configuration except for one option:
|
||||
|
||||
| Option | `ajv` | `ajvQuery` |
|
||||
|-------------------|---------|------------|
|
||||
| `coerceTypes` | `false` | `true` |
|
||||
| `allowUnionTypes` | `true` | `true` |
|
||||
| `code.source` | `true` | `true` |
|
||||
| `discriminator` | `true` | `true` |
|
||||
|
||||
In short:
|
||||
|
||||
- **`ajv`**: does not change data types; values must already match the types expected by the schema.
|
||||
- **`ajvQuery`**: attempts to **coerce** types when the schema expects `number`, `integer`, or `boolean` (e.g. the string `"50"` becomes the number `50`).
|
||||
|
||||
Custom formats (`addFormats`) and keywords (e.g. `isNotEmpty`) are registered on both instances.
|
||||
|
||||
---
|
||||
|
||||
## Why two instances?
|
||||
|
||||
### Query params are always strings
|
||||
|
||||
In HTTP requests, **query parameters** (everything after `?` in the URL) reach the server as **strings**. HTTP does not carry type information; the server receives, for example:
|
||||
|
||||
- `?count=25` → `count` is the string `"25"`
|
||||
- `?open=true` → `open` is the string `"true"`
|
||||
|
||||
If the schema expects `count` as `number` or `open` as `boolean`, a validator that does **not** coerce will reject:
|
||||
|
||||
- `"25"` is not of type `number` → error like "must be number" / "invalid-params".
|
||||
- `"true"` is not of type `boolean` → validation error.
|
||||
|
||||
### Body can already be typed
|
||||
|
||||
For the **body** (JSON in POST/PUT/PATCH etc.), the client sends JSON. Parsing (e.g. `JSON.parse`) already yields numbers and booleans. In that case we do not want the validator to mutate values; we use the instance **without** coercion (`ajv`).
|
||||
|
||||
---
|
||||
|
||||
## When to use each
|
||||
|
||||
### Use `ajvQuery.compile` when:
|
||||
|
||||
- The validator is used for **query parameters** (query string).
|
||||
- **GET** routes (or any method that only reads query params).
|
||||
- The schema has properties of type `number`, `integer`, or `boolean` that come from the URL.
|
||||
|
||||
Examples of validators that should use `ajvQuery`:
|
||||
|
||||
- Pagination: `count`, `offset` (typically numbers).
|
||||
- Flags: `open`, `readThreads` (booleans).
|
||||
- Any numeric or boolean parameter the client sends in the query string.
|
||||
|
||||
```ts
|
||||
// GET /v1/livechat/rooms?count=25&offset=0
|
||||
export const isGETLivechatRoomsParams = ajvQuery.compile<GETLivechatRoomsParams>(GETLivechatRoomsParamsSchema);
|
||||
```
|
||||
|
||||
### Use `ajv.compile` when:
|
||||
|
||||
- The validator is used for the request **body** (POST, PUT, PATCH, etc.).
|
||||
- Data is already parsed JSON (numbers and booleans are already typed).
|
||||
|
||||
Examples:
|
||||
|
||||
- Create/update resources (JSON body).
|
||||
- Response validators or internal structures that do not come from the query string.
|
||||
|
||||
```ts
|
||||
// POST /v1/livechat/room/close — JSON body
|
||||
export const isPOSTLivechatRoomCloseParams = ajv.compile<POSTLivechatRoomCloseParams>(...);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick reference
|
||||
|
||||
| Data source | Instance | Reason |
|
||||
|--------------------------------|------------|--------|
|
||||
| **Query string** (GET, query params) | `ajvQuery` | Query values are strings; `coerceTypes: true` converts to number/boolean when the schema expects it. |
|
||||
| **Body** (JSON in POST/PUT/PATCH) | `ajv` | JSON already has types; strict validation without mutating values. |
|
||||
|
||||
---
|
||||
|
||||
## Response schemas and `nullable`
|
||||
|
||||
Response schemas also use `ajv` (coerceTypes: false). In **test mode**, the Router validates every response against its declared schema (`options.response[statusCode]`). If validation fails, the Router returns a **400** with `errorType: "error-invalid-body"` instead of the original response.
|
||||
|
||||
With `coerceTypes: true` (old behavior), `null` values were silently coerced (e.g. `null` → `""` for strings). With `coerceTypes: false`, any field that can be `null` **must** declare `nullable: true` in the schema — otherwise the response validator rejects it.
|
||||
|
||||
### Example
|
||||
|
||||
A video conference user object may have `avatarETag: null`. The response schema must account for this:
|
||||
|
||||
```ts
|
||||
// WRONG — fails when avatarETag is null
|
||||
{ type: 'string' }
|
||||
|
||||
// CORRECT
|
||||
{ type: 'string', nullable: true }
|
||||
```
|
||||
|
||||
### `oneOf` / discriminator schemas
|
||||
|
||||
Schemas using `oneOf` with strict enum discriminators (e.g. `type: { enum: ['direct'] }`) also become stricter without coercion. If the actual data has a `type` value not listed in any branch, the `oneOf` fails. Ensure all possible discriminator values are covered, or relax the items schema (e.g. `{ type: 'object' }`) when full type-level validation is not needed at runtime.
|
||||
|
||||
---
|
||||
|
||||
## Common mistakes
|
||||
|
||||
1. **Using `ajv` for query params** when the schema expects `number` or `boolean`:
|
||||
- Client sends `?count=25`.
|
||||
- Validator expects `number` but receives the string `"25"`.
|
||||
- Result: "must be number" / "invalid-params" error.
|
||||
|
||||
2. **Using `ajvQuery` for body**:
|
||||
- Usually works, but coercion can hide incorrect types (e.g. string where a number was expected). For body, the standard is to use `ajv` and require the client to send the correct types in the JSON.
|
||||
|
||||
3. **Response schemas with `null` fields** (test mode only):
|
||||
- With `coerceTypes: false`, `null` is no longer coerced to `""` or `0`.
|
||||
- Fields that can be `null` must use `nullable: true`.
|
||||
- Symptom: tests get 400 with `errorType: "error-invalid-body"` even though the endpoint logic succeeds.
|
||||
|
||||
---
|
||||
|
||||
## Where the instances are defined
|
||||
|
||||
- **File:** `packages/rest-typings/src/v1/Ajv.ts`
|
||||
- **Export:** `export { ajv, ajvQuery };`
|
||||
- **Usage:** In each rest-typings module, import the appropriate instance and call `.compile(schema)` to obtain the validator used by the API routes.
|
||||
97
docs/anchor-navigation.md
Normal file
97
docs/anchor-navigation.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Anchor Navigation
|
||||
|
||||
This document describes the pattern for navigating directly to a specific field within a page using URL hash fragments (e.g., `/account/accessibility-and-appearance#clockMode`).
|
||||
|
||||
## Problem
|
||||
|
||||
Rocket.Chat uses a custom SPA router that intercepts link clicks via `e.preventDefault()` and navigates with `history.pushState()`. This means the browser's native hash-based scrolling does not work for cross-page navigation. When a user clicks a link like `/page-b#field`, the router renders the new page but does not scroll to the target element.
|
||||
|
||||
## Solution
|
||||
|
||||
Scrolling to hash targets is handled automatically by `RouterProvider` via `useRouterScrollToHash`. Any element with an `id` matching the URL hash will be scrolled into view on navigation.
|
||||
|
||||
For pages that need to adjust layout based on the hash (e.g., expanding an accordion section), `useLocationHash()` from `@rocket.chat/ui-contexts` provides the current hash value.
|
||||
|
||||
### Key files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `apps/meteor/client/hooks/useRouterScrollToHash.ts` | Scroll logic, called by `RouterProvider` |
|
||||
| `packages/ui-contexts/src/hooks/useLocationHash.ts` | Generic `useLocationHash()` hook from `@rocket.chat/ui-contexts` |
|
||||
|
||||
## How it works
|
||||
|
||||
1. A **source page** links to a target page with a hash fragment in the `href`
|
||||
2. The **target page** renders elements with matching `id` attributes
|
||||
3. `useRouterScrollToHash` (in `RouterProvider`) scrolls to the element matching the hash via `scrollIntoView`
|
||||
4. If the target page needs to adjust layout (e.g., expand a collapsed section), it reads `useLocationHash()` to check if a hash is present
|
||||
|
||||
```text
|
||||
PreferencesMessagesSection AccessibilityPage
|
||||
┌──────────────────────────┐ ┌──────────────────────────┐
|
||||
│ Message TimeFormat │ │ Adjustable layout │
|
||||
│ [Go to accessibility] ────┼── #clockMode ───> │ <Field id="clockMode"> │
|
||||
│ │ │ │
|
||||
│ Hide usernames │ │ <Field id="hideUsernames"> │
|
||||
│ [Go to accessibility] ────┼── #hideUsernames ───> │ │
|
||||
└──────────────────────────┘ └──────────────────────────┘
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### 1. Add the `id` to the target element
|
||||
|
||||
On the destination page, set the `id` on the element you want to scroll to:
|
||||
|
||||
```tsx
|
||||
<Field id='myNewField'>
|
||||
<FieldLabel>{t('My_New_Field')}</FieldLabel>
|
||||
{/* ... */}
|
||||
</Field>
|
||||
```
|
||||
|
||||
### 2. Expand collapsed sections (if needed)
|
||||
|
||||
If the target field is inside a collapsed section, use `useLocationHash()` to expand it:
|
||||
|
||||
```tsx
|
||||
import { useLocationHash } from '@rocket.chat/ui-contexts';
|
||||
|
||||
const MyPage = () => {
|
||||
const shouldExpand = useLocationHash().length > 1;
|
||||
|
||||
return (
|
||||
<Accordion>
|
||||
<AccordionItem defaultExpanded={shouldExpand} title={t('Section')}>
|
||||
<Field id='myNewField'>
|
||||
{/* ... */}
|
||||
</Field>
|
||||
</AccordionItem>
|
||||
</Accordion>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### 3. Link from the source page
|
||||
|
||||
Use the hash fragment in the `href`:
|
||||
|
||||
```tsx
|
||||
<FieldLink href='/account/accessibility-and-appearance#myNewField'>
|
||||
{t('Go_to_accessibility_and_appearance')}
|
||||
</FieldLink>
|
||||
```
|
||||
|
||||
## Why `scrollIntoView` is needed
|
||||
|
||||
The browser natively scrolls to hash targets on full page loads and un-intercepted anchor clicks. However, the custom router calls `e.preventDefault()` on cross-page links and uses `history.pushState()`, which by spec does not trigger scroll. `useRouterScrollToHash` compensates by calling `scrollIntoView` after each navigation.
|
||||
|
||||
Same-page hash navigation (clicking `#anchor` when already on that page) still works natively — the router detects matching pathnames and lets the browser handle it.
|
||||
|
||||
## Checklist
|
||||
|
||||
When adding a new anchor:
|
||||
|
||||
- [ ] `id` attribute set on the target element
|
||||
- [ ] Source `href` includes the `#anchor` fragment
|
||||
- [ ] `useLocationHash()` can be called in target page to identify if theres hash and control collapsed content, if layout adjustment is needed
|
||||
637
docs/api-endpoint-migration.md
Normal file
637
docs/api-endpoint-migration.md
Normal file
@@ -0,0 +1,637 @@
|
||||
# API Endpoint Migration Guide
|
||||
|
||||
Migration from the legacy `API.v1.addRoute()` pattern to the new `API.v1.get()` / `.post()` / `.put()` / `.delete()` pattern with request/response validation.
|
||||
|
||||
## Why
|
||||
|
||||
- **Response validation** in test mode catches mismatches between code and types
|
||||
- **Type safety** with AJV-compiled schemas for request and response
|
||||
- **OpenAPI docs** generated automatically from schemas
|
||||
- **Consistent error format** across all endpoints
|
||||
|
||||
## Legacy Pattern (BEFORE)
|
||||
|
||||
```typescript
|
||||
import { isChannelsAddAllProps } from '@rocket.chat/rest-typings';
|
||||
|
||||
import { API } from '../api';
|
||||
|
||||
API.v1.addRoute(
|
||||
'channels.addAll',
|
||||
{
|
||||
authRequired: true,
|
||||
validateParams: isChannelsAddAllProps,
|
||||
},
|
||||
{
|
||||
async post() {
|
||||
const { activeUsersOnly, ...params } = this.bodyParams;
|
||||
const findResult = await findChannelByIdOrName({ params, userId: this.userId });
|
||||
|
||||
await addAllUserToRoomFn(this.userId, findResult._id, activeUsersOnly === 'true' || activeUsersOnly === 1);
|
||||
|
||||
return API.v1.success({
|
||||
channel: await findChannelByIdOrName({ params, userId: this.userId }),
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
Source: `apps/meteor/app/api/server/v1/channels.ts`
|
||||
|
||||
## New Pattern (AFTER)
|
||||
|
||||
```typescript
|
||||
import {
|
||||
ajv,
|
||||
isReportHistoryProps,
|
||||
validateBadRequestErrorResponse,
|
||||
validateUnauthorizedErrorResponse,
|
||||
validateForbiddenErrorResponse,
|
||||
} from '@rocket.chat/rest-typings';
|
||||
|
||||
import { API } from '../api';
|
||||
import { getPaginationItems } from '../helpers/getPaginationItems';
|
||||
|
||||
// See "Known Pitfall: Date | string unions" — IModerationAudit uses a relaxed inline schema
|
||||
const paginatedReportsResponseSchema = ajv.compile<{ reports: IModerationAudit[]; count: number; offset: number; total: number }>({
|
||||
type: 'object',
|
||||
properties: {
|
||||
reports: {
|
||||
type: 'array',
|
||||
items: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
userId: { type: 'string' },
|
||||
username: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
message: { type: 'string' },
|
||||
msgId: { type: 'string' },
|
||||
ts: { type: 'string' },
|
||||
rooms: { type: 'array', items: { type: 'object' } },
|
||||
roomIds: { type: 'array', items: { type: 'string' } },
|
||||
count: { type: 'number' },
|
||||
isUserDeleted: { type: 'boolean' },
|
||||
},
|
||||
required: ['userId', 'ts', 'rooms', 'roomIds', 'count', 'isUserDeleted'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
count: { type: 'number' },
|
||||
offset: { type: 'number' },
|
||||
total: { type: 'number' },
|
||||
success: { type: 'boolean', enum: [true] },
|
||||
},
|
||||
required: ['reports', 'count', 'offset', 'total', 'success'],
|
||||
additionalProperties: false,
|
||||
});
|
||||
|
||||
API.v1.get(
|
||||
'moderation.reportsByUsers',
|
||||
{
|
||||
authRequired: true,
|
||||
permissionsRequired: ['view-moderation-console'],
|
||||
query: isReportHistoryProps,
|
||||
response: {
|
||||
200: paginatedReportsResponseSchema,
|
||||
400: validateBadRequestErrorResponse,
|
||||
401: validateUnauthorizedErrorResponse,
|
||||
403: validateForbiddenErrorResponse,
|
||||
},
|
||||
},
|
||||
async function action() {
|
||||
const { latest: _latest, oldest: _oldest, selector = '' } = this.queryParams;
|
||||
const { count = 20, offset = 0 } = await getPaginationItems(this.queryParams);
|
||||
const { sort } = await this.parseJsonQuery();
|
||||
|
||||
const latest = _latest ? new Date(_latest) : new Date();
|
||||
const oldest = _oldest ? new Date(_oldest) : new Date(0);
|
||||
|
||||
const reports = await ModerationReports.findMessageReportsGroupedByUser(latest, oldest, escapeRegExp(selector), {
|
||||
offset,
|
||||
count,
|
||||
sort,
|
||||
}).toArray();
|
||||
|
||||
return API.v1.success({
|
||||
reports,
|
||||
count: reports.length,
|
||||
offset,
|
||||
total: reports.length === 0 ? 0 : await ModerationReports.getTotalUniqueReportedUsers(latest, oldest, escapeRegExp(selector), true),
|
||||
});
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
Source: `apps/meteor/app/api/server/v1/moderation.ts`
|
||||
|
||||
## Step-by-Step Migration
|
||||
|
||||
### 1. Identify the HTTP method
|
||||
|
||||
Look at the handler keys inside the `addRoute` call:
|
||||
|
||||
| Handler key | New method |
|
||||
| ---------------- | -------------------- |
|
||||
| `async get()` | `API.v1.get(...)` |
|
||||
| `async post()` | `API.v1.post(...)` |
|
||||
| `async put()` | `API.v1.put(...)` |
|
||||
| `async delete()` | `API.v1.delete(...)` |
|
||||
|
||||
### 2. Replace `addRoute` with the HTTP method
|
||||
|
||||
```typescript
|
||||
// BEFORE
|
||||
API.v1.addRoute('endpoint.name', options, { async get() { ... } });
|
||||
|
||||
// AFTER
|
||||
API.v1.get('endpoint.name', options, async function action() { ... });
|
||||
```
|
||||
|
||||
The handler becomes a standalone `async function action()` (named function, not arrow function).
|
||||
|
||||
### 3. Move `validateParams` to `query` or `body`
|
||||
|
||||
| HTTP method | Option name |
|
||||
| ----------- | -------------------- |
|
||||
| GET, DELETE | `query: validatorFn` |
|
||||
| POST, PUT | `body: validatorFn` |
|
||||
|
||||
```typescript
|
||||
// BEFORE
|
||||
{
|
||||
validateParams: isSomeEndpointProps;
|
||||
}
|
||||
|
||||
// AFTER (GET)
|
||||
{
|
||||
query: isSomeEndpointProps;
|
||||
}
|
||||
|
||||
// AFTER (POST)
|
||||
{
|
||||
body: isSomeEndpointProps;
|
||||
}
|
||||
```
|
||||
|
||||
The `validateParams` option is deprecated. Do not use it in new code.
|
||||
|
||||
### 4. Create response schemas
|
||||
|
||||
Define response schemas using `ajv.compile<T>()` **before** the endpoint registration. Every response schema must include the `success` field. When the response contains complex types from `@rocket.chat/core-typings`, prefer using `$ref` instead of `{ type: 'object' }` (see [Using Typia `$ref` for Complex Types](#using-typia-ref-for-complex-types)).
|
||||
|
||||
```typescript
|
||||
const myResponseSchema = ajv.compile<{ items: SomeType[]; count: number }>({
|
||||
type: 'object',
|
||||
properties: {
|
||||
items: { type: 'array', items: { $ref: '#/components/schemas/SomeType' } },
|
||||
count: { type: 'number' },
|
||||
success: { type: 'boolean', enum: [true] },
|
||||
},
|
||||
required: ['items', 'count', 'success'],
|
||||
additionalProperties: false,
|
||||
});
|
||||
```
|
||||
|
||||
For endpoints that return only `{ success: true }`:
|
||||
|
||||
```typescript
|
||||
const successResponseSchema = ajv.compile<void>({
|
||||
type: 'object',
|
||||
properties: { success: { type: 'boolean', enum: [true] } },
|
||||
required: ['success'],
|
||||
additionalProperties: false,
|
||||
});
|
||||
```
|
||||
|
||||
### 5. Add the `response` object
|
||||
|
||||
Always include error schemas for relevant status codes:
|
||||
|
||||
```typescript
|
||||
response: {
|
||||
200: myResponseSchema,
|
||||
400: validateBadRequestErrorResponse,
|
||||
401: validateUnauthorizedErrorResponse,
|
||||
}
|
||||
```
|
||||
|
||||
Add `403: validateForbiddenErrorResponse` when the endpoint has `permissionsRequired`.
|
||||
|
||||
### 6. Update imports
|
||||
|
||||
```typescript
|
||||
import {
|
||||
ajv,
|
||||
isMyEndpointProps, // request validator (from rest-typings)
|
||||
validateBadRequestErrorResponse,
|
||||
validateUnauthorizedErrorResponse,
|
||||
validateForbiddenErrorResponse,
|
||||
} from '@rocket.chat/rest-typings';
|
||||
```
|
||||
|
||||
## Using Typia `$ref` for Complex Types
|
||||
|
||||
For response fields that use complex types already defined in `@rocket.chat/core-typings` (like `IMessage`, `ISubscription`, `ICustomSound`, `IPermission`, etc.), **do not rewrite the JSON schema manually**. Instead, use `$ref` to link to the typia-generated schema.
|
||||
|
||||
### How it works
|
||||
|
||||
1. **`packages/core-typings/src/Ajv.ts`** generates JSON schemas from TypeScript types at compile time using typia:
|
||||
|
||||
```typescript
|
||||
import typia from 'typia';
|
||||
import type { ICustomSound } from './ICustomSound';
|
||||
// ...
|
||||
|
||||
export const schemas = typia.json.schemas<
|
||||
[
|
||||
ISubscription | IInvite | ICustomSound | IMessage | IOAuthApps | IPermission | IMediaCall,
|
||||
CallHistoryItem,
|
||||
ICustomUserStatus,
|
||||
SlashCommand,
|
||||
],
|
||||
'3.0'
|
||||
>();
|
||||
```
|
||||
|
||||
2. **`apps/meteor/app/api/server/ajv.ts`** registers all generated schemas into the shared AJV instance:
|
||||
|
||||
```typescript
|
||||
import { schemas } from '@rocket.chat/core-typings';
|
||||
import { ajv } from '@rocket.chat/rest-typings';
|
||||
|
||||
const components = schemas.components?.schemas;
|
||||
if (components) {
|
||||
for (const key in components) {
|
||||
if (Object.prototype.hasOwnProperty.call(components, key)) {
|
||||
ajv.addSchema(components[key], `#/components/schemas/${key}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Endpoints** reference the schema by `$ref` instead of writing it inline:
|
||||
|
||||
```typescript
|
||||
const customSoundsResponseSchema = ajv.compile<PaginatedResult<{ sounds: ICustomSound[] }>>({
|
||||
type: 'object',
|
||||
properties: {
|
||||
sounds: {
|
||||
type: 'array',
|
||||
items: { $ref: '#/components/schemas/ICustomSound' },
|
||||
},
|
||||
count: { type: 'number' },
|
||||
offset: { type: 'number' },
|
||||
total: { type: 'number' },
|
||||
success: { type: 'boolean', enum: [true] },
|
||||
},
|
||||
required: ['sounds', 'count', 'offset', 'total', 'success'],
|
||||
additionalProperties: false,
|
||||
});
|
||||
```
|
||||
|
||||
Source: `apps/meteor/app/api/server/v1/custom-sounds.ts`
|
||||
|
||||
### Available `$ref` schemas
|
||||
|
||||
These types are already registered and available via `$ref`:
|
||||
|
||||
- `#/components/schemas/ISubscription`
|
||||
- `#/components/schemas/IInvite`
|
||||
- `#/components/schemas/ICustomSound`
|
||||
- `#/components/schemas/IMessage`
|
||||
- `#/components/schemas/IOAuthApps`
|
||||
- `#/components/schemas/IPermission`
|
||||
- `#/components/schemas/IMediaCall`
|
||||
- `#/components/schemas/IEmailInbox`
|
||||
- `#/components/schemas/IImport`
|
||||
- `#/components/schemas/IIntegrationHistory`
|
||||
- `#/components/schemas/ICalendarEvent`
|
||||
- `#/components/schemas/IRole`
|
||||
- `#/components/schemas/IRoom`
|
||||
- `#/components/schemas/IUser`
|
||||
- `#/components/schemas/IModerationAudit`
|
||||
- `#/components/schemas/IModerationReport`
|
||||
- `#/components/schemas/IBanner`
|
||||
- `#/components/schemas/CallHistoryItem`
|
||||
- `#/components/schemas/ICustomUserStatus`
|
||||
- `#/components/schemas/SlashCommand`
|
||||
- `#/components/schemas/VideoConferenceCapabilities`
|
||||
- `#/components/schemas/CloudRegistrationIntentData`
|
||||
- `#/components/schemas/CloudRegistrationStatus`
|
||||
|
||||
**Union types** — these TypeScript types are unions, so typia generates individual schemas for each variant instead of a single named schema. Use `oneOf` to reference them (see below):
|
||||
|
||||
| TypeScript type | Generated schemas |
|
||||
| ----------------------------- | ----------------------------------------------------------------------------------------------------- |
|
||||
| `IIntegration` | `IIncomingIntegration`, `IOutgoingIntegration` |
|
||||
| `VideoConference` | `IDirectVideoConference`, `IGroupVideoConference`, `ILivechatVideoConference`, `IVoIPVideoConference` |
|
||||
| `VideoConferenceInstructions` | `DirectCallInstructions`, `ConferenceInstructions`, `LivechatInstructions` |
|
||||
| `CloudConfirmationPollData` | `CloudConfirmationPollDataPending`, `CloudConfirmationPollDataSuccess` |
|
||||
|
||||
Plus any sub-types that these reference internally (e.g., `MessageMention`, `IVideoConferenceUser`, `VideoConferenceStatus`, etc.).
|
||||
|
||||
### Handling union types with `oneOf`
|
||||
|
||||
When a TypeScript type is a union (e.g., `IIntegration = IIncomingIntegration | IOutgoingIntegration`), typia generates separate schemas for each variant but **no single named schema** for the union itself. Use `oneOf` to reference the variants:
|
||||
|
||||
```typescript
|
||||
// Single field
|
||||
integration: {
|
||||
oneOf: [
|
||||
{ $ref: '#/components/schemas/IIncomingIntegration' },
|
||||
{ $ref: '#/components/schemas/IOutgoingIntegration' },
|
||||
],
|
||||
},
|
||||
|
||||
// Array of union type
|
||||
integrations: {
|
||||
type: 'array',
|
||||
items: {
|
||||
oneOf: [
|
||||
{ $ref: '#/components/schemas/IIncomingIntegration' },
|
||||
{ $ref: '#/components/schemas/IOutgoingIntegration' },
|
||||
],
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
### Handling nullable types
|
||||
|
||||
When a field can be `null`, combine `nullable: true` with `$ref`:
|
||||
|
||||
```typescript
|
||||
// Nullable $ref
|
||||
report: { nullable: true, $ref: '#/components/schemas/IModerationReport' },
|
||||
|
||||
// Nullable union
|
||||
integration: {
|
||||
nullable: true,
|
||||
oneOf: [
|
||||
{ $ref: '#/components/schemas/IIncomingIntegration' },
|
||||
{ $ref: '#/components/schemas/IOutgoingIntegration' },
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
### Handling intersection types with `allOf`
|
||||
|
||||
When a response intersects a type with additional properties (e.g., `VideoConferenceInstructions & { providerName: string }`), use `allOf`:
|
||||
|
||||
```typescript
|
||||
data: {
|
||||
allOf: [
|
||||
{
|
||||
oneOf: [
|
||||
{ $ref: '#/components/schemas/DirectCallInstructions' },
|
||||
{ $ref: '#/components/schemas/ConferenceInstructions' },
|
||||
{ $ref: '#/components/schemas/LivechatInstructions' },
|
||||
],
|
||||
},
|
||||
{ type: 'object', properties: { providerName: { type: 'string' } }, required: ['providerName'] },
|
||||
],
|
||||
},
|
||||
```
|
||||
|
||||
This also applies when the API spreads a type at root level alongside `success` (e.g., `API.v1.success(emailInbox)` producing `{ ...emailInbox, success: true }`):
|
||||
|
||||
```typescript
|
||||
{
|
||||
allOf: [
|
||||
{ $ref: '#/components/schemas/IEmailInbox' },
|
||||
{ type: 'object', properties: { success: { type: 'boolean', enum: [true] } }, required: ['success'] },
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
### Types that are NOT good `$ref` candidates
|
||||
|
||||
Some TypeScript types cannot (or should not) be represented as a single `$ref`:
|
||||
|
||||
- **Complex union types with many variants**: `ISetting` (union of `ISettingBase | ISettingEnterprise | ISettingColor | ...`) generates too many individual schemas without a clean single reference.
|
||||
- **`Pick<>` / `Omit<>` types**: `Pick<IUser, 'username' | 'name' | '_id'>` is not registered in typia as a standalone schema. Leave these as `{ type: 'object' }` or `{ type: ['object', 'null'] }`.
|
||||
- **Intersection + union types**: `LoginServiceConfiguration` and similar complex composed types.
|
||||
- **Types with `Date | string` fields**: See [Known Pitfall: `Date | string` unions](#known-pitfall-date--string-unions) below.
|
||||
|
||||
For these cases, keep using `{ type: 'object' }` as a placeholder.
|
||||
|
||||
### Known Pitfall: `Date | string` unions
|
||||
|
||||
Some core-typings define timestamp fields as `Date | string` (e.g., `IModerationAudit.ts`, `IModerationReport.ts`). When typia generates the JSON Schema for these fields, it creates a `oneOf` with two branches: one for `Date` (which maps to `{ type: "string", format: "date-time" }`) and one for `string` (which maps to `{ type: "string" }`).
|
||||
|
||||
The problem: an ISO date string like `"2026-03-11T16:07:21.755Z"` satisfies **both** schemas simultaneously, causing AJV to fail with:
|
||||
|
||||
```
|
||||
must match exactly one schema in oneOf (passingSchemas: 0,1)
|
||||
```
|
||||
|
||||
This happens because `oneOf` requires **exactly one** match, but the value is both a valid `date-time` string and a valid `string`.
|
||||
|
||||
**Workaround**: Use a relaxed inline schema instead of `$ref` for these types, defining `ts` as `{ type: 'string' }`. Add a `TODO` comment referencing this issue so it can be tracked:
|
||||
|
||||
```typescript
|
||||
// TODO: IModerationAudit defines `ts` as `Date | string` which generates a oneOf in JSON Schema.
|
||||
// When the aggregation returns `ts` as an ISO date string, it matches both `Date` (format: "date-time")
|
||||
// and `string` schemas simultaneously, causing AJV oneOf validation to fail with "passingSchemas: 0,1".
|
||||
// Until the core-typings type is revised (either narrowing `ts` to `string` to match what MongoDB
|
||||
// aggregation actually returns, or adjusting the AJV schema generation for union types), we use a
|
||||
// relaxed inline schema here that accepts `ts` as a string.
|
||||
```
|
||||
|
||||
**Long-term fix**: Revise the core-typings to narrow `ts` to `string` (which is what MongoDB aggregation pipelines and `JSON.stringify` actually return), or adjust the AJV/typia schema generation to handle `Date | string` unions correctly (e.g., using `anyOf` instead of `oneOf`, or collapsing `Date` into `string`).
|
||||
|
||||
### Adding a new type to typia
|
||||
|
||||
If you need a `$ref` for a type that is not yet registered:
|
||||
|
||||
1. Edit `packages/core-typings/src/Ajv.ts`
|
||||
2. Import the type and add it to the `typia.json.schemas<[...]>()` type parameter list
|
||||
3. Rebuild `core-typings`: `yarn workspace @rocket.chat/core-typings run build`
|
||||
4. The new schema will be automatically registered at startup via `apps/meteor/app/api/server/ajv.ts`
|
||||
|
||||
## Chaining Endpoints and Type Augmentation
|
||||
|
||||
Migrated endpoints **must always be chained** when a file registers multiple endpoints. Store the result in a variable, then use `ExtractRoutesFromAPI` to extract the route types and augment the `Endpoints` interface in `rest-typings`. This is what makes the endpoints fully typed across the entire codebase (SDK, client, tests).
|
||||
|
||||
### Full example
|
||||
|
||||
```typescript
|
||||
import type { IInvite } from '@rocket.chat/core-typings';
|
||||
import {
|
||||
ajv,
|
||||
isFindOrCreateInviteParams,
|
||||
validateBadRequestErrorResponse,
|
||||
validateUnauthorizedErrorResponse,
|
||||
} from '@rocket.chat/rest-typings';
|
||||
|
||||
import type { ExtractRoutesFromAPI } from '../ApiClass';
|
||||
import { API } from '../api';
|
||||
|
||||
// Chain all endpoints from this file into a single variable
|
||||
const invites = API.v1
|
||||
.get(
|
||||
'listInvites',
|
||||
{
|
||||
authRequired: true,
|
||||
response: {
|
||||
200: ajv.compile<IInvite[]>({
|
||||
type: 'array',
|
||||
items: { $ref: '#/components/schemas/IInvite' },
|
||||
additionalProperties: false,
|
||||
}),
|
||||
401: validateUnauthorizedErrorResponse,
|
||||
},
|
||||
},
|
||||
async function action() {
|
||||
const result = await listInvites(this.userId);
|
||||
return API.v1.success(result);
|
||||
},
|
||||
)
|
||||
.post(
|
||||
'findOrCreateInvite',
|
||||
{
|
||||
authRequired: true,
|
||||
body: isFindOrCreateInviteParams,
|
||||
response: {
|
||||
200: findOrCreateInviteResponseSchema,
|
||||
400: validateBadRequestErrorResponse,
|
||||
401: validateUnauthorizedErrorResponse,
|
||||
},
|
||||
},
|
||||
async function action() {
|
||||
const { rid, days, maxUses } = this.bodyParams;
|
||||
return API.v1.success((await findOrCreateInvite(this.userId, { rid, days, maxUses })) as IInvite);
|
||||
},
|
||||
);
|
||||
|
||||
// Extract route types from the chained result
|
||||
type InvitesEndpoints = ExtractRoutesFromAPI<typeof invites>;
|
||||
|
||||
// Augment the Endpoints interface so the SDK, client hooks, and tests see these routes
|
||||
declare module '@rocket.chat/rest-typings' {
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, @typescript-eslint/no-empty-interface
|
||||
interface Endpoints extends InvitesEndpoints {}
|
||||
}
|
||||
```
|
||||
|
||||
Source: `apps/meteor/app/api/server/v1/invites.ts`
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Always chain**: Every `.get()` / `.post()` / `.put()` / `.delete()` call in the same file should be chained on the same variable (e.g., `const myEndpoints = API.v1.get(...).post(...).get(...)`).
|
||||
2. **Store in a `const`**: The chained result must be stored in a variable so `typeof` can extract its type.
|
||||
3. **Extract with `ExtractRoutesFromAPI`**: Use `type MyEndpoints = ExtractRoutesFromAPI<typeof myEndpoints>` to get the typed route definitions.
|
||||
4. **Augment `Endpoints`**: Use `declare module '@rocket.chat/rest-typings'` to merge the extracted types into the global `Endpoints` interface. This is what makes `useEndpoint('listInvites')` and similar SDK calls type-safe.
|
||||
5. **Import `ExtractRoutesFromAPI`** from `'../ApiClass'` (relative to the endpoint file in `v1/`).
|
||||
|
||||
### What augmentation enables
|
||||
|
||||
Once the `Endpoints` interface is augmented, the entire stack benefits:
|
||||
|
||||
- **Client SDK**: `useEndpoint('listInvites')` gets typed params and response
|
||||
- **REST client**: `api.get('/v1/listInvites')` is type-checked
|
||||
- **Tests**: response shape is inferred from the endpoint definition
|
||||
- **OpenAPI**: routes appear in the generated documentation
|
||||
|
||||
## Endpoints with Multiple HTTP Methods
|
||||
|
||||
When an `addRoute` registers both GET and POST (or other combinations), split them into separate calls:
|
||||
|
||||
```typescript
|
||||
// BEFORE
|
||||
API.v1.addRoute('endpoint', { authRequired: true, validateParams: { GET: isGetProps, POST: isPostProps } }, {
|
||||
async get() { ... },
|
||||
async post() { ... },
|
||||
});
|
||||
|
||||
// AFTER
|
||||
API.v1.get('endpoint', {
|
||||
authRequired: true,
|
||||
query: isGetProps,
|
||||
response: { 200: getResponseSchema, 400: validateBadRequestErrorResponse, 401: validateUnauthorizedErrorResponse },
|
||||
}, async function action() { ... });
|
||||
|
||||
API.v1.post('endpoint', {
|
||||
authRequired: true,
|
||||
body: isPostProps,
|
||||
response: { 200: postResponseSchema, 400: validateBadRequestErrorResponse, 401: validateUnauthorizedErrorResponse },
|
||||
}, async function action() { ... });
|
||||
```
|
||||
|
||||
## Error Handling (thrown errors)
|
||||
|
||||
Every route registered through `API.v1.get/post/put/delete` is wrapped by `ApiClass`'s internal handler (`_internalRouteActionHandler`), which runs the action inside a `try/catch`. A thrown `Meteor.Error` (or any core-services error carrying `error`/`message` by shape) is mapped to the matching HTTP failure:
|
||||
|
||||
- `error-too-many-requests` → `429`
|
||||
- `unauthorized` / `error-unauthorized` → `401` (or `403` pre-breaking-changes)
|
||||
- `forbidden` / `error-forbidden` → `403`
|
||||
- anything else → `API.v1.failure(message, errorType, stack, details)` → `400`
|
||||
|
||||
So a migrated handler should **just throw** to signal a client error — exactly as the legacy DDP methods and `addRoute` handlers did. Do **not** wrap every handler in a `try/catch` that returns `API.v1.failure(...)`: it is redundant with the global wrapper and, worse, flattens `401`/`403`/`429` into `400`.
|
||||
|
||||
```typescript
|
||||
async function action() {
|
||||
const room = await findChannelByIdOrName({ params: this.bodyParams }); // throws error-room-not-found
|
||||
// ...
|
||||
return API.v1.success({ channel: room });
|
||||
}
|
||||
```
|
||||
|
||||
Declare in the `response` block every error status the handler (or the wrapper) can produce, so response validation under `TEST_MODE` accepts them — `400: validateBadRequestErrorResponse` for thrown/failure errors, plus `401`/`403` when `authRequired`/`permissionsRequired` are set.
|
||||
|
||||
Add an explicit `catch` only when you need behaviour the global wrapper does not provide (e.g. returning a specific `API.v1.failure(...)` payload for a known condition, or mapping a status differently); otherwise let the error propagate.
|
||||
|
||||
## Test Changes
|
||||
|
||||
Migrating an endpoint changes how validation errors are returned. Tests must be updated accordingly.
|
||||
|
||||
### `errorType` changes for query parameter validation
|
||||
|
||||
The new router returns a different `errorType` for query parameter validation errors:
|
||||
|
||||
```typescript
|
||||
// BEFORE (addRoute with validateParams)
|
||||
expect(res.body).to.have.property('errorType', 'invalid-params');
|
||||
|
||||
// AFTER (.get() with query)
|
||||
expect(res.body).to.have.property('errorType', 'error-invalid-params');
|
||||
```
|
||||
|
||||
This only affects **query** parameter validation (GET/DELETE). Body parameter validation (POST/PUT) keeps `'invalid-params'`.
|
||||
|
||||
### Error message format changes
|
||||
|
||||
The `[invalid-params]` suffix is removed from error messages:
|
||||
|
||||
```typescript
|
||||
// BEFORE
|
||||
expect(res.body).to.have.property('error', "must have required property 'platform' [invalid-params]");
|
||||
|
||||
// AFTER
|
||||
expect(res.body).to.have.property('error', "must have required property 'platform'");
|
||||
```
|
||||
|
||||
### Summary of test changes per endpoint
|
||||
|
||||
When migrating an endpoint, search for its tests and update:
|
||||
|
||||
1. `errorType` from `'invalid-params'` to `'error-invalid-params'` (for query params only)
|
||||
2. Remove `' [invalid-params]'` suffix from `error` message assertions
|
||||
3. Verify that status codes remain the same (400 for validation errors)
|
||||
|
||||
## Reference Files
|
||||
|
||||
| Pattern | File |
|
||||
| -------------------------------- | ------------------------------------------------ |
|
||||
| Chaining + augmentation | `apps/meteor/app/api/server/v1/invites.ts` |
|
||||
| Chaining + augmentation + `$ref` | `apps/meteor/app/api/server/v1/custom-sounds.ts` |
|
||||
| GET with `$ref` to typia schemas | `apps/meteor/app/api/server/v1/custom-sounds.ts` |
|
||||
| GET with pagination | `apps/meteor/app/api/server/v1/moderation.ts` |
|
||||
| POST endpoint | `apps/meteor/app/api/server/v1/import.ts` |
|
||||
| Multiple endpoints (misc) | `apps/meteor/app/api/server/v1/misc.ts` |
|
||||
| GET with permissions | `apps/meteor/app/api/server/v1/permissions.ts` |
|
||||
| Typia schema generation | `packages/core-typings/src/Ajv.ts` |
|
||||
| AJV schema registration | `apps/meteor/app/api/server/ajv.ts` |
|
||||
| Error response validators | `packages/rest-typings/src/v1/Ajv.ts` |
|
||||
| Request validators (examples) | `packages/rest-typings/src/v1/moderation/` |
|
||||
| Router implementation | `packages/http-router/src/Router.ts` |
|
||||
83
docs/apps-engine-migration.md
Normal file
83
docs/apps-engine-migration.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# Apps-Engine migration to microservice
|
||||
|
||||
## Goal
|
||||
|
||||
Isolate apps execution into a microservice to decouple monolith scaling from apps-engine scaling. However, the current package architecture ends up making this transition quite difficult, so we'll move in phases.
|
||||
|
||||
Here is an overview of the action items required to achieve our goal:
|
||||
|
||||
- **Migrate app management code from @rocket.chat/apps-engine to private @rocket.chat/apps** - Stop publishing code that handles app management on the server alongside the public surface that apps actually use. Anything living _outside_ the `@rocket.chat/apps-engine/definition/` directory should either move to the internal `@rocket.chat/apps` package or be removed.
|
||||
|
||||
- **Consolidate all interaction with the apps internals on the server to the AppsEngineService** - Backend code handling apps business rules has to happen via a single entrypoint: the AppsEngineService class. This will give us the flexibility of serving the functionality either locally or through NATS.
|
||||
|
||||
- **Break the file upload flow for async validation step** - Decouple upload validation from the actual file upload request. This allows us to run the Apps-Engine `IPreFileUpload` event asynchronously and gives us a longer window for the execution, which means we're not forced to transfer file contents over NATS.
|
||||
|
||||
- **Create Apps-Engine service Docker image**
|
||||
|
||||
## Part 1: Migrate app management code from @rocket.chat/apps-engine to private @rocket.chat/apps
|
||||
|
||||
The Apps-Engine originally lived in its own repository, so the colocation of domain there made some sense. However, especially after the introduction of the deno-runtime, a large portion of the code published is never used by the apps themselves, reserved only for server usage. As we moved the package into the monorepo, the initial colocation stopped being a requirement.
|
||||
|
||||
Besides that, migrating app management code away from a public package enables us to integrate more closely with the private packages available in the monorepo, so we can effectivelly reuse more code and standardize approaches like `@rocket.chat/logger`.
|
||||
|
||||
To make this migration easier to understand and review, we're using a stacked PR approach on Github - similar to a feature branch but disallowing sibling PRs. They are:
|
||||
|
||||
- [40395](https://github.com/RocketChat/Rocket.Chat/pull/40395) The feature branch itself. It will accumulate the changes of the whole stack (⟶ `develop`)
|
||||
- [40183](https://github.com/RocketChat/Rocket.Chat/pull/40183) Replaces `AppPackageParser.getEngineVersion()` - which resolved the version by traversing the filesystem relative to `__dirname` - with a direct import of `ENGINE_VERSION`. This will support the migration of the `AppPackageParser` class itself (⟶ #40395)
|
||||
- [40184](https://github.com/RocketChat/Rocket.Chat/pull/40184) Copies all relevant source files from `packages/apps-engine/src/server`, `packages/apps-engine/src/client`, `packages/apps-engine/deno-runtime`, `packages/apps-engine/tests` and `packages/apps-engine/scripts` into their corresponding path at `packages/apps` (⟶ #40183)
|
||||
- [40185](https://github.com/RocketChat/Rocket.Chat/pull/40185) Adapts the path resolution of the apps-engine package for the deno runtime (⟶ #40184)
|
||||
- [40186](https://github.com/RocketChat/Rocket.Chat/pull/40186) Flips the switch; changes code in `apps/meteor` and `packages/core-services` that pointed to `@rocket.chat/apps-engine` and makes it point to `@rocket.chat/apps`. This is the turning point in what is actually executed (⟶ #40185)
|
||||
- [40343](https://github.com/RocketChat/Rocket.Chat/pull/40343) Removes old files from the `@rocket.chat/apps-engine` package (⟶ #40186)
|
||||
|
||||
### Architecturally Relevant Decisions
|
||||
|
||||
#### 1. `@rocket.chat/apps-engine` becomes a definition-only public package
|
||||
|
||||
The most consequential decision in this migration is that `@rocket.chat/apps-engine` is narrowed to expose **only** its `definition/` directory. All server-side management code, the client UI host, the deno-runtime, scripts, and tests were removed from the package. The `files` field in `package.json` went from listing `client/**`, `definition/**`, `deno-runtime/**`, `lib/**`, `scripts/**`, and `server/**` down to just `definition/**`. The package description was updated to reflect this: "The public API and type definitions for Rocket.Chat App development."
|
||||
|
||||
This preserves the public contract for external app developers while keeping all server-side complexity private and under active control inside the monorepo.
|
||||
|
||||
#### 2. `@rocket.chat/apps` becomes the operational package
|
||||
|
||||
`@rocket.chat/apps` (already private, `"private": true`) absorbs everything that was removed from `apps-engine`. Its `package.json` gained the runtime dependencies that `apps-engine` shed (`adm-zip`, `debug`, `esbuild`, `jose`, `jsonrpc-lite`, `lodash.clonedeep`, `semver`, `stack-trace`) and its build pipeline was extended to also run `deno-cache` and produce `deno-runtime/` and `.deno-cache/` artifacts alongside the TypeScript `dist/`.
|
||||
|
||||
The package also received its own `turbo.json` that declares those additional output directories, ensuring Turborepo cache invalidation covers the deno artefacts.
|
||||
|
||||
#### 3. `ENGINE_VERSION` is exported from the definition layer
|
||||
|
||||
`AppPackageParser` validated app compatibility by reading the engine's `package.json` via filesystem traversal relative to `__dirname`. That assumption breaks the moment the code moves to a different package. The fix is a dedicated `packages/apps-engine/src/definition/version.ts` that exports a single constant `ENGINE_VERSION`. It uses a `require()` (not a static import) with a path determined at runtime by checking whether `__dirname` ends with `src/definition` (source/ts-node context) or not (compiled context), resolving to `../../package.json` or `../package.json` respectively. Using `require()` prevents TypeScript from resolving the path at compile time, making the two-level distinction safe.
|
||||
|
||||
#### 4. Dynamic Deno import map generation at subprocess spawn time
|
||||
|
||||
The previous static `deno.jsonc` contained `"@rocket.chat/apps-engine/": "./../src/"` — a relative path that only worked when `deno-runtime/` and `apps-engine/src/` were siblings inside the same package. After the move, `deno-runtime/` lives in `@rocket.chat/apps` while the type definitions it imports live in `@rocket.chat/apps-engine`.
|
||||
|
||||
The solution is to drop the static entry and instead generate a temporary `deno_runtime.jsonc` file into the subprocess temp directory before each Deno process spawn. `DenoRuntimeSubprocessController` resolves the apps-engine source directory at runtime via `require.resolve('@rocket.chat/apps-engine/package.json')` and injects it as an absolute `file:` URL into the import map. This strategy works correctly in any environment (monorepo dev, Meteor bundle, CI) without any assumptions about directory layout.
|
||||
|
||||
Additionally, `--sloppy-imports` was added to the Deno subprocess flags, allowing the runtime to import `.ts` files from `apps-engine` without requiring explicit `.ts` extensions in every import specifier.
|
||||
|
||||
#### 5. Cross-boundary types promoted to the definition layer
|
||||
|
||||
Several types that were previously only exported from `server/` were needed by code that must stay in the public API. Rather than giving consumers a path that crosses into `@rocket.chat/apps`, these types were moved into `definition/`:
|
||||
|
||||
- `GetMessagesOptions`, `GetRoomsFilters`, `GetRoomsOptions`, and `GetMessagesSortableFields` moved from `server/bridges/RoomBridge.ts` into `definition/rooms/IGetMessagesOptions.ts` and re-exported from `definition/rooms/index.ts`.
|
||||
- `OAuth2Client` moved from `server/oauth2/OAuth2Client.ts` into `definition/oauth2/OAuth2Client.ts`; the server copy became a re-export pointing to the definition location.
|
||||
- `IExternalComponentRoomInfo` and `IExternalComponentUserInfo` moved from `client/definition/` into `definition/externalComponent/`, keeping `IExternalComponentState` (which depends on them) coherent within the definition layer.
|
||||
- `IRoomRead` and other accessors that referenced the moved types had their import paths updated accordingly.
|
||||
|
||||
The guiding principle: anything that crosses the public/private boundary must live in `definition/`.
|
||||
|
||||
#### 6. Deep `@rocket.chat/apps/dist/` path imports
|
||||
|
||||
An attempt to use the modern `exports` field in `@rocket.chat/apps/package.json` (to define clean subpath entry points like `./server/*` and `./client/*`) was reverted because Meteor's bundler does not support the `exports` field, and thus could not resolve import paths as expected. The package fell back to `main` + `types` for the root entry point, and server-only types are imported via deep `dist/` paths (e.g., `@rocket.chat/apps/dist/server/IGetAppsFilter`). `typesVersions` was also removed for the same reason.
|
||||
|
||||
#### 7. TypeScript strict mode relaxed for the migrated code
|
||||
|
||||
The migrated code from `apps-engine` was written without strict TypeScript checks. Rather than fixing every existing violation upfront, `packages/apps/tsconfig.json` explicitly disables `strict`, `noUnusedParameters`, `noImplicitOverride`, and `noImplicitReturns`. This is intentional technical debt: the migration is a lift-and-shift first; tightening the type-checking can happen incrementally afterwards.
|
||||
|
||||
#### 8. CI and Dockerfile changes flow to the new owner
|
||||
|
||||
The `esbuild` platform-specific binary cleanup that runs inside Docker builds previously targeted `@rocket.chat/apps-engine/node_modules/@esbuild`; this was updated to `@rocket.chat/apps/node_modules/@esbuild`. Likewise, the microservice Dockerfiles (account-service, authorization-service, ddp-streamer, etc.) that copied `packages/apps-engine/client` into the image dropped that directory, as it no longer exists in the package. The `packages/apps-engine/definition` copy was kept because those types are still consumed by core-services and other packages that run inside those microservices.
|
||||
|
||||
#### 9. ESLint rules unified across both packages
|
||||
|
||||
The root `eslint.config.mjs` was updated to apply the same rule overrides (permissive `@typescript-eslint` settings, test-file non-null assertion relaxation, and built artefact `ignores`) to both `packages/apps-engine/**/*` and `packages/apps/**/*`, treating the two packages as a coherent unit under the same lint policy.
|
||||
123
docs/backend-folder-structure.md
Normal file
123
docs/backend-folder-structure.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Backend Folder Structure
|
||||
|
||||
This document describes how the server-side code of the main app (`apps/meteor/`) is organized, what each folder is responsible for, and — most importantly — **how to decide where new code goes**. It replaces the migration plan that drove the 2026 restructure (PRs #40259, #41115, #41126, #41155, #41225, #41315, #41381), which moved everything out of the old `app/*/server/` feature folders.
|
||||
|
||||
## The organizing principle
|
||||
|
||||
```
|
||||
server/<responsibility>/<domain>/<file>
|
||||
```
|
||||
|
||||
Code is grouped **by responsibility first** (what *kind* of code it is: an API endpoint, a hook, a domain function, a Meteor method) **and by domain second** (what *feature area* it belongs to: rooms, users, messages, omnichannel…). This is the opposite of the old layout, which grouped by feature first (`app/livechat/server/…`) and made responsibilities (methods, hooks, API, lib) repeat inside every feature.
|
||||
|
||||
Practical consequence: when you look for code, ask "what kind of thing is it?" before "what feature is it for?". A livechat REST endpoint is in `server/api/v1/omnichannel/`, a livechat hook in `server/hooks/omnichannel/`, a livechat domain function in `server/lib/omnichannel/`.
|
||||
|
||||
## Top-level map of `apps/meteor/`
|
||||
|
||||
| Path | What it is |
|
||||
| ----------- | ---------- |
|
||||
| `server/` | **All community server code.** Described in detail below. |
|
||||
| `ee/server/` | Enterprise server code — an exact mirror of `server/`'s responsibility layout. See [License boundary](#the-license-boundary-ee). |
|
||||
| `client/`, `ee/client/` | Frontend (React) code. |
|
||||
| `app/` | **Legacy remnant, do not add server code here.** What's left is genuinely shared client+server code (`app/*/lib/`), client-only feature code (`app/*/client/`), and `app/apps/server/` (Apps-Engine bridges/converters, kept intentionally). |
|
||||
| `lib/` | Small utilities shared between client and server (isomorphic, no Meteor/server imports). |
|
||||
| `imports/` | Legacy Meteor-style feature folders (`personal-access-tokens`, message-read-receipt client parts). Avoid adding to it. |
|
||||
| `packages/` (repo root) | Workspace packages (`@rocket.chat/*`). Cross-app logic, typings (`core-typings`, `rest-typings`), and services SDK (`core-services`) live here — prefer a package when code must be shared beyond `apps/meteor`. |
|
||||
| `tests/` | Test suites — see [Tests](#tests). |
|
||||
|
||||
## Inside `server/`
|
||||
|
||||
### Entry points
|
||||
|
||||
- `main.ts` — the server entry point. Import order here matters (models → settings definitions → startup → `importPackages` → methods → publications).
|
||||
- `importPackages.ts` — the big list of side-effect imports that load every feature at startup. If you create a module that must run at boot and nothing imports it, register it here (or in the closer aggregator: `meteor-methods/index.ts`, `startup/index.ts`).
|
||||
- `models.ts` — registers the MongoDB model classes from `@rocket.chat/models`.
|
||||
|
||||
### Responsibility folders
|
||||
|
||||
| Folder | Responsibility | Domain layout |
|
||||
| ------ | -------------- | ------------- |
|
||||
| `api/` | REST API. `ApiClass.ts`/`router.ts` are the Hono-based framework, `v1/` holds one file per endpoint group (`users.ts`, `chat.ts`, …), `v1/omnichannel/` the livechat endpoints, `v1/middlewares/` auth/cors/metrics, `lib/` request helpers, `validation/` the ajv setup, `webhooks.ts` the `/hooks/*` incoming-webhook API. Endpoint payload types live in `packages/rest-typings`. |
|
||||
| `services/` | Internal services registered on the `@rocket.chat/core-services` broker (one folder per service: `room/`, `team/`, `omnichannel/`, `federation/`, …). Callable cross-process in microservices deployments. **Preferred home for new business logic** — see the decision guide. |
|
||||
| `lib/` | Everything that is a plain function/class library: domain functions and shared server utilities. See [`server/lib/` domains](#serverlib-domains). |
|
||||
| `hooks/` | Event handlers registered on the `callbacks` system (`callbacks.add('afterSaveMessage', …)`). Domain subfolders: `messages/`, `rooms/`, `auth/`, `omnichannel/`; a few cross-cutting ones sit flat (`afterUserActions.ts`, `sauMonitorHooks.ts`). A hook file only registers listeners — real logic should live in `lib/` or `services/` and be called from the hook. |
|
||||
| `meteor-methods/` | **Deprecated.** Meteor DDP RPC handlers, kept only for backward compatibility with existing clients. Domain subfolders (`users/`, `rooms/`, `messages/`, `auth/`, `omnichannel/`, `settings/`, `platform/`, `media/`, `import/`, `integrations/`). Registration happens via side-effect import from `meteor-methods/index.ts` — a method file that nobody imports **silently stops existing** (lint/tsc/tests all stay green), so never remove an import without removing the method. Do not add new methods; add a REST endpoint instead. |
|
||||
| `publications/` | **Deprecated.** Meteor DDP publications. Same story as methods: don't add, use REST + streams. |
|
||||
| `settings/` | The settings system. `index.ts` is the **registry barrel** (`settings`, `settingsRegistry` — what everything imports); `definitions.ts` loads the per-domain setting definition files (`accounts.ts`, `email.ts`, `omnichannel.ts`, …) that call `settingsRegistry.addGroup(...)`; `theme/` is the theme compiler; `lib/` settings-specific helpers. New setting? Add it to the matching definitions file, not to the registry. |
|
||||
| `slashcommands/` | One file per slash command (`ban.ts`, `invite.ts`, …). |
|
||||
| `bridges/` | Adapters to external systems: `irc/`, `slack/` (slackbridge), `smarsh/`, `webdav/`, `nextcloud/`. Leaf code — imports the core, nothing imports it. |
|
||||
| `cron/` | Scheduled jobs. |
|
||||
| `startup/` | Boot-time configuration and one-off initialization (`initialData.ts`, `rateLimiter.js`, `robots.js`, `migrations/`). |
|
||||
| `routes/` | Non-REST HTTP routes (avatar serving, etc.). |
|
||||
| `database/` | Mongo connection utilities (`trash`, `readSecondaryPreferred`, transaction helpers). |
|
||||
| `modules/` | Larger self-contained subsystems (core-apps, listeners, notifications, streamer). |
|
||||
| `features/` | Feature-flag style subsystems (e.g. `EmailInbox/`). |
|
||||
| `configuration/` | Runtime configuration glue (OAuth, CAS, LDAP wiring). |
|
||||
| `email/`, `ufs/`, `oauth2-server/`, `deasync/` | Infrastructure kept as-is: mailer transport, Upload-File-System storage engine, OAuth2 provider implementation, deasync shim. |
|
||||
|
||||
### `server/lib/` domains
|
||||
|
||||
`lib/` is the largest folder. Its immediate children are **domains** (plus a handful of flat cross-cutting utilities like `callbacks.ts`, `i18n.ts`, `notifyListener.ts`, `RateLimiter.js`, `debug.js`, `bugsnag.ts`, `deprecationWarningLogger.ts`, `validateEmailDomain.js`, `defaultBlockedDomainsList.ts`).
|
||||
|
||||
| Domain | Contents |
|
||||
| ------ | -------- |
|
||||
| `users/` | User domain functions: `deleteUser`, `setRealName`, `saveUserIdentity`, `getFullUserData`, availability checks, `status/` (custom user status)… |
|
||||
| `rooms/` | Room domain functions: `createRoom`, `deleteRoom`, `addUserToRoom`, `roomCoordinator`, plus `settings/` (save-room-* functions), `invites/`, `retention/` (retention-policy cron). |
|
||||
| `messages/` | Message domain functions: `sendMessage`, `insertMessage`, `deleteMessage`, `updateMessage`, `loadMessageHistory`… |
|
||||
| `messaging/` | Messaging *features* around messages: `threads/`, `discussions/`, `reactions/`, `pins/`, `stars/`, `unread/`, `mentions/`, `markdown/`, `emoji`, `msgStream`, `Message.ts` (formatting), `getHiddenSystemMessages`, `validateCustomMessageFields`. Rule of thumb: `messages/` = lifecycle of a message document; `messaging/` = features built on top of messages. |
|
||||
| `omnichannel/` | The livechat/omnichannel engine: `QueueManager`, `RoutingManager` (+ `routing/` strategies), `business-hour/`, `contacts/`, `analytics/`, transcripts, visitors, guests, departments… The biggest domain. |
|
||||
| `authorization/` | Permission/role checks: `hasPermission`, `canAccessRoom`, `getRoles`, `constant/permissions.ts` (the permission list), `streamer/` (permission change notifications). |
|
||||
| `auth/` | Authentication utilities: `passwordPolicy`, `generatePassword`, login attempt logging/restriction, `oauth2-server/` (OAuth app admin), `token-login`. |
|
||||
| `auth-providers/` | One folder/file per external login provider: `apple/`, `crowd/`, `custom-oauth/`, `oauth/` (core OAuth glue), `github.ts`, `gitlab.ts`, `google.js`, `wordpress.ts`, … |
|
||||
| `saml/`, `cas/`, `ldap/`, `2fa/`, `e2e/` | Protocol-specific auth/crypto subsystems. |
|
||||
| `notifications/` | Notification delivery: `push/`, `push-config/`, `email/` (Mailer API), `mail-messages/`, `queue/`, `core/` (Notifications streams), `message/` (per-message desktop/email/mobile decision helpers), direct-reply email processing. |
|
||||
| `media/` | Files and media: `file-upload/` (FileUpload + storage configs), `file/`, `emoji-custom/`, `emoji-native/`, `custom-sounds/`, `assets/`. |
|
||||
| `import/` | Importer framework (`classes/`, converters) + one folder per importer (`csv/`, `slack/`, `slack-users/`, `omnichannel-contacts/`, `pending-avatars/`, `pending-files/`). |
|
||||
| `integrations/` | Incoming/outgoing webhook integrations engine (the REST surface is `server/api/webhooks.ts`). |
|
||||
| `cloud/` | Rocket.Chat Cloud connectivity, `version-check/`, license sync. |
|
||||
| `search/`, `autotranslate/`, `statistics/`, `metrics/`, `moderation/`, `dataExport/`, `oauth/`, `roles/`, `bot-helpers/`, `cors/`, `error-handler/`, `ui-master/` | Smaller single-purpose domains, named after what they do. |
|
||||
| `utils/` | Generic server utilities without a domain (`getURL`, `slashCommand` registry, `restrictions`, JWT helper, timezone…). If a function clearly belongs to a domain above, it does **not** go here. |
|
||||
| `shared/` | Tiny validators shared across domains (`validateName`, `validateNameChars`…). |
|
||||
|
||||
## The EE tree (`ee/server/`) and the license boundary
|
||||
|
||||
`ee/server/` mirrors the same responsibility layout: `api/`, `hooks/`, `lib/` (with `omnichannel/`, `license/`, `ldap/`, `canned-responses/`, `abac/`, `audit/`, …), `meteor-methods/`, `settings/`, `cron/`, `models/`, `patches/`, `startup/`, `configuration/`, and `local-services/` (EE internal services — note: `ee/server/services/` is docker/build scaffolding for the microservices images, **not** a code folder).
|
||||
|
||||
**The directory boundary is the license boundary.** Code under an `ee/` path is governed by the Enterprise license (`apps/meteor/ee/LICENSE`); everything else is community-licensed:
|
||||
|
||||
- Never move a file across the `ee/` boundary in either direction — that silently relicenses it.
|
||||
- Imports may cross the boundary (community code imports EE modules behind license checks and vice versa); files may not.
|
||||
- EE features are gated at runtime via `License.onLicense(...)` / license modules — moving a file into `ee/` is not what gates it, but EE-licensed code must live there.
|
||||
- Standalone EE workspaces (`ee/apps/*`, `ee/packages/*` at the repo root) and EE subtrees inside packages (`packages/*/src/ee/`) are separate from `apps/meteor/ee/`.
|
||||
|
||||
## Where do I put new code?
|
||||
|
||||
1. **A REST endpoint** → `server/api/v1/<group>.ts` (omnichannel ones in `v1/omnichannel/`), payload schemas in `packages/rest-typings`. This is the default way to expose anything to clients.
|
||||
2. **Business logic** → prefer a **service** in `server/services/` (registered via `@rocket.chat/core-services`) when the logic has a clear service boundary or must be callable cross-process. Plain **domain functions** in `server/lib/<domain>/` are fine for logic tied to one domain that services and endpoints compose. Long-term direction: business logic trends toward `services/`; `lib/<domain>/` is the pragmatic middle ground.
|
||||
3. **Reacting to an event** ("when a message is saved…", "after a user logs in…") → `server/hooks/<domain>/`. Keep the hook file thin; put the logic in `lib/`/`services/`.
|
||||
4. **A Meteor method or publication** → don't. Both are deprecated; expose a REST endpoint (+ streamer event if clients need pushes). If you must touch an existing method, it lives in `server/meteor-methods/<domain>/` and must stay imported by `meteor-methods/index.ts`.
|
||||
5. **A setting** → the matching definitions file under `server/settings/`.
|
||||
6. **A slash command** → `server/slashcommands/`.
|
||||
7. **A scheduled job** → `server/cron/`.
|
||||
8. **Startup-only wiring** → `server/startup/` (and an import in `main.ts`/`importPackages.ts` if nothing else loads it).
|
||||
9. **Code shared with the frontend** → `apps/meteor/lib/` for app-local isomorphic helpers, or a `packages/*` package if shared beyond the app. Never import server code from client code.
|
||||
10. **Enterprise-only** → same decision tree inside `ee/server/`, gated by license checks.
|
||||
11. **A new domain?** If your code doesn't fit an existing `lib/` domain, create a new `server/lib/<domain>/` folder rather than dropping files flat in `lib/` or stretching `utils/`. Name it after the business concept, not the implementing tech.
|
||||
|
||||
Anti-patterns to avoid:
|
||||
|
||||
- Adding server code under `app/` (the folder is frozen; only shared/client code remains there).
|
||||
- Barrel `index.ts` files that just re-export — import the concrete file. (Existing barrels like `settings/index.ts` and `authorization/index.ts` are legacy API surfaces, kept until their consumers are rewritten.)
|
||||
- Putting logic in a hook/method/endpoint file instead of `lib`/`services` — those files should be thin entry points.
|
||||
- "Utility" dumping: if it has a domain, it goes in the domain.
|
||||
|
||||
## Tests
|
||||
|
||||
- **Co-located specs** (`server/**/**.spec.ts`) and **mirrored specs** (`tests/unit/server/**` mirrors the source tree; EE code is mirrored under `ee/tests/unit/**`) both exist. When you move source, move its mirror.
|
||||
- Two runners with **separate globs**: mocha (`.mocharc.js` `spec` list) and jest (`jest.config.ts` `testMatch`). A spec in a folder not matched by any glob silently never runs; a jest spec caught by a mocha glob throws `jest is not defined`.
|
||||
- `proxyquire`/`jest.mock` keys are **string literals matched against the loaded module's import specifiers** — lint and tsc cannot validate them. If you move a module, every mock key referencing it (or referencing its dependencies' specifiers) must be updated, and the affected suites actually run. A stale key silently loads the real dependency; if that dependency reaches `server/settings/index.ts`, the whole mocha run dies with a top-level-await transform error — that error is the signature of a stale settings mock.
|
||||
- Specs must not depend on suite execution order (e.g. on another spec having installed a mocked core-services broker) — mock `@rocket.chat/core-services` locally.
|
||||
|
||||
## History
|
||||
|
||||
The structure above is the result of a 7-phase migration (finished 2026-07) that dissolved the old Meteor-package-style `app/*/server/` folders: slash commands (#40259), bridges (#41115), REST API (#41126), domain functions (#41155), meteor-methods (#41225), lib/hooks/feature code (#41315), omnichannel + final cleanup (#41381). Files were moved as-is, so `git log --follow` works across the moves. The old `app/<feature>/server/` path of any file can be found in those PRs' manifests if needed.
|
||||
174
docs/bundle-optimization-react-aria.md
Normal file
174
docs/bundle-optimization-react-aria.md
Normal file
@@ -0,0 +1,174 @@
|
||||
# Bundle Optimization: barrel import patches (react-aria, react-stately, zod)
|
||||
|
||||
## The Problem
|
||||
|
||||
Meteor's bundler (`standard-minifier-js`) does **not** perform tree-shaking. When any code imports from the `react-aria` barrel package:
|
||||
|
||||
```typescript
|
||||
import { FocusScope } from 'react-aria';
|
||||
```
|
||||
|
||||
Meteor resolves the barrel's entry point, which re-exports **all 43 sub-packages** (`@react-aria/dnd`, `@react-aria/calendar`, `@react-aria/table`, etc.). Since there is no dead-code elimination, every sub-package ends up in the main JS bundle — even if only `FocusScope` is used.
|
||||
|
||||
The same applies to `react-stately` (20 sub-packages).
|
||||
|
||||
**Measured impact before the fix:** 832 KB of react-aria + react-stately in the main bundle, of which ~700 KB was unused code.
|
||||
|
||||
## The Solution
|
||||
|
||||
Two complementary changes:
|
||||
|
||||
### 1. Yarn patches on the barrel packages
|
||||
|
||||
`yarn patch` replaces the barrel entry points (`dist/main.js`, `dist/module.js`, `dist/import.mjs`) with slim versions that only re-export the sub-packages actually used.
|
||||
|
||||
The patches live in `.yarn/patches/` and are referenced in `package.json` resolutions. They are applied automatically on every `yarn install`.
|
||||
|
||||
**react-aria** — retained sub-packages:
|
||||
`button`, `focus`, `i18n`, `interactions`, `listbox`, `menu`, `overlays`, `select`, `separator`, `slider`, `utils`, `visually-hidden`, `dialog`
|
||||
|
||||
**react-stately** — retained sub-packages:
|
||||
`collections`, `menu`, `overlays`, `select`, `slider`, `tree`
|
||||
|
||||
### 2. Direct sub-package imports in source code
|
||||
|
||||
All imports across `apps/meteor` and workspace packages (`gazzodown`, `ui-client`, `ui-contexts`, `ui-voip`) were changed from barrel to direct sub-package imports:
|
||||
|
||||
```typescript
|
||||
// Before
|
||||
import { FocusScope } from 'react-aria';
|
||||
import { useOverlayTriggerState } from 'react-stately';
|
||||
|
||||
// After
|
||||
import { FocusScope } from '@react-aria/focus';
|
||||
import { useOverlayTriggerState } from '@react-stately/overlays';
|
||||
```
|
||||
|
||||
This is the correct long-term fix, but alone it was not enough because `@rocket.chat/fuselage` (published npm package) still uses `require("react-aria")` internally via its UMD build. The yarn patch covers that case.
|
||||
|
||||
## Results (react-aria + react-stately)
|
||||
|
||||
| Metric | Before | After | Delta |
|
||||
|---|---|---|---|
|
||||
| Main JS (minified) | 3688 KB | 3199 KB | **-489 KB (-13%)** |
|
||||
| Main JS (gzip) | 939 KB | 841 KB | **-98 KB (-10%)** |
|
||||
|
||||
---
|
||||
|
||||
## Zod v4 locale barrel
|
||||
|
||||
### The Problem
|
||||
|
||||
Zod v4 re-exports all 50 locale files from its main entry point:
|
||||
|
||||
```js
|
||||
// zod/v4/classic/external.js
|
||||
export * as locales from "../locales/index.js";
|
||||
```
|
||||
|
||||
This means `import { z } from 'zod'` pulls in error messages for Arabic, Hebrew, Thai, Russian, and 46 other languages — even though Rocket.Chat only uses the English locale (loaded by default via `config(en())`).
|
||||
|
||||
**Measured impact:** 147 KB (50 locale files) out of 278 KB total for zod — **53% of the zod bundle was unused locale data**.
|
||||
|
||||
### The Solution
|
||||
|
||||
A `yarn patch` on `zod@4.3.6` removes the `export * as locales` line from the barrel. The English locale (`en.js`) remains loaded since it's imported separately by `config(en())`. All other locales remain available for explicit import if needed:
|
||||
|
||||
```typescript
|
||||
// Still works:
|
||||
import pt from 'zod/v4/locales/pt';
|
||||
import { config } from 'zod';
|
||||
config(pt());
|
||||
```
|
||||
|
||||
### What requires attention
|
||||
|
||||
#### Upgrading zod
|
||||
|
||||
When upgrading zod, the patch must be re-created:
|
||||
|
||||
1. Remove the old patch reference from `package.json` resolutions
|
||||
2. Delete `.yarn/patches/zod-*.patch`
|
||||
3. Run `yarn install`
|
||||
4. Run `yarn patch zod@npm:<new-version>`
|
||||
5. Remove the `export * as locales` line from `v4/classic/external.js`
|
||||
6. Remove the `exports.locales` require from `v4/classic/external.cjs`
|
||||
7. Run `yarn patch-commit -s <patch-folder>`
|
||||
|
||||
#### If zod locales are needed at runtime
|
||||
|
||||
If a feature requires localized zod error messages (e.g., form validation in the user's language), import the specific locale directly instead of relying on the barrel:
|
||||
|
||||
```typescript
|
||||
import de from 'zod/v4/locales/de';
|
||||
import { config } from 'zod';
|
||||
config(de());
|
||||
```
|
||||
|
||||
This loads only the one locale needed (~3 KB) instead of all 50 (~147 KB).
|
||||
|
||||
## What requires attention
|
||||
|
||||
### Adding new react-aria hooks or components
|
||||
|
||||
If you need to use a react-aria hook or component that is **not** in the slim barrel (e.g., `useCalendar`, `useTable`, `useDrag`), you must:
|
||||
|
||||
1. **Import from the sub-package directly** (preferred):
|
||||
```typescript
|
||||
import { useCalendar } from '@react-aria/calendar';
|
||||
```
|
||||
|
||||
2. **Or update the yarn patch** if the import comes from a dependency you don't control (like `@rocket.chat/fuselage`):
|
||||
```bash
|
||||
yarn patch react-aria@npm:3.37.0
|
||||
# Add the missing export to dist/import.mjs, dist/module.js, and dist/main.js
|
||||
yarn patch-commit -s <patch-folder>
|
||||
```
|
||||
|
||||
### Upgrading react-aria / react-stately versions
|
||||
|
||||
When upgrading these packages, the yarn patches must be re-created for the new version:
|
||||
|
||||
1. Remove the old patch references from `package.json` resolutions
|
||||
2. Delete the old `.yarn/patches/react-aria-*.patch` and `react-stately-*.patch` files
|
||||
3. Run `yarn install` to get the unpatched version
|
||||
4. Run `yarn patch react-aria@npm:<new-version>` and re-apply the slim barrel
|
||||
5. Run `yarn patch react-stately@npm:<new-version>` and re-apply the slim barrel
|
||||
6. Verify no new exports are needed by running the app and checking for runtime errors
|
||||
|
||||
### Upgrading @rocket.chat/fuselage
|
||||
|
||||
When the fuselage package is updated, check if it uses any new react-aria exports. You can verify with:
|
||||
|
||||
```bash
|
||||
grep -oE 'react_aria_1\.[a-zA-Z]+' node_modules/@rocket.chat/fuselage/dist/fuselage.development.js | sort -u
|
||||
grep -oE 'react_stately_1\.[a-zA-Z]+' node_modules/@rocket.chat/fuselage/dist/fuselage.development.js | sort -u
|
||||
```
|
||||
|
||||
If there are new exports, add them to the yarn patch.
|
||||
|
||||
### The long-term fix
|
||||
|
||||
The react-aria/react-stately yarn patches are a workaround. The proper fix is for `@rocket.chat/fuselage` to import from sub-packages directly instead of the barrel, which eliminates the need for those patches entirely. Once that is done, only the direct sub-package imports in source code are needed.
|
||||
|
||||
The zod patch is a workaround for a design choice in zod v4 (`export * as locales` in the main barrel). This may be addressed upstream — track https://github.com/colinhacks/zod for changes to the locale export strategy.
|
||||
|
||||
## How to analyze the bundle
|
||||
|
||||
Meteor generates a `.stats.json` file alongside each build. To inspect it:
|
||||
|
||||
```bash
|
||||
# Find the latest stats file
|
||||
ls -t .meteor/local/build/programs/web.browser/*.stats.json | head -1
|
||||
|
||||
# Quick summary
|
||||
python3 -c "
|
||||
import json
|
||||
with open('<stats-file>') as f:
|
||||
data = json.load(f)
|
||||
print(f'Total: {data[\"totalMinifiedBytes\"]/1024:.0f} KB')
|
||||
print(f'Gzip: {data[\"totalMinifiedGzipBytes\"]/1024:.0f} KB')
|
||||
"
|
||||
```
|
||||
|
||||
The stats file contains a per-package breakdown in `minifiedBytesByPackage`, with `packages/modules.js` containing a nested tree of every npm module included in the main bundle.
|
||||
125
docs/coverage.md
Normal file
125
docs/coverage.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# Code Coverage
|
||||
|
||||
This document explains how code coverage instrumentation works in Rocket.Chat's build and CI pipeline.
|
||||
|
||||
## Overview
|
||||
|
||||
Coverage is collected during E2E test runs (API, UI, Livechat) to measure how much of the server-side code is exercised by tests. The instrumentation uses [Istanbul](https://istanbul.js.org/)-compatible tooling, which injects counters (`__coverage__`) into the compiled code at build time.
|
||||
|
||||
## Architecture
|
||||
|
||||
The coverage pipeline has three components:
|
||||
|
||||
1. **Build-time instrumentation** - injects coverage counters into the code during Meteor build
|
||||
2. **Runtime collection** - the `rocketchat:coverage` Meteor package collects `__coverage__` data on process exit
|
||||
3. **CI reporting** - test workflows merge coverage data and upload reports
|
||||
|
||||
```
|
||||
Build (SWC + plugin) --> Run tests --> Process exit triggers report --> Merge & upload
|
||||
```
|
||||
|
||||
## Build-time instrumentation
|
||||
|
||||
### Modern build stack (SWC)
|
||||
|
||||
Rocket.Chat uses Meteor's modern build stack with [SWC](https://swc.rs/) as the transpiler. For coverage builds, the [`swc-plugin-coverage-instrument`](https://github.com/kwonoj/swc-plugin-coverage-instrument) plugin is injected into `.swcrc` at build time.
|
||||
|
||||
This is configured in `.github/actions/meteor-build/action.yml`:
|
||||
|
||||
```yaml
|
||||
env:
|
||||
BABEL_ENV: ${{ inputs.type }} # "production" or "coverage"
|
||||
```
|
||||
|
||||
When `BABEL_ENV=coverage`, the build script:
|
||||
|
||||
1. Adds `rocketchat:coverage` to `.meteor/packages`
|
||||
2. Injects `swc-plugin-coverage-instrument` into `.swcrc` via a node script:
|
||||
|
||||
```js
|
||||
const swcrc = JSON.parse(fs.readFileSync('./apps/meteor/.swcrc', 'utf8'));
|
||||
swcrc.jsc.experimental = swcrc.jsc.experimental || {};
|
||||
swcrc.jsc.experimental.plugins = swcrc.jsc.experimental.plugins || [];
|
||||
swcrc.jsc.experimental.plugins.push(['swc-plugin-coverage-instrument', {}]);
|
||||
fs.writeFileSync('./apps/meteor/.swcrc', JSON.stringify(swcrc, null, 2) + '\n');
|
||||
```
|
||||
|
||||
This approach ensures the same build pipeline (SWC) is used for both regular and coverage builds, avoiding behavioral differences between build modes.
|
||||
|
||||
### Legacy build stack (Babel)
|
||||
|
||||
Before the modern build stack, coverage was handled via `babel-plugin-istanbul` configured in `.babelrc`:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"coverage": {
|
||||
"plugins": [
|
||||
["istanbul", { "exclude": ["**/*.spec.js", "**/*.test.js"] }]
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This section is still present in `.babelrc` as a fallback for files that fall back to Babel compilation (e.g., SWC-incompatible code).
|
||||
|
||||
## Runtime collection: `rocketchat:coverage`
|
||||
|
||||
The `rocketchat:coverage` Meteor package (`apps/meteor/packages/rocketchat-coverage/`) is only added to the build during coverage runs. It:
|
||||
|
||||
1. Registers a `process.on('exit')` handler
|
||||
2. Reads `globalThis['__coverage__']` (populated by the instrumentation)
|
||||
3. Generates a coverage report using `istanbul-lib-coverage` and `istanbul-reports`
|
||||
|
||||
Configuration via environment variables:
|
||||
|
||||
| Variable | Description | Example |
|
||||
|---|---|---|
|
||||
| `COVERAGE_DIR` | Output directory for reports | `/tmp/coverage/api` |
|
||||
| `COVERAGE_FILE_NAME` | Report filename | `api-1.json` |
|
||||
| `COVERAGE_REPORTER` | Istanbul reporter format | `json`, `lcov` |
|
||||
|
||||
## CI workflow
|
||||
|
||||
Coverage is collected in the `ci-test-e2e.yml` workflow:
|
||||
|
||||
1. **Build**: `meteor-build` action runs with `type: coverage`, producing a Docker image with instrumented code
|
||||
2. **Test**: E2E tests run against the instrumented server. On each test shard:
|
||||
- `COVERAGE_DIR`, `COVERAGE_FILE_NAME`, and `COVERAGE_REPORTER` are set
|
||||
- When the Rocket.Chat process exits after tests, the coverage plugin writes a JSON report
|
||||
3. **Merge**: `nyc merge` combines per-shard JSON reports into a single coverage file
|
||||
4. **Upload**: Coverage data is uploaded to Codecov
|
||||
|
||||
## Local development
|
||||
|
||||
To build with coverage locally:
|
||||
|
||||
```bash
|
||||
cd apps/meteor
|
||||
|
||||
# Inject the SWC coverage plugin into .swcrc
|
||||
node -e "
|
||||
const fs = require('fs');
|
||||
const swcrc = JSON.parse(fs.readFileSync('.swcrc', 'utf8'));
|
||||
swcrc.jsc.experimental = { plugins: [['swc-plugin-coverage-instrument', {}]] };
|
||||
fs.writeFileSync('.swcrc', JSON.stringify(swcrc, null, 2));
|
||||
"
|
||||
|
||||
# Add the coverage package
|
||||
echo -e "rocketchat:coverage\n" >> .meteor/packages
|
||||
|
||||
# Set env vars and run
|
||||
COVERAGE_DIR=/tmp/coverage COVERAGE_FILE_NAME=local.json COVERAGE_REPORTER=lcov yarn dev
|
||||
```
|
||||
|
||||
Remember to restore `.swcrc` and `.meteor/packages` after testing.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Purpose |
|
||||
|---|---|
|
||||
| `swc-plugin-coverage-instrument` | SWC plugin for Istanbul-compatible instrumentation |
|
||||
| `istanbul-lib-coverage` | Coverage map creation (used by `rocketchat:coverage`) |
|
||||
| `istanbul-lib-report` | Report context creation |
|
||||
| `istanbul-reports` | Report formatters (json, lcov, etc.) |
|
||||
215
docs/experimental-api-endpoints-plan.md
Normal file
215
docs/experimental-api-endpoints-plan.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Plan: Experimental REST API endpoints
|
||||
|
||||
## Goal
|
||||
|
||||
Allow REST endpoints to ship to production but **change or be removed in any release
|
||||
without a major-version bump**. The mechanism must be general-purpose (any team can
|
||||
use it), not specific to one feature.
|
||||
|
||||
## The contract
|
||||
|
||||
> Endpoints under `/api/experimental/...` are unstable. They may change shape or be
|
||||
> removed in any release, without notice and without a deprecation cycle. No semver
|
||||
> promise attaches to this namespace.
|
||||
|
||||
The namespace **is** the contract. A caller hitting `/api/experimental/*` has opted
|
||||
into instability by the URL alone. `/v1` keeps its implicit semver-stability promise,
|
||||
untouched.
|
||||
|
||||
**Only the new typed API is allowed on experimental routes.** Endpoints must be
|
||||
registered with `.get()` / `.post()` / `.put()` / `.delete()` (with AJV `body` / `query`
|
||||
/ `response` validators). The deprecated `.addRoute()` must not be used — new surface
|
||||
area should not be born on the legacy registration path. This is a documented rule, not a
|
||||
compiler-enforced one: `API.experimental` is a plain `APIClass`, and `.addRoute()` already
|
||||
carries `@deprecated` everywhere it is reachable.
|
||||
|
||||
## Why this design (key findings from the current code)
|
||||
|
||||
- `createApi({ version })` turns the `version` string into the URL path segment, so a
|
||||
new instance with `version: 'experimental'` mounts at `/api/experimental/<name>`
|
||||
without changing any router internals — the new router still has to be mounted in
|
||||
`startRestAPI` (see Step 2). See `apps/meteor/server/api/api.ts` (the `API` object and
|
||||
`createApi`) and `apps/meteor/server/api/ApiClass.ts` (`apiPath` composition in the
|
||||
`APIClass` constructor).
|
||||
- The typed route methods `.get()/.post()/.put()/.delete()` are generic over
|
||||
`TSubPathPattern extends string` (`ApiClass.ts`, `APIClass.method()` and its
|
||||
per-verb wrappers) — they are **not** gated on `keyof Endpoints`. Routes accumulate
|
||||
*outward* into the instance's `TOperations`, read back by `ExtractApiClassEndpoints`
|
||||
(`apps/meteor/server/api/api.ts`). So a separate experimental instance works *with*
|
||||
the type system.
|
||||
- `PathPattern`, `Method`, `Path`, and the typed client are all derived from the
|
||||
`Endpoints` interface (`packages/rest-typings/src/index.ts`). Keeping
|
||||
experimental paths **out** of that interface keeps the stable client surface clean
|
||||
and forces explicit opt-in for experimental ones.
|
||||
- The deprecation framework already writes `x-deprecation-*` response headers
|
||||
(`writeDeprecationHeader` in `apps/meteor/server/lib/deprecationWarningLogger.ts`). We
|
||||
mirror that pattern for an `x-experimental` / `Warning` signal.
|
||||
- Auth, permissions, rate limiting, CORS, AJV validation, and metrics all come from
|
||||
`createApi` + the middleware chain in `startRestAPI` — experimental endpoints get
|
||||
them for free.
|
||||
|
||||
---
|
||||
|
||||
## Commit constraint
|
||||
|
||||
**Each implementation step below ships as exactly one commit.** No step is split across
|
||||
multiple commits, and no commit spans more than one step. This keeps the history
|
||||
bisectable, makes each phase independently reviewable and revertable, and maps the PR
|
||||
review 1:1 onto the plan.
|
||||
|
||||
- A step's commit must leave the tree in a compiling, lint-clean state (`yarn lint
|
||||
--quiet` passes) — partial work is squashed before committing.
|
||||
- Commit message subject names the step, e.g. `feat(api): add experimental API instance
|
||||
(step 1)`.
|
||||
- If a step turns out to require a prerequisite not in the plan, add it to that step's
|
||||
single commit rather than introducing an out-of-band commit.
|
||||
|
||||
## Implementation steps
|
||||
|
||||
### Step 1 — Add the `experimental` API instance
|
||||
|
||||
**File:** `apps/meteor/server/api/api.ts`
|
||||
|
||||
1. In the `API` object literal, add:
|
||||
```ts
|
||||
experimental: createApi({ version: 'experimental', useDefaultAuth: true }),
|
||||
```
|
||||
Place it between `v1` and `default`.
|
||||
2. Add an `experimental` entry to the `API` type annotation so it is typed:
|
||||
`APIClass<'/experimental'>`. Hiding `addRoute()` from that type was considered and
|
||||
dropped: the typed methods return `this`, so a restricted surface only holds until the
|
||||
first chained registration, and closing that hole means duplicating every typed
|
||||
signature and widening the route-extraction types that pattern-match `APIClass`. Not
|
||||
worth it for a rule `@deprecated` already signals.
|
||||
3. Refreshing experimental routes when settings change is a **required** parity
|
||||
condition, not an optional extra — the contract above promises experimental
|
||||
endpoints get rate limiting "for free", which only holds if the refresh callbacks
|
||||
cover them. The `settings.watch(...)` callbacks in this file that must also update
|
||||
`API.experimental`:
|
||||
- `API_Enable_Rate_Limiter_Limit_Time_Default` → `reloadRoutesToRefreshRateLimiter()`
|
||||
- `API_Enable_Rate_Limiter_Limit_Calls_Default` → `reloadRoutesToRefreshRateLimiter()`
|
||||
- `Accounts_CustomFields` → `setLimitedCustomFields()`
|
||||
|
||||
**Known gap:** the rate-limiter watchers are at parity; the `Accounts_CustomFields`
|
||||
watcher still updates `API.v1` only. That is currently harmless — no experimental
|
||||
endpoint returns user objects — but it must be closed before one does.
|
||||
|
||||
**Acceptance:** `API.experimental.get('ping', { ... }, handler)` compiles and serves at
|
||||
`GET /api/experimental/ping`.
|
||||
|
||||
**Commit (1 of 5):** `feat(api): add experimental API instance`
|
||||
|
||||
### Step 2 — Mount it in the request pipeline
|
||||
|
||||
**File:** `apps/meteor/server/api/api.ts`, `startRestAPI`
|
||||
|
||||
1. Insert `.use(API.experimental.router)` into the chain, **before**
|
||||
`.use(API.default.router)`. Order matters: `default` is the catch-all.
|
||||
2. Add a second `metricsMiddleware` block pointed at `API.experimental` so experimental
|
||||
traffic is measured. Metrics are the canary used later to decide whether an endpoint is
|
||||
ready for promotion to `/v1`. Because every block shares the same `/api` mount, each one
|
||||
needs a guard or a request is sampled more than once: the versioned blocks opt in via
|
||||
`basePathRegex`, and a catch-all block for `API.default` (`/api/info`, `/api/docs/json`,
|
||||
unmatched `/api/*`) opts out via `excludePathRegex`. Without that catch-all block the
|
||||
guards silently drop default-router traffic that used to be sampled.
|
||||
|
||||
**Acceptance:** experimental requests appear in the REST API Prometheus metrics labelled
|
||||
`version=experimental` — specifically that label, not merely a distinguishable one. A
|
||||
change that only adjusted the path regex while leaving experimental traffic under the
|
||||
`v1` version label does not satisfy this. `/api/v1/*` and default-router traffic each
|
||||
still record exactly one sample under their own label.
|
||||
|
||||
**Commit (2 of 5):** `feat(api): mount experimental router and metrics`
|
||||
|
||||
### Step 3 — Runtime "unstable" signal (mirror deprecation headers)
|
||||
|
||||
**New file:** `apps/meteor/server/api/v1/middlewares/experimental.ts`, colocated with the
|
||||
existing middlewares.
|
||||
|
||||
1. Write a middleware that sets, on every response from the experimental instance:
|
||||
```
|
||||
Warning: 299 - "experimental: endpoint is unstable and may change without notice"
|
||||
x-experimental: true
|
||||
```
|
||||
`x-experimental: true` is the **supported programmatic signal** — clients should detect
|
||||
experimental responses with it. `Warning: 299` is a legacy compatibility signal only:
|
||||
warn code 299 came from RFC 7234, which RFC 9111 has since obsoleted along with the
|
||||
`Warning` header itself, so modern clients are not expected to generate or interpret
|
||||
it. It is emitted for the benefit of tooling that still surfaces it, and may be dropped
|
||||
without it being a breaking change. Model the header-writing on `writeDeprecationHeader`
|
||||
in `apps/meteor/server/lib/deprecationWarningLogger.ts`.
|
||||
2. Register the middleware on the shared `/api` mount in `startRestAPI`, **ahead of**
|
||||
`cors`, scoped to `/api/experimental` by a `basePathRegex` (same shape as the metrics
|
||||
middleware guard). It cannot live on `API.experimental.router`: `cors` answers rejected
|
||||
preflights with 403/405 without calling `next()`, so a router-scoped middleware would
|
||||
never run for those responses.
|
||||
|
||||
**Acceptance:** every `/api/experimental/*` response carries both headers — including 404s
|
||||
and CORS preflight rejections; `/api/v1/*` responses do not.
|
||||
|
||||
**Commit (3 of 5):** `feat(api): add experimental unstable-signal middleware`
|
||||
|
||||
### Step 4 — Separate, opt-in SDK typings
|
||||
|
||||
**File:** `packages/rest-typings/src/index.ts` (+ a new file for the declarations)
|
||||
|
||||
1. Create `packages/rest-typings/src/experimental/index.ts` (new folder) and declare:
|
||||
```ts
|
||||
export type ExperimentalEndpoints = {
|
||||
'/experimental/<name>': {
|
||||
GET: (params: ...) => ...;
|
||||
};
|
||||
// ...
|
||||
};
|
||||
```
|
||||
Follow the existing per-resource endpoint style (e.g.
|
||||
`packages/rest-typings/src/v1/channels/channels.ts`).
|
||||
2. Export `ExperimentalEndpoints` from the package root, but **do NOT** add it to the
|
||||
`interface Endpoints extends ...` union. This keeps `PathPattern`,
|
||||
`Method`, `Path`, and the stable typed client free of experimental paths.
|
||||
3. Consumers who want typed experimental calls import `ExperimentalEndpoints`
|
||||
explicitly.
|
||||
|
||||
**Acceptance:** `import type { Endpoints } from '@rocket.chat/rest-typings'` does NOT
|
||||
include experimental paths; `import type { ExperimentalEndpoints }` does.
|
||||
|
||||
**Commit (4 of 5):** `feat(rest-typings): add opt-in ExperimentalEndpoints`
|
||||
|
||||
### Step 5 — Guardrails (because it is a general mechanism)
|
||||
|
||||
1. **No path in both unions.** A type-level CI guard for this was considered and dropped:
|
||||
union keys are full paths, so `/experimental/x` and `/v1/x` never collide, and the
|
||||
transition window described below does not produce a collision either. Keep the rule in
|
||||
the docs instead — promotion means *moving* the declaration to a stable `*Endpoints`
|
||||
type, not leaving a copy behind.
|
||||
2. **Promotion path:** document that stabilizing an endpoint means copying it to `/v1`
|
||||
(optionally keeping the experimental path forwarding for a transition window).
|
||||
Removal needs no deprecation cycle — but log removals for courtesy.
|
||||
3. **Docs/CONTRIBUTING note:** state the no-semver guarantee and how to add an
|
||||
experimental endpoint, so the mechanism is discoverable.
|
||||
4. **OpenAPI/doc generation:** decide deliberately whether generated API docs scan only
|
||||
`Endpoints` (experimental endpoints hidden — probably desirable) or also
|
||||
`ExperimentalEndpoints`.
|
||||
|
||||
**Commit (5 of 5):** `chore(api): add experimental guardrails and docs`
|
||||
|
||||
---
|
||||
|
||||
## Test checklist
|
||||
|
||||
- [ ] `GET /api/experimental/<name>` resolves and returns the `x-experimental` + `Warning` headers.
|
||||
- [ ] `/api/v1/*` responses are unchanged (no experimental headers).
|
||||
- [ ] Auth / permissions / rate limiting enforced on an experimental route exactly as on `/v1`.
|
||||
- [ ] `Endpoints` type does not include experimental paths; `ExperimentalEndpoints` does.
|
||||
- [ ] Experimental requests show up in REST API metrics.
|
||||
|
||||
## Files touched (summary)
|
||||
|
||||
| File | Change |
|
||||
| ---- | ------ |
|
||||
| `apps/meteor/server/api/api.ts` | Add `experimental` instance, type entry, mount in `startRestAPI`, metrics blocks |
|
||||
| `apps/meteor/server/api/v1/middlewares/experimental.ts` (new) | `x-experimental` / `Warning` header middleware |
|
||||
| `apps/meteor/server/api/v1/middlewares/metrics.ts` | `basePathRegex` / `excludePathRegex` sampling guards |
|
||||
| `packages/rest-typings/src/experimental/index.ts` (new) | `ExperimentalEndpoints` type, NOT merged into `Endpoints` |
|
||||
| `packages/rest-typings/src/index.ts` | Export `ExperimentalEndpoints` |
|
||||
| docs / CONTRIBUTING | Document the contract + promotion path |
|
||||
116
docs/experimental-api-endpoints.md
Normal file
116
docs/experimental-api-endpoints.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Experimental REST API endpoints
|
||||
|
||||
> Developer guide. For how the mechanism is built, see
|
||||
> [experimental-api-endpoints-plan.md](experimental-api-endpoints-plan.md).
|
||||
|
||||
## What they are
|
||||
|
||||
Experimental endpoints live under `/api/experimental/...` and carry an explicit
|
||||
stability contract:
|
||||
|
||||
> Endpoints under `/api/experimental/...` are **unstable**. They may change shape or be
|
||||
> removed in **any** release — without notice and without a deprecation cycle. No semver
|
||||
> promise attaches to this namespace.
|
||||
|
||||
Compare with `/api/v1/...`, which is the official, stable surface: its endpoints follow
|
||||
semver, breaking changes require a major-version bump, and removals go through a
|
||||
deprecation cycle.
|
||||
|
||||
## Why they exist
|
||||
|
||||
We sometimes need to ship something to production *before* its API shape has settled:
|
||||
|
||||
- A new feature whose request/response contract is still being learned from real usage.
|
||||
- An endpoint built for a specific client (e.g. our own UI) where we are not yet ready
|
||||
to commit to it as a public, supported interface.
|
||||
- Something we want behind a clear "use at your own risk" sign while it matures.
|
||||
|
||||
Without an experimental lane, the only choices are bad ones: either freeze a design we
|
||||
are not confident in onto `/v1` (and then carry it forever, or break it with a major
|
||||
bump), or keep the feature out of production until the API is perfect. Experimental
|
||||
endpoints give a third path — ship now, iterate freely, commit later.
|
||||
|
||||
## Should I use one? — decision guide
|
||||
|
||||
**Use an experimental endpoint when:**
|
||||
|
||||
- The request/response shape is likely to change as the feature matures.
|
||||
- You want production traffic / real feedback before committing to a contract.
|
||||
- The consumer is internal or opted-in, and can tolerate breaking changes between
|
||||
releases.
|
||||
- You would otherwise be tempted to "just put it on `/v1` for now and fix it later."
|
||||
|
||||
**Do NOT use an experimental endpoint when:**
|
||||
|
||||
- The endpoint is meant for third-party integrators who expect stability. They should
|
||||
not have to track breakage release-to-release.
|
||||
- The contract is already well understood and unlikely to change — put it on `/v1`.
|
||||
- You are tempted to use `experimental` as a permanent home to avoid the discipline of
|
||||
a stable API. It is a staging area, not a dumping ground (see below).
|
||||
|
||||
## Expectations if you publish one
|
||||
|
||||
- **It is not forever.** Every experimental endpoint is expected to either be
|
||||
**elevated to `/v1`** once its contract stabilizes, or be **removed** if it does not
|
||||
pan out. An endpoint that sits in `experimental` indefinitely is a smell — it means a
|
||||
decision is overdue.
|
||||
- **Callers are warned at runtime.** Every experimental response carries
|
||||
`x-experimental: true` — that is the supported signal to detect and surface in client
|
||||
code. Responses also carry a `Warning: 299 ...` header, kept only for legacy tooling
|
||||
that still reads it: RFC 9111 obsoletes the `Warning` header and its warn codes, so do
|
||||
not build new client logic on it.
|
||||
- **Typed clients must opt in.** Experimental endpoints are declared in a separate
|
||||
`ExperimentalEndpoints` type, not in the main `Endpoints` union, so the stable SDK
|
||||
surface stays honest. Consumers import them deliberately.
|
||||
- **Use the typed API only.** Register with `.get()` / `.post()` / `.put()` /
|
||||
`.delete()` and AJV validators. Do not use `.addRoute()`: it is already deprecated
|
||||
across the whole API, and a namespace created to iterate on new contracts is the last
|
||||
place that should add to the legacy path.
|
||||
|
||||
## Lifecycle: experimental → official
|
||||
|
||||
```text
|
||||
stabilizes
|
||||
experimental ───────────────▶ v1 (official, semver-stable)
|
||||
(/api/experimental/x) (/api/v1/x)
|
||||
│
|
||||
│ does not pan out
|
||||
▼
|
||||
removed (no deprecation cycle needed)
|
||||
```
|
||||
|
||||
**Elevating to `/v1`:**
|
||||
|
||||
1. Confirm the contract is stable and you are ready to support it under semver.
|
||||
2. Add the endpoint under `/v1`: register it on `API.v1` and declare its types in the
|
||||
appropriate `*Endpoints` type that *is* part of the `Endpoints` union.
|
||||
3. Optionally keep the experimental path forwarding to the new `/v1` path for a
|
||||
transition window so existing callers are not broken on the day of promotion.
|
||||
4. Remove the experimental declaration once the transition window closes.
|
||||
|
||||
**Removing an experimental endpoint** needs no deprecation cycle — that freedom is the
|
||||
whole point of the namespace. Still, log the removal and give a heads-up to any known
|
||||
consumers as a courtesy.
|
||||
|
||||
## Guardrails & tooling
|
||||
|
||||
- **No path lives in both unions.** A duplicate key would silently attach a semver
|
||||
obligation to a path advertised as unstable, so promotion means *moving* the declaration
|
||||
to a stable `*Endpoints` type, not leaving a copy behind. Nothing enforces this — the
|
||||
`/experimental/` path prefix keeps the two unions from overlapping in practice.
|
||||
- **Generated API docs intentionally skip experimental endpoints.** OpenAPI /
|
||||
doc generation scans the `Endpoints` union, which experimental paths are
|
||||
deliberately kept out of, so they do not appear in public API docs. This is
|
||||
by design: an unstable surface should not be advertised as part of the
|
||||
documented contract. The runtime `x-experimental` / `Warning` headers and
|
||||
this guide are how the namespace is surfaced instead.
|
||||
- **Metrics are the promotion signal.** Experimental traffic is recorded in the
|
||||
REST API Prometheus metrics under `version=experimental`, so real usage can
|
||||
inform whether an endpoint is ready to graduate to `/v1` or should be removed.
|
||||
|
||||
## TL;DR
|
||||
|
||||
Experimental endpoints let you ship an API to production while its shape is still in
|
||||
flux, without locking yourself into semver. They are a **staging area, not a permanent
|
||||
home**: every one is expected to graduate to `/v1` or be removed. If you need stability
|
||||
guarantees, use `/v1`. If a third party will depend on it, use `/v1`.
|
||||
116
docs/features/ban-user.md
Normal file
116
docs/features/ban-user.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Ban User
|
||||
|
||||
## Overview
|
||||
|
||||
Banning prevents a user from participating in a specific room. Unlike kicking (which deletes the subscription), banning **keeps the subscription record** with `status: 'BANNED'`, creating a persistent access barrier.
|
||||
|
||||
## Ban Flow
|
||||
|
||||
1. A user with `ban-user` permission (roles: `admin`, `owner`, `moderator`) triggers the ban via UI, API (`POST /v1/rooms.banUser`), or slash command (`/ban @username`).
|
||||
2. **Validations** (`banUserFromRoomMethod` in `server/lib/banUserFromRoom.ts`):
|
||||
- Checks `ban-user` permission scoped to the room.
|
||||
- Checks if the room type allows the action (via `roomDirectives.allowMemberAction`).
|
||||
- Checks if the banning user has access to the room.
|
||||
- Checks if the target user exists and is in the room.
|
||||
- Rejects ban if the target is already banned.
|
||||
- Rejects ban if the target is the last owner.
|
||||
3. **Execution** (`performUserBan` in `app/lib/server/functions/banUserFromRoom.ts`):
|
||||
- Updates the subscription to `status: 'BANNED'` (does not delete the record).
|
||||
- Removes the room from the user's `__rooms` array.
|
||||
- Decrements the room's `usersCount`.
|
||||
- Removes room-scoped roles (`moderator`, `owner`, `leader`) in channels and groups.
|
||||
- If the room is a team's main room, removes the member from the team.
|
||||
- Saves a `user-banned` system message.
|
||||
- Notifies the client with a `removed` event on the subscription (so the client drops the stream).
|
||||
4. **Callback** `afterBanFromRoom` fires (used by Matrix federation to propagate the ban).
|
||||
|
||||
## Unban Flow
|
||||
|
||||
1. Triggered via UI (contextual bar "Banned Users"), API (`POST /v1/rooms.unbanUser`), or slash command (`/unban @username`).
|
||||
2. Finds the subscription via `findOneBannedSubscription`.
|
||||
3. **Removes the subscription entirely** (`Subscriptions.removeById`) — does not restore it to active status.
|
||||
4. Saves a `user-unbanned` system message.
|
||||
5. **Callback** `afterUnbanFromRoom` fires (federation).
|
||||
|
||||
**Important:** after unban the user **does not become a member** of the room again. The banned subscription is deleted. The user must be invited or join again.
|
||||
|
||||
## Join / Invite / Re-entry Behavior
|
||||
|
||||
A banned user **cannot** re-enter the room through any path. The ban must be explicitly lifted first. Below is how each entry point enforces this for both normal and federated rooms.
|
||||
|
||||
### Invite via API / UI (`groups.invite`, `channels.invite`, "Add Users")
|
||||
|
||||
`addUsersToRoom` checks for a `BANNED` subscription before calling `addUserToRoom`:
|
||||
- Returns `error-user-is-banned` — the invite is rejected.
|
||||
- The UI shows a warning modal asking the admin to unban first.
|
||||
- Applies equally to normal and federated rooms (the check is in the method layer, before the room-type branch).
|
||||
|
||||
### Invite link (`useInviteToken`)
|
||||
|
||||
`useInviteToken` checks for a `BANNED` subscription before saving the invite token or calling `addUserToRoom`:
|
||||
- Returns `error-user-is-banned` — the token is not consumed.
|
||||
- Because the check runs before `Users.updateInviteToken`, the secondary path through `setUsername` (for users who register via invite link) is also blocked.
|
||||
|
||||
### Direct join (`channels.join`, `joinRoom`)
|
||||
|
||||
`Room.join` calls `canAccessRoom` before `addUserToRoom`:
|
||||
- For **public rooms** and **public rooms inside teams**, the `canAccessRoom` validators explicitly check `findOneBannedSubscription` and deny access.
|
||||
- For **private rooms**, `countByRoomIdAndUserId` excludes `BANNED` subscriptions (`status: { $exists: false }`), so the "already joined" validator returns false and access is denied.
|
||||
|
||||
### Federation invite events
|
||||
|
||||
When a Matrix homeserver sends an invite for a user who is banned locally:
|
||||
- `handleInvite` in `federation-matrix/src/events/member.ts` finds the existing (banned) subscription and returns early without creating a new one.
|
||||
- The user never receives an `INVITED` subscription, so `handleJoin` is never reached.
|
||||
|
||||
### Expected flow
|
||||
|
||||
1. **Unban** the user via `POST /v1/rooms.unbanUser`, `/unban @username`, or the "Banned Users" contextual bar. This deletes the banned subscription.
|
||||
2. **Invite or join** — the user can now be invited (API, UI, invite link) or join (public rooms) normally.
|
||||
|
||||
## Access Control
|
||||
|
||||
The `canAccessRoom` validators check for bans in two public room scenarios:
|
||||
- **Public rooms inside teams** — if banned, access is denied.
|
||||
- **Regular public rooms** — if banned, access is denied.
|
||||
|
||||
For private rooms, access is controlled by the subscription: `countByRoomIdAndUserId` excludes `BANNED` subscriptions, so a banned user has no valid subscription and cannot access the room.
|
||||
|
||||
## UI
|
||||
|
||||
- **Ban action:** appears in the user info panel (inside a room), gated by `ban-user` permission + `roomCanBan` + federation rules.
|
||||
- **Banned users list:** "Banned Users" tab in the room toolbox (icon: `ban`, order: 13, requires `ban-user`), with virtualized scroll and infinite pagination via `GET /v1/rooms.bannedUsers`.
|
||||
- **Unban action:** context menu on each item in the banned users list.
|
||||
- **Confirmation:** both actions show a `GenericModal` with `danger` variant.
|
||||
|
||||
## System Messages
|
||||
|
||||
| Key | When |
|
||||
|-----|------|
|
||||
| `user-banned` | A user is banned from the room |
|
||||
| `user-unbanned` | A user is unbanned (including via re-addition) |
|
||||
|
||||
## REST Endpoints
|
||||
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| POST | `/v1/rooms.banUser` | Ban a user (accepts `userId` or `username` + `roomId`) |
|
||||
| POST | `/v1/rooms.unbanUser` | Unban a user |
|
||||
| GET | `/v1/rooms.bannedUsers` | List banned users (paginated) |
|
||||
|
||||
## Key Files
|
||||
|
||||
| Layer | File |
|
||||
|-------|------|
|
||||
| API routes | `app/api/server/v1/rooms.ts` |
|
||||
| Validation & permissions | `server/lib/banUserFromRoom.ts` |
|
||||
| Core ban logic | `app/lib/server/functions/banUserFromRoom.ts` |
|
||||
| Core unban logic | `app/lib/server/functions/executeUnbanUserFromRoom.ts` |
|
||||
| Slash commands | `app/slashcommands-ban/server/ban.ts`, `unban.ts` |
|
||||
| Client ban hook | `client/views/room/hooks/useBanUser.tsx` |
|
||||
| Client unban hook | `client/views/room/hooks/useUnbanUser.tsx` |
|
||||
| Ban action (user info) | `client/views/room/hooks/useUserInfoActions/actions/useBanUserAction.tsx` |
|
||||
| Banned users UI | `client/views/room/contextualBar/BannedUsers/` |
|
||||
| Subscription types | `packages/core-typings/src/ISubscription.ts` |
|
||||
| REST typings | `packages/rest-typings/src/v1/rooms.ts` |
|
||||
| Model typings | `packages/model-typings/src/models/ISubscriptionsModel.ts` |
|
||||
243
docs/features/custom-sidebar-categories.md
Normal file
243
docs/features/custom-sidebar-categories.md
Normal file
@@ -0,0 +1,243 @@
|
||||
# Custom Sidebar Categories
|
||||
|
||||
## Overview
|
||||
|
||||
Custom categories let each user group their sidebar rooms into personal, named collections that appear
|
||||
alongside the system groups (Favorites, Teams, Channels, Direct messages, …). A user can create, rename,
|
||||
delete and reorder categories; move rooms in and out of them via context menus; and toggle whether each
|
||||
category keeps showing unread rooms while collapsed.
|
||||
|
||||
Categories are **per-user** and **client-only in meaning** — they are a presentation grouping of the user's
|
||||
own subscriptions. They carry **no access, security or membership semantics**: moving a room into a category
|
||||
never changes who can see the room or the room itself. Assignment is **exclusive** — a room lives in exactly
|
||||
one place at a time (one custom category, *or* Favorites, *or* its system group).
|
||||
|
||||
Requires the `experimental-enterprise-features` license module.
|
||||
|
||||
## Data Model
|
||||
|
||||
### Category record — `ISidebarCategory`
|
||||
|
||||
```ts
|
||||
// packages/core-typings/src/IUser.ts
|
||||
export interface ISidebarCategory {
|
||||
_id: string; // Random.id() for custom; system group key for defaults
|
||||
name: string; // display name (trimmed; max 30 chars)
|
||||
default?: boolean; // true for system-group entries managed by the platform
|
||||
showUnreads?: boolean; // when collapsed, keep listing unread rooms (default: false)
|
||||
keepUnreadsOnTop?: boolean; // when expanded, sort unread rooms to the top (default: false)
|
||||
}
|
||||
```
|
||||
|
||||
### Room assignment
|
||||
|
||||
Room-to-category assignment is stored on the **subscription**, not on the category:
|
||||
|
||||
```ts
|
||||
// Subscription.category: string | null
|
||||
```
|
||||
|
||||
Set via `POST /experimental/rooms.setCategory`. The subscription field is the source of truth for which
|
||||
category a room belongs to; the category record only carries display metadata and ordering.
|
||||
|
||||
## Persistence
|
||||
|
||||
### `sidebarCategories` preference (unified group ordering + category metadata)
|
||||
|
||||
The preference `sidebarCategories: ISidebarCategory[]` is the **single source of truth** for both the
|
||||
ordered list of all visible groups and the metadata of custom categories. It contains entries for custom
|
||||
categories *and* for any system group whose metadata has been touched (show-unreads, keep-unreads-on-top,
|
||||
or position).
|
||||
|
||||
**Invariant enforced on every write:** all five dynamic groups (`Incoming_Calls`, `Incoming_Livechats`,
|
||||
`Open_Livechats`, `On_Hold_Chats`, `Unread`) are always stored first, in the order prescribed by
|
||||
`sidebarSectionsOrder`. Custom and static system groups follow. This is enforced by `withDynamicFirst`
|
||||
in `useCategoryList.ts`, called by every write hook.
|
||||
|
||||
All mutations go through `usePersistCategoriesMutation`, which calls:
|
||||
|
||||
```http
|
||||
POST /v1/users.setPreferences { data: { sidebarCategories: ISidebarCategory[] } }
|
||||
```
|
||||
|
||||
### Room assignment — `POST /experimental/rooms.setCategory`
|
||||
|
||||
```http
|
||||
POST /experimental/rooms.setCategory
|
||||
{ roomIds: string[], category: string | null }
|
||||
```
|
||||
|
||||
- Requires `experimental-enterprise-features` license.
|
||||
- Rejects system group keys (`SIDEBAR_SYSTEM_GROUP_KEYS`) as the target category.
|
||||
- Verifies the target category exists in the user's `sidebarCategories` preference before writing.
|
||||
- Writes to `subscription.category` and fires change notifications.
|
||||
- `category: null` unassigns the room, returning it to its system group.
|
||||
|
||||
## Group Ordering
|
||||
|
||||
### `sidebarSectionsOrder` preference
|
||||
|
||||
An **admin-configurable** ordered list of system group keys that defines the default group order when
|
||||
`sidebarCategories` has no explicit entry for a key. Falls back to `SIDEBAR_SYSTEM_GROUP_KEYS`.
|
||||
|
||||
### `mergeWithSectionsOrder(explicitIds, sectionsOrder)` — READ side
|
||||
|
||||
Used in `useCategoryList` when building the visible category list. Inserts any key missing from
|
||||
`explicitIds` before its first successor that is already present, maintaining the `sectionsOrder`
|
||||
relative ordering for system groups.
|
||||
|
||||
### `withDynamicFirst(ids, sectionsOrder)` — WRITE side
|
||||
|
||||
Called by every write hook before persisting `sidebarCategories`. Guarantees:
|
||||
1. All five dynamic group keys appear first (in `sectionsOrder` canonical order), regardless of whether
|
||||
they are currently visible.
|
||||
2. Static ids (custom + system) follow, merged with `sectionsOrder` via `mergeWithSectionsOrder`.
|
||||
|
||||
This makes moving logic simple: swapping two adjacent static groups is always a valid operation — dynamic
|
||||
groups can never appear below a static group in the stored list, so a DOWN move can never cross one.
|
||||
|
||||
### `SIDEBAR_DYNAMIC_GROUP_KEYS`
|
||||
|
||||
```ts
|
||||
export const SIDEBAR_DYNAMIC_GROUP_KEYS: readonly string[] = [
|
||||
'Incoming_Calls', 'Incoming_Livechats', 'Open_Livechats', 'On_Hold_Chats', 'Unread',
|
||||
];
|
||||
```
|
||||
|
||||
Groups that are hidden from the sidebar when empty. They are excluded from user reordering
|
||||
(`canMoveGroup` returns `false` for them) and are always stored first by `withDynamicFirst`.
|
||||
|
||||
## Behaviors / Flows
|
||||
|
||||
### Create — `useCreateCustomCategory`
|
||||
|
||||
1. Generates `_id` with `Random.id()`.
|
||||
2. Inserts the new entry at the first position after the last dynamic group entry in the stored list
|
||||
(using `findLastIndex`), so it appears first among static groups.
|
||||
3. Optionally assigns rooms via `useSetCategory` in the same call (`Create and move` flow).
|
||||
|
||||
### Move room — `useMoveRoomCategory`
|
||||
|
||||
Accepts a `target` (category `_id`, `'favorites'` sentinel, or `undefined` to remove).
|
||||
- `'favorites'`: calls `POST /v1/rooms.favorite` and removes the category assignment.
|
||||
- Custom category `_id`: calls `POST /experimental/rooms.setCategory`.
|
||||
- `undefined` / same as current: removes assignment (unfavorites or sets category to `null`).
|
||||
- Dispatches a toast unless `silent: true` or the room has no name.
|
||||
|
||||
### Move category position — `useMoveCategoryPosition`
|
||||
|
||||
Takes `currentKeys` (visible group keys in render order), a `key` to move, and a direction.
|
||||
Performs an adjacent swap in `currentKeys`, then calls `withDynamicFirst` on the result before
|
||||
persisting — so dynamic groups are always re-sorted to the front regardless of what the swap produced.
|
||||
|
||||
### Toggle show-unreads / keep-unreads-on-top — `useToggleUnreads`
|
||||
|
||||
`upsertGroupEntry(id, patch)` — updates the matching `ISidebarCategory` entry in `sidebarCategories`
|
||||
(or creates a new one if the entry doesn't exist yet). Always calls `withDynamicFirst` before persisting.
|
||||
|
||||
### Delete — `useDeleteCategory`
|
||||
|
||||
Removes the category entry from `sidebarCategories` and calls `POST /experimental/rooms.setCategory`
|
||||
with `category: null` for all rooms that were in it — returning them to their system group.
|
||||
|
||||
### Name validation — `useValidateCategoryName`
|
||||
|
||||
Returns an i18n error string or `undefined`. Checks in order:
|
||||
1. Empty / whitespace → `Please_enter_a_category_name`
|
||||
2. Longer than `MAX_CATEGORY_NAME_LENGTH` (30) → `Category_name_is_too_long__max__maxLength__characters`
|
||||
3. Matches a translated system group name (case-insensitive) → `Category_name_conflicts_with_system_group`
|
||||
4. Duplicate among existing custom categories (case-insensitive, excluding `excludeId`) → `A_category_with_this_name_already_exists`
|
||||
|
||||
### Favorites interaction
|
||||
|
||||
`FAVORITES_TARGET = 'favorites'` is a sentinel for the Favorites system group. Moving a room to Favorites
|
||||
calls `POST /v1/rooms.favorite` and strips the category assignment. Moving a room from Favorites into a
|
||||
category unfavorites it. Assignment is exclusive.
|
||||
|
||||
### Visibility in the room list
|
||||
|
||||
`filterGroupVisibility` (in `useCategoryList.ts`) applies when building the room list:
|
||||
- **Dynamic groups** — only shown when non-empty.
|
||||
- **All other groups** (custom + static system) — always shown when EE is active, even if empty.
|
||||
|
||||
This means custom categories are always present in the visible group list with EE, so no category entry
|
||||
in `sidebarCategories` can be silently dropped during a move.
|
||||
|
||||
## Sidebar UI
|
||||
|
||||
### Category collapser kebab — `CategoryMenu.tsx`
|
||||
|
||||
**Custom category:**
|
||||
- Order: Move up / Move down / New channel
|
||||
- Manage: Rename / Delete / New category
|
||||
- When collapsed: Show unreads toggle
|
||||
- When expanded: Keep unreads on top toggle
|
||||
|
||||
**System group:**
|
||||
- Order: Move up / Move down (dynamic groups have both disabled)
|
||||
- When collapsed: Show unreads toggle
|
||||
- When expanded: Keep unreads on top toggle
|
||||
|
||||
### Room context menu — `CategoryRoomMenu.tsx`
|
||||
|
||||
For rooms inside a custom category, adds a "Remove from {category}" item.
|
||||
|
||||
### Room kebab — `useCategoryMenuItems.ts`
|
||||
|
||||
Builds the "Move to" submenu: Favorites + each custom category with a checkmark on the current one.
|
||||
Appended with a "New category" item and "Remove from {category}" when applicable.
|
||||
|
||||
### Room-header grouping menu — `RoomHeaderCategoryMenu.tsx`
|
||||
|
||||
Placed before the room title in `RoomHeader`. Icon: `star-filled` (favorited) / `folder` (custom category) / `star` (default). Same item list as the room kebab submenu.
|
||||
|
||||
### Modals — `categories/`
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `CreateCategoryModal.tsx` | Create or "create and move" — single name field, max 30 chars. |
|
||||
| `ManageCategoryModal.tsx` | Rename with duplicate/empty validation. |
|
||||
| `DeleteCategoryModal.tsx` | Confirmation + "rooms will return to default groups" warning. |
|
||||
| `hooks/useCategoryModals.tsx` | Single entry point: `openCreate(room?)` / `openManage(category)` / `openDelete(category)`. |
|
||||
|
||||
### Navbar create menu
|
||||
|
||||
`useCreateNewMenu.ts` adds a "Category" item (icon: `folder`) in the navbar `+` menu.
|
||||
|
||||
## REST Contract
|
||||
|
||||
| Method | Endpoint | Purpose |
|
||||
|--------|----------|---------|
|
||||
| POST | `/v1/users.setPreferences` | Persist `sidebarCategories` (ordering + metadata) |
|
||||
| POST | `/experimental/rooms.setCategory` | Assign / unassign rooms to a category |
|
||||
| POST | `/v1/rooms.favorite` | Favorite / unfavorite (used by the Favorites target) |
|
||||
|
||||
## Key Files
|
||||
|
||||
| Layer | File |
|
||||
|-------|------|
|
||||
| Types | `packages/core-typings/src/IUser.ts` (`ISidebarCategory`, `SIDEBAR_SYSTEM_GROUP_KEYS`) |
|
||||
| Category list + ordering | `apps/meteor/client/sidebar/hooks/useCategoryList.ts` |
|
||||
| Room list | `apps/meteor/client/sidebar/hooks/useRoomList.ts` |
|
||||
| Persist mutation | `apps/meteor/client/sidebar/categories/hooks/usePersistCategoriesMutation.ts` |
|
||||
| Create category | `apps/meteor/client/sidebar/categories/hooks/useCreateCustomCategory.ts` |
|
||||
| Move room | `apps/meteor/client/sidebar/categories/hooks/useMoveRoomCategory.ts` |
|
||||
| Move category position | `apps/meteor/client/sidebar/categories/hooks/useMoveCategoryPosition.ts` |
|
||||
| Toggle unreads | `apps/meteor/client/sidebar/categories/hooks/useToggleUnreads.ts` |
|
||||
| Name validation | `apps/meteor/client/sidebar/categories/hooks/useValidateCategoryName.ts` |
|
||||
| Custom categories list | `apps/meteor/client/sidebar/categories/hooks/useCustomCategories.ts` |
|
||||
| Set category (client) | `apps/meteor/client/sidebar/categories/hooks/useSetCategory.ts` |
|
||||
| Server endpoint | `apps/meteor/server/api/experimental/rooms.setCategory.ts` |
|
||||
| Category UI | `apps/meteor/client/sidebar/categories/` |
|
||||
| Room kebab | `apps/meteor/client/sidebar/RoomMenu.tsx` |
|
||||
| Navbar create menu | `apps/meteor/client/navbar/NavBarPagesGroup/hooks/useCreateNewMenu.ts` |
|
||||
| Room-header grouping | `apps/meteor/client/views/room/Header/icons/RoomGroupingMenu.tsx` |
|
||||
| i18n | `packages/i18n/src/locales/en.i18n.json` |
|
||||
|
||||
## Tests
|
||||
|
||||
| Kind | File |
|
||||
|------|------|
|
||||
| API (preference persistence + validation) | `apps/meteor/tests/end-to-end/api/sidebar-custom-categories.ts` |
|
||||
| E2E (Playwright) | `apps/meteor/tests/e2e/sidebar-custom-categories.spec.ts` |
|
||||
| E2E page object | `apps/meteor/tests/e2e/page-objects/fragments/sidebar.ts` (custom-category helpers) |
|
||||
150
docs/features/mcp-server.md
Normal file
150
docs/features/mcp-server.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# MCP Server (Model Context Protocol)
|
||||
|
||||
> **Status: Alpha.** Off by default. Its capabilities and configuration may evolve as we gather feedback.
|
||||
|
||||
## Overview
|
||||
|
||||
The MCP server exposes the Rocket.Chat REST API to [Model Context Protocol](https://modelcontextprotocol.io) clients (Claude Desktop/Code, IDE agents, custom agents). It speaks **JSON-RPC 2.0 over Streamable HTTP** and turns existing REST endpoints into MCP **tools**, so an AI client can drive a workspace (post messages, search, manage rooms, look up users, …) using the user's own credentials and permissions.
|
||||
|
||||
It is **enterprise (protected) code** under `apps/meteor/ee/`. The design goal is _native + minimum changes_: it mounts on the existing Hono API router and reuses its authentication, permissions, license checks, remote-address resolution, logging, metrics, tracing, CORS, and generated typed-route metadata. Tool execution goes through the REST API, including each target endpoint's validation, authorization, and rate limiting. The endpoint adds MCP-specific Origin validation for DNS-rebinding protection. There is **no new runtime dependency** — the JSON-RPC layer is implemented directly.
|
||||
|
||||
## Endpoint
|
||||
|
||||
| Method | Endpoint | Behavior |
|
||||
| ------ | ------------- | ------------------------------------------------------------------------------------------------------------- |
|
||||
| POST | `/api/v1/mcp` | JSON-RPC 2.0 message. The `2025-03-26` revision also accepts batch arrays. Response is `application/json`. |
|
||||
| GET | `/api/v1/mcp` | `405` with `Allow: POST` — this transport does not offer a server-initiated SSE stream. |
|
||||
|
||||
The endpoint is attached directly to the existing `/api/v1` Hono router because its JSON-RPC envelopes are defined by the MCP specification rather than Rocket.Chat's REST response contract. It still uses the standard authentication, `access-mcp` permission, and AI license middleware. When `MCP_Enabled` is off the action returns `404`.
|
||||
|
||||
## Request lifecycle
|
||||
|
||||
The MCP handshake is the standard JSON-RPC flow:
|
||||
|
||||
1. **`initialize`** — client and server exchange protocol version + capabilities. The server negotiates the handshake-based Streamable HTTP revisions it supports (`2025-03-26`, `2025-06-18`, and `2025-11-25`) and advertises `{ tools: { listChanged: false } }`. The transport is stateless and does not issue a session id.
|
||||
2. **`tools/list`** — returns the available tools (name, description, `inputSchema`).
|
||||
3. **`tools/call`** — runs a tool by name with arguments and returns its result as `content`.
|
||||
|
||||
Also handled: `ping`, and the `notifications/initialized` / `notifications/cancelled` notifications (acknowledged with `202`, no body).
|
||||
|
||||
After initialization, clients should send the negotiated version in the `MCP-Protocol-Version` header. Requests with an unsupported version are rejected with `400`; when the header is omitted, the transport follows the specification's `2025-03-26` compatibility default. JSON-RPC batching is accepted only for that revision because later Streamable HTTP revisions require one message per POST.
|
||||
The shared CORS middleware advertises this request header only while MCP is enabled.
|
||||
|
||||
## Authentication
|
||||
|
||||
Authentication is **reused from the REST layer** — no MCP-specific credential type:
|
||||
|
||||
- The client sends a **Personal Access Token** as `X-User-Id` + `X-Auth-Token` headers on every request (including `initialize`), because the route is `authRequired: true`.
|
||||
- For POST requests, the standard auth middleware resolves the user; the MCP action then verifies that the matching login-token record is a Personal Access Token. Missing, invalid, and session tokens are rejected with `401`. GET is still covered by route authentication but returns `405` without performing the additional token-type lookup because it cannot execute tools.
|
||||
- Every MCP action additionally requires the **`access-mcp`** permission (`permissionsRequired: ['access-mcp']`), enforced by the standard permissions middleware. Without it the request is rejected with `403`. The permission is granted to `admin` by default; admins can grant it to other roles from the Permissions admin page.
|
||||
|
||||
> When creating the PAT, tick **"Ignore Two Factor Authentication"**, otherwise header auth is rejected with a 2FA challenge.
|
||||
|
||||
## Transport security
|
||||
|
||||
The endpoint validates the `Origin` header to protect browser-accessible deployments from DNS-rebinding attacks. Requests without `Origin` are accepted for native MCP clients. Browser requests are accepted only when their normalized origin matches `Site_Url` or an explicit entry in `API_CORS_Origin` while CORS is enabled. The wildcard (`*`) does not authorize a browser origin for MCP.
|
||||
|
||||
Tool dispatch is restricted to server-generated, allow-listed REST paths on `127.0.0.1`; client input cannot select a URL. Redirects are rejected, calls time out after 20 seconds, and each REST response and final encoded MCP response is capped at 5 MiB. Batches run at most four calls concurrently and share the same 5 MiB streaming response budget.
|
||||
|
||||
## Licensing
|
||||
|
||||
The feature is gated behind the **Rocket.Chat AI add-on** (`chat.rocket.rc-ai`):
|
||||
|
||||
- The route is registered with `license: [AI_LICENSE_MODULE]`, so requests are rejected unless the workspace license includes the module.
|
||||
- The `MCP_*` settings use the same module and an `invalidValue` of `false`, so without the add-on they fall back to **off** and the feature cannot be enabled.
|
||||
|
||||
## Tool catalog
|
||||
|
||||
Two **bounded** sets are exposed, selected by the `MCP_Expose_Extended_API` setting. The full unfiltered API is **never** exposed.
|
||||
|
||||
| `MCP_Expose_Extended_API` | Exposed tools |
|
||||
| ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **off** (default) | **Minimal curated set** — a small hand-picked list (`post_chat_postMessage`, `get_chat_getMessage`, `post_channels_create`, `get_channels_list_joined`, `get_rooms_get`, `get_users_info`). |
|
||||
| **on** | **Extended set** — the full catalog filtered by the `ALLOWED_TOOL_NAMES` allow-list (~100 routes), still excluding routes tagged `Missing Documentation`. |
|
||||
|
||||
Both sets are built from registered typed-route metadata. The extended set filters generated base names through the allow-list, while the curated set filters routes through its smaller explicit list and supplies fallback descriptions where metadata is incomplete.
|
||||
|
||||
### Tool naming & variant expansion
|
||||
|
||||
A route's base tool name is `toolNameFor(path, method)` — e.g. `GET /api/v1/users.info` → `get_users_info`. Curated and extended catalogs use the same names, so enabling the extended catalog only adds tools.
|
||||
|
||||
When a route's request schema is a `oneOf`/`anyOf` of object sub-schemas with **distinct discriminators** (its `required` keys), each branch becomes its **own tool**, named `<base>_by_<discriminator>`:
|
||||
|
||||
- `chat.postMessage` → `post_chat_postMessage_by_channel`, `post_chat_postMessage_by_roomId`
|
||||
- `users.info` → `get_users_info_by_userId`, `get_users_info_by_username`, `get_users_info_by_importId`, …
|
||||
|
||||
The allow-list is matched on the **base name**, so a single entry (`get_users_info`) admits all of that route's variants.
|
||||
|
||||
### Schema handling
|
||||
|
||||
MCP / the Anthropic tools API require each `inputSchema` to be a plain object schema and reject several JSON-Schema constructs. `mcpSafeSchema` normalizes a route's schema for the tool view (the REST validator is untouched):
|
||||
|
||||
- strips OpenAPI-only `nullable` and `not`;
|
||||
- resolves `oneOf`/`anyOf`/`allOf` by adopting the first branch (so a value union like `string | string[]` keeps a concrete type, and a sub-schema union yields a clean object);
|
||||
- forces a top-level `type: "object"`.
|
||||
|
||||
Tool **descriptions** are sourced from the route schema's own `description` (added in `rest-typings`), falling back to a per-route string.
|
||||
|
||||
## Dispatch
|
||||
|
||||
`tools/call` executes the target endpoint **as the authenticated user** via a loopback HTTP call to the local REST API (`http://127.0.0.1:<PORT><ROOT_URL_PATH_PREFIX>/api/v1/<route>`), forwarding:
|
||||
|
||||
- `X-User-Id` + `X-Auth-Token` (the caller's PAT), so all validation and permission checks run exactly as for a real REST client — **zero duplicated business logic**;
|
||||
- `X-Real-IP` set to the resolved client address (`this.requestIp`), so the target endpoint's per-route rate limiter keys on the real client rather than the loopback address.
|
||||
|
||||
The REST response is wrapped as MCP `content` (`type: "text"`); a non-2xx REST response is returned with `isError: true`. Internal calls time out after 20 seconds. REST bodies and final encoded MCP responses are capped at 5 MiB; batched calls use bounded concurrency and share a single streaming body budget to limit retained response data.
|
||||
|
||||
## Rate limiting
|
||||
|
||||
The MCP endpoint uses Rocket.Chat's **built-in per-route rate limiter** with a limit of 60 requests per minute (enabled by `API_Enable_Rate_Limiter`, honoring `api-bypass-rate-limit`). Every tool call is also subject to its target REST endpoint's rate limit. The resolved client address is propagated to that endpoint rather than being counted as loopback traffic. MCP protocol requests are additionally bounded to 20 batch entries, four concurrent dispatches, 20-second calls, and a shared 5 MiB response budget. As with every Rocket.Chat API route, deployments behind a proxy must configure `HTTP_FORWARDED_COUNT` and trusted forwarding headers correctly.
|
||||
|
||||
## Settings
|
||||
|
||||
Registered under **Admin → AI Center → MCP** (`server/settings/ai.ts`):
|
||||
|
||||
| Setting | Default | Description |
|
||||
| ------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `MCP_Enabled` | `false` | Enables the `/api/v1/mcp` endpoint. Flagged **alpha** via an admin warning callout (`MCP_Alpha_Alert`). |
|
||||
| `MCP_Expose_Extended_API` | `false` | When on, exposes the extended allow-listed toolset instead of the minimal curated one. Gated behind `MCP_Enabled`. The full API is never exposed. |
|
||||
|
||||
## Connecting a client
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http rocketchat http://<host>/api/v1/mcp \
|
||||
--header "X-User-Id: <userId>" \
|
||||
--header "X-Auth-Token: <personalAccessToken>"
|
||||
```
|
||||
|
||||
Raw smoke test:
|
||||
|
||||
```bash
|
||||
H=(-H "Content-Type: application/json" -H "X-User-Id: <uid>" -H "X-Auth-Token: <token>")
|
||||
curl -s "${H[@]}" http://localhost:3000/api/v1/mcp -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
|
||||
curl -s "${H[@]}" http://localhost:3000/api/v1/mcp \
|
||||
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"post_chat_postMessage_by_channel","arguments":{"channel":"#general","text":"hello from MCP"}}}'
|
||||
```
|
||||
|
||||
> The client **must** send `Content-Type: application/json` on POST — the shared router only parses the body for that content type.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Alpha**, off by default.
|
||||
- Implements the handshake-based Streamable HTTP lifecycle through protocol version `2025-11-25`; newer lifecycle methods are not yet supported.
|
||||
- Requires the **Rocket.Chat AI add-on** — both the route and the settings are gated by it.
|
||||
- One HTTP response per POST. The `2025-03-26` compatibility revision accepts a bounded batch of up to 20 messages; later revisions accept one message per POST. There is no server-initiated SSE stream (`GET` → `405`).
|
||||
- The official `@modelcontextprotocol/sdk` is intentionally not used; the files are structured so it can be dropped into the transport/server layer later for SSE and richer session handling.
|
||||
|
||||
## Key Files
|
||||
|
||||
| Layer | File |
|
||||
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
|
||||
| Hono route registration (`/api/v1/mcp`) + `MCP_Enabled` gate | `apps/meteor/ee/server/api/mcp/index.ts` |
|
||||
| JSON-RPC handlers (`initialize`/`tools/list`/`tools/call`/…) | `apps/meteor/ee/server/api/mcp/server.ts` |
|
||||
| Tool catalog (curated + extended allow-list, variants, schema normalization) | `apps/meteor/ee/server/api/mcp/catalog.ts` |
|
||||
| Tool dispatch (loopback to REST as the user) | `apps/meteor/ee/server/api/mcp/dispatch.ts` |
|
||||
| License-gated permission seed (`access-mcp`) | `apps/meteor/ee/server/startup/mcp.ts` |
|
||||
| EE module load | `apps/meteor/ee/server/api/index.ts` |
|
||||
| Settings (license-gated) | `apps/meteor/server/settings/ai.ts` |
|
||||
| License module | `packages/ai-search/src/constants.ts` (`AI_LICENSE_MODULE`) |
|
||||
| Schema descriptions reused as tool docs | `packages/rest-typings/src/v1/chat.ts`, `packages/rest-typings/src/v1/users/UsersInfoParamsGet.ts` |
|
||||
| i18n | `packages/i18n/src/locales/en.i18n.json` (`MCP_*` keys) |
|
||||
182
docs/features/slackbridge-manual-tests.md
Normal file
182
docs/features/slackbridge-manual-tests.md
Normal file
@@ -0,0 +1,182 @@
|
||||
# SlackBridge Manual Test Plan
|
||||
|
||||
The SlackBridge is a bi-directional bridge between a Rocket.Chat workspace and one or more Slack
|
||||
workspaces — each token line in the SlackBridge settings gets its own connection and adapter. It
|
||||
has no automated end-to-end coverage: exercising it requires a real Slack workspace, a real bot
|
||||
installation, and a human watching both sides at once. This document is the checklist to walk
|
||||
through when touching the bridge.
|
||||
|
||||
**Scope:** the matrices below assume a **single** bridged Slack workspace — configure exactly one
|
||||
token per setting while running them. Bridging several workspaces at once is a supported
|
||||
configuration that this plan does not cover; see [Coverage gaps](#coverage-gaps).
|
||||
|
||||
Code lives in [apps/meteor/server/bridges/slack/](../../apps/meteor/server/bridges/slack/) —
|
||||
`SlackAdapter.ts` handles Slack → Rocket.Chat, `RocketAdapter.ts` handles Rocket.Chat → Slack
|
||||
(hooked in through the `afterSaveMessage`, `afterDeleteMessage`, `afterSetReaction` and
|
||||
`afterUnsetReaction` callbacks). Settings are declared in
|
||||
[apps/meteor/server/settings/slackbridge.ts](../../apps/meteor/server/settings/slackbridge.ts).
|
||||
|
||||
## Setup
|
||||
|
||||
Configure the bridge in **Admin → Settings → SlackBridge**:
|
||||
|
||||
| Setting | Notes |
|
||||
| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
|
||||
| `SlackBridge_Enabled` | Master switch. Must be on. |
|
||||
| `SlackBridge_UseLegacy` | Chooses the connection mode — see below. Run the matrices in the mode you are testing. |
|
||||
| `SlackBridge_APIToken` | Legacy mode only (RTM). |
|
||||
| `SlackBridge_BotToken`, `SlackBridge_SigningSecret`, `SlackBridge_AppToken` | App mode only (Bolt / Socket Mode). |
|
||||
| `SlackBridge_Out_Enabled` | Required for anything to flow Rocket.Chat → Slack. |
|
||||
| `SlackBridge_Out_All` / `SlackBridge_Out_Channels` | Which Rocket.Chat rooms are bridged out. |
|
||||
| `SlackBridge_Reactions_Enabled` | Gates reaction propagation in both directions. |
|
||||
| `SlackBridge_AliasFormat` | Optional, Slack → Rocket.Chat only. `%s` is the Slack username; test with `%s (via Slack)`. |
|
||||
| `SlackBridge_FileUpload_Enabled` | Gates file transfer. |
|
||||
|
||||
Two connection modes exist and they use different Slack event plumbing, so a regression can
|
||||
appear in one and not the other. **Run the full matrix in both** when changing anything in the
|
||||
connect/event-subscription path:
|
||||
|
||||
- **Legacy (`SlackBridge_UseLegacy: true`)** — RTM connection with an API token.
|
||||
- **App (`SlackBridge_UseLegacy: false`)** — Slack app using bot token, signing secret and app
|
||||
token.
|
||||
|
||||
You also need:
|
||||
|
||||
- A Slack workspace where you can create channels and install/invite the bot (referred to below
|
||||
as **rocketbot**).
|
||||
- At least one channel that exists on both sides **with** rocketbot in it, and one that exists on
|
||||
both sides **without** rocketbot in it. Several cases below hinge on that difference.
|
||||
- A second Slack user, if possible — some behaviors only show up when the message author is not
|
||||
the same account as the bot.
|
||||
|
||||
Watch `SlackBridge` logs (`slackLogger` / `rocketLogger`) while testing; several expected
|
||||
failures are only visible there.
|
||||
|
||||
## How to read the matrices
|
||||
|
||||
- **Origin** — where the message being acted on was originally posted.
|
||||
- **Action performed on** — the side where the tester performs the action.
|
||||
- Every matrix must also be repeated **on a channel that rocketbot is not a member of**. Nothing
|
||||
should propagate in either direction for those channels, and nothing should error.
|
||||
|
||||
## Sending messages
|
||||
|
||||
| # | Origin | Action performed on | Expected result |
|
||||
| --- | ----------- | ------------------- | ------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | Slack | Slack | Message appears in the linked Rocket.Chat room, under the alias `SlackBridge_AliasFormat` produces. |
|
||||
| 2 | Rocket.Chat | Rocket.Chat | Message appears in the linked Slack channel, posted by rocketbot with the sender's username and avatar. |
|
||||
|
||||
`SlackBridge_AliasFormat` only affects case 1: `addAliasToMsg` sets `alias` on the **Rocket.Chat**
|
||||
message, so with `%s (via Slack)` a message from Slack user `alice` shows as `alice (via Slack)`.
|
||||
Case 2 carries the sender's identity through `chat.postMessage`'s `username`/`icon_url` and ignores
|
||||
the setting entirely. Set the format, run case 1 with it set and again with it empty, then **restore
|
||||
the original value** before moving on — the later matrices assume the workspace's normal setup.
|
||||
|
||||
Repeat both on a channel without rocketbot: neither message crosses over.
|
||||
|
||||
## Editing messages
|
||||
|
||||
Slack does not allow an app to edit messages it did not post, and it does not allow humans to
|
||||
edit bot messages. Two of the four cases are therefore **expected failures** — they are listed
|
||||
because "nothing crashes and the local side stays consistent" is the requirement.
|
||||
|
||||
| # | Origin | Action performed on | Expected result |
|
||||
| --- | ----------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | Slack | Slack | Edit propagates to Rocket.Chat. |
|
||||
| 2 | Rocket.Chat | Rocket.Chat | Edit propagates to Slack. |
|
||||
| 3 | Slack | Rocket.Chat | **Expected failure.** Rocket.Chat shows the edit; Slack is unchanged. `chat.update` is rejected because the message is not the bot's. |
|
||||
| 4 | Rocket.Chat | Slack | **Expected failure.** Slack does not even offer the edit affordance for a bot message. |
|
||||
|
||||
Repeat on a channel without rocketbot: edits stay local on both sides.
|
||||
|
||||
## Deleting messages
|
||||
|
||||
Case 3 needs care about _who_ deletes. A Rocket.Chat-origin message is authored on Slack by
|
||||
rocketbot, and Slack offers no delete affordance for an app's message to a regular member, so
|
||||
pick one of:
|
||||
|
||||
- Delete it from the Slack UI as a **workspace owner or admin** — they can delete any message,
|
||||
including an app's.
|
||||
- Call `chat.delete` with the very token the bridge is configured with, so the caller is
|
||||
rocketbot itself: `SlackBridge_APIToken` in legacy/RTM mode, `SlackBridge_BotToken` in
|
||||
app/Socket Mode. Any other token fails with `cant_delete_message`, which is case 4's failure,
|
||||
not case 3's.
|
||||
|
||||
Both routes emit the same `message_deleted` event, so either is a valid way to run the case.
|
||||
|
||||
| # | Origin | Action performed on | Expected result |
|
||||
| --- | ----------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 1 | Slack | Slack | Deletion propagates to Rocket.Chat. |
|
||||
| 2 | Rocket.Chat | Rocket.Chat | Deletion propagates to Slack. |
|
||||
| 3 | Rocket.Chat | Slack | Deletion propagates to Rocket.Chat — deleted as a workspace admin or via `chat.delete` with rocketbot's own token, per the note above. |
|
||||
| 4 | Slack | Rocket.Chat | **Expected failure.** The message is deleted in Rocket.Chat only; Slack keeps it, since `chat.delete` cannot remove another user's message. |
|
||||
|
||||
Repeat on a channel without rocketbot: deletions stay local on both sides.
|
||||
|
||||
## Reactions
|
||||
|
||||
Requires `SlackBridge_Reactions_Enabled`. Reactions sent to Slack are added by the bot account,
|
||||
so on the Slack side every Rocket.Chat reaction shows up as rocketbot's reaction regardless of
|
||||
who actually reacted.
|
||||
|
||||
| # | Origin | Action performed on | Expected result |
|
||||
| --- | ----------- | ------------------- | ----------------------------------------------------- |
|
||||
| 1 | Slack | Slack | Reaction appears on the Rocket.Chat message. |
|
||||
| 2 | Rocket.Chat | Slack | Reaction appears on the Rocket.Chat message. |
|
||||
| 3 | Rocket.Chat | Rocket.Chat | Reaction appears on the Slack message (as rocketbot). |
|
||||
| 4 | Slack | Rocket.Chat | Reaction appears on the Slack message (as rocketbot). |
|
||||
|
||||
Also check removal of each reaction, and confirm a reaction is not echoed back to its origin
|
||||
(the bridge de-duplicates through its reactions map — a double-count means that broke).
|
||||
|
||||
Repeat on a channel without rocketbot: reactions stay local.
|
||||
|
||||
### `SlackBridge_Reactions_Enabled` toggle
|
||||
|
||||
| Setting state | Expected result |
|
||||
| ------------- | -------------------------------------------------------------------------------------------------------------- |
|
||||
| Disabled | React to both a Slack-origin and a Rocket.Chat-origin message from both sides — nothing propagates, no errors. |
|
||||
| Enabled | Re-run the same reactions — they propagate as in the matrix above, without restarting the server. |
|
||||
|
||||
## Channel lifecycle
|
||||
|
||||
These cases cover how the bridge reacts to channels appearing, disappearing, and the bot joining
|
||||
or leaving. Send a message **from each side** after every step and verify propagation.
|
||||
|
||||
### Channel exists on both sides, rocketbot not in the Slack channel
|
||||
|
||||
1. Confirm nothing propagates in either direction.
|
||||
2. Invite rocketbot to the Slack channel.
|
||||
3. Send from Slack and from Rocket.Chat — both directions now work.
|
||||
|
||||
### Channel exists on Rocket.Chat only
|
||||
|
||||
1. Create the matching channel in Slack and invite rocketbot.
|
||||
2. Send from Rocket.Chat — it reaches Slack.
|
||||
3. Send from Slack — it reaches Rocket.Chat.
|
||||
|
||||
### Channel exists on Slack only
|
||||
|
||||
1. Invite rocketbot to the Slack channel.
|
||||
2. Send from Slack — the Rocket.Chat channel is created and receives the message.
|
||||
3. Send from Rocket.Chat — it reaches Slack.
|
||||
|
||||
### Channel exists on both sides with rocketbot
|
||||
|
||||
1. Delete the Rocket.Chat channel, then send from Slack — behaves like the "Slack only" case
|
||||
above (the channel is recreated).
|
||||
2. Remove rocketbot from the Slack channel, then send from Slack and from Rocket.Chat — nothing
|
||||
propagates in either direction, and no errors are logged.
|
||||
|
||||
## Coverage gaps
|
||||
|
||||
The matrices above are the historical manual suite and deliberately mirror it. They do **not**
|
||||
cover, and it is worth checking manually when relevant:
|
||||
|
||||
- File uploads (`SlackBridge_FileUpload_Enabled`).
|
||||
- Slack history import (`/slackbridge-import`, `slackbridge_import.server.ts`).
|
||||
- Threads (`thread_ts` mapping in `postMessage`).
|
||||
- Private channels and DMs.
|
||||
- `SlackBridge_ExcludeBotnames` filtering.
|
||||
- The **Remove Channel Links** admin action (`SlackBridge_Remove_Channel_Links`).
|
||||
- Multiple Slack workspaces bridged at once (one token per line in each token setting).
|
||||
235
docs/form-validation.md
Normal file
235
docs/form-validation.md
Normal file
@@ -0,0 +1,235 @@
|
||||
# Form Validation Guidelines
|
||||
|
||||
This document outlines the standardized form validation patterns and guidelines established in PR [#39590](https://github.com/RocketChat/Rocket.Chat/pull/39590) to ensure consistent user experience across Rocket.Chat forms.
|
||||
|
||||
## Overview
|
||||
|
||||
The form validation standardization aims to:
|
||||
- **Improve accessibility** by keeping submit buttons enabled and letting validation run on submit
|
||||
- **Provide consistent UX** with validation triggered on form submission and re-validation on field changes
|
||||
- **Prevent unnecessary API calls** by using dirty-checks and appropriate revalidation modes
|
||||
- **Enhance user feedback** with clear error messages and proper ARIA attributes
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Submit-First validation (`mode: 'onSubmit'`)
|
||||
|
||||
Forms should use `mode: 'onSubmit'` in react-hook-form to trigger initial validation only when the user attempts to submit the form.
|
||||
|
||||
**Why:** This approach improves accessibility by:
|
||||
- Keeping submit buttons enabled (allowing screen readers and keyboard users to discover validation requirements)
|
||||
- Avoiding premature error messages that can confuse users
|
||||
- Letting users complete the form at their own pace before seeing validation feedback
|
||||
|
||||
**Example:**
|
||||
```tsx
|
||||
const {
|
||||
control,
|
||||
formState: { errors, isDirty, isSubmitting },
|
||||
handleSubmit,
|
||||
} = useForm<FormData>({
|
||||
mode: 'onSubmit', // This can be omitted, `onSubmit` it's the default mode value
|
||||
defaultValues: initialData,
|
||||
});
|
||||
```
|
||||
|
||||
### 2. Smart revalidation strategy
|
||||
|
||||
After the first submit attempt, forms should revalidate fields intelligently:
|
||||
|
||||
#### Default: `reValidateMode: 'onChange'`
|
||||
For most forms, use the default onChange revalidation to provide immediate feedback as users correct errors.
|
||||
|
||||
#### Exception: `reValidateMode: 'onBlur'` for Async Validation
|
||||
For forms with **async validation** (e.g., username availability, email uniqueness checks), explicitly set `reValidateMode: 'onBlur'` to avoid excessive API calls.
|
||||
|
||||
**Example with async validation:**
|
||||
```tsx
|
||||
const {
|
||||
control,
|
||||
formState: { errors, isDirty },
|
||||
handleSubmit,
|
||||
} = useForm<FormData>({
|
||||
reValidateMode: 'onBlur', // Avoid API calls on every keystroke
|
||||
defaultValues: initialData,
|
||||
});
|
||||
```
|
||||
|
||||
### 3. Dirty-check with `useFormSubmitWithDirtyCheck`
|
||||
|
||||
Use the `useFormSubmitWithDirtyCheck` hook to provide user-friendly feedback when attempting to save unchanged forms.
|
||||
|
||||
Usually applicable on edit forms, where fields are already populated.
|
||||
|
||||
**Purpose:**
|
||||
- Prevents unnecessary save operations on unchanged data
|
||||
- Shows informative toast message: "No changes to save"
|
||||
- Maintains accessibility by keeping buttons enabled
|
||||
|
||||
**Signature:**
|
||||
|
||||
Receives a callback as the first parameter (your submit handler), and an object as the second parameter containing `isDirty` and an optional `noChangesMessage` translation key, to be dispatched in the info toast.
|
||||
|
||||
**Usage:**
|
||||
|
||||
```tsx
|
||||
import { useFormSubmitWithDirtyCheck } from '/hooks/useFormSubmitWithDirtyCheck';
|
||||
|
||||
const handleSave = useFormSubmitWithDirtyCheck(
|
||||
async (data: FormData) => {
|
||||
try {
|
||||
await saveData(data);
|
||||
dispatchToastMessage({ type: 'success', message: t('Saved') });
|
||||
} catch (error) {
|
||||
dispatchToastMessage({ type: 'error', message: error });
|
||||
}
|
||||
},
|
||||
{ isDirty }
|
||||
);
|
||||
|
||||
// In JSX:
|
||||
<form onSubmit={handleSubmit(handleSave)}>
|
||||
```
|
||||
|
||||
**When to use dirty-check:**
|
||||
|
||||
This hook is recommended when the same form component is used for both creation (new) and editing existing data. The hook intelligently handles both scenarios:
|
||||
- ✅ **Create mode** (no existing data): Allows submission without dirty check
|
||||
- ✅ **Edit mode** (existing data): Shows "No changes to save" toast when form is unchanged
|
||||
- ✅ **Unified component**: Simplifies logic by handling both create and edit in one place
|
||||
|
||||
|
||||
## Form Implementation Patterns
|
||||
|
||||
### Basic Form Structure
|
||||
|
||||
```tsx
|
||||
import { useForm, Controller } from 'react-hook-form';
|
||||
import { useFormSubmitWithDirtyCheck } from '../../../hooks/useFormSubmitWithDirtyCheck';
|
||||
|
||||
type FormData = {
|
||||
name: string;
|
||||
email: string;
|
||||
};
|
||||
|
||||
const MyForm = ({ data, onSave }: FormProps) => {
|
||||
const { t } = useTranslation();
|
||||
|
||||
|
||||
const {
|
||||
control,
|
||||
formState: { errors, isDirty, isSubmitting },
|
||||
handleSubmit,
|
||||
} = useForm<FormData>({
|
||||
defaultValues: data || {},
|
||||
});
|
||||
|
||||
const handleFormSubmit = useFormSubmitWithDirtyCheck(
|
||||
async (formData: FormData) => {
|
||||
await onSave(formData);
|
||||
},
|
||||
{ isDirty }
|
||||
);
|
||||
|
||||
return (
|
||||
<form onSubmit={handleSubmit(handleFormSubmit)} id={formId}>
|
||||
{/* Form fields */}
|
||||
</form>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
## Button State Management
|
||||
|
||||
### Submit Button States
|
||||
|
||||
```tsx
|
||||
<Button
|
||||
primary
|
||||
type='submit'
|
||||
form={formId}
|
||||
loading={isSubmitting}
|
||||
>
|
||||
{t('Save')}
|
||||
</Button>
|
||||
```
|
||||
|
||||
**Key Points:**
|
||||
- Use `loading={isSubmitting}` to show loading state during submission
|
||||
- Never disable the save button (keep enabled for a11y)
|
||||
- Always connect button to form via `form={formId}` attribute
|
||||
|
||||
|
||||
|
||||
## Basic checklist
|
||||
|
||||
When updating an existing form to follow these guidelines:
|
||||
|
||||
- [ ] Use `mode` to `'onSubmit'` in `useForm`
|
||||
- [ ] Add `reValidateMode: 'onBlur'` if form has async validation
|
||||
- [ ] Wrap submit handler with `useFormSubmitWithDirtyCheck` (for create and edit forms)
|
||||
- [ ] Add ARIA attributes: `aria-describedby`, `aria-invalid`, `role='alert'` when applicable
|
||||
- [ ] Button states: `loading={isSubmitting}`, but never `disabled`
|
||||
- [ ] Verify accessibility with screen reader testing
|
||||
|
||||
## Basic DOs and DON'Ts
|
||||
|
||||
### ❌ Don't: Disable buttons based on form validity
|
||||
|
||||
```tsx
|
||||
// Bad - prevents discovery of validation requirements
|
||||
<Button disabled={!isValid || !isDirty}>Save</Button>
|
||||
```
|
||||
|
||||
### ✅ Do: Keep buttons enabled, let validation run on submit
|
||||
|
||||
```tsx
|
||||
// Good - accessible and provides feedback
|
||||
<Button
|
||||
type='submit'
|
||||
disabled={existingId ? !isDirty : false}
|
||||
loading={isSubmitting}
|
||||
>
|
||||
Save
|
||||
</Button>
|
||||
```
|
||||
|
||||
### ❌ Don't: Use `mode: 'onChange'` for initial validation
|
||||
|
||||
```tsx
|
||||
// Bad - shows errors immediately, poor UX
|
||||
useForm({ mode: 'onChange' })
|
||||
```
|
||||
|
||||
### ✅ Do: Use `mode: 'onSubmit'` for initial validation
|
||||
|
||||
```tsx
|
||||
// Good - validates on submit, revalidates on change
|
||||
useForm({ mode: 'onSubmit' })
|
||||
```
|
||||
|
||||
### ❌ Don't: Use `reValidateMode: 'onChange'` with async validation
|
||||
|
||||
```tsx
|
||||
// Bad - causes API call on every keystroke
|
||||
useForm({
|
||||
mode: 'onSubmit',
|
||||
// Uses default 'onChange' revalidation - too many API calls!
|
||||
})
|
||||
```
|
||||
|
||||
### ✅ Do: Use `reValidateMode: 'onBlur'` with async validation
|
||||
|
||||
```tsx
|
||||
// Good - reduces API calls while maintaining feedback
|
||||
useForm({
|
||||
mode: 'onSubmit',
|
||||
reValidateMode: 'onBlur',
|
||||
})
|
||||
```
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [React Hook Form Documentation](https://react-hook-form.com/)
|
||||
- [WCAG 2.1 Form Guidelines](https://www.w3.org/WAI/WCAG21/quickref/?showtechniques=332#error-identification)
|
||||
- [PR #39590 - Form Validation Standardization](https://github.com/RocketChat/Rocket.Chat/pull/39590)
|
||||
480
docs/frontend/building-components.md
Normal file
480
docs/frontend/building-components.md
Normal file
@@ -0,0 +1,480 @@
|
||||
# Building components
|
||||
|
||||
A component is a reusable piece of code that represents a single UI element. Components vary by complexity and type, existing at either the **application** level or the **Fuselage library** level:
|
||||
|
||||
- **Application components** remain specific to particular Rocket.Chat applications, without cross-application reusability.
|
||||
- **Fuselage library components** are reusable across all Rocket.Chat applications and represent the recommended approach.
|
||||
|
||||
## Component rules matrix
|
||||
|
||||
| Combination | Fuselage level | Application level |
|
||||
| ---------------- | -------------- | ----------------- |
|
||||
| Simple & Visual | ✅ | ❌ |
|
||||
| Complex & Visual | ✅ | ✅ |
|
||||
| Simple & Logical | ❌ | ❌ |
|
||||
| Complex & Logical| ❌ | ✅ |
|
||||
|
||||
## Simple components
|
||||
|
||||
Simple components represent atomic UI elements like buttons or text fields.
|
||||
|
||||
### Prefer variation over styles
|
||||
|
||||
Use prop names suggesting component variation rather than style-based props. Instead of `color="blue"`, use `variation="primary"` for improved readability and consistency.
|
||||
|
||||
### Avoid hardcoded values or magic numbers
|
||||
|
||||
Preferred:
|
||||
|
||||
```jsx
|
||||
<Button small square>
|
||||
<Icon name='circle-arrow-down' size='x24' />
|
||||
</Button>
|
||||
```
|
||||
|
||||
Not recommended:
|
||||
|
||||
```jsx
|
||||
<Button height='50px' width='50px' square>
|
||||
<Icon name='circle-arrow-down' size='x24' />
|
||||
</Button>
|
||||
```
|
||||
|
||||
### Customize via CSS variables
|
||||
|
||||
Prioritize CSS variable customization over arbitrary values:
|
||||
|
||||
```scss
|
||||
$modal-margin: theme('modal-margin', auto);
|
||||
|
||||
.rcx-modal {
|
||||
position: static;
|
||||
display: flex;
|
||||
width: 100%;
|
||||
max-height: 100%;
|
||||
margin: $modal-margin;
|
||||
}
|
||||
```
|
||||
|
||||
### Document variations in Storybook
|
||||
|
||||
Display all possible variations with descriptive explanations for non-obvious options.
|
||||
|
||||
### Unit test all behaviors
|
||||
|
||||
Comprehensive unit tests ensure reliability across all intended scenarios:
|
||||
|
||||
```jsx
|
||||
describe('[Menu Component]', () => {
|
||||
it('should renders without crashing', () => {
|
||||
render(<Simple {...Simple.args} />);
|
||||
});
|
||||
|
||||
it('should open options when click', async () => {
|
||||
const { getByTestId } = render(<Simple {...Simple.args} />);
|
||||
const button = getByTestId('menu');
|
||||
userEvent.click(button);
|
||||
expect(await screen.findByText('Make Admin')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('should have no options when click twice', async () => {
|
||||
const { getByTestId } = render(<Simple {...Simple.args} />);
|
||||
const button = getByTestId('menu');
|
||||
await userEvent.click(button);
|
||||
await userEvent.click(button);
|
||||
expect(screen.queryByText('Make Admin')).toBeNull();
|
||||
});
|
||||
|
||||
it('should have no options when click on menu and then elsewhere', async () => {
|
||||
const { getByTestId } = render(<Simple {...Simple.args} />);
|
||||
const button = getByTestId('menu');
|
||||
await userEvent.click(button);
|
||||
await userEvent.click(document.body);
|
||||
expect(screen.queryByText('Make Admin')).toBeNull();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Avoid "Boxed" components
|
||||
|
||||
The `Box` component works as a wildcard primarily for simple or complex components during prototyping. For simple components specifically, construct using HTML tags instead.
|
||||
|
||||
## Complex components
|
||||
|
||||
Complex components combine multiple simple components for sophisticated UI elements like modals or tables.
|
||||
|
||||
### Visual only, no logic
|
||||
|
||||
Concentrate solely on the user interface design, ensuring it is poised to incorporate the required logic seamlessly.
|
||||
|
||||
```jsx
|
||||
export const Default = () => (
|
||||
<Modal>
|
||||
<ModalHeader>
|
||||
<ModalHeaderText>
|
||||
<ModalTitle>Modal Header</ModalTitle>
|
||||
</ModalHeaderText>
|
||||
<ModalClose />
|
||||
</ModalHeader>
|
||||
<ModalContent>Modal Body</ModalContent>
|
||||
<ModalFooter>
|
||||
<ModalFooterControllers>
|
||||
<Button>Cancel</Button>
|
||||
<Button primary onClick={action('click')}>
|
||||
Submit
|
||||
</Button>
|
||||
</ModalFooterControllers>
|
||||
</ModalFooter>
|
||||
</Modal>
|
||||
);
|
||||
```
|
||||
|
||||
### Split components for clarity
|
||||
|
||||
Structure components into understandable, logical segments.
|
||||
|
||||
### Develop with Storybook first
|
||||
|
||||
Start with Storybook to separate interface from logic:
|
||||
|
||||
```tsx
|
||||
export const CallingDM: ComponentStory<typeof VideoConfMessage> = () => (
|
||||
<VideoConfMessage>
|
||||
<VideoConfMessageRow>
|
||||
<VideoConfMessageIcon variant='incoming' />
|
||||
<VideoConfMessageText>Calling...</VideoConfMessageText>
|
||||
</VideoConfMessageRow>
|
||||
<VideoConfMessageFooter>
|
||||
<VideoConfMessageAction primary>Join</VideoConfMessageAction>
|
||||
<VideoConfMessageFooterText>Waiting for answer</VideoConfMessageFooterText>
|
||||
</VideoConfMessageFooter>
|
||||
</VideoConfMessage>
|
||||
);
|
||||
|
||||
export const CallEndedDM: ComponentStory<typeof VideoConfMessage> = () => (
|
||||
<VideoConfMessage>
|
||||
<VideoConfMessageRow>
|
||||
<VideoConfMessageIcon />
|
||||
<VideoConfMessageText>Call ended</VideoConfMessageText>
|
||||
</VideoConfMessageRow>
|
||||
<VideoConfMessageFooter>
|
||||
<VideoConfMessageAction>Call Back</VideoConfMessageAction>
|
||||
<VideoConfMessageFooterText>Call was not answered</VideoConfMessageFooterText>
|
||||
</VideoConfMessageFooter>
|
||||
</VideoConfMessage>
|
||||
);
|
||||
```
|
||||
|
||||
### Child components must remain scoped
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```tsx
|
||||
export const MyComponent: ComponentStory<typeof VideoConfMessage> = () => (
|
||||
<Box display='flex'>
|
||||
<form>
|
||||
<VideoConfMessageAction>Call ended</VideoConfMessageAction>
|
||||
</form>
|
||||
</Box>
|
||||
);
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```tsx
|
||||
export const MyComponent: ComponentStory<typeof VideoConfMessage> = () => (
|
||||
<Box display='flex'>
|
||||
<form>
|
||||
<VideoConfMessage>
|
||||
<VideoConfMessageFooter>
|
||||
<VideoConfMessageAction>Call ended</VideoConfMessageAction>
|
||||
</VideoConfMessageFooter>
|
||||
</VideoConfMessage>
|
||||
</form>
|
||||
</Box>
|
||||
);
|
||||
```
|
||||
|
||||
### HTML elements, box, and box props should be encapsulated
|
||||
|
||||
Encapsulation ensures predictable behavior and streamlined debugging by accessing HTML elements and Box props solely through the Box component's API.
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```tsx
|
||||
export const VideoConfMessage = () => (
|
||||
<Box mbs='x4' maxWidth='345px' borderWidth={2} borderColor='neutral-200' borderRadius='x4'>
|
||||
<Box p='x16' display='flex' alignItems='center'>
|
||||
<Icon name='link' />
|
||||
<div>My Text</div>
|
||||
</Box>
|
||||
</Box>
|
||||
);
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```tsx
|
||||
export type VideoConfMessageProps = Omit<AllHTMLAttributes<HTMLDivElement>, 'is'>;
|
||||
|
||||
const VideoConfMessage = (props: VideoConfMessageProps) => (
|
||||
<Box mbs='x4' maxWidth='345px' borderWidth={2} borderColor='neutral-200' borderRadius='x4' {...props} />
|
||||
);
|
||||
```
|
||||
|
||||
### Provide hooks as helpers
|
||||
|
||||
In the following example, the `useVideoConfControllers` hook is provided as a helper to manage the state of the popup's controllers:
|
||||
|
||||
```ts
|
||||
export const useVideoConfControllers = (
|
||||
initialPreferences: controllersConfigProps = { mic: true, cam: false },
|
||||
): { controllersConfig: controllersConfigProps; handleToggleMic: () => void; handleToggleCam: () => void } => {
|
||||
const [controllersConfig, setControllersConfig] = useState(initialPreferences);
|
||||
|
||||
const handleToggleMic = useCallback((): void => {
|
||||
setControllersConfig((prevState) => ({ ...prevState, mic: !prevState.mic }));
|
||||
}, []);
|
||||
|
||||
const handleToggleCam = useCallback((): void => {
|
||||
setControllersConfig((prevState) => ({ ...prevState, cam: !prevState.cam }));
|
||||
}, []);
|
||||
|
||||
return {
|
||||
controllersConfig,
|
||||
handleToggleMic,
|
||||
handleToggleCam,
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
```tsx
|
||||
const { controllersConfig } = useVideoConfControllers();
|
||||
|
||||
return (
|
||||
<VideoConfPopup>
|
||||
<VideoConfPopupHeader>
|
||||
<VideoConfPopupTitle text={t('Calling')} counter />
|
||||
<VideoConfPopupControllers>
|
||||
<VideoConfController
|
||||
active={controllersConfig.cam}
|
||||
title={controllersConfig.cam ? t('Cam_on') : t('Cam_off')}
|
||||
icon={controllersConfig.cam ? 'video' : 'video-off'}
|
||||
disabled
|
||||
/>
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
</VideoConfPopupControllers>
|
||||
</VideoConfPopupHeader>
|
||||
</VideoConfPopup>
|
||||
);
|
||||
```
|
||||
|
||||
## Understanding the component and defining the scope
|
||||
|
||||
New components typically emerge from requirements put forth by the Product Design Team. The frontend engineer holds the responsibility of assessing the genuine necessity of such components. Due to the substantial effort involved in creating a new component, it is prudent to collaborate with product managers and designers. It is advisable to explore the feasibility of employing Complex Components as an MVP to validate concepts and user flows. After successful validation, the progression to developing a new Fuselage level component can be considered.
|
||||
|
||||
**How do I know my component should be part of the Fuselage library?**
|
||||
|
||||
Consider the `VerticalBar` component as a clear example. It began as a Complex Component for a single application but has now advanced to the Fuselage level. This shift is driven by its usefulness in multiple applications, like Rocket.Chat and Cloud Portal. This case demonstrates how components can grow from specific solutions to versatile tools with broader applications.
|
||||
|
||||
## Logical components
|
||||
|
||||
### Use child components to compose a logical complex component
|
||||
|
||||
Leverage the integration of child components to construct a unified and logical complex component:
|
||||
|
||||
```tsx
|
||||
const OutgoingPopup = ({ room, onClose, id }: OutgoingPopupProps): ReactElement => {
|
||||
const t = useTranslation();
|
||||
const videoConfPreferences = useVideoConfPreferences();
|
||||
const { controllersConfig } = useVideoConfControllers();
|
||||
|
||||
return (
|
||||
<VideoConfPopup>
|
||||
<VideoConfPopupHeader>
|
||||
<VideoConfPopupTitle text={t('Calling')} counter />
|
||||
<VideoConfPopupControllers>
|
||||
<VideoConfController
|
||||
active={controllersConfig.cam}
|
||||
title={controllersConfig.cam ? t('Cam_on') : t('Cam_off')}
|
||||
icon={controllersConfig.cam ? 'video' : 'video-off'}
|
||||
disabled
|
||||
/>
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
</VideoConfPopupControllers>
|
||||
</VideoConfPopupHeader>
|
||||
</VideoConfPopup>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### Customization through variations
|
||||
|
||||
Provide users with the ability to customize a component's appearance or behavior by selecting from predefined variations or options. This approach enhances user experience and flexibility in adapting components to specific requirements.
|
||||
|
||||
```tsx
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
```
|
||||
|
||||
```tsx
|
||||
const VideoConfController = ({ icon, active, secondary, disabled, small = true, ...props }: VideoConfControllerProps): ReactElement => {
|
||||
const id = useUniqueId();
|
||||
|
||||
return (
|
||||
<IconButton
|
||||
small={small}
|
||||
icon={icon}
|
||||
id={id}
|
||||
info={active}
|
||||
disabled={disabled}
|
||||
secondary={secondary || active || disabled}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
### Avoid direct styles
|
||||
|
||||
Refrain from applying direct styles to components. By avoiding inline styling, the code maintains a cleaner structure and promotes better separation of concerns, enhancing maintainability and readability.
|
||||
|
||||
❌ Avoid:
|
||||
|
||||
```tsx
|
||||
<VideoConfPopup>
|
||||
<VideoConfPopupHeader>
|
||||
<VideoConfPopupTitle text={t('Calling')} counter />
|
||||
<VideoConfPopupControllers>
|
||||
<Box display='flex' alignItems='center'>
|
||||
<VideoConfController
|
||||
width='50px'
|
||||
height='50px'
|
||||
active={controllersConfig.cam}
|
||||
title={controllersConfig.cam ? t('Cam_on') : t('Cam_off')}
|
||||
icon={controllersConfig.cam ? 'video' : 'video-off'}
|
||||
disabled
|
||||
/>
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
</Box>
|
||||
</VideoConfPopupControllers>
|
||||
</VideoConfPopupHeader>
|
||||
</VideoConfPopup>
|
||||
```
|
||||
|
||||
### Don't write CSS styles in JS files
|
||||
|
||||
This approach separates your component's logic from styling, promoting better code organization and maintainability while avoiding inline CSS-in-JS styling.
|
||||
|
||||
Define your custom styling in an external CSS file:
|
||||
|
||||
```css
|
||||
/* styles.css */
|
||||
|
||||
.customClass {
|
||||
border: 1px solid black;
|
||||
padding: 1.5rem;
|
||||
}
|
||||
```
|
||||
|
||||
Then, apply the class to your component:
|
||||
|
||||
```tsx
|
||||
import './styles.css';
|
||||
|
||||
return (
|
||||
<VideoConfPopup>
|
||||
<VideoConfPopupHeader>
|
||||
<VideoConfPopupTitle text={t('Calling')} counter />
|
||||
<VideoConfPopupControllers>
|
||||
<VideoConfController
|
||||
className='customClass'
|
||||
active={controllersConfig.cam}
|
||||
title={controllersConfig.cam ? t('Cam_on') : t('Cam_off')}
|
||||
icon={controllersConfig.cam ? 'video' : 'video-off'}
|
||||
disabled
|
||||
/>
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
</VideoConfPopupControllers>
|
||||
</VideoConfPopupHeader>
|
||||
</VideoConfPopup>
|
||||
);
|
||||
```
|
||||
|
||||
### Use the states of the component
|
||||
|
||||
By using component states to conditionally render different complex components, you maintain a clear and organized structure in your code, enhancing readability and maintainability.
|
||||
|
||||
```tsx
|
||||
if (isReceiving) {
|
||||
return <IncomingPopup room={room} id={id} position={position} onClose={onClose} onMute={handleMute} onConfirm={handleConfirm} />;
|
||||
}
|
||||
|
||||
if (isCalling) {
|
||||
return <OutgoingPopup room={room} id={id} onClose={onClose} />;
|
||||
}
|
||||
|
||||
return <StartCallPopup loading={starting} room={room} id={id} onClose={dismissOutgoing} onConfirm={handleStartCall} />;
|
||||
```
|
||||
|
||||
Each state should render the proper Complex Component:
|
||||
|
||||
```tsx
|
||||
const OutgoingPopup = ({ room, onClose, id }: OutgoingPopupProps): ReactElement => {
|
||||
const t = useTranslation();
|
||||
const videoConfPreferences = useVideoConfPreferences();
|
||||
const { controllersConfig } = useVideoConfControllers();
|
||||
|
||||
return (
|
||||
<VideoConfPopup>
|
||||
<VideoConfPopupHeader>
|
||||
<VideoConfPopupTitle text={t('Calling')} counter />
|
||||
<VideoConfPopupControllers>
|
||||
<VideoConfController
|
||||
active={controllersConfig.cam}
|
||||
title={controllersConfig.cam ? t('Cam_on') : t('Cam_off')}
|
||||
icon={controllersConfig.cam ? 'video' : 'video-off'}
|
||||
disabled
|
||||
/>
|
||||
<VideoConfController
|
||||
active={controllersConfig.mic}
|
||||
title={controllersConfig.mic ? t('Mic_on') : t('Mic_off')}
|
||||
icon={controllersConfig.mic ? 'mic' : 'mic-off'}
|
||||
disabled
|
||||
/>
|
||||
</VideoConfPopupControllers>
|
||||
</VideoConfPopupHeader>
|
||||
</VideoConfPopup>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
## Visual components
|
||||
|
||||
Visual components are responsible for the appearance of a UI element. They define the element's style, layout, and other visual properties.
|
||||
|
||||
Adhering to the guidelines in Fuselage's componentization offers the value of modular, reusable, and maintainable UI components. This approach enables efficient development, ensures consistent behavior, and supports the evolution of solutions from specific contexts to broader applications. By encapsulating logic, avoiding direct styles, and leveraging API-driven customization, developers can create a streamlined and user-centered experience.
|
||||
94
docs/frontend/i18n.md
Normal file
94
docs/frontend/i18n.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Translations (i18n) in the client
|
||||
|
||||
Client-side conventions only. How keys are stored, named, and written — naming, interpolation, plurals, formatters, what the linter enforces — is in [docs/i18n.md](../i18n.md), and applies here too.
|
||||
|
||||
## Reading translations
|
||||
|
||||
Use `useTranslation` from `react-i18next`:
|
||||
|
||||
```tsx
|
||||
import { useTranslation } from 'react-i18next';
|
||||
|
||||
const DeleteRoomButton = () => {
|
||||
const { t } = useTranslation();
|
||||
|
||||
return <Button danger>{t('Delete_room')}</Button>;
|
||||
};
|
||||
```
|
||||
|
||||
❌ Not `useTranslation` from `@rocket.chat/ui-contexts` — it is `@deprecated` and re-renders every consumer of the translation context on language change. ~125 client modules still import it against ~890 for the `react-i18next` one; don't add to the count.
|
||||
|
||||
[`TranslationProvider`](../../apps/meteor/client/providers/TranslationProvider.tsx) bundles `en` statically and sets it as `fallbackLng`, so English keys resolve immediately. Only a non-English active locale is fetched over HTTP (`partialBundledLanguages: true`); until it arrives, and for any key it omits, the English string is used. The server, by contrast, holds all 68 locales in memory.
|
||||
|
||||
## Never build keys at runtime
|
||||
|
||||
`t()` does not reject unknown keys (see [docs/i18n.md](../i18n.md#types-are-generated-from-en)), so a computed key has nothing checking it — and it makes the key invisible to anyone grepping for it.
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```tsx
|
||||
{t(`App_status_${instance.status}` as TranslationKey)}
|
||||
```
|
||||
|
||||
✅ Correct — map the value to a literal key:
|
||||
|
||||
```tsx
|
||||
const APP_STATUS_LABELS = {
|
||||
installed: 'App_status_installed',
|
||||
updating: 'App_status_updating',
|
||||
} as const satisfies Record<AppStatus, RocketchatI18nKeys>;
|
||||
|
||||
{t(APP_STATUS_LABELS[instance.status])}
|
||||
```
|
||||
|
||||
The map is the thing that tells you a key is missing when a new status is added, and `satisfies` is what makes the map itself checkable. `as TranslationKey` appears about 70 times in the client; treat every one as debt rather than precedent.
|
||||
|
||||
## Rich text: the `Trans` component
|
||||
|
||||
When a sentence contains a link, bold run, or nested element, don't split it into several keys. Use `Trans` and put the markup in the translation:
|
||||
|
||||
```json
|
||||
{
|
||||
"Unique_ID_change_detected_learn_more_link": "Read the <a>documentation</a> before proceeding."
|
||||
}
|
||||
```
|
||||
|
||||
```tsx
|
||||
import { Trans } from 'react-i18next';
|
||||
|
||||
<Trans i18nKey='Unique_ID_change_detected_learn_more_link' components={{ a: <ExternalLink to={links.go.fingerPrintChangedFaq} /> }} />;
|
||||
```
|
||||
|
||||
Name your tags. Indexed placeholders (`<1>`, `<3>`) appear in older keys and are effectively unreadable for a translator, who can't tell what index 3 wraps:
|
||||
|
||||
❌ Avoid:
|
||||
|
||||
```json
|
||||
{ "Limits_reached": "Your workspace reached the <1>{{val}}</1> limit. <3>Manage subscription</3>." }
|
||||
```
|
||||
|
||||
✅ Prefer:
|
||||
|
||||
```json
|
||||
{ "Limits_reached": "Your workspace reached the <bold>{{val}}</bold> limit. <link>Manage subscription</link>." }
|
||||
```
|
||||
|
||||
The base locale is currently split about evenly between the two (~50 values each), so this is a direction, not an established majority.
|
||||
|
||||
Pass interpolation values through `values` and `t` options through `tOptions`:
|
||||
|
||||
```tsx
|
||||
<Trans i18nKey='Airgapped_workspace_warning' values={{ remainingDays }} />
|
||||
```
|
||||
|
||||
## Interpolated values are not HTML-escaped
|
||||
|
||||
`escapeValue` is `false`. That is safe only because React escapes strings when it renders them as text nodes — it is not a property of the translation itself.
|
||||
|
||||
So if a translated string is injected as HTML, sanitize it, as the two existing call sites do:
|
||||
|
||||
```tsx
|
||||
<FieldHint dangerouslySetInnerHTML={{ __html: DOMPurify.sanitize(t(provider.description)) }} />
|
||||
```
|
||||
|
||||
Prefer `Trans` over injected HTML wherever possible — it needs no sanitizing because the components are real React elements, not parsed markup.
|
||||
61
docs/frontend/migrating-from-javascript.md
Normal file
61
docs/frontend/migrating-from-javascript.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# Migrating from JavaScript
|
||||
|
||||
## TypeScript is a superset of JavaScript
|
||||
|
||||
TypeScript is an extension of JavaScript, meaning that when transitioning from JavaScript to TypeScript, you can employ the identical syntax used in JavaScript. Errors flagged by the TypeScript compiler (`tsc`) and ESLint enforce best practices, and both gate CI — [`ci-code-check`](../../.github/workflows/ci-code-check.yml) runs `turbo run typecheck` and `yarn lint` — so resolve them rather than leaving them in place.
|
||||
|
||||
When a diagnostic is genuinely wrong about the code, suppress it explicitly and narrowly:
|
||||
|
||||
- `@ts-expect-error` with a short reason, rather than `@ts-ignore` — it fails once the underlying error is gone, so the suppression can't outlive its cause;
|
||||
- `eslint-disable-next-line`, naming the specific rule rather than disabling the line wholesale.
|
||||
|
||||
## JSDoc
|
||||
|
||||
When `allowJs` is enabled in `tsconfig.json`, JSDoc comments can document types within JavaScript code. This is particularly useful during gradual migration when `tsc` struggles with type inference.
|
||||
|
||||
Example with `@typedef`:
|
||||
|
||||
```js
|
||||
// module.js
|
||||
|
||||
/**
|
||||
* @typedef {Object} Foo
|
||||
* @property {string} bar
|
||||
* @property {string} [qux]
|
||||
*/
|
||||
|
||||
/** @type {Foo} */
|
||||
export const foo = { bar: 'baz' };
|
||||
|
||||
foo.qux = 'quux';
|
||||
```
|
||||
|
||||
Alternative using the `@type` tag:
|
||||
|
||||
```js
|
||||
// module.js
|
||||
|
||||
/**
|
||||
* @type {{ bar: string; qux?: string }}
|
||||
*/
|
||||
export const foo = { bar: 'baz' };
|
||||
|
||||
foo.qux = 'quux';
|
||||
```
|
||||
|
||||
Both approaches help ensure TypeScript accurately recognizes the complete type structure.
|
||||
|
||||
## Declare a `*.d.ts` file
|
||||
|
||||
Creating a `.d.ts` declaration file is strongly recommended when migrating large JavaScript modules. It describes the module's public surface for type checking and is easier to read as a whole than JSDoc scattered through the implementation.
|
||||
|
||||
A declaration file is types only: it is erased at build time and creates no runtime code. Place it alongside the JavaScript module it describes, with the same basename, and declare only exports that module actually has — TypeScript will trust the declaration over the `.js`, so anything you declare that isn't really exported will typecheck at the import site and be `undefined` at runtime.
|
||||
|
||||
```ts
|
||||
// hugeModule.d.ts — sits next to hugeModule.js
|
||||
|
||||
export function foo(): void; // hugeModule.js exports `foo`
|
||||
export function bar(): void; // hugeModule.js exports `bar`
|
||||
```
|
||||
|
||||
Writing the file is still a good way to plan the shape you're migrating toward, but it only records exports; it can't create or relocate them. If an export belongs in another module, move the implementation first, then declare it beside its new home.
|
||||
258
docs/frontend/react.md
Normal file
258
docs/frontend/react.md
Normal file
@@ -0,0 +1,258 @@
|
||||
# React
|
||||
|
||||
> Most of the recommendations here are based on Alex Kondov's [Tao of React](https://alexkondov.com/tao-of-react/).
|
||||
|
||||
## Prefer functional components
|
||||
|
||||
React initially introduced class components to leverage JavaScript class syntax for managing state and component lifecycles. However, class components have significant drawbacks:
|
||||
|
||||
- they tend to be verbose;
|
||||
- they often involve a misuse of the inheritance mechanism through `extends` and `super`.
|
||||
|
||||
Hooks were introduced to provide an alternative approach to declaring state and effects. They maintain the core concept of the component's render function without the need for classes, streamlining the development process.
|
||||
|
||||
## Declare one component per file
|
||||
|
||||
While colocation is a commendable concept, it's not consistently followed for React components within a single file. The main reason is that we've noticed people misusing this approach before. It might start with something as straightforward as adding a basic modal component alongside a page component, but it can quickly lead to a confusing jumble of components that becomes difficult to manage.
|
||||
|
||||
## Name components
|
||||
|
||||
Failing to name a component is a common mistake that can lead to prolonged debugging efforts. It results in less informative error stacks and challenges while navigating components in React Dev Tools. There are two approaches to properly name a component:
|
||||
|
||||
1. By assigning the function straight to a binding. The arrow function is itself anonymous, but assigning it to a `const` makes JavaScript infer the name from the variable:
|
||||
|
||||
```jsx
|
||||
const Foo = () => {
|
||||
return <div>Foo</div>;
|
||||
};
|
||||
|
||||
console.log('The component name is:', Foo.name);
|
||||
```
|
||||
|
||||
2. By setting the `displayName` property. Once a wrapper call sits between the binding and the function, there is nothing for inference to attach to — the arrow is an argument to `memo`, and `memo` returns an object rather than a function:
|
||||
|
||||
```jsx
|
||||
const Foo = memo(() => {
|
||||
return <div>Foo</div>;
|
||||
});
|
||||
|
||||
Foo.displayName = 'Foo'; // `Foo.name` is `undefined`
|
||||
|
||||
console.log('The component name is:', Foo.displayName);
|
||||
```
|
||||
|
||||
## Use default export at the end of file
|
||||
|
||||
While named exports are often preferred, using default export enhances code readability, especially when dealing with Higher Order Components (HOCs) like `memo` and `forwardRef`, and it aligns neatly with code splitting using `lazy`.
|
||||
|
||||
```tsx
|
||||
// Component.tsx
|
||||
import { memo } from 'react';
|
||||
|
||||
type ComponentProps = {
|
||||
name: string;
|
||||
};
|
||||
|
||||
// Anonymous arrow, but the const assignment infers the name "Component"
|
||||
const Component = (props: ComponentProps) => {
|
||||
return <div>Hello, {props.name}</div>;
|
||||
};
|
||||
|
||||
export default memo(Component); // the component name is preserved
|
||||
```
|
||||
|
||||
```ts
|
||||
// index.ts
|
||||
import { lazy } from 'react';
|
||||
|
||||
const Component = lazy(() => import('./Component'));
|
||||
```
|
||||
|
||||
The same example based on named exports is less readable:
|
||||
|
||||
```tsx
|
||||
// Component.tsx
|
||||
import { memo } from 'react';
|
||||
|
||||
type ComponentProps = {
|
||||
name: string;
|
||||
};
|
||||
|
||||
// Anonymous arrow in an argument position — no name is inferred
|
||||
export const Component = memo((props: ComponentProps) => {
|
||||
return <div>Hello, {props.name}</div>;
|
||||
});
|
||||
|
||||
Component.displayName = 'Component'; // needed for React Dev Tools
|
||||
```
|
||||
|
||||
```ts
|
||||
// index.ts
|
||||
import { lazy } from 'react';
|
||||
|
||||
const Component = lazy(async () => {
|
||||
const { Component } = await import('./Component');
|
||||
return { default: Component }; // you need to reconstruct the default export
|
||||
});
|
||||
```
|
||||
|
||||
## Extract helper functions
|
||||
|
||||
A drawback associated with the adoption of React Hooks is the tendency for individuals to define helper functions within the component. This is facilitated by the fact that there is no requirement to pass arguments; instead, variables from the encompassing scope can be directly accessed and used.
|
||||
|
||||
```jsx
|
||||
const Component = () => {
|
||||
const value = useMyHook();
|
||||
const isValueOK = () => value === 'OK';
|
||||
return isValueOK() ? <>OK</> : null;
|
||||
};
|
||||
```
|
||||
|
||||
Typically, effective helper functions adhere to the principle of being pure, which makes them simpler to debug. However, when you bind variables into the helper's scope, you're making it impure. Furthermore, with each rendering cycle, the function value gets redefined. While this process is efficient in terms of CPU and memory usage, it can lead to scenarios where you must rely on techniques like `useCallback` to prevent unnecessary re-renders of child components that receive your helper function as a prop.
|
||||
|
||||
Here's the ideal case:
|
||||
|
||||
```jsx
|
||||
const isValueOK = (value) => value === 'OK';
|
||||
|
||||
const Component = () => {
|
||||
const value = useMyHook();
|
||||
return isValueOK(value) ? <>OK</> : null;
|
||||
};
|
||||
```
|
||||
|
||||
## Do not refer to the `children` prop explicitly
|
||||
|
||||
Children should always be actual children, not passed in as a prop. Use the `eslint-plugin-react` rule [`react/no-children-prop`](https://github.com/jsx-eslint/eslint-plugin-react/blob/master/docs/rules/no-children-prop.md) as reference.
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```jsx
|
||||
export const MyComponent = ({ children }) => <div children={children} />;
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```jsx
|
||||
export const MyComponent = ({ children }) => <div>{children}</div>;
|
||||
```
|
||||
|
||||
## Declare props type explicitly
|
||||
|
||||
Each component named, for instance, `Component` must have an _exported_ type declaration for `ComponentProps`.
|
||||
|
||||
```tsx
|
||||
export type MyComponentProps = {
|
||||
// ...
|
||||
};
|
||||
|
||||
const MyComponent = (props: MyComponentProps) => /* ... */;
|
||||
|
||||
export default MyComponent;
|
||||
```
|
||||
|
||||
## Use generic types when appropriate
|
||||
|
||||
Generic types are a great way to ensure type safety once they enable us to not fall into the unsoundness of type assertions.
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```tsx
|
||||
// MyComponent.tsx
|
||||
export type MyComponentProps = {
|
||||
value: unknown;
|
||||
onChange: (newValue: unknown) => void;
|
||||
};
|
||||
|
||||
const MyComponent = ({ value, onChange }: MyComponentProps) => /* ... */;
|
||||
|
||||
export default MyComponent;
|
||||
|
||||
// MyApp.tsx
|
||||
import MyComponent from './MyComponent';
|
||||
|
||||
const MyApp = () => {
|
||||
const [value, setValue] = useState('');
|
||||
const handleChange = (value: unknown) => {
|
||||
setValue(value as string); // unsound type assertion, as unknown can be anything
|
||||
};
|
||||
return <MyComponent value={value} onChange={handleChange} />;
|
||||
};
|
||||
|
||||
export default MyApp;
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```tsx
|
||||
// MyComponent.tsx
|
||||
export type MyComponentProps<TValue> = {
|
||||
value: TValue;
|
||||
onChange: (newValue: TValue) => void;
|
||||
};
|
||||
|
||||
const MyComponent = <TValue,>({ value, onChange }: MyComponentProps<TValue>) => /* ... */;
|
||||
|
||||
export default MyComponent;
|
||||
|
||||
// MyApp.tsx
|
||||
import MyComponent from './MyComponent';
|
||||
|
||||
const MyApp = () => {
|
||||
const [value, setValue] = useState('');
|
||||
const handleChange = (value: string) => {
|
||||
setValue(value);
|
||||
};
|
||||
// since `value` is string, `onChange` type will be inferred
|
||||
// as `(value: string) => void`
|
||||
return <MyComponent value={value} onChange={handleChange} />;
|
||||
};
|
||||
|
||||
export default MyApp;
|
||||
```
|
||||
|
||||
## Avoid naming identifiers with their value type/kind
|
||||
|
||||
This is a common programming convention, but it seems easier to violate when working with React concepts. In general, identifiers should not include information about the data structure they refer to.
|
||||
|
||||
```ts
|
||||
// Examples of bad identifiers
|
||||
const amountConst = 123; // instead of "amount"
|
||||
let countNumber = 0; // instead of "count"
|
||||
function logFunction(lineString: string) {
|
||||
// instead of "log" and "line"
|
||||
console.log(lineString);
|
||||
}
|
||||
```
|
||||
|
||||
Identifiers like this mimic the old-fashioned Hungarian notation. In React, this happens quite often:
|
||||
|
||||
```tsx
|
||||
// Examples of bad identifiers in React constructs
|
||||
function useThingHook() {
|
||||
// ...
|
||||
} // instead of "useThing"
|
||||
|
||||
const useThingContext = () => useContext(ThingContext);
|
||||
// "ThingContext" already exposes itself as a context that provides a "Thing".
|
||||
// "useThing" is good enough, since it doesn't need to expose its context dependency.
|
||||
|
||||
const MyComponent = () => <div />;
|
||||
// Fine for code examples and components passed by props*, but in general
|
||||
// capital cased function names used in JSX already communicate they are of
|
||||
// "Component" kind
|
||||
|
||||
// * when "Component" is okay as a suffix
|
||||
type MyPanelProps = {
|
||||
panelComponent: ElementType<{}>;
|
||||
};
|
||||
|
||||
function MyPanel({ panelComponent: PanelComponent }: MyPanelProps) {
|
||||
// "PanelComponent" is fine here
|
||||
return (
|
||||
<div className='panel'>
|
||||
<PanelComponent />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
```
|
||||
199
docs/frontend/typescript-conventions.md
Normal file
199
docs/frontend/typescript-conventions.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# TypeScript general conventions
|
||||
|
||||
## Don't use CommonJS features
|
||||
|
||||
Avoid using CommonJS features such as `require` and `module` alongside ES module constructs like `import` and `export`. The preference is for ES modules due to their enhanced portability and user-friendly nature. However, keep in mind that synchronous conditional imports aren't possible in ES modules.
|
||||
|
||||
Example of CommonJS (not recommended):
|
||||
|
||||
```ts
|
||||
// commonjs.ts
|
||||
if (condition) {
|
||||
const foo = require('foo');
|
||||
module.exports = foo;
|
||||
} else {
|
||||
module.exports = {};
|
||||
}
|
||||
```
|
||||
|
||||
Invalid ES module pattern:
|
||||
|
||||
```ts
|
||||
// esmodule.ts
|
||||
if (condition) {
|
||||
import foo from 'foo';
|
||||
export default foo;
|
||||
} else {
|
||||
export default {};
|
||||
}
|
||||
```
|
||||
|
||||
## Prefer `import type` over `import`
|
||||
|
||||
Prioritize using `import type` for type-only imports. While a regular `import` works for both JavaScript and TypeScript, it can include module output code in the bundle. The `import type` construct is specific to TypeScript compilation and avoids unnecessary code inclusion.
|
||||
|
||||
```ts
|
||||
// Foo.ts
|
||||
export class Foo {
|
||||
bar: string;
|
||||
|
||||
constructor(bar: string) {
|
||||
this.bar = bar;
|
||||
}
|
||||
}
|
||||
|
||||
// Bar.ts
|
||||
export class Bar {
|
||||
foo: Foo;
|
||||
|
||||
constructor(foo: Foo) {
|
||||
this.foo = foo;
|
||||
}
|
||||
}
|
||||
|
||||
// index.ts
|
||||
import { Foo } from './Foo';
|
||||
import type { Bar } from './Bar';
|
||||
|
||||
export const foo = new Foo('baz');
|
||||
declare const bar: Bar;
|
||||
|
||||
// index.js (transpiled from index.ts)
|
||||
import { Foo } from './Foo';
|
||||
export const foo = new Foo('baz');
|
||||
```
|
||||
|
||||
`Foo` is constructed, so its import is emitted; `Bar` is only ever a type, so `import type { Bar }` disappears entirely.
|
||||
|
||||
Note that TypeScript also elides a plain `import` whose bindings are only used in type positions, so writing `import` where `import type` belongs is usually not what bloats the output. `import type` is preferred because it states the intent, fails loudly if someone later uses the binding as a value, and is what tooling that transpiles file-by-file — `isolatedModules`, `verbatimModuleSyntax`, esbuild, SWC — needs in order to strip the import without consulting the type checker.
|
||||
|
||||
## Know the difference between `type` and `interface`
|
||||
|
||||
An `interface` serves as a type declaration similar to a class construct, while a `class` functions as an actual class declaration.
|
||||
|
||||
```ts
|
||||
interface IThing {
|
||||
prop: string;
|
||||
method(): void;
|
||||
}
|
||||
|
||||
class Thing implements IThing {
|
||||
public prop = 'foo';
|
||||
|
||||
method(): void {
|
||||
console.log('bar');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A `type` construct is similar to an interface but offers greater flexibility. The `type` construct permits declaration of union types:
|
||||
|
||||
```ts
|
||||
type Foo = string | number;
|
||||
```
|
||||
|
||||
An `interface` can declare generic types:
|
||||
|
||||
```ts
|
||||
interface IFoo<T> {
|
||||
prop: T;
|
||||
}
|
||||
```
|
||||
|
||||
A `type` also supports generics and conditional types, providing more adaptability:
|
||||
|
||||
```ts
|
||||
type Foo<T> = T extends string ? { foo: number } : { bar: number };
|
||||
```
|
||||
|
||||
## Avoid using classes as namespaces
|
||||
|
||||
Pattern to avoid:
|
||||
|
||||
```ts
|
||||
// foo.ts
|
||||
class Foo {
|
||||
bar(): void {
|
||||
// ...
|
||||
}
|
||||
}
|
||||
|
||||
export const foo = new Foo();
|
||||
|
||||
// index.ts
|
||||
import { foo } from './foo';
|
||||
|
||||
foo.bar();
|
||||
```
|
||||
|
||||
Preferred approach (when no state is needed):
|
||||
|
||||
```ts
|
||||
// foo.ts
|
||||
export function bar(): void {
|
||||
// ...
|
||||
}
|
||||
|
||||
// index.ts
|
||||
import * as foo from './foo';
|
||||
|
||||
foo.bar();
|
||||
```
|
||||
|
||||
Valid use case (when managing state):
|
||||
|
||||
```ts
|
||||
// foo.ts
|
||||
class Foo {
|
||||
baz: number;
|
||||
|
||||
bar(): void {
|
||||
// perform actions referencing and modifying `this.baz`
|
||||
}
|
||||
}
|
||||
|
||||
export const foo = new Foo();
|
||||
|
||||
// index.ts
|
||||
import { foo } from './foo';
|
||||
|
||||
foo.bar();
|
||||
```
|
||||
|
||||
Classes are reasonable when they encapsulate state and provide a controlled interface for modification.
|
||||
|
||||
## Avoid using the `any` type except when it's used as a constraint
|
||||
|
||||
Refrain from using `any` under most circumstances. Instead, follow this model:
|
||||
|
||||
- `unknown` serves as the universal type, encompassing all potential values;
|
||||
- `any` should not be seen as an actual type, but rather as a means to disable TypeScript's type checking.
|
||||
|
||||
Using `any` is discouraged as it indicates unawareness regarding the type being manipulated. Working with `unknown` necessitates type narrowing, leading to more robust code.
|
||||
|
||||
```ts
|
||||
// Avoid using any
|
||||
declare const foo: any;
|
||||
foo.bar(); // No compilation error
|
||||
|
||||
// Prefer using unknown
|
||||
declare const bar: unknown;
|
||||
bar.baz(); // Compilation error
|
||||
|
||||
const hasBaz = (bar: unknown): bar is { baz(): void } =>
|
||||
typeof bar === 'object' && bar !== null && 'baz' in bar && typeof (bar as { baz: unknown }).baz === 'function';
|
||||
|
||||
if (hasBaz(bar)) {
|
||||
bar.baz(); // No compilation error
|
||||
}
|
||||
```
|
||||
|
||||
Exception — generic type constraints:
|
||||
|
||||
```ts
|
||||
type X<F> = F extends (x: unknown) => void ? true : false;
|
||||
type Y<F> = F extends (x: any) => void ? true : false;
|
||||
|
||||
type A = X<(x: string) => void>; // `false`, because x is not `unknown`
|
||||
type B = Y<(x: string) => void>; // `true`, because x is anything
|
||||
```
|
||||
237
docs/i18n.md
Normal file
237
docs/i18n.md
Normal file
@@ -0,0 +1,237 @@
|
||||
# Translations (i18n)
|
||||
|
||||
How translation keys are stored, named, and written. This applies to every runtime that consumes `@rocket.chat/i18n`.
|
||||
|
||||
Client-specific guidance — hooks, the `Trans` component, escaping — lives in [docs/frontend/i18n.md](frontend/i18n.md).
|
||||
|
||||
## Where translations live
|
||||
|
||||
All translations are flat JSON files in [`packages/i18n/src/locales/`](../packages/i18n/src/locales/), one per language, named `<language>.i18n.json` (68 locales at the time of writing).
|
||||
|
||||
`en.i18n.json` is the **base language**. It is the only file you should edit when adding a feature:
|
||||
|
||||
- keys absent from a locale fall back to `en` (`fallbackLng: 'en'` in every runtime);
|
||||
- keys present in a locale but *not* in `en` are treated as stale and removed by the linter (`wipe-extra-keys`);
|
||||
- key order in every other locale is derived from `en`, so reordering `en` reorders all of them.
|
||||
|
||||
Translations for other languages arrive separately. Don't hand-write them for your own feature.
|
||||
|
||||
### Types are generated from `en`
|
||||
|
||||
`packages/i18n/src/resources.ts` is a checked-in dummy. The real key union is generated into `dist/resources.d.ts` from `en.i18n.json` at build time. At the time of writing it doesn't restrict `t`'s signature — the declaration adds overloads to i18next's `TFunction` rather than narrowing it, and narrowing was avoided for typecheck performance. **A misspelled key is not a compile error**, so verify keys by grepping the base locale.
|
||||
|
||||
A key you just added won't appear in the generated types until the package is rebuilt:
|
||||
|
||||
```sh
|
||||
yarn workspace @rocket.chat/i18n build
|
||||
```
|
||||
|
||||
## Naming keys
|
||||
|
||||
The dominant convention is capitalized snake case — `Sentence_case_with_underscores`:
|
||||
|
||||
```json
|
||||
{
|
||||
"Cam_on": "Camera on",
|
||||
"Delete_room": "Delete room",
|
||||
"You_are_offline_please_reconnect": "You are offline, please reconnect"
|
||||
}
|
||||
```
|
||||
|
||||
Write the key to describe the *meaning*, not the value, and not where it is rendered. `Delete_room` survives a copy change; `Delete_room_red_button` does not.
|
||||
|
||||
Reuse an existing key when the meaning is identical. Do not reuse one because the English happens to match — languages that inflect by context will need them separated, and splitting a shared key later is a breaking change for every locale.
|
||||
|
||||
### Namespaces
|
||||
|
||||
A key may be prefixed with one of exactly five namespaces, separated by a dot (`nsSeparator: '.'`):
|
||||
|
||||
`core` (the default) · `onboarding` · `registration` · `cloud` · `subscription`
|
||||
|
||||
```json
|
||||
{
|
||||
"onboarding.component.form.action.next": "Next",
|
||||
"subscription.callout.title.limitsReached": "Limits reached"
|
||||
}
|
||||
```
|
||||
|
||||
An unprefixed key lands in `core`. Namespaces exist so a client can load a subset of the bundle, not as general grouping. The set is defined in [`packages/i18n/src/index.ts`](../packages/i18n/src/index.ts). Within a namespace, keys use lowercase dotted paths rather than snake case, following the existing entries.
|
||||
|
||||
## Interpolation
|
||||
|
||||
Use i18next named placeholders, `{{likeThis}}`, with camelCase names:
|
||||
|
||||
```json
|
||||
{
|
||||
"Room_removed": "Room {{roomName}} removed from ABAC management"
|
||||
}
|
||||
```
|
||||
|
||||
Two obsolete forms still exist in the base locale and must not be copied:
|
||||
|
||||
| Form | Status |
|
||||
| ---- | ------ |
|
||||
| `{{name}}` | ✅ correct |
|
||||
| `__name__` | ❌ auto-rewritten by the linter (`replace-2-underscores`) |
|
||||
| `%s` | ❌ legacy sprintf, positional; ~75 remaining, tracked by `find-sprintf-params` |
|
||||
|
||||
The sprintf form still works at runtime — both the client and the Meteor server install `i18next-sprintf-postprocessor` and wrap `t` with `addSprinfToI18n` — but it is positional, so a translator reordering a sentence silently swaps the values. Don't add new ones.
|
||||
|
||||
Some *key names* also embed the old marker — `Added__username__to_team`, `__count__result_found`. That's cosmetic legacy in the name only; the values use `{{...}}`. Don't imitate it in new keys.
|
||||
|
||||
### Never assemble a sentence from fragments
|
||||
|
||||
Word order is not universal, and a translator only sees the pieces:
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```ts
|
||||
`${t('Deleted')} ${count} ${t('messages')}`;
|
||||
```
|
||||
|
||||
✅ Correct — one key holds the whole sentence:
|
||||
|
||||
```ts
|
||||
t('Messages_deleted', { count });
|
||||
```
|
||||
|
||||
A key carrying a count needs plural forms too, so `Messages_deleted` would be defined as an object — see [Pluralization](#pluralization).
|
||||
|
||||
Composing a `Label: value` pair from a label key and a runtime value is fine; splitting prose across keys is not.
|
||||
|
||||
### Formatters
|
||||
|
||||
Placeholders accept an i18next formatter after a comma. The Intl-backed built-ins work in every runtime:
|
||||
|
||||
```json
|
||||
{
|
||||
"Exceeded_limits": "Your workspace exceeded the {{val, list}} license limits.",
|
||||
"Seats_used": "{{count, number}} seats used"
|
||||
}
|
||||
```
|
||||
|
||||
One custom formatter, `capitalize`, is registered **only in the client** ([`TranslationProvider`](../apps/meteor/client/providers/TranslationProvider.tsx)). It exists so that a language needing a different word order can capitalize whichever word ends up first, inside the translation file rather than in code. No base key uses it yet. Don't reach for it in a key the server also renders — there it would pass through unformatted.
|
||||
|
||||
## Pluralization
|
||||
|
||||
Give the key an object of plural forms and pass `count`. i18next picks the form using the locale's CLDR rules:
|
||||
|
||||
```json
|
||||
{
|
||||
"message_counter": {
|
||||
"one": "{{count}} message",
|
||||
"other": "{{count}} messages"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For `en` that means `one` and `other`. Other locales have different sets — Arabic has six — which is exactly why this must not be hand-rolled:
|
||||
|
||||
❌ Incorrect:
|
||||
|
||||
```ts
|
||||
count === 1 ? t('message_counter_one') : t('message_counter_other');
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```ts
|
||||
t('message_counter', { count });
|
||||
```
|
||||
|
||||
i18next also honours a special `zero` form, used when a dedicated empty-state phrasing reads better than "0 items":
|
||||
|
||||
```json
|
||||
{
|
||||
"Calls_in_queue": {
|
||||
"zero": "Queue is empty",
|
||||
"one": "{{count}} call in queue",
|
||||
"other": "{{count}} calls in queue"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Only add `zero` when the wording genuinely differs; `other` already covers 0 in English.
|
||||
|
||||
Plural forms are validated per language: forms outside that language's CLDR set are stripped (`wipe-invalid-plurals`), and a locale missing a form that `en` defines is reported (`find-missing-plurals`).
|
||||
|
||||
## Server-side usage
|
||||
|
||||
Three runtimes initialise the shared resources independently:
|
||||
|
||||
| Runtime | Init |
|
||||
| ------- | ---- |
|
||||
| Client | [`client/providers/TranslationProvider.tsx`](../apps/meteor/client/providers/TranslationProvider.tsx) — `en` bundled, a non-English active locale fetched over HTTP |
|
||||
| Meteor server | [`server/lib/i18n.ts`](../apps/meteor/server/lib/i18n.ts) — eager, all 68 locales in memory |
|
||||
| `omnichannel-transcript` service | [`ee/apps/omnichannel-transcript/src/i18n.ts`](../ee/apps/omnichannel-transcript/src/i18n.ts) — same eager shape |
|
||||
|
||||
On the server, import the instance rather than creating one:
|
||||
|
||||
```ts
|
||||
import { i18n } from '../../app/utils/lib/i18n';
|
||||
```
|
||||
|
||||
### Always pass `lng`
|
||||
|
||||
> This actually exposes a gap in the server-side i18n design.
|
||||
|
||||
The server instance is initialised with `lng: 'en'` and there is no per-request language context. Omitting `lng` is not an error — it silently returns English. Only about a third of the ~200 server call sites pass it, so the surrounding code is not a reliable guide here.
|
||||
|
||||
❌ Incorrect — English regardless of who receives it:
|
||||
|
||||
```ts
|
||||
i18n.t('Username_and_message_must_not_be_empty');
|
||||
```
|
||||
|
||||
✅ Correct:
|
||||
|
||||
```ts
|
||||
i18n.t('Username_and_message_must_not_be_empty', { lng: user.language || settings.get('Language') || 'en' });
|
||||
```
|
||||
|
||||
That fallback chain — recipient's language, then the workspace `Language` setting, then `en` — is the established pattern. There is no shared helper for it yet, so it is written out at each call site.
|
||||
|
||||
Pick the language of whoever **reads** the string, which is not always the acting user: a notification, an email, or an export is rendered for its recipient.
|
||||
|
||||
### Don't translate at the API boundary
|
||||
|
||||
Prefer returning a key from an endpoint and translating in the client, which is what most endpoints already do. The client knows the reader's language; the server has to be told.
|
||||
|
||||
## Separate systems
|
||||
|
||||
[`packages/livechat`](../packages/livechat/) has its **own** translations in `src/i18n/`, unrelated to `@rocket.chat/i18n`: 59 locales, plain `<language>.json`, nested under a single `translation` root key, `lower_snake_case` keys, and plurals as `_one`/`_other` key suffixes instead of nested objects.
|
||||
|
||||
None of the rules on this page — the linter included — apply there. Don't copy conventions in either direction.
|
||||
|
||||
## What the linter enforces
|
||||
|
||||
`yarn workspace @rocket.chat/i18n lint` runs ESLint plus [`src/scripts/check.mts`](../packages/i18n/src/scripts/check.mts). Most findings are auto-fixable with `lint:fix`.
|
||||
|
||||
| Task | Rule |
|
||||
| ---- | ---- |
|
||||
| `sort-base-keys` | `en` keys sorted alphabetically, case-insensitive |
|
||||
| `sort-keys` | every locale follows `en`'s key order |
|
||||
| `wipe-extra-keys` | locales may not hold keys absent from `en` |
|
||||
| `wipe-invalid-plurals` | plural forms must be valid for that language (plus `zero`) |
|
||||
| `find-missing-plurals` | a locale must define every plural form `en` defines |
|
||||
| `replace-2-underscores` | `__name__` → `{{name}}` |
|
||||
| `missing-placeholders` / `extra-placeholders` | placeholders must match `en` exactly |
|
||||
| `find-duplicate-keys` | no duplicate JSON keys |
|
||||
| `trim-eof` | no trailing whitespace at end of file |
|
||||
|
||||
`sort-base-keys` runs before `sort-keys`, since every other locale's order is derived from `en`. A new key can go anywhere in `en`; `lint:fix` moves it into place and reorders the other 67 files to match.
|
||||
|
||||
Two details of the order are not alphabetical, and both are forced by JavaScript rather than chosen:
|
||||
|
||||
- integer-like keys come first (`"500"`), because `JSON.parse` hoists them to the front of the object no matter where the file puts them;
|
||||
- keys differing only in case (`Private` / `private`, 69 such pairs) tie under the case-insensitive comparison and are broken by a plain code-point comparison, so the order stays canonical.
|
||||
|
||||
One task is defined but excluded from the default run, so it won't fail your build: `find-sprintf-params`, which flags the leftover `%s` values in `en` (6 keys today). It is excluded because of an existing backlog, and shouldn't be "fixed" as a side effect of a feature PR. You can inspect it without changing anything:
|
||||
|
||||
```sh
|
||||
cd packages/i18n && node --experimental-transform-types ./src/scripts/check.mts -t find-sprintf-params
|
||||
```
|
||||
|
||||
## Committing
|
||||
|
||||
Translation-only changes use the `i18n:` commit type, per the [pull request template](../.github/PULL_REQUEST_TEMPLATE.md).
|
||||
34
docs/meteor-modern-stack.md
Normal file
34
docs/meteor-modern-stack.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# Meteor's Modern Build Stack
|
||||
|
||||
We now use Meteor's [Modern Build Stack](https://docs.meteor.com/about/modern-build-stack.html) to build and run the project. This is a set of components that improve the building process as a whole, with better speed and new features and plugin support.
|
||||
|
||||
## Issues with File Watching
|
||||
|
||||
The modern stack uses [@parcel/watcher](https://github.com/parcel-bundler/watcher) to watch for file changes. This is a module that supports multiple watcher backends and will use the best option available, following this priority order:
|
||||
|
||||
- `FSEvents` on macOS
|
||||
- `Watchman` if installed
|
||||
- `inotify` on Linux
|
||||
- `ReadDirectoryChangesW` on Windows
|
||||
- `kqueue` on FreeBSD, or as an alternative to FSEvents on macOS
|
||||
|
||||
However, Watchman seems to [have a problem with Meteor's file structure](https://forums.meteor.com/t/modern-watcher-had-failed-to-start-watcher-for-error-message/63830) and it might not work properly.
|
||||
|
||||
If you ever had to build our ReactNative app, you likely have watchman installed on your computers. And so, if you're not using a Mac, the modern stack may fail to build and/or watch files. In this case, your options are:
|
||||
|
||||
1. Uninstalling watchman
|
||||
2. Disabling Meteor's modern watcher on the package.json file:
|
||||
|
||||
```
|
||||
"meteor": {
|
||||
"modern": {
|
||||
"watcher": false
|
||||
}
|
||||
},
|
||||
```
|
||||
|
||||
### Additional Issues with TurboRepo
|
||||
|
||||
TurboRepo has an issue where the subprocesses it spawns are not always terminated along with the parent process. You may have noticed this before, as you often need to hit Ctrl+C twice to completely shutdown all processes. When Watchman is used as the watcher backend, this TurboRepo issue is aggravated and you now need to kill the watchman and node processes manually any time you want to stop the Rocket.Chat server.
|
||||
|
||||
TurboRepo has [just recently released](https://github.com/vercel/turborepo/pull/12607) an improvement related to this, but we have not updated to it yet.
|
||||
144
docs/npm-publishing.md
Normal file
144
docs/npm-publishing.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# npm publishing
|
||||
|
||||
The public `@rocket.chat/*` packages under `packages/` are published to npm by
|
||||
`.github/workflows/release.yml`, which delegates to the local `packages/release-action`.
|
||||
Everything else in the workspace is `private: true` and never published.
|
||||
|
||||
## How a publish happens
|
||||
|
||||
`release.yml` is the **only** workflow that publishes. It carries three triggers and maps each one
|
||||
to a `release-action` mode:
|
||||
|
||||
| Trigger | `action` | Publishes? | dist-tag |
|
||||
| ----------------------------- | --------------- | ---------- | --------------- |
|
||||
| `push` to `master` | `publish-final` | yes | `latest` |
|
||||
| `schedule` (monthly, 20th) | `next` | yes | `rc` |
|
||||
| `workflow_dispatch` → `cut` | `cut` | yes | `latest` / `rc` |
|
||||
| `workflow_dispatch` → `next` | `next` | yes | `rc` |
|
||||
| `workflow_dispatch` → `patch` | `patch` | no | — |
|
||||
|
||||
`publishRelease.ts` and `bumpNextVersion.ts` both end with `yarn changeset publish --no-git-tag`.
|
||||
Changesets then spawns `npm publish <packageDir> --access public --tag <tag>` from the repo root, so
|
||||
**the npm CLI on `PATH` is what authenticates** — not yarn.
|
||||
|
||||
## Trusted publishing (OIDC)
|
||||
|
||||
Publishing authenticates via [npm trusted publishing](https://docs.npmjs.com/trusted-publishers/)
|
||||
rather than a long-lived token. GitHub Actions mints a short-lived, job-scoped credential at publish
|
||||
time, and npm attaches a provenance attestation to every published tarball.
|
||||
|
||||
Two hard requirements, both handled in `release.yml`:
|
||||
|
||||
- `permissions: id-token: write` on the publishing job — without it the runner never exposes
|
||||
`ACTIONS_ID_TOKEN_REQUEST_URL` and the OIDC exchange cannot happen.
|
||||
- **npm >= 11.5.1** (the `Setup npm` step installs it). `engines.node` is 22.x, which bundles npm 10.
|
||||
|
||||
### How `NPM_TOKEN` interacts with the exchange
|
||||
|
||||
`npm publish` runs the OIDC exchange *before* it reads credentials, and on success it overrides the
|
||||
`~/.npmrc` auth line with the freshly minted token. A present `NPM_TOKEN` therefore does **not**
|
||||
prevent trusted publishing — the exchange still wins whenever it succeeds.
|
||||
|
||||
The token matters only when the exchange fails. npm's `oidc()` is written to never throw: every
|
||||
failure path (missing `id-token` permission, package not registered, exchange rejected) returns
|
||||
quietly, and `publish` then proceeds with the npmrc token — no provenance, no error, exit code 0.
|
||||
|
||||
That makes the token a **migration aid and a hazard at the same time**:
|
||||
|
||||
- While packages are being registered one at a time, it keeps releases from hard-failing on the
|
||||
ones that are not registered yet.
|
||||
- Once everything is registered, it hides regressions — a broken registration keeps publishing
|
||||
under token auth instead of failing.
|
||||
|
||||
So `release.yml` keeps `NPM_TOKEN` only until every package publishes with provenance, then drops
|
||||
both references (the `setup-node` input and the `Release` step's `env`).
|
||||
|
||||
### Migrating packages incrementally
|
||||
|
||||
npm allows one trusted publisher per package, so packages move over one at a time:
|
||||
|
||||
1. Register the package (see below). Watch for packages previously released from another repo —
|
||||
`@rocket.chat/emitter`'s last provenance came from `RocketChat/fuselage`'s `cd.yml`, and that
|
||||
registration has to be re-pointed here before this workflow can publish it.
|
||||
2. Let a release run. The job log is not evidence — a token fallback looks identical to success.
|
||||
3. Confirm with `npm view @rocket.chat/<name> --json dist.attestations`; non-null means the package
|
||||
is publishing through OIDC.
|
||||
4. When all published packages are attested, remove `NPM_TOKEN` from `release.yml`.
|
||||
|
||||
### Why there is only one release workflow
|
||||
|
||||
npm allows **one trusted publisher per package**, and it validates the _entry-point_ workflow
|
||||
filename — the `workflow_ref` OIDC claim. `workflow_call` reusable workflows inherit the caller's
|
||||
`workflow_ref`, so splitting the publish step into a shared workflow does not help: every caller
|
||||
would need its own registration. Hence the single `release.yml` with three triggers.
|
||||
|
||||
**Renaming or moving `release.yml` breaks publishing** until every package's trusted publisher is
|
||||
updated on npmjs.com.
|
||||
|
||||
### Registering a newly published package
|
||||
|
||||
npm cannot configure a trusted publisher for a package that does not exist yet, so a brand-new
|
||||
package needs one manual first publish before it can be registered.
|
||||
|
||||
Register from a machine with npm >= 11.15.0 and account-level 2FA (granular tokens with the
|
||||
bypass-2FA option are rejected):
|
||||
|
||||
```sh
|
||||
npm trust github @rocket.chat/<name> \
|
||||
--file release.yml \
|
||||
--repo RocketChat/Rocket.Chat \
|
||||
--allow-publish
|
||||
|
||||
npm trust list @rocket.chat/<name>
|
||||
```
|
||||
|
||||
Or on `npmjs.com/package/@rocket.chat/<name>/access`:
|
||||
|
||||
| Field | Value |
|
||||
| ----------------- | -------------------------------------- |
|
||||
| Provider | GitHub Actions |
|
||||
| Organization | `RocketChat` |
|
||||
| Repository | `Rocket.Chat` |
|
||||
| Workflow filename | `release.yml` (filename only, no path) |
|
||||
| Environment | _(blank)_ |
|
||||
| Allowed actions | `npm publish` |
|
||||
|
||||
No GitHub Environment is configured: the monthly RC cron and the push-to-master final release both
|
||||
run unattended, and an environment with required reviewers would stall them.
|
||||
|
||||
### Provenance requires a correct `repository` field
|
||||
|
||||
npm generates provenance automatically for public packages published from a public repo via OIDC,
|
||||
and **rejects the publish (422) when `repository.url` does not match the repository the workflow ran
|
||||
in**. Every published package therefore needs:
|
||||
|
||||
```json
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/RocketChat/Rocket.Chat.git",
|
||||
"directory": "packages/<name>"
|
||||
}
|
||||
```
|
||||
|
||||
The org/repo casing (`RocketChat/Rocket.Chat`) has to match too.
|
||||
|
||||
Verify after a release:
|
||||
|
||||
```sh
|
||||
npm view @rocket.chat/<name> --json dist.attestations
|
||||
```
|
||||
|
||||
A non-null result is proof the publish went through OIDC — token-based publishes from this repo
|
||||
never carried provenance.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **`E404` / `ENEEDAUTH` on `/-/npm/v1/oidc/token/exchange/package/…`** — almost always a
|
||||
trusted-publisher mismatch: wrong repository, or the workflow filename entered with a path
|
||||
(`.github/workflows/release.yml`) instead of bare `release.yml`.
|
||||
- **`E401` / `ENEEDAUTH` with no token configured** — the exchange failed and there was nothing to
|
||||
fall back to. Run with `--loglevel verbose`: `oidc()` logs its reason (`Skipped because incorrect
|
||||
permissions`, `Failed token exchange request…`) instead of throwing.
|
||||
- **422 on publish** — provenance mismatch; check the package's `repository` field.
|
||||
- **Silent fallback to token auth** — if a valid token is present npm may authenticate with it
|
||||
instead. Confirm via `dist.attestations` rather than the job log.
|
||||
285
docs/proposals/apps-converters-zod/README.md
Normal file
285
docs/proposals/apps-converters-zod/README.md
Normal file
@@ -0,0 +1,285 @@
|
||||
# Proposal: Migrate the Apps-Engine converters to Zod codecs
|
||||
|
||||
## Status
|
||||
|
||||
In progress (PR #41205). Phases 0–5 landed; `transformMappedData` removed. See the per-phase
|
||||
notes below, including the messages/threads deviation in Phase 4.
|
||||
|
||||
## Problem
|
||||
|
||||
`apps/meteor/app/apps/server/converters/` translates data between two models in both
|
||||
directions:
|
||||
|
||||
- **Rocket.Chat → Apps-Engine** ("to app"): Mongo documents (`IUser`, `IRoom`, `IMessage`, …)
|
||||
become Apps-Engine objects (`IAppsUser`, `IAppsRoom`, `IAppsMessage`, …). Methods such as
|
||||
`convertToApp` / `convertRoom` / `convertMessage`.
|
||||
- **Apps-Engine → Rocket.Chat** ("from app"): the reverse. Methods such as `convertToRocketChat` /
|
||||
`convertApp*`.
|
||||
|
||||
Today the two directions are implemented by two unrelated mechanisms:
|
||||
|
||||
- The **to-app** direction is driven by `transformMappedData`, a declarative field remapper
|
||||
(`{ to: 'from' | fn | { from, map, list } }`) that also collects everything it did not map into an
|
||||
`_unmappedProperties_` bucket.
|
||||
- The **from-app** direction is hand-written object construction with long chains of
|
||||
`...(cond && { … })` spreads, re-merging `_unmappedProperties_` at the end.
|
||||
|
||||
This split has real costs:
|
||||
|
||||
1. **Two sources of truth per entity.** The field mapping is expressed once in a `map` and again,
|
||||
inverted, in a hand-written builder. They drift.
|
||||
2. **No runtime validation at the boundary.** The from-app path trusts whatever an app sends and
|
||||
spreads it onto a Mongo document.
|
||||
3. **Weak typing.** Several converters were untyped `.js`; the mapping is stringly-typed and the
|
||||
relationship between the two directions is invisible to the compiler.
|
||||
|
||||
### Inventory
|
||||
|
||||
| Converter | Was | To-app | From-app | Async / DB | Notable hazards |
|
||||
|--------------------|-----------|:------:|:--------:|:----------:|-----------------|
|
||||
| `settings` | js → ts | ✅ | — | sync | `SettingType` enum |
|
||||
| `roles` | ts | ✅ | — | sync | trivial |
|
||||
| `videoConferences` | ts | ✅ | ✅ | sync | pass-through clone |
|
||||
| `visitors` | js → ts | ✅ | ✅ | sync | `_unmappedProperties_` bucket |
|
||||
| `departments` | js → ts | ✅ | ✅ | sync | `_unmappedProperties_` bucket |
|
||||
| `users` | js → ts | ✅ | ✅ | sync | `UserType` / `UserStatusConnection` enums, contextual `console.warn` |
|
||||
| `uploads` | js → ts | ✅ | ✅ | **async** | cross-converter (rooms/users/visitors) |
|
||||
| `rooms` | js → ts | ✅ | ✅ | **async** | cross-converter fan-out, `isPartial`, `secureFieldsMapper`, `RoomType` enum |
|
||||
| `messages` | js → ts | ✅ | ✅ | **async** | `WeakMap` + `cachedFunction` memoization, `isPartial`, attachment sub-map, visitor-sender fallback |
|
||||
| `threads` | ts | ✅ | — | **async** | duplicates the messages attachment map |
|
||||
| `contacts` | ts | ✅ | ✅ | sync | deep nested `list` reverse map |
|
||||
|
||||
Infrastructure: `transformMappedData.ts`, `cachedFunction.ts`, `convertMessageFiles.ts`.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
Model each entity as a **Zod codec** — a single bidirectional artifact that replaces the split
|
||||
"map + hand-written inverse" pair and adds runtime validation.
|
||||
|
||||
Zod 4.3 (`~4.3.6`) is already a dependency, and the repo already ships a codec —
|
||||
`TimestampSchema` in `packages/core-typings/src/utils.ts`:
|
||||
|
||||
```ts
|
||||
export const TimestampSchema = z.codec(z.iso.datetime(), z.date(), {
|
||||
encode: (date) => date.toISOString(),
|
||||
decode: (str) => new Date(str),
|
||||
});
|
||||
```
|
||||
|
||||
`z.codec(RcSchema, AppSchema, { decode, encode })` maps directly onto our two directions:
|
||||
|
||||
- `decode` : Rocket.Chat → Apps-Engine (today's `convertToApp` / `convertRoom` / …)
|
||||
- `encode` : Apps-Engine → Rocket.Chat (today's `convertToRocketChat` / `convertApp*`)
|
||||
|
||||
`z.decode(codec, x)` / `z.encode(codec, x)` run them synchronously; `z.decodeAsync` / `z.encodeAsync`
|
||||
run them when the transforms are async.
|
||||
|
||||
The converter **classes stay** as the public façade (they implement the `IAppXConverter` interfaces
|
||||
and are reached via `orch.getConverters().get('x')`, used in ~25 files). Codecs are an internal
|
||||
implementation detail those methods delegate to.
|
||||
|
||||
## Key design decisions
|
||||
|
||||
1. **Async transforms → `z.decodeAsync` / `z.encodeAsync`.** rooms/messages/uploads/threads perform
|
||||
DB lookups inside the mapping. Their codec transforms are async and must be driven with the async
|
||||
entry points; the sync `z.decode` throws `$ZodAsyncError` on an async codec. Sync converters
|
||||
(settings, roles, users, visitors, departments, contacts, videoConferences) use plain
|
||||
`z.decode` / `z.encode`. *Validated against the installed Zod build during Phase 0.*
|
||||
|
||||
2. **Orchestrator dependency → codec factories.** Cross-converter converters cannot be static
|
||||
singletons — they need `orch`. Expose `createRoomCodec(orch)`, `createUploadsCodec(orch)`, etc.,
|
||||
returning a closure-bound codec. DB-free converters export a static codec constant.
|
||||
|
||||
3. **Preserve the `_unmappedProperties_` contract — do not drop it.** It is load-bearing: the reverse
|
||||
converters merge it back, the EE redactor (`ee/server/apps/lib/redactor.ts`) references the path,
|
||||
and `RoomBridge` reads it. A plain `z.object` strips unknown keys; `z.looseObject` keeps them
|
||||
inline but *without* the bucket. We will build a small reusable helper (working name
|
||||
`mappedCodec`) that reproduces `transformMappedData`'s bucket semantics exactly, so output is
|
||||
byte-identical. This keeps the migration behaviour-preserving rather than a behaviour change.
|
||||
The helper is intentionally **not** built up-front — it will be co-designed with the first
|
||||
bidirectional converter (Phase 2) to avoid guessing the abstraction.
|
||||
|
||||
4. **`isPartial` (rooms/messages from-app) stays in the class method, not the codec.** Partial mode
|
||||
skips required-field generation, skips the unmapped merge, and strips `undefined`. Model it as the
|
||||
class calling either `z.encode` (full) or a partial path; do not encode the flag into the schema.
|
||||
|
||||
5. **Enum conversions → shared codec constants.** `UserType`, `UserStatusConnection`, `RoomType`,
|
||||
`SettingType` become small codecs (like `TimestampSchema`) in `converters/codecs/enums.ts`,
|
||||
reproducing the current `switch` logic including the pass-through/upper-case fallbacks.
|
||||
Contextual `console.warn` calls that need data unavailable to a pure enum mapping (e.g. the
|
||||
affected user's id/username in the status-connection warning) stay in the converter layer.
|
||||
|
||||
6. **Memoization (messages/threads) stays.** The `WeakMap` + `cachedFunction` dedup of user/room
|
||||
lookups within a single conversion is preserved by constructing the message codec per-conversion
|
||||
through the factory, passing in the memoized lookups.
|
||||
|
||||
7. **Loose validation on the from-app path.** Apps send arbitrary data today; strict schemas would
|
||||
reject payloads that currently pass. Use `.loose()` / `.optional()` generously on the app-side
|
||||
schemas so the migration introduces no rejections. Tightening is a deliberate, separate follow-up.
|
||||
|
||||
8. **Preserve the public surface.** The `IAppXConverter` interfaces and the
|
||||
`orch.getConverters().get('x')` usage must not change.
|
||||
|
||||
### Direction convention (illustrative)
|
||||
|
||||
```ts
|
||||
const UserCodec = z.codec(UserRocketChatSchema, AppsUserSchema, {
|
||||
decode: (user) => ({ id: user._id, /* … */ }), // convertToApp
|
||||
encode: (appUser) => removeEmpty({ _id: appUser.id, /* … */ }), // convertToRocketChat
|
||||
});
|
||||
|
||||
// RC -> App: z.decode(UserCodec, user)
|
||||
// App -> RC: z.encode(UserCodec, appUser)
|
||||
```
|
||||
|
||||
## Migration phases
|
||||
|
||||
Each phase is independently shippable, keeps the class façade, and is gated by the golden tests.
|
||||
|
||||
### Phase 0 — Scaffolding & de-risk (done, PR #41205)
|
||||
|
||||
- Converted the 7 remaining `.js` converters to `.ts`, behaviour-preserving. `rooms`/`messages` keep
|
||||
loose typing on their transform maps for now; that tightens when each is codec-ified.
|
||||
- Fixed three pre-existing latent bugs that surfaced once the converters were typed (see
|
||||
[Incidental fixes](#incidental-fixes)). These are the only intentional behaviour changes in the
|
||||
phase and are each pinned by a golden/focused test.
|
||||
- Added `converters/codecs/` with the first shared primitives: bidirectional enum codecs
|
||||
(`UserType`, `UserStatusConnection`, `RoomType`, `SettingType`).
|
||||
- Added the **behavioural safety net**: enum-codec unit tests (asserting parity with the legacy
|
||||
helpers) and **golden-snapshot tests** locking the current RC ↔ Apps-Engine field mapping,
|
||||
including `_unmappedProperties_` bucketing, for settings, users, visitors, departments, roles,
|
||||
videoConferences, contacts and uploads.
|
||||
- Validated the async-codec approach (`z.decodeAsync` / `z.encodeAsync`, `$ZodAsyncError`) that
|
||||
Phases 3–4 depend on.
|
||||
|
||||
#### Incidental fixes
|
||||
|
||||
Typing the converters exposed three latent bugs in the legacy code. They are small, self-contained,
|
||||
and fixed in Phase 0 rather than carried forward as "behaviour to preserve":
|
||||
|
||||
1. **`users` — `utfOffset` typo (from-app).** `convertToRocketChat` read the misspelled
|
||||
`appUser.utfOffset`, so a round-tripped user always lost its `utcOffset`. It now reads
|
||||
`utcOffset` first and keeps `utfOffset` only as a compatibility fallback for any app still
|
||||
emitting the old key. The golden round-trip fixture uses `utcOffset`; a focused test pins the
|
||||
legacy fallback.
|
||||
2. **`messages` — dead visitor-sender fallback (to-app).** The `sender` resolver deleted
|
||||
`message.u` *before* the "old system message without token" fallback re-read it, so the fallback
|
||||
always received `undefined`. It now captures the sender before deletion. A focused test drives a
|
||||
missing primary lookup and asserts the fallback resolves the original sender.
|
||||
3. **`messages` — null editor deref (from-app).** `convertAppMessage` dereferenced
|
||||
`Users.findOneById(editor.id)` through a non-null assertion, throwing if the editor no longer
|
||||
exists. It now falls back to the editor data carried on the app payload, mirroring the adjacent
|
||||
sender handling.
|
||||
|
||||
Alongside these, a few defensive guards were added where a destructuring or lookup could hit an
|
||||
optional value that the types claimed was always present (`uploads` `rid`/`room`, `users`
|
||||
`convertByUsername`, `rooms` `visitorChannelInfo`). These only turn a latent `TypeError` into the
|
||||
already-intended empty result and match the guards their sibling fields already had, so they change
|
||||
no covered output.
|
||||
|
||||
### Phase 1 — Trivial, one-way, sync
|
||||
|
||||
`settings`, `roles`, `videoConferences`. Proves the codec + enum-codec pattern end-to-end at the
|
||||
lowest risk.
|
||||
|
||||
### Phase 2 — Self-contained, bidirectional, sync
|
||||
|
||||
`visitors`, `departments`, `users`. Introduces and hardens the `mappedCodec` unmapped-bucket helper
|
||||
and exercises the enum codecs with the `console.warn` side effects preserved in both directions.
|
||||
|
||||
### Phase 3 — Async, cross-converter
|
||||
|
||||
`uploads`, then `rooms`. Introduces the codec-factory pattern, `z.decodeAsync` / `z.encodeAsync`,
|
||||
`isPartial` handled in the class layer, and `secureFieldsMapper` integration.
|
||||
|
||||
### Phase 4 — Async + memoized + nested (done)
|
||||
|
||||
`messages`, `threads` and `contacts`. `contacts` became a clean bidirectional `ContactCodec`
|
||||
(`decode` clones; `encode` maps every attribute via the nested field map). `mappedDecodeAsync` was
|
||||
extended with the nested `{ from, map, list }` branch so it fully reproduces `transformMappedData`.
|
||||
|
||||
**Deviation — messages and threads are not standalone codecs.** Unlike every other converter, these
|
||||
two were migrated by swapping their `transformMappedData` calls for the shared `mappedDecodeAsync`
|
||||
helper *in place*, rather than being wrapped in `create*Codec` factories. Reasons:
|
||||
|
||||
1. **Test harness.** `messages.tests.js` uses `proxyquire` to stub `@rocket.chat/models`,
|
||||
`@rocket.chat/random` and `@rocket.chat/core-typings` (`isMessageFromVisitor`) on the `messages.ts`
|
||||
module. `proxyquire` only replaces a module's *own* `require`s, so moving `convertAppMessage`'s
|
||||
model/`Random` usage into a separate `codecs/messages.ts` module would bypass the stubs and break
|
||||
the test (real `Rooms.findOneById` → no `GENERAL` room → the "proper schema"/"invalid room" cases
|
||||
fail). Keeping that usage in the class preserves the harness without editing it.
|
||||
2. **Poor fit for the codec shape.** `convertMessage` builds its field map per call, closing over a
|
||||
per-message memoization cache (`WeakMap` + `cachedFunction`), the source `msgObj`
|
||||
(`isMessageFromVisitor`) and `mainFile`; `convertAppMessage` adds an `isPartial` flag and
|
||||
`Random.id`/`new Date()` defaults. None of this expresses cleanly as a static bidirectional
|
||||
`z.codec`, and forcing it would add casts and risk for no real gain.
|
||||
|
||||
The migration goal for these two is therefore narrower but still met: the `transformMappedData`
|
||||
dependency is removed and the mapping logic is the shared, tested `mappedDecodeAsync`. The
|
||||
attachment sub-maps stayed per-converter because messages' and threads' variants differ (threads has
|
||||
extra `author`/`timestamp`/`fileId` guards) — sharing them would change behaviour. Wrapping messages
|
||||
in a codec later remains possible (e.g. via `proxyquire`'s `@global` stubs) if desired.
|
||||
|
||||
### Phase 5 — Cleanup (done)
|
||||
|
||||
Every converter is off `transformMappedData`, so `transformMappedData.ts` was deleted and its
|
||||
(previously importer-located) spec was retired — its coverage now lives in the `mappedDecodeAsync`
|
||||
tests under `tests/unit/app/apps/server/codecs/`. `cachedFunction` stays: it is still used by the
|
||||
messages and threads memoization, which remained in the class layer (see the Phase 4 deviation).
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Golden snapshots** (added in Phase 0) are the equivalence oracle: each phase must keep them green
|
||||
while swapping internals. Output is compared after `JSON.parse(JSON.stringify(...))` so Dates
|
||||
normalise to ISO strings and `undefined` fields drop — matching how payloads cross the app bridge.
|
||||
- **Enum-codec unit tests** assert `decode`/`encode` match the legacy `_convert*` helpers for every
|
||||
input, including fallbacks.
|
||||
- The existing `rooms.tests.ts` and `messages.tests.js` remain the oracle for the two hardest
|
||||
converters.
|
||||
- New codecs get their own focused tests as they are introduced.
|
||||
|
||||
## Risks
|
||||
|
||||
- **`_unmappedProperties_` fidelity** — the single biggest behavioural trap; the `mappedCodec` helper
|
||||
plus golden tests are the mitigation.
|
||||
- **From-app validation rejecting live app payloads** — mitigated by loose schemas; do not tighten
|
||||
during the migration.
|
||||
- **Async codec ergonomics** — de-risked in Phase 0, but the factory wiring for rooms/messages should
|
||||
still be spiked before committing to Phase 3.
|
||||
- **Test rewrite** — the existing tests use `proxyquire.noCallThru()` against module paths; codec-ing
|
||||
each converter means updating those loaders.
|
||||
|
||||
## Type strategy for the mapping helpers
|
||||
|
||||
The mapping helpers in `converters/codecs/mappedData.ts` are split by how well the source type is
|
||||
known, which is what makes generic typing worthwhile in some cases and not others:
|
||||
|
||||
- **Sync string maps → fully generic and inferred.** `mappedDecode<Source>(fieldMap)` and
|
||||
`createMappedCodec<Source>(fieldMap)` are generic over the source document type. The field map is
|
||||
typed `FieldMap<Source> = Record<string, Extract<keyof Source, string>>`, so a renamed or
|
||||
misspelled source field is a **compile error** (a `@ts-expect-error` test locks this). The decode
|
||||
result is the inferred `Decoded<Source, Map>` — each renamed target as an optional property plus the
|
||||
`_unmappedProperties_` bucket typed as `Omit<Source, mappedKeys>` — rather than `Record<string, any>`.
|
||||
`mappedEncode` is the inverse and returns `Partial<Source>`. `Source` defaults to a loose record, so
|
||||
untyped call sites (e.g. tests) still work. Consumers with clean document types (`visitors`,
|
||||
`departments`, `roles`) pass their `Source` and get the typo-safety for free.
|
||||
|
||||
- **Async/function/nested maps → loose map, generic result.** `mappedDecodeAsync` keeps its
|
||||
`AsyncFieldMap` loosely typed on purpose: its consumers (`rooms`, `messages`, `uploads`, `contacts`)
|
||||
map many optional/livechat-only fields that are *not* on the base document types (which is why the
|
||||
original converters used dynamic access), and the return shape depends on arbitrary function/nested
|
||||
entries. Fully inferring that would resurrect the large conditional type the old helper avoided. It
|
||||
does take a `Result` type parameter — `mappedDecodeAsync<IAppsUpload>(data, map)` — so call sites
|
||||
assert the produced shape once instead of trailing every call with `as unknown as Promise<…>`.
|
||||
|
||||
- **Each apps-type boundary keeps a single, documented cast.** The Apps-Engine interfaces
|
||||
(`IAppsVisitor`, …) are hand-authored and don't structurally equal the produced `Decoded`/mapped
|
||||
shape, so each affected codec boundary — supplied role, department, user and visitor — retains one
|
||||
localized `as unknown as <AppsType>` where the codec meets that interface. Those casts are the
|
||||
deliberate loose-validation seams (see design decision 7), not incidental looseness.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changing the `IAppXConverter` interfaces or any app-facing behaviour.
|
||||
- Tightening the from-app validation surface (tracked as a separate follow-up).
|
||||
- Migrating converters that are not in the directory above.
|
||||
104
docs/proposals/compact-ast-format.md
Normal file
104
docs/proposals/compact-ast-format.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# Proposal: Compact AST Format for message-parser
|
||||
|
||||
## Status
|
||||
|
||||
Draft
|
||||
|
||||
## Problem
|
||||
|
||||
The current message-parser AST is verbose — every node carries its full text content as self-contained strings. For a message like `**hello** world`, the AST stores `"hello"` inside the Bold node and `" world"` in a separate Plain node, even though both substrings already exist in the original message. This redundancy inflates payload size, especially for messages with deeply nested formatting (bold inside italic inside strikethrough, etc.).
|
||||
|
||||
In high-traffic environments, parsed messages are stored and transmitted frequently. Reducing the AST footprint has a direct impact on storage costs, cache efficiency, and wire transfer size.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
Introduce a **compact AST format** that replaces self-contained string values with **span references** (`[start, end]`) into the original message text.
|
||||
|
||||
### Core Idea
|
||||
|
||||
Instead of:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "PARAGRAPH",
|
||||
"value": [
|
||||
{ "type": "BOLD", "value": [{ "type": "PLAIN_TEXT", "value": "hello" }] },
|
||||
{ "type": "PLAIN_TEXT", "value": " world" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The compact format stores:
|
||||
|
||||
```json
|
||||
{ "t": "p", "c": [{ "t": "b", "c": [[2, 7]] }, [8, 14]] }
|
||||
```
|
||||
|
||||
Plain text nodes become simple `[start, end]` tuples. Structural nodes use short type keys (`b`, `i`, `s`, `p`, `h`, etc.) and reference children via the same span mechanism.
|
||||
|
||||
### Key Operations
|
||||
|
||||
| Function | Description |
|
||||
|---|---|
|
||||
| `compactify(ast, msg)` | Converts a verbose AST + original message into a compact AST |
|
||||
| `expand(compactAst, msg)` | Reconstructs the full verbose AST from a compact AST + original message |
|
||||
| `validateRoundtrip(ast, msg)` | Verifies `expand(compactify(ast, msg), msg)` equals the original AST |
|
||||
|
||||
### Compact Type Mapping
|
||||
|
||||
| Verbose Type | Compact Key | Notes |
|
||||
|---|---|---|
|
||||
| `PLAIN_TEXT` | `[start, end]` | Span tuple, no wrapper object |
|
||||
| `BOLD` | `b` | |
|
||||
| `ITALIC` | `i` | |
|
||||
| `STRIKE` | `s` | |
|
||||
| `SPOILER` | `\|\|` | |
|
||||
| `INLINE_CODE` | `` ` `` | |
|
||||
| `MENTION_USER` | `@` | |
|
||||
| `MENTION_CHANNEL` | `#` | |
|
||||
| `INLINE_KATEX` | `$` | |
|
||||
| `LINK` | `a` | |
|
||||
| `IMAGE` | `img` | |
|
||||
| `EMOJI` | `:` | |
|
||||
| `TIMESTAMP` | `ts` | |
|
||||
| `COLOR` | `c` | Stores RGBA as `[r, g, b, a]` |
|
||||
| `PARAGRAPH` | `p` | |
|
||||
| `HEADING` | `h` | Includes level `l: 1..4` |
|
||||
| `CODE` | ```` ``` ```` | |
|
||||
| `BLOCKQUOTE` | `>` | |
|
||||
| `QUOTE` | `q` | |
|
||||
| `SPOILER_BLOCK` | `\|\|\|` | |
|
||||
| `ORDERED_LIST` | `ol` | |
|
||||
| `UNORDERED_LIST` | `ul` | |
|
||||
| `TASKS` | `tl` | |
|
||||
| `KATEX` | `$$` | |
|
||||
| `LINE_BREAK` | `br` | |
|
||||
| `BIG_EMOJI` | `E` | |
|
||||
|
||||
## Trade-offs
|
||||
|
||||
### Advantages
|
||||
|
||||
- Significant size reduction on typical messages (observed 30-60% in initial tests)
|
||||
- Lossless — roundtrip conversion preserves the full AST
|
||||
- The original message is already stored alongside the AST, so no additional data is needed
|
||||
- Short keys further reduce payload size
|
||||
|
||||
### Concerns
|
||||
|
||||
- Requires the original message text to be available at expansion time
|
||||
- Adds a conversion layer — any bug in `compactify`/`expand` could corrupt message rendering
|
||||
- Span-based references are fragile if the message text is modified after compaction
|
||||
- Increases complexity in the message-parser package
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Storage strategy** — Should compact ASTs replace verbose ones in the database, or coexist (e.g., compact for wire transfer, verbose for rendering)?
|
||||
2. **Migration path** — How do we handle existing messages already stored with verbose ASTs?
|
||||
3. **Rendering integration** — Should gazzodown learn to render compact ASTs directly, or always expand first?
|
||||
4. **Message edits** — When a message is edited, do we reparse and recompact, or invalidate the compact form?
|
||||
5. **Performance budget** — Is the compactify/expand overhead acceptable on the hot path, or should it be deferred to a background job?
|
||||
|
||||
## Reference
|
||||
|
||||
A working proof-of-concept implementation exists with full bidirectional conversion and roundtrip validation tests. It covers all current AST node types including BigEmoji, lists, tasks, code blocks, KaTeX, and color nodes.
|
||||
114
docs/proposals/offline-license-capability-enforcement.md
Normal file
114
docs/proposals/offline-license-capability-enforcement.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Proposal: License Capability Enforcement via Branded Proofs
|
||||
|
||||
## Status
|
||||
|
||||
Draft — v2. Supersedes the earlier `CloudConnection`-only draft: the single-purpose connection capability is generalized into one proof mechanism that can guard any license-controlled capability, with the specific entitlement captured in the type.
|
||||
|
||||
## Problem
|
||||
|
||||
License enforcement today is a runtime side-condition the compiler knows nothing about, in two distinct places:
|
||||
|
||||
1. **Module entitlements.** `License.hasModule('auditing')` returns a `boolean`. Nothing ties that boolean to the code it guards: a feature entry point can be called without any check, a check for the *wrong* module still typechecks, and a refactor that moves code out from under its `if` block breaks enforcement silently.
|
||||
2. **Offline (air-gapped) licenses.** When `license.information.offline` is true, the workspace must never initiate outbound connections to Rocket.Chat-owned endpoints. This is enforced by ~15 scattered `hasOfflineLicense()` checks (sync, marketplace, telemetry, push gateway, Gravatar). Any new feature can `import { serverFetch }` and ship a compliance violation no type error catches — and two real bugs of exactly this class were found during QA (a startup race in the usage report, and a stale verdict held by the push retry chain).
|
||||
|
||||
Both are the same underlying flaw: **the license check produces no evidence**. Nothing forces the check to happen, to happen for the right entitlement, or to happen at the right time.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
Make every license check return an unforgeable **proof value**, and make guarded code demand that proof in its signature. The pattern has three parts.
|
||||
|
||||
### 1. The lock — branded proof types
|
||||
|
||||
One brand mechanism, parameterized by what it proves. The generic parameter captures the exact literal passed to the checker, so proofs for different modules are not interchangeable:
|
||||
|
||||
```ts
|
||||
// packages/core-typings/src/license/LicenseModule.ts (already exists)
|
||||
export type InternalModuleName = (typeof CoreModules)[number];
|
||||
export type ExternalModuleName = `${string}.${string}`;
|
||||
export type LicenseModule = InternalModuleName | ExternalModuleName;
|
||||
|
||||
// ee/packages/license/src/proofs.ts (new)
|
||||
declare const LicenseAuthorized: unique symbol; // NOT exported — unforgeable outside the package
|
||||
|
||||
/** Proof that the current license grants module M. */
|
||||
export type ModuleProof<M extends LicenseModule> = {
|
||||
readonly [LicenseAuthorized]: M;
|
||||
};
|
||||
|
||||
/** Proof that the current license permits outbound calls to Rocket.Chat-owned
|
||||
* endpoints (absent when the license carries the offline flag). */
|
||||
export type CloudEgressProof = {
|
||||
readonly [LicenseAuthorized]: 'cloud:egress';
|
||||
};
|
||||
```
|
||||
|
||||
The brand property is phantom — no such field exists at runtime. Proofs are frozen empty objects: zero allocation cost (a shared singleton per kind), zero serialization surface.
|
||||
|
||||
### 2. The keymaker — the license package owns construction
|
||||
|
||||
The only casts live inside `ee/packages/license`, next to the state they attest to:
|
||||
|
||||
```ts
|
||||
// on LicenseManager — additive API; the existing boolean hasModule() stays
|
||||
public proveModule<M extends LicenseModule>(module: M): ModuleProof<M> | undefined {
|
||||
return this.hasModule(module) ? (PROOF as ModuleProof<M>) : undefined;
|
||||
}
|
||||
|
||||
public proveCloudEgress(): CloudEgressProof | undefined {
|
||||
return this.hasOfflineLicense() ? undefined : (PROOF as CloudEgressProof);
|
||||
}
|
||||
```
|
||||
|
||||
Design decisions:
|
||||
|
||||
- **Additive, not a signature change.** `hasModule(): boolean` has hundreds of call sites, including display logic and the REST surface the client consumes. Those keep the boolean. `proveModule()` is what *server-side entry points* migrate to.
|
||||
- **Synchronous.** License state is in memory; proofs are free to acquire, which matters for the per-attempt rule below.
|
||||
- **The offline gate is a proof family, not a module.** The offline flag has inverted semantics — it *revokes* a permission rather than granting a feature — but the same brand expresses it: `proveCloudEgress()` returns `undefined` exactly when the offline license forbids egress. A `proveCloudEgressOrThrow(message?)` variant throws the existing `CloudOfflineLicenseError` for interactive flows (registration, cloud login, billing), preserving today's error contract.
|
||||
|
||||
### 3. The guard — signatures demand proofs
|
||||
|
||||
```ts
|
||||
// Module-gated feature entry point:
|
||||
function initAuditing(proof: ModuleProof<'auditing'>) { ... }
|
||||
|
||||
initAuditing(); // ❌ compile error: expected 1 argument
|
||||
const token = License.proveModule('auditing');
|
||||
initAuditing(token); // ❌ compile error: possibly undefined
|
||||
if (token) {
|
||||
initAuditing(token); // ✅ narrowed to ModuleProof<'auditing'>
|
||||
}
|
||||
|
||||
// Cloud I/O — the single fetch wrapper in apps/meteor/server/lib/cloud/cloudClient.ts:
|
||||
export function cloudFetch(proof: CloudEgressProof, input: string, options?: ExtendedFetchOptions): Promise<Response>;
|
||||
```
|
||||
|
||||
The developer experience does the enforcement: to call the function you need the token; the only way to get the token is the checker whose name and JSDoc explain the rule; the `| undefined` return forces an explicit decision about the denied case (skip silently vs. throw); and truthiness narrowing makes the happy path read naturally.
|
||||
|
||||
The proof is compile-time evidence, not the runtime gate itself: `cloudFetch` **re-validates `hasOfflineLicense()` immediately before dispatching** and drops the request if the license changed while the operation was in flight. The proof parameter guarantees the check can't be forgotten; the dispatch-time re-check guarantees it can't go stale.
|
||||
|
||||
Inner helpers that are only reachable from a guarded entry point take the proof as a parameter rather than re-checking — the signature documents "this function performs cloud I/O / module-X work", and the compiler walks the requirement up the call graph to wherever the proof is legitimately acquired.
|
||||
|
||||
## The staleness rule: acquire per attempt, never cache
|
||||
|
||||
Licenses change at runtime — swapped, upgraded, invalidated, or an offline license applied mid-flight. A proof is a **point-in-time verdict**, and both QA-discovered bugs in the offline work were staleness bugs:
|
||||
|
||||
- the usage report evaluated its gate at function entry, then spent seconds generating statistics before fetching (the verdict predated license application);
|
||||
- the push gateway retry chain captured its decision in `setTimeout` closures that outlived a license change.
|
||||
|
||||
Rules, to be stated in the proofs' JSDoc and enforced in review:
|
||||
|
||||
1. Acquire the proof at the top of each operation *attempt*; retry closures re-acquire on every attempt (the factories are sync — this costs nothing).
|
||||
2. Never store a proof on a class field, module scope, or queue payload.
|
||||
3. A proof is not a subscription. Long-lived module features (services started when a module is granted) must still use the existing lifecycle events — `License.onValidFeature` / `onInvalidFeature` / `onToggledFeature` — for startup and teardown. Proofs guard entry points; events manage lifetimes.
|
||||
4. Proofs are server-only and must never cross the API boundary; the client keeps boolean checks driven by `licenses.info`.
|
||||
|
||||
## Migration
|
||||
|
||||
1. **Cloud egress first** (the offline license feature, already enforced at runtime in ~27 files): introduce `proveCloudEgress()` and `cloudFetch(proof, ...)`, then convert the four established patterns — interactive flows (`OrThrow`), background jobs (`undefined` → keep side effects, skip), the marketplace client (proof acquired per `fetch()` call, never stored on the instance), and retry closures (per-attempt). The existing `hasOfflineLicense()` helpers remain for behavior gates that don't perform I/O themselves (`shouldUseGateway()`, cron-body skips, Gravatar suggestion filtering).
|
||||
2. **Module proofs opportunistically**: as EE features are touched, their server entry points gain `ModuleProof<'...'>` parameters, starting with features whose checks have historically drifted from their code. No big-bang rewrite; the boolean API keeps working throughout.
|
||||
|
||||
## Limitations
|
||||
|
||||
TypeScript cannot forbid an import: a new file can still `import { serverFetch }` directly and hardcode an endpoint, and `{} as ModuleProof<'auditing'>` defeats the brand (both are greppable and glaring in review — the cast requires importing a type whose only documented constructor is the license manager). What the type system guarantees is narrower but real: **code routed through proof-demanding signatures cannot skip the check, cannot check the wrong module, and must handle the denied case explicitly** — and violations shrink to two obvious review signals: a raw `serverFetch` near a Rocket.Chat domain, or a forged cast.
|
||||
|
||||
A directory-scoped lint rule (`no-restricted-imports` on `@rocket.chat/server-fetch` with `cloudClient.ts` exempt) or a CI grep could close the import hole; both were considered and deliberately left out in favor of a types-only approach.
|
||||
Reference in New Issue
Block a user