snapshot of https://github.com/zulip/zulip.git @ 48c69911c7dd4e03b20552eb3bd77ad8151c1357 (2026-09-15)

.claude/skills/visual-test/SKILL.md

@@ -0,0 +1,374 @@
+---
+name: visual-test
+description: "Visually verify UI changes using Puppeteer screenshots. Use when you need to check layout, colors, positioning, or other visual aspects of a UI change."
+---
+
+# Visual Test
+
+Runs a real browser against the Zulip test server and takes
+screenshots you can read as images to verify layout, colors,
+positioning, text content, etc.
+
+## Steps
+
+### 1. Write the puppeteer test script
+
+Create `web/e2e-tests/_claude_<feature>_test.test.ts` using this template:
+
+```typescript
+import type {Page} from "puppeteer";
+
+import * as common from "./lib/common.ts";
+
+async function visual_test(page: Page): Promise<void> {
+    await common.log_in(page);
+    await common.screenshot(page, "step-1-logged-in");
+
+    // Navigate, interact, and screenshot each significant state.
+    // See "Available helpers" below.
+}
+
+await common.run_test(visual_test);
+```
+
+Adapt the body to exercise whatever UI you need to verify. Take a
+screenshot at every visually significant state using descriptive names
+like `step-2-color-picker-open`, `step-3-color-selected`.
+
+**Important patterns:**
+
+These patterns are derived from the existing Puppeteer tests in
+`web/e2e-tests/`. Follow them to write reliable, non-flaky tests.
+
+#### Waiting: never use hardcoded timeouts
+
+The existing test suite has essentially zero `setTimeout` calls
+(the two in `common.ts` are explicitly commented workarounds for
+specific animation flakes). Always wait for the specific condition
+you expect instead. The three main waiting primitives, in order of
+preference:
+
+- **`waitForSelector`** — wait for an element to appear or disappear.
+  This is the most common pattern in the test suite (100+ uses):
+
+  ```typescript
+  // Wait for element to be visible (most common)
+  await page.waitForSelector("#left-sidebar", {visible: true});
+
+  // Wait for element to disappear (e.g., overlay closed, row deleted)
+  await page.waitForSelector("#subscription_overlay", {hidden: true});
+  ```
+
+- **`waitForFunction`** — wait for a condition that can't be
+  expressed as a single selector (text content, element count,
+  attribute value, application state):
+
+  ```typescript
+  // Wait for specific text content
+  await page.waitForFunction(
+      () => document.querySelector(".save-button")?.textContent?.trim() === "Save changes",
+  );
+
+  // Wait for element count after filtering
+  await page.waitForFunction(
+      () => document.querySelectorAll(".linkifier_row").length === 4,
+  );
+
+  // Wait for an input's value to update
+  await page.waitForFunction(
+      () => document.querySelector<HTMLInputElement>("#full_name")?.value === "New name",
+  );
+
+  // Wait for focus to land on a specific element
+  await page.waitForFunction(
+      () => document.activeElement?.classList?.contains("search") === true,
+  );
+
+  // Wait for internal app state via zulip_test
+  await page.waitForFunction(
+      (content) => {
+          const last_msg = zulip_test.current_msg_list?.last();
+          return last_msg !== undefined && last_msg.raw_content === content
+              && !last_msg.locally_echoed;
+      },
+      {},
+      content,
+  );
+  ```
+
+- **`waitForNavigation`** — only for actual full-page navigations
+  (form submits, reloads). Wrap with `Promise.all` when the
+  navigation is triggered by an action:
+  ```typescript
+  await Promise.all([
+      page.waitForNavigation(),
+      page.$eval("form#login_form", (form) => { form.submit(); }),
+  ]);
+  ```
+
+#### Interacting with elements
+
+- **`page.click(selector)`** is the standard for clicking. When it's
+  unreliable (overlapping elements, timing), fall back to clicking
+  via `evaluate` — several existing tests do this with a comment
+  explaining why:
+
+  ```typescript
+  // When page.click() is unreliable, click via the DOM directly
+  await page.evaluate(() => {
+      document.querySelector<HTMLElement>(".dialog_submit_button")?.click();
+  });
+  ```
+
+- **`page.type(selector, text)`** for typing. Use `{delay: 100}`
+  when typing triggers a typeahead or filter that needs per-keystroke
+  updates:
+
+  ```typescript
+  await page.type('[name="user_list_filter"]', "ot", {delay: 100});
+  ```
+
+- **`common.clear_and_type(page, selector, text)`** to replace
+  existing input content (triple-click + Delete + type).
+
+- **`common.fill_form(page, selector, params)`** to fill multiple
+  form fields at once — handles text inputs, checkboxes (by
+  toggling), and `<select>` elements.
+
+- **`common.select_item_via_typeahead(page, selector, str, item)`**
+  to type into a field and pick a typeahead suggestion.
+
+- **`page.keyboard.press("KeyC")`** for Zulip keyboard shortcuts.
+  After pressing, wait for the resulting UI change:
+
+  ```typescript
+  await page.keyboard.press("KeyC");
+  await page.waitForSelector("#compose-textarea", {visible: true});
+  ```
+
+- **Hover before clicking action buttons** that only appear on
+  hover (e.g., message action icons):
+  ```typescript
+  const msg = (await page.$$(".message_row")).at(-1)!;
+  await msg.hover();
+  await page.waitForSelector(".message-actions-menu-button", {visible: true});
+  await page.click(".message-actions-menu-button");
+  ```
+
+#### Navigating within the app
+
+- **Click sidebar items** for in-app navigation:
+
+  ```typescript
+  await page.click(".narrow-filter[data-stream-id='...'] .stream-name");
+  await page.waitForSelector("#message_view_header .zulip-icon-hashtag", {visible: true});
+  ```
+
+- **`page.goto(url)`** for hash-route navigation:
+
+  ```typescript
+  await page.goto(`http://zulip.zulipdev.com:9981/#channels/${stream_id}/Denmark`);
+  ```
+
+- **`common.manage_organization(page)`** to navigate to org settings,
+  **`common.open_personal_menu(page)`** to open the personal menu.
+
+#### Reading state with page.evaluate
+
+Use `page.evaluate()` to read internal application state or DOM
+properties not accessible through selectors:
+
+```typescript
+// Read internal Zulip state via the zulip_test global
+const stream_id = await page.evaluate(
+    () => zulip_test.get_sub("Verona")!.stream_id,
+);
+
+// Read DOM properties
+const page_language = await page.evaluate(
+    () => document.documentElement.lang,
+);
+```
+
+#### Changing user settings via the API
+
+Use `page.evaluate` with `fetch()` and reload, rather than clicking
+through the settings UI:
+
+```typescript
+await page.evaluate(async () => {
+    const csrfToken = document.querySelector<HTMLInputElement>(
+        'input[name="csrfmiddlewaretoken"]',
+    )?.value ?? "";
+    await fetch("/json/settings", {
+        method: "PATCH",
+        headers: {
+            "Content-Type": "application/x-www-form-urlencoded",
+            "X-CSRFToken": csrfToken,
+        },
+        body: "user_list_style=1",
+    });
+});
+await page.reload({waitUntil: "networkidle2"});
+```
+
+#### XPath selectors for text matching
+
+When you need to match elements by text content, use XPath with
+`common.has_class_x()`:
+
+```typescript
+await page.waitForSelector(
+    `xpath///*[${common.has_class_x("stream-name")} and normalize-space()="Verona"]`,
+);
+```
+
+#### Assertions for visual tests
+
+For visual test scripts, prefer a soft-assertion pattern that
+reports all failures rather than aborting on the first:
+
+```typescript
+const results: string[] = [];
+function check(name: string, ok: boolean): void {
+    results.push(`${ok ? "PASS" : "FAIL"}: ${name}`);
+    console.log(`${ok ? "PASS" : "FAIL"}: ${name}`);
+}
+// ... run checks ...
+const failures = results.filter((r) => r.startsWith("FAIL"));
+console.log(`\n${results.length - failures.length}/${results.length} tests passed`);
+```
+
+This keeps the test running through failures so you see all results,
+unlike `assert` which aborts on the first failure.
+
+#### Verifying alignment
+
+When using Puppeteer to verify visual alignment, do not rely on
+eyeballing screenshots — especially small full-page ones. Instead:
+
+- Use `page.evaluate()` with `getBoundingClientRect()` to measure
+  actual pixel positions of the elements you need aligned, and print
+  them to the console. Compare the numbers.
+- Always take **both** a full-page screenshot and a zoomed clip of
+  the area of interest.
+- For zoomed clips, calculate the clip region from non-fixed elements;
+  fixed/sticky elements may report bounding-box positions that don't
+  match their visual location on the page.
+- Be aware that CSS nesting can scope styles to a specific parent
+  (e.g., `.parent .my-class`) — reusing the same class name in a
+  different context may not pick up the expected styles.
+- To verify keyboard-focus styles, use real keyboard navigation
+  (`page.keyboard.press`); programmatic `.focus()` doesn't reliably
+  trigger `:focus-visible` and may be overridden by view-level focus
+  management.
+- Focus rings drawn as `::before` / `::after` pseudo-elements aren't
+  visible in `getComputedStyle` of the focused element — verify them
+  in a screenshot, not via computed styles.
+- For visual changes, produce before/after screenshot pairs by writing
+  one test and running it twice with a `SCREENSHOT_SUFFIX` env var
+  (`-old` on `main`, `-updated` on your branch).
+
+### 2. Run the test
+
+```bash
+./tools/test-js-with-puppeteer _claude_<feature>_test
+```
+
+The runner matches test file names by prefix, so you don't need the
+full filename or `.test.ts` suffix. This starts a fresh test server
+on port 9981, runs the script, and saves screenshots to
+`var/puppeteer/`. The test database is reset between test files.
+
+On **aarch64 (ARM) hosts**, you must set `PUPPETEER_EXECUTABLE_PATH`
+(see "Environment details" below):
+
+```bash
+PUPPETEER_EXECUTABLE_PATH=$(echo ~/.cache/ms-playwright/chromium-*/chrome-linux/chrome) \
+    ./tools/test-js-with-puppeteer _claude_<feature>_test
+```
+
+To run all existing Puppeteer tests, omit the test name argument.
+
+**Timeout:** Tests can take 1–3 minutes each. Use a 300000ms timeout
+for the Bash tool.
+
+### 3. Read the screenshots
+
+Use the Read tool on each `var/puppeteer/step-*.png` file. Claude's
+multimodal vision will show the rendered page.
+
+### 4. Analyze and report
+
+Describe what you see — layout, colors, text content, positioning,
+any issues. Compare against what was expected.
+
+Zulip displays any JS exceptions encountered as a pop-up, but you should
+also be able to get them from the puppeteer output.
+
+### 5. Iterate if needed
+
+If something is wrong, fix the source code (or adjust the test
+script), then re-run from step 2.
+
+### 6. Clean up
+
+Leave test files as untracked `_claude_*` files so you can reuse them
+when rebasing or iterating on the pull request. The `_claude_` prefix
+is a convention to distinguish these from Zulip's committed test
+files. Do not commit them.
+
+## Environment details
+
+- **Architecture:** On x86_64, Puppeteer's bundled Chrome works. On
+  **aarch64**, it does NOT (fails with rosetta/ld-linux errors). Use
+  `uname -m` to check. For aarch64, install Playwright's Chromium
+  and point `PUPPETEER_EXECUTABLE_PATH` at it (shown in step 2).
+- **If the Playwright Chromium is missing**, install it:
+  ```bash
+  npx --yes playwright install chromium
+  ```
+- **Headless mode:** Tests run headless (`headless: true` in
+  `common.ts`). There is no display server.
+
+## Test infrastructure facts
+
+- **Server URL:** `http://zulip.zulipdev.com:9981/`
+- **Login:** `common.log_in(page)` uses credentials from
+  `var/puppeteer/test_credentials.json` (auto-generated by the test
+  harness). Default user is Desdemona (realm owner).
+- **Known users:** `common.fullname.cordelia`, `.othello`, `.hamlet`
+- **Screenshots:** saved to `var/puppeteer/<name>.png`
+- **Window size:** 1400 x 1024
+- **Test data:** The test database includes channels like "Verona",
+  "Denmark", "Scotland" with topics and messages. Non-system user
+  group `hamletcharacters` (members: Cordelia, Hamlet).
+- **`zulip_test` global:** Only a limited set of internal functions
+  are exposed — see `web/src/zulip_test.ts`. Functions like
+  `get_stream_id` and `get_user_id_from_name` are available, but
+  `user_groups` is not. Navigate to groups via URL hash routes
+  or by clicking list items instead.
+- **Database reset:** The test runner calls
+  `reset_zulip_test_database()` and `POST /flush_caches` between
+  test files, so each test file starts with a clean state.
+- **`common.run_test()`** handles browser lifecycle, console log
+  forwarding with source-map resolution, automatic failure
+  screenshots, and logout at the end.
+
+## Available helpers (from `web/e2e-tests/lib/common.ts`)
+
+| Helper                                        | Purpose                                  |
+| --------------------------------------------- | ---------------------------------------- |
+| `common.log_in(page)`                         | Log in as the default user (Desdemona)   |
+| `common.screenshot(page, "name")`             | Save `var/puppeteer/name.png`            |
+| `common.clear_and_type(page, selector, text)` | Clear input and type                     |
+| `common.fill_form(page, selector, params)`    | Fill multiple form fields at once        |
+| `common.wait_for_micromodal_to_open(page)`    | Wait for modal open animation            |
+| `common.wait_for_micromodal_to_close(page)`   | Wait for modal close animation           |
+| `common.get_stream_id(page, name)`            | Get a stream's ID                        |
+| `common.get_user_id_from_name(page, name)`    | Get a user's ID                          |
+| `common.open_personal_menu(page)`             | Open the personal menu                   |
+| `common.manage_organization(page)`            | Navigate to org settings                 |
+| `common.send_message(page, type, params)`     | Send a stream or DM message              |
+| `common.send_multiple_messages(page, msgs)`   | Send several messages in sequence        |
+| `common.select_item_via_typeahead(page, ...)` | Type into a field and select a typeahead |