snapshot of https://github.com/zulip/zulip.git @ 48c69911c7dd4e03b20552eb3bd77ad8151c1357 (2026-09-15)

docs/processes/api-design.md

@@ -0,0 +1,453 @@
+# Designing changes to the Zulip API
+
+One of Zulip's strengths is its clean, [well-documented][api-docs-doc]
+API which clients use to talk to the server.
+[This HTTP-based API][api-docs] is used equally by the Zulip web/desktop app,
+the official mobile app and terminal app, and
+numerous third-party integrations, bots, and tools
+written by many different Zulip users.
+
+Many new Zulip features involve making changes to the API.
+Because the Zulip API is so widely used, it's important that we
+do a good job of those changes, in several respects:
+
+- All API changes must be _compatible_, meaning they
+  don't cause existing Zulip clients
+  (e.g., existing installs of the Zulip mobile app)
+  to break.
+
+- All API changes should be _clean APIs_, meaning
+  the new API features have crisp meanings,
+  will support clients implementing the feature in a natural way,
+  and have clear names that point accurately to the meanings.
+
+- All API changes should be _clearly documented_
+  and follow our [API documentation conventions][api-docs-doc].
+
+The last point is covered in [its own document][api-docs-doc].
+This document describes how we accomplish the other two:
+compatible API changes, and clean APIs.
+
+[api-docs-doc]: ../documentation/api.md
+[api-docs]: https://zulip.com/api/
+[#api design]: https://chat.zulip.org/#narrow/channel/378-api-design
+
+## API change process, in short
+
+There are two essential rules for how we make API changes:
+
+1. When considering an API change,
+   **always discuss it in the [#api design][] channel**.
+
+2. Before merging a PR with an API change,
+   always make sure the change has been approved in that channel.
+
+As long as you follow those two rules,
+the maintainers who manage that channel
+can guide you through all the other aspects of the process,
+as described in this document.
+
+(For what counts as an "API change", see the next section.)
+
+## What is an API change?
+
+An "API change" is anything that
+adds, removes, or changes the meaning of
+anything that the server sends to clients,
+or that the server accepts from clients.
+
+For example, this includes:
+
+- adding or removing an endpoint (editing `zproject/urls.py`)
+- adding or removing parameters of an endpoint
+- adding or removing any fields in the initial snapshot
+  (the [/api/register-queue][] response)
+  or any other endpoint's response
+- changing the type of any of the above,
+  including adding new values to an enum type
+- renaming or moving any of the above
+- changing the meaning of any existing parameter or field
+
+In general, almost any change that touches
+the API spec file `zerver/openapi/zulip.yaml`
+will be an API change.
+(The few exceptions are things like clarifying the wording
+of existing documentation there.)
+Conversely, for most — but not all! — API changes,
+if you write tests that exercise the changes,
+then our test suite will ensure there's a corresponding change
+in the API spec file.
+
+Some types of API changes that one might not realize are API changes:
+
+- Any change to the structure of "[Zulip content HTML][]",
+  the HTML that the server emits for message content
+  (and a few other places like channel descriptions).
+  This includes:
+
+  - new CSS classes
+  - new `data-*` attributes or other attributes
+  - existing CSS classes or attributes appearing in new places,
+    or disappearing from some existing places
+
+- Any change to [push notification payloads][].
+
+All types of API changes need to
+follow the principles in this document
+and be discussed in [#api design][].
+
+[/api/register-queue]: https://zulip.com/api/register-queue
+[Zulip content HTML]: https://zulip.com/api/message-formatting
+[push notification payloads]: https://zulip.com/api/mobile-notifications
+
+## API design approval
+
+The [#api design][] channel has a few maintainers who have taken on
+the specific responsibility of managing that channel's discussions.
+Typically these include the lead of the Zulip mobile team,
+and one or two Zulip maintainers who work primarily in
+the server and/or web app.
+
+A given API change proposal is approved when
+one of the channel's maintainers says,
+in the change's thread in the channel,
+that the change is approved.
+
+Typically this will happen only after reaching consensus
+between both a mobile maintainer and a server or web maintainer,
+each agreeing that the change looks good from their perspective.
+
+If one of the channel maintainers has said the change looks good
+and you're not sure if they mean only from their perspective
+(so are waiting for a perspective from
+a maintainer of the other system)
+or they mean the change is approved and ready to merge,
+please ask so they can make the status more explicit.
+
+## API compatibility
+
+All changes to the Zulip API must be compatible:
+they must not cause existing Zulip clients to break.
+
+For some kinds of API changes this is easy.
+For others it takes more work.
+
+### Changes that are automatically compatible
+
+Some API changes are automatically compatible
+with no further effort.
+
+A first question for any API change is to determine
+whether this happy circumstance applies,
+because it saves us from needing to do the
+API compatibility work described in the next section.
+
+Some of the principles that maintainers will use
+in determining if a change is automatically compatible are:
+
+- Adding new endpoints, or new parameters on an endpoint,
+  is always compatible because existing clients
+  won't interact with these.
+
+  - This is true only so long as the meaning of existing requests
+    doesn't change, though.
+    For example, if a new parameter has a default
+    which is different from the endpoint's old behavior,
+    then that's a change in the meaning of the endpoint
+    when the new parameter isn't sent,
+    and therefore may not be compatible.
+
+- Adding new fields to an endpoint's response,
+  or a JSON object nested inside a response,
+  is always compatible.
+  A correct Zulip client already ignores any field it doesn't expect.
+
+  - Conversely, this means that if an existing piece of API
+    starts including by default data objects it previously didn't,
+    that change is _not_ automatically compatible,
+    even if the new objects have some new field to distinguish them —
+    because existing clients will ignore that field.
+
+    For example, when we started including archived channels in the
+    lists of channels in [/api/register-queue][], we didn't just start
+    including them while adding an `"is_archived": true` flag to them,
+    because if we did that then existing clients would have innocently
+    treated those as additional non-archived channels (and therefore
+    wrongly shown them in the list-of-channels screen, etc.).
+    Instead more compatibility work was required, per the next section.
+
+- In [Zulip content HTML][],
+  adding new attributes is compatible.
+  Adding new CSS classes, however, is _not_ automatically compatible.
+  In general, most changes to Zulip content HTML
+  are not automatically compatible.
+
+  Relevant discussion: [#api design > HTML pattern for truly inline images @ 💬](https://chat.zulip.org/#narrow/channel/378-api-design/topic/HTML.20pattern.20for.20truly.20inline.20images/near/2348105)
+
+- If a change affects only API features that are used only by the
+  Zulip web app and no other clients, then the change is compatible
+  even if it does things like remove or rename a field.
+
+  Typically this is true if a feature is used only in the settings UI.
+  It's sometimes true of other features too.
+
+  These changes are compatible because the web app is deployed with
+  the server, and clients automatically reload into the new web app
+  version soon after a server upgrade. Managing such changes typically
+  at most requires the Zulip Cloud team to split the web and
+  server changes across two consecutive deployments, which does not
+  justify permanent compatibility code.
+
+  To determine if this is the case:
+
+  - Check that the mobile app doesn't use the affected feature.
+    Always confirm this explicitly before deciding the feature
+    isn't used; occasionally the app does depend on an API feature
+    one wouldn't have expected.
+
+    This check needs to apply to both the latest version of the app
+    and any previous version that is
+    [still supported][supported-clients].
+    For example, if the mobile app previously used a given feature
+    and then stopped doing so, we can't remove the feature until
+    about 12 months after the mobile release that stopped using it.
+    (The mobile team lead makes removal decisions based on data
+    the app stores provide about the distribution of versions).
+
+  - Quickly check if the latest zulip/zulip-terminal might use the feature,
+    and file an issue if it might. It's typically a quick `git grep`.
+    We do not block API changes on terminal app compatibility.
+
+  - Consider whether it's likely that any third-party
+    integrations, bots, tools, or other clients
+    might use the feature.
+    Typically we figure they'll definitely use
+    certain key endpoints like [/api/send-message][];
+    won't use admin-only settings;
+    and things in between are less clear.
+    When in doubt, assume a feature might be used.
+    Zulip Cloud maintainers have access to logs that can help determine
+    to what extent an API endpoint is currently used by third-party code
+    in that environment, but we have no access to similar data
+    for self-hosted environments.
+
+[supported-clients]: ../overview/release-lifecycle.md#server-and-client-app-compatibility
+[/api/send-message]: https://zulip.com/api/send-message
+
+### Making a change compatible
+
+When an API change isn't automatically compatible
+(as described in the preceding section),
+we have to do some more work to make it compatible.
+
+The fundamental strategy here is to make the change **opt-in**.
+This means that existing clients, those unaware of the change,
+don't see the change.
+
+The basic way to make a change opt-in is to have the affected endpoint
+take a new parameter that enables the change, defaulting to off.
+For example, [`allow_empty_topic_name` on /api/get-messages][]
+
+When an API change affects the content of events,
+we need to use a variation on that approach:
+we add one more flag to the [`client_capabilities`][] parameter
+of [/api/register-queue][], again defaulting to off.
+The resulting event queue then applies the given "client capability",
+affecting all requests to [/api/get-events][] that use that queue.
+
+When a change is opt-in
+(through either of these ways or any other variation),
+it incurs an ongoing complexity cost in the server codebase
+because the server must continue to support the old form
+as well as the new form.
+For this reason it's good to eventually come back and
+remove support for the old form. You should add a
+`TODO/compatibility` marker in the server codebase detailing
+the plan for compatibility code that you expect to be able to
+delete when a given condition has occurred. You should review
+several examples of this pattern using `git grep TODO/compat -A10`
+before writing such a marker.
+
+That removal is itself an API change, though,
+so it needs to go through the process like any other change.
+
+Clients can help reduce this maintenance cost for the server
+implementation by promptly upgrading to request and properly handle
+the new form of recently changed APIs. (Even if the client app team
+doesn't have time to fully implement the new feature that motivated
+the API change).
+
+[`allow_empty_topic_name` on /api/get-messages]: https://zulip.com/api/get-messages#parameter-allow_empty_topic_name
+[`client_capabilities`]: https://zulip.com/api/register-queue#parameter-client_capabilities
+[/api/get-events]: https://zulip.com/api/get-events
+
+## Clean APIs
+
+Designing an API change cleanly isn't as immediately critical
+as making sure it's compatible:
+an API not being clean won't immediately cause anything to break.
+
+It's valuable to the project, though, that most of our API
+meets a fairly high standard of cleanliness and clarity.
+(Not all of it; we've made some mistakes.)
+
+This has helped us keep space for making more and more
+API changes without boxing ourselves in.
+
+It's also an essential ingredient in enabling the developers
+of the Zulip mobile app to efficiently implement each new feature,
+without having to reverse-engineer the server's behavior
+or work around awkward quirks
+that complicate a client's implementation.
+
+Similarly, one of Zulip's selling points for users is that
+our API lets users write custom tools that can do anything
+the official clients can do.
+For this promise to work out in practice,
+it's important that the API be reasonably clear
+and free of pitfalls.
+
+Writing a clean API is a matter of taste, experience,
+and engineering judgement,
+and can't straightforwardly be codified.
+With that in mind, though, here are some broad principles
+and questions to ask when reviewing a proposed API change
+for cleanliness and clarity:
+
+- Think through what a client-side implementation of the feature
+  (e.g., for web, or for mobile)
+  would look like.
+  Are there API revisions that would make that simpler or clearer?
+  Should logic for the API feature be implemented in the
+  server, the client, or both?
+
+- Think through the proposed semantics of the new API.
+  Make sure they make sense and seem like they'll solve
+  the intended problem.
+
+- Think through the proposed data structures,
+  and the names of parameters, fields, and enum values.
+  Make sure they're clear, they match the semantics,
+  and they aren't likely to accidentally suggest a meaning
+  that's different from the actual meaning.
+
+- Think about possible future extensions of the API feature
+  that is being implemented. Could the API design being
+  proposed force us to make an incompatible change if we
+  want to add a natural extension of the current feature?
+  (For example: Some early API fields were implemented as
+  tuples, rather than objects, and all of those resulted in
+  painful migration work to replace the tuples with objects
+  as additional fields were desired).
+
+## API change process
+
+The essential thing is that all API changes
+are discussed and approved in the [#api design][] channel.
+See [the description above](#api-change-process-in-short).
+
+More details on the process are below.
+
+### As a PR author
+
+If you're the author of a PR, please go ahead and
+start the [#api design][] thread yourself.
+Then put a link to it in the PR description.
+
+For any discussion that isn't specifically related to the API changes,
+please use a different channel such as [#backend][]. (You can
+cross-link, if a data model discussion has implications for the API,
+for example).
+
+Keeping [#api design][] focused is important for other participants
+such as the mobile team, so that
+they can be sure to follow the discussions on [#api design][]
+that are about API changes and are important for them.
+
+[#backend]: https://chat.zulip.org/#narrow/channel/3-backend
+
+### As a PR reviewer
+
+If you're reviewing a PR and it seems to make changes to the API,
+check the PR description for a link to an [#api design][] thread.
+If there isn't one already, ask about it in your review.
+You can also start the thread yourself.
+
+If you're reviewing a PR and it's near ready to merge,
+check to see if the [#api design][] thread has concluded
+with approval of a proposed change.
+Check also that the PR's API changes match the agreed-on plan.
+If the chat thread hasn't concluded, please post there
+to try to bring it toward a conclusion.
+
+### As an API reviewer
+
+After an API change is approved on [#api design][],
+the author on the server side prepares or revises a PR
+to implement the change.
+
+If you were involved in approving the API change —
+and particularly if maintaining our API quality is
+an important part of your work,
+e.g., because you're the lead of the mobile team —
+then it's helpful to review that PR for its API docs changes.
+That means totally ignoring the code and reading only
+`api_docs/` and `zerver/openapi/`.
+
+- Depending on the author's skills and their experience with
+  Zulip development, it's common for the draft docs to be
+  unclear or inaccurate.
+  Feedback on those is very helpful.
+
+- Sometimes the draft docs reveal a misunderstanding or
+  miscommunication about the API semantics that were agreed to.
+  Those cases are important to catch.
+
+### As a channel maintainer of [#api design][]
+
+If you're one of the maintainers responsible for the
+[#api design][] channel,
+try to catch up on threads there each workday.
+
+If a thread needs input from the system you work on
+(mobile, web, server),
+try to provide it or to identify someone to do so.
+
+If the proposed changes look good from your perspective
+but still need feedback from another perspective
+(e.g., server/web vs. mobile),
+try to make that clear, so the PR's author or reviewer
+doesn't mistakenly think the changes are already approved. An example
+remark might be:
+
+> This looks good to me, but we need someone from the mobile team
+> to confirm ...
+
+Conversely, if all perspectives are either already represented
+or you believe they're unnecessary for the particular change,
+then say explicitly that the API changes are approved
+so that the PR's author and reviewer know they can move forward.
+
+Once an API change is approved,
+consider filing issues to implement it for mobile and other clients.
+(Or asking, on the thread, for someone else to file those.)
+This is useful for high-priority changes in order to move quickly.
+For most changes, we wait until the server PR has been merged
+and a bot posts about it in [#api documentation][].
+
+[#api documentation]: https://chat.zulip.org/#narrow/channel/412-api-documentation
+
+### As the mobile team lead
+
+If you're leading the mobile team, that probably means
+you're a maintainer of the [#api design][] channel
+and an API reviewer for most API changes.
+See the corresponding subsections above.
+
+After each API change is merged into the server,
+a bot will post a new thread in [#api documentation][].
+You should make sure you or someone else
+replies to explicitly decide if there should be a mobile issue
+to track making a corresponding change in the mobile app,
+and files or updates such an issue if needed.
+(See [#api documentation > new feature level: 454 @ 💬](https://chat.zulip.org/#narrow/channel/412-api-documentation/topic/new.20feature.20level.3A.20454/near/2360660).)

docs/processes/api-release-checklist.md

@@ -0,0 +1,93 @@
+# Zulip PyPI packages release checklist
+
+Zulip manages the following three PyPI packages from the
+[zulip/python-zulip-api][python-zulip-api] repository:
+
+- [zulip][zulip-package]: The package containing the
+  [Zulip API](https://zulip.com/api/) Python bindings.
+- [zulip_bots][zulip-bots-package]: The package containing
+  [Zulip's interactive bots](https://zulip.com/api/running-bots).
+- [zulip_botserver][zulip-botserver-package]: The package for Zulip's Botserver.
+
+The `python-zulip-api` packages are often released all together. Here is a
+checklist of things one must do before making a PyPI release:
+
+1. Increment `__version__` in `zulip/__init__.py`, `ZULIP_BOTS_VERSION` in
+   `zulip_bots/setup.py`, and `ZULIP_BOTSERVER_VERSION` in
+   `zulip_botserver/setup.py`. They should all be set to the same version
+   number.
+
+2. Set `IS_PYPA_PACKAGE` to `True` in `zulip_bots/setup.py`. **Note**:
+   Setting this constant to `True` prevents `setup.py` from including content
+   that should not be a part of the official PyPI release, such as logos,
+   assets and documentation files. However, this content is required by the
+   [Zulip server repo][zulip-repo] to render the interactive bots on
+   [Zulip's integrations page](https://zulip.com/integrations/). The server
+   repo installs the `zulip_bots` package
+   directly from the GitHub repository so that this extra
+   content is included in its installation of the package.
+
+3. Follow PyPI's instructions in
+   [Generating distribution archives][generating-dist-archives] to generate the
+   archives for each package. It is recommended to manually inspect the build output
+   for the `zulip_bots` package to make sure that the extra files mentioned above
+   are not included in the archives.
+
+4. Follow PyPI's instructions in [Uploading distribution archives][upload-dist-archives]
+   to upload each package's archives to TestPyPI, which is a separate instance of the
+   package index intended for testing and experimentation. **Note**: You need to
+   [create a TestPyPI](https://test.pypi.org/account/register/) account for this step.
+
+5. Follow PyPI's instructions in [Installing your newly uploaded package][install-pkg]
+   to test installing all three packages from TestPyPI.
+
+6. If everything goes well in step 5, you may repeat step 4, except this time, upload
+   the packages to the actual Python Package Index.
+
+7. Once the packages are uploaded successfully, set `IS_PYPA_PACKAGE` to `False` in
+   `zulip_bots/setup.py` and commit your changes with the version increments. Push
+   your commit to `python-zulip-api`. Create a release tag and push the tag as well.
+   See [the tag][example-tag] and [the commit][example-commit] for the 0.8.1 release
+   to see an example.
+
+Now it is time to [update the dependencies](../subsystems/dependencies.md) in the
+[Zulip server repository][zulip-repo]:
+
+1. Increment `PROVISION_VERSION` in `version.py`. A minor version bump should suffice in
+   most cases.
+
+2. Update the release tags in the Git URLs for `zulip` and `zulip_bots` in
+   `pyproject.toml`.
+
+3. Run `uv lock` to update the Python lock file.
+
+4. Commit your changes and submit a PR! **Note**: See
+   [this example commit][example-zulip-commit] to get an idea of what the final change
+   looks like.
+
+## Other PyPI packages maintained by Zulip
+
+Zulip also maintains two additional PyPI packages:
+
+- [fakeldap][fakeldap]: A simple package for mocking LDAP backend servers
+  for testing.
+- [virtualenv-clone][virtualenvclone]: A package for cloning a non-relocatable
+  virtualenv.
+
+The release process for these two packages mirrors the release process for the
+`python-zulip-api` packages, minus the steps specific to `zulip_bots` and the
+update to dependencies required in the [Zulip server repo][zulip-repo].
+
+[zulip-repo]: https://github.com/zulip/zulip
+[python-zulip-api]: https://github.com/zulip/python-zulip-api
+[zulip-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip
+[zulip-bots-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip_bots
+[zulip-botserver-package]: https://github.com/zulip/python-zulip-api/tree/main/zulip_botserver
+[generating-dist-archives]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives
+[upload-dist-archives]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#uploading-the-distribution-archives
+[install-pkg]: https://packaging.python.org/en/latest/tutorials/packaging-projects/#installing-your-newly-uploaded-package
+[example-tag]: https://github.com/zulip/python-zulip-api/releases/tag/0.8.1
+[example-commit]: https://github.com/zulip/python-zulip-api/commit/fec8cc50c42f04c678a0318f60a780d55e8f382b
+[example-zulip-commit]: https://github.com/zulip/zulip/commit/0485aece4e58a093cf45163edabe55c6353a0b3a#
+[fakeldap]: https://github.com/zulip/fakeldap
+[virtualenvclone]: https://pypi.org/project/virtualenv-clone/

docs/processes/index.md

@@ -0,0 +1,12 @@
+# Project processes
+
+```{toctree}
+---
+maxdepth: 3
+---
+
+api-design
+integration-review
+release-checklist
+api-release-checklist
+```

docs/processes/integration-review.md

@@ -0,0 +1,189 @@
+# Integration review
+
+This page details the process of integration review for pull requests.
+Integration review serves several purposes:
+
+- A second opinion to the maintainer review.
+- An end-to-end check that a PR's changes have the necessary approvals as
+  part of our process of getting the attention of subject-matter experts on
+  changes that could result in expensive mistakes.
+- An end-to-end check that potential risks have been suitably managed.
+- An opportunity to kick off follow-up work.
+
+Key resources that you should understand well before doing integration
+review on general changes:
+
+- Our general maintainer review resources, including the [review
+  process](../contributing/review-process.md)
+- The process for [design
+  discussions](../contributing/design-discussions.md), and how to
+  determine whether the design/UI component of a project has
+  consensus/approval. It is very common for eager contributors to
+  ask you to merge something that would seriously regress the UI and
+  has not been suitably discussed.
+- How [API design](../processes/api-design.md) decisions are made. It is
+  the responsibility of the integration reviewer to confirm that an API
+  change has been approved before integrating a change. It is common
+  for an integration review result to be "This looks great, except I can't
+  tell if we have API design approval for this field name."
+- Zulip's [testing philosophy](../testing/philosophy.md).
+- [Database migrations](../subsystems/schema-migrations.md), including the
+  additional considerations related to Cloud (online migrations due to
+  large data size) and self-hosting (migrations need to Just Work or they
+  will generate ~infinite unpleasant support work as installations upgrade).
+- Our [tools](../git/zulip-tools.md) for working with contributor PRs.
+- What failure modes you can expect existing [automated
+  tests](../testing/testing.md) to reliably catch, so that you can
+  focus your attention on failure modes that they cannot. If you
+  ever see a bug in a PR that you thought the testing system should
+  have caught, report that as a tooling bug.
+- How to tell whether a change to the Zulip data model is a good one.
+- Familiarity with the full set of pages in this ReadTheDocs site
+  is extremely helpful. You don't need to have read every page, but
+  you should know what exists, or have habits of searching/grepping
+  that ensure you are making decisions with full access to
+  relevant project context and history.
+
+(Obviously, use your judgement. If you're only merging CSS changes,
+it's not critical that you understand how to check if a database
+schema change is risky, just to know that's a category of change that
+you should hand off to a different reviewer.)
+
+## Risks to manage
+
+### General
+
+- Is this a user experience checkpoint that we'd be OK with going out
+  to production by accident, or reaching self-hosters who upgrade to
+  `main`? Would a UX change expose users to risk or breakage that we'd
+  find embarrassing?
+
+### Readability
+
+- Function reuse land mines, where a function does not do exactly what
+  its name suggests it will do.
+- Cross-linking of documentation to help understanding. E.g., if we
+  have a nice document explaining an algorithm we just wrote, is it
+  linked to from all the places that should? (Code comments in that
+  code path, related docs, etc.)
+- Code that is hard to read. If you notice yourself spending a bit
+  trying to figure out what some code does, and succeed, always ask:
+  "How could this code be improved or commented so that the next
+  person doesn't find it hard to follow?"
+
+### Code quality
+
+- Excessive testing costs, either for humans to read or requiring
+  excessive runtime.
+- Logic that violates the product's security model.
+- Security land mines, where security depends on each caller of a
+  function checking separately the properties central to the
+  correctness of a code path.
+- Caching and performance mistakes. E.g., it is far easier to catch
+  accidentally quadratic code before you've merged it.
+
+### Server
+
+- Transaction model issues, either work that should be atomic
+  but needlessly isn't, or multi-minute transactions.
+- Audit logs and information destruction risks.
+
+### API documentation
+
+- Occasionally, we will want to merge the API and backend changes for
+  a new feature without also implementing that API/feature in the web
+  app. In this case, any new API endpoints should be labeled with the
+  `x-pending-web-implementation` tag and you'll want to open an issue
+  for the web app implementation of the feature so that it's tracked.
+
+### Web app
+
+- Cross-site scripting bugs. Pay special attention to `{{{` in
+  Handlebars templates and manual HTML encoding/decoding.
+- New Puppeteer tests should have a convincing case for why they are
+  worth their runtime.
+- Any changes involving browser local storage can have major
+  compatibility issues that break the app.
+- UI churn risk if we're still iterating on something in core
+  workflows (not just settings).
+- Error handling that could make a future error/mistake worse. E.g.,
+  anything touching the logic that reloads the web app.
+- API documentation reflects that the feature is implemented in the
+  web app; see API changes point above.
+
+## Checklist
+
+Here are some mental tests to consider applying when merging a pull
+request:
+
+- Could this PR fail CI after being rebased onto `upstream/main`? When
+  in doubt, manually rebase and configure to merge once CI passes,
+  instead of using an immediate "Rebase and merge".
+- Are there any follow-ups to this work that we'd be sad to discover
+  in 4 months hadn't been done? If so, what is the concrete
+  plan for ensuring those are integrated soon? Are follow-ups tracked on the
+  release board, or assigned to an owner who can be counted on to do
+  it?
+- Did you remember to express appreciation for the work, including
+  making sure the project has thanked the original reporter?
+- If we discover a serious problem tomorrow that requires reverting
+  this, what will happen?
+- Do commit messages correctly and helpfully explain the commits we're
+  actually merging? (Sometimes contributors redo the change a lot
+  without editing these).
+- Are there any risks with sending this to Cloud or self-hosted
+  systems that the review process this far may not have validated?
+
+## Mistakes
+
+We are all human, and you will definitely at some point integrate
+something you didn't intend to or later discover was a mistake. It's
+important to avoid disrupting others' work, and a broken `main` branch
+is a good reason to push changes to `main` without waiting for CI to
+run, if you are confident your fix is correct.
+
+You have a few options:
+
+- **Fix forward**. This is best when the mistake is a simple CI
+  failure, like a lint issue: just fix the linter and push a fix
+  promptly. It is also a good choice for less urgent mistakes that
+  can happily be a follow-up improvement.
+- **Revert the change** in a new commit. This is recommended in any
+  case where you cannot resolve an issue that will impact others' work
+  via a fix forward quickly. Note that `git revert` accepts a range of
+  commits. It may feel ugly, but it's fine.
+
+The following option also exists:
+
+- **Force-push**. This is necessary if the incorrectly merged changes
+  are problematic to have in the project's history. (Say, you checked
+  in a 2GB video or your Driver's License). Note that Weblate will
+  require manual fixing if you force-push in `zulip/zulip`, so be sure
+  to check the translation channel to manage this.
+
+Some common mistakes to watch out for:
+
+- Pushing revisions to the wrong PR. If you run
+  `tools/push-to-pull-request <ID>` and paste in the wrong pull
+  request ID, you'll have overwritten code was on that other PR. The
+  console output will contain the old commit ID for the overwritten
+  PR, which can sometimes let you recover if you have a copy of
+  that. But typically, you have to just ask the PR's author to
+  re-upload the correct code to their pull request.
+
+  (If you did `--merge`, GitHub offers no undo for a merged PR, so the
+  only option is to post comments cross-linking and explaining what
+  happened).
+
+- Merging a pull request with API changes without running
+  `tools/merge-api-changelogs` to allocate a feature level number. CI
+  should complain, and you can then run the merge command in a
+  follow-up commit. If the commit history has many separate feature-level
+  commits when merging a PR that stacked several API changes from
+  different commits in a feature level, it's not a big deal. (But
+  typically, we squash the `merge-api-changelogs` commit when there's
+  only one commit in a PR touching the API, to keep the history clean).
+
+- Merging a pull request with migrations without rebasing the work
+  and running `tools/renumber-migrations`. PRs older than a couple
+  weeks are very likely to have duplicate migration numbers.

docs/processes/release-checklist.md

@@ -0,0 +1,163 @@
+# Zulip server release checklist
+
+This document has reminders of things one might forget to do when
+preparing a new release.
+
+### A week before the release
+
+- _Major releases only (e.g., 4.0):_
+  - Upgrade all Python dependencies in
+    `requirements` to latest upstream versions so they can burn in (use
+    `pip list --outdated`).
+  - Upgrade all puppet dependencies in `puppet/deps.yaml`
+  - Upgrade all puppet-installed dependencies (e.g., Smokescreen, go,
+    etc) in `puppet/zulip/manifests/common.pp`
+  - [Post a message to
+    Weblate](https://hosted.weblate.org/projects/zulip/#announcement)
+    inviting translators to translate new strings.
+  - Prepare draft updates to the [changelog](../overview/changelog.md)
+    with changes since the last release. The `/write-changelog` experimental
+    Claude skill may be a better starting point than the original
+    `git log --stat 12.x..upstream/main` marathon workflow.
+  - Use updating the changelog and drafting the release blog post as review
+    passes to identify:
+    - Projects that require follow-up work.
+    - Documentation that needs to be updated.
+  - Update /features and other pages (e.g., /for/business) with major new
+    features, and important changes to existing features.
+  - Draft newsletter.
+  - Draft email and social media announcements.
+- Create a burn-down list of issues that need to be fixed before we can
+  release, and make sure all of them are being worked on.
+- Draft the release blog post (a.k.a. the release notes) in Paper. In
+  it, list the important changes in the release, from most to least
+  notable (or organized by category for major releases).
+- Collect a list of folks who would like to be mentioned in the blog
+  post for their non-code contributions during the release cycle. See
+  [CZO thread for previous releases][non-code-contributions-topic].
+
+### Final release preparation
+
+- Update the Paper blog post draft with any new commits.
+- Merge updated translations from Weblate (using the appropriate
+  branch for the release).
+- Send around the Paper blog post draft for review.
+- Move the blog post draft to Astro:
+  - Use "··· > Export > Markdown" to get a pretty good Markdown
+    conversion, and save it in `src/posts` with a filename appropriate
+    for a URL slug.
+  - Add the needed YAML frontmatter.
+  - Move any images into `public` and update their references.
+  - Proofread, especially for formatting.
+  - If the draft post should remain secret until release, avoid using
+    a guessable Git branch name for the pull request (the deployment
+    preview URL is based on the branch name).
+- _Major releases only (e.g., 4.0):_ Schedule team members to provide
+  extra responsive #production help support following the release.
+
+### Executing the release
+
+- Create the release commit, on `main` (for major releases) or on the
+  release branch (for minor releases):
+  - Copy the Markdown release notes for the release into
+    `docs/overview/changelog.md`.
+  - Verify the changelog passes lint, and has the right release date.
+  - _Major releases only:_ Adjust the `changelog.md` heading to have
+    the stable release series boilerplate.
+  - Update `ZULIP_VERSION` and `LATEST_RELEASE_VERSION` in `version.py`.
+  - _Major releases only:_ Update `API_FEATURE_LEVEL` to a feature
+    level for the final release, and document a reserved range.
+- Run `tools/release` with the release version.
+- Update the [Docker image](https://github.com/zulip/docker-zulip):
+  - Commit the Docker updates:
+    - Update `ZULIP_GIT_REF` in `Dockerfile`
+    - Update `README.md`
+    - Update the image in `docker-compose.yml`, as well as the `ZULIP_GIT_REF`
+  - Commit the Helm updates:
+    - Add a new entry to `kubernetes/chart/zulip/CHANGELOG.md`
+    - Update the `appVersion` in `kubernetes/chart/zulip/Chart.yaml`
+    - Update the `tag` in `kubernetes/chart/zulip/values.yaml`
+    - Update the docs by running `helm-docs`
+    - Update the `image` in `kubernetes/manual/zulip-rc.yml`
+  - Build the image: `docker build --pull . -t zulip/docker-zulip:4.11-0 --no-cache`
+  - Also tag it with `latest`: `docker build . -t zulip/docker-zulip:latest`
+  - Push those tags: `docker push zulip/docker-zulip:4.11-0; docker push zulip/docker-zulip:latest`
+  - Push the commits to `main`.
+- Merge the blog post PR.
+- Announce the release, pointing to the blog post, via:
+  - Email to [zulip-announce](https://groups.google.com/g/zulip-announce)
+  - Email to [zulip-blog-announce](https://groups.google.com/a/zulip.com/g/zulip-blog-announce)
+  - Message in [#announce](https://chat.zulip.org/#narrow/channel/1-announce)
+  - Post from [@zulip.bsky.social](https://bsky.app/profile/zulip.bsky.social).
+  - Toot from [fosstodon.org/@zulip](https://fosstodon.org/@zulip)
+  - Post from [Kandra Labs LinkedIn](https://linkedin.com/company/zulip-by-kandra-labs/)
+
+### Post-release
+
+- The DigitalOcean one-click image will report in an internal channel
+  once it is built, and how to test it. Verify it, then publish it to
+  DigitalOcean marketplace.
+- _Major releases only:_
+  - Create a release branch (e.g., `4.x`).
+  - On the release branch, update `ZULIP_VERSION` in `version.py` to
+    the present release with a `+git` suffix, e.g., `4.0+git`.
+  - On `main`, update `ZULIP_VERSION` to the future major release with
+    a `-dev+git` suffix, e.g., `5.0-dev+git`. Make a Git tag for this
+    update commit with a `-dev` suffix, e.g., `5.0-dev`. Push the tag
+    to both zulip.git and zulip-internal.git to get a correct version
+    number for future Cloud deployments.
+  - Add the new release to `.github/ISSUE_TEMPLATE/2_bug_report.md`.
+  - Consider removing a few old releases from the issue template and
+    ReadTheDocs; we keep about two years of back-versions.
+  - Update Weblate to add a component on the release branch for
+    Django; then add a parallel Frontend component by using "Duplicate
+    this component" on the Django release branch component.
+  - In Weblate, remove the previous stable components.
+  - Add a new CI production upgrade target:
+    - Build a docker image: `cd tools/ci && docker build --pull . -f Dockerfile.prod --build-arg=BASE_IMAGE=zulip/ci:bookworm --build-arg=VERSION=7.0 --tag=zulip/ci:bookworm-7.0 && docker push zulip/ci:bookworm-7.0`
+    - Add a new line to the `production_upgrade` matrix in
+      `.github/workflows/production-suite.yml`.
+  - Update /history page in `templates/corporate/history.md`, adding the release
+    and any other major news from the release cycle.
+  - Inspect all `TODO/compatibility` comments for whether we can
+    remove any backwards-compatibility code following this release.
+  - Review possible improvements to API bindings to better match the
+    defaults and features of the new release.
+- _Minor releases only (e.g., 3.2):_
+  - On the release branch, update `ZULIP_VERSION` to the present
+    release with a `+git` suffix, e.g., `3.2+git`.
+  - On main, update `LATEST_RELEASE_VERSION` with the released
+    version, as well as the changelog changes from the release branch.
+- _Prereleases only (e.g., 7.0-beta3):_
+  - Atop the prerelease commit (e.g., `7.0-beta3`), make a commit
+    updating `ZULIP_VERSION` to the prerelease version with a `+git`
+    suffix, e.g., `7.0-beta3+git`. Push this to `main`. (If `main` has
+    already diverged from the prerelease, a merge commit will be
+    needed here.)
+  - Delete the prerelease branch (e.g., `7.0-beta3-branch`); it's now
+    an ancestor of `main` and thus unnecessary.
+
+### Important permissions for executing the release
+
+Note that the main steps of pushing the release through `tools/release` require
+several distinct permissions. Ensure that the person executing the release has
+these permissions - this is particularly important for a security release, to
+avoid getting stuck in a half-way state where the final commits have already
+been pushed but the release can't be finalized.
+
+- AWS credentials that authorize you to upload the tarball to
+  `download.zulip.com`. This is needed by the `tools/upload-release`
+  step. You will need `aws-cli` installed and logged in. The simplest
+  way to set up credentials for use by `upload-release` is to export
+  temporary credentials into your environment, obtained via
+  `aws configure export-credentials --format env`.
+- `git` permission to push to the `<major version>.x` branch.
+- `git` permission to push the `<major version>.<minor version>` tag.
+  Note that tags matching this specific release tag format can be
+  subject to GitHub rules restricting who can push them - so check
+  this and don't assume that you can push these tags just because you
+  can push ordinary tags such as `foo`.
+- The GitHub `gh` CLI tool, logged into your account, with permission
+  to create releases.
+
+[non-code-contributions-topic]: https://chat.zulip.org/#narrow/channel/1-announce/topic/recognition.20for.20non-code.20contributions/with/2443350