snapshot of https://github.com/zulip/zulip.git @ 48c69911c7dd4e03b20552eb3bd77ad8151c1357 (2026-09-15)

docs/documentation/api.md

@@ -0,0 +1,631 @@
+# Documenting REST API endpoints
+
+This document explains the system for documenting [Zulip's REST
+API](https://zulip.com/api/rest). See also the [API design
+documentation](../processes/api-design.md) which covers how to safely
+make changes to the Zulip API.
+
+Zulip's API documentation is an essential resource both for users and
+for the developers of Zulip's mobile and terminal apps. Our vision is
+for the documentation to be sufficiently good that developers of
+Zulip's apps should never need to look at the server's implementation
+to answer questions about the API's semantics.
+
+To achieve these goals, Zulip leverages the popular OpenAPI format as
+the data source to ensure that Zulip's API documentation is correct
+and remains so as Zulip's API evolves.
+
+In particular, the top goal for this system is that all mistakes in
+verifiable content (i.e. not the English explanations) should cause
+the Zulip test suite to fail. This is incredibly important, because
+once you notice one error in API documentation, you no longer trust it
+to be correct, which ends up wasting the time of its users.
+
+Since it's very difficult to not make little mistakes when writing any
+untested code, the only good solution to this is a way to test
+the documentation. We found dozens of errors in the process of adding
+the validation Zulip has today.
+
+Our API documentation is defined by a few sets of files:
+
+- The primary source of our API documentation is the Zulip server's
+  [OpenAPI description](openapi.md) at
+  `zerver/openapi/zulip.yaml`.
+- The documentation is written the same Markdown framework that powers
+  our [integration docs](integrations.md), with some special
+  extensions for rendering nice code blocks and example
+  responses. Most API endpoints share a common template,
+  `api_docs/api-doc-template.md`, which renders the
+  OpenAPI description of the API endpoint. A handful of endpoints that
+  require special content, as well as pages that document general API
+  details rather than specific endpoints, live at `api_docs/*.md`.
+- We have an extensive set of tests designed to validate that the data
+  in the OpenAPI file matching the implementation. Specifically,
+  `zerver/tests/test_openapi.py` compares every endpoint's accepted
+  parameters in `views` code with those declared in `zulip.yaml`. And
+  the [backend test suite](../testing/testing-with-django.md) checks
+  that every API response served during our extensive backend test
+  suite matches one the declared OpenAPI schema for that endpoint.
+- The text for the Python examples comes from a test suite for the
+  Python API documentation (`zerver/openapi/python_examples.py`; run via
+  `tools/test-api`). The `generate_code_example` macro will magically
+  read content from that test suite and render it as the code example.
+  This structure ensures that Zulip's API documentation is robust to a
+  wide range of possible typos and other bugs in the API
+  documentation.
+- The JavaScript examples are similarly generated and tested using
+  `zerver/openapi/javascript_examples.js`.
+- The cURL examples are generated and tested using
+  `zerver/openapi/curl_param_value_generators.py`.
+- The REST API index
+  (`api_docs/include/rest-endpoints.md`) in the broader
+  /api left sidebar (`api_docs/sidebar_index.md`).
+
+This first section is focused on explaining how the API documentation
+system is put together; when actually documenting an endpoint, you'll
+want to also read the [Step by step guide](#step-by-step-guide).
+
+## How it works
+
+Let's use the existing documentation for one of our REST API endpoints
+to show how the documentation system works:
+[POST /messages/render](https://zulip.com/api/render-message).
+We highly recommend looking at these resources while reading the above
+documentation page:
+
+- `api_docs/api-doc-template.md`
+- `zerver/openapi/zulip.yaml`, specifically the section with
+  `operationId: render-message`
+- `zerver/openapi/python_examples.py`
+
+If you look at the documentation for existing endpoints, you'll notice
+that a typical endpoint's documentation is divided into four sections:
+
+- **Title and description**
+- **Usage examples**
+- **Parameters**
+- **Response with examples**
+
+The rest of this guide describes how each of these sections works.
+
+### Title and description
+
+The first line of `api-doc-template.md` generates a lot of key
+information for our API endpoint documentation:
+
+```
+{generate_api_header(API_ENDPOINT_NAME)}
+```
+
+At the top of the endpoint documentation page is the title, and it
+comes from the `summary` parameter in the OpenAPI data,
+`zerver/openapi/zulip.yaml`.
+
+The endpoint `description` in the OpenAPI data explains what the
+endpoint does in clear English. It should include details on how to
+use the endpoint correctly or what it's good or bad for, with links
+to any alternative endpoints the user might want to consider.
+
+The description should often contain a link to the documentation of
+the relevant feature in the [help center](helpcenter.md), and should
+include **Changes** notes for all feature level updates documented
+in the [API changelog](https://zulip.com/api/changelog), see
+`api_docs/changelog.md`, that reference the endpoint.
+
+The description itself should always describe the behavior of
+current servers, so that a reader never needs to consult a
+**Changes** note to understand it. The **Changes** notes exist for
+understanding the behavior of older servers, starting from that of
+current servers. So when documenting a change to existing behavior,
+write the note as a diff going backward in time, giving only the
+details of the old behavior: for example, "Before Zulip 12.0
+(feature level 483), the server did not automatically remove legacy
+registrations", rather than "As of Zulip 12.0 (feature level 483),
+the server now automatically removes...". The forward framing
+misleadingly suggests that the note carries all the details of the
+new behavior, when those belong in the main description.
+
+Endpoints that only administrators can use should be tagged with the
+custom `x-requires-administrator` field in the OpenAPI definition.
+
+All of this information is rendered via a Markdown preprocessor,
+specifically the `APIHeaderPreprocessor` class defined in
+`zerver/openapi/markdown_extension.py`.
+
+### Usage examples
+
+We display usage examples in three languages: Python, JavaScript and
+curl; we may add more in the future. Every endpoint should have
+Python and curl documentation; JavaScript is optional as we don't
+consider that API library to be fully supported.
+
+The examples are defined using a special Markdown extension, see
+`zerver/openapi/markdown_extension.py`. Here's the Markdown file
+block that uses this in `api-doc-template.md`:
+
+```md
+{start_tabs}
+
+{generate_code_example(python)|API_ENDPOINT_NAME|example}
+
+{generate_code_example(javascript)|API_ENDPOINT_NAME|example}
+
+{tab|curl}
+
+{generate_code_example(curl)|API_ENDPOINT_NAME|example}
+
+{end_tabs}
+```
+
+In some cases, one wants to configure specific parameters to be
+included or excluded from the example curl requests for readability
+reasons. One can do that using the `x-curl-examples-parameters`
+parameter in the OpenAPI data.
+
+#### Writing Python examples
+
+For the Python examples, you'll write the example in
+`zerver/openapi/python_examples.py`, and it'll be run and verified
+automatically in Zulip's automated test suite. The code for our
+example API endpoint looks like this:
+
+```python
+@openapi_test_function('/messages/render:post')
+def render_message(client: Client) -> None:
+    # {code_example|start}
+    # Render a message
+    request = {
+        'content': '**foo**'
+    }
+    result = client.render_message(request)
+    # {code_example|end}
+
+    validate_against_openapi_schema(result, '/messages/render', 'post', '200')
+```
+
+This is an actual Python function which will be run as part of the
+`tools/test-api` test suite. The `validate_against_openapi_schema`
+function will verify that the result of that request is as defined in
+the examples in `zerver/openapi/zulip.yaml`.
+
+To run as part of the test suite, the `render_message` function needs
+to be called from `test_messages` (or one of the other functions at
+the bottom of the file). The final function, `test_the_api`, is what
+actually runs the tests. Tests with the `openapi_test_function`
+decorator that are not called will fail tests, as will new endpoints
+that are not covered by an `openapi_test_function`-decorated test.
+
+You will still want to manually test the example using Zulip's Python
+API client by copy-pasting from the website; it's easy to make typos
+and other mistakes where variables are defined outside the tested
+block, and the tests are not foolproof.
+
+The code that renders API documentation pages will extract the block
+between the `# {code_example|start}` and `# {code_example|end}` comments,
+and substitute it in place of
+`{generate_code_example(python)|/messages/render:post|example}`. Note
+that here the `API_ENDPOINT_NAME` has been filled in with our example
+endpoint's information.
+
+Additional Python imports can be added using the custom
+`x-python-examples-extra-imports` field in the OpenAPI definition.
+
+### Parameters
+
+We have a separate Markdown extension to document the parameters that
+an API endpoint supports. Implemented in
+`zerver/lib/markdown/api_arguments_table_generator.py`, you can see
+this in `api-doc-template.md` after the **Parameters** header:
+
+```
+{generate_api_arguments_table|zulip.yaml|API_ENDPOINT_NAME}
+```
+
+This generates the information from the endpoint's parameter
+definition in the OpenAPI data.
+
+Additional content that you'd like to appear in the parameter
+description area can be declared using the custom
+`x-parameter-description` field in the OpenAPI definition.
+
+### Response with examples
+
+Similar to the parameters section above, there is a separate Markdown
+extension to document the endpoint's return values and generate the
+example response(s) from the OpenAPI data. Implemented in
+`zerver/lib/markdown/api_return_values_table_generator.py`, you can
+see this in after the **Response** header in `api-doc-template.md`:
+
+```
+{generate_return_values_table|zulip.yaml|API_ENDPOINT_NAME}
+```
+
+To generate the example responses from the OpenAPI data, we again
+use the special Markdown extension from the **Usage examples**
+discussed above, except with the `fixture` argument instead of the
+`example` argument:
+
+```
+{generate_code_example|API_ENDPOINT_NAME|fixture}
+```
+
+Additional content that you'd like to appear in the responses part of
+the page can be added using the custom `x-response-description` field
+in the OpenAPI definition.
+
+## Step by step guide
+
+This section offers a step-by-step process for adding documentation
+for a new API endpoint. It assumes you've read and understood the
+above.
+
+1. Start by adding [OpenAPI format](openapi.md)
+   data to `zerver/openapi/zulip.yaml` for the endpoint. If you
+   copy-paste (which is helpful to get the indentation structure
+   right), be sure to update all the content that you copied to
+   correctly describe your endpoint!
+
+   In order to do this, you need to figure out how the endpoint in
+   question works by reading the code! To understand how arguments
+   are specified in Zulip backend endpoints, read our [REST API
+   tutorial][rest-api-tutorial], paying special attention to the
+   details of `typed_endpoint`.
+
+   Once you understand that, the best way to determine the supported
+   arguments for an API endpoint is to find the corresponding URL
+   pattern in `zproject/urls.py`, look up the backend function for
+   that endpoint in `zerver/views/`, and inspect its keyword-only
+   arguments.
+
+   You can check your formatting using these helpful tools.
+
+   - `tools/check-openapi.ts` will verify the syntax of `zerver/openapi/zulip.yaml`.
+   - `tools/test-backend zerver/tests/test_openapi.py`; this test compares
+     your documentation against the code and can find many common
+     mistakes in how arguments are declared.
+   - `test-backend`: The full Zulip backend test suite will fail if
+     any actual API responses generated by the tests don't match your
+     defined OpenAPI schema. Use `test-backend --rerun` for a fast
+     edit/refresh cycle when debugging.
+
+   [rest-api-tutorial]: ../tutorials/writing-views.md#writing-api-rest-endpoints
+
+1. Add a function for the endpoint you'd like to document to
+   `zerver/openapi/python_examples.py`, decorated with
+   `@openapi_test_function`. `render_message` is a good example to
+   follow. There are generally two key pieces to your test: (1) doing
+   an API query and (2) verifying its result has the expected format
+   using `validate_against_openapi_schema`.
+
+1. Make the desired API call inside the function. If our Python
+   bindings don't have a dedicated method for a specific API call,
+   you may either use `client.call_endpoint` or add a dedicated
+   function to the [zulip PyPI
+   package](https://github.com/zulip/python-zulip-api/tree/main/zulip).
+   Ultimately, the goal is for every endpoint to be documented the
+   latter way, but it's useful to be able to write working
+   documentation for an endpoint that isn't supported by
+   `python-zulip-api` yet.
+
+1. Add the function to one of the `test_*` functions at the end of
+   `zerver/openapi/python_examples.py`; this will ensure your
+   function will be called when running `test-api`.
+
+1. Capture the JSON response returned by the API call (the test
+   "fixture"). The easiest way to do this is add an appropriate print
+   statement (usually `json.dumps(result, indent=4, sort_keys=True)`),
+   and then run `tools/test-api`. You can also use
+   <https://jsonformatter.curiousconcept.com/> to format the JSON
+   fixtures. Add the fixture to the `example` subsection of the
+   `responses` section for the endpoint in
+   `zerver/openapi/zulip.yaml`.
+
+1. Run `./tools/test-api` to make sure your new test function is being
+   run and the tests pass.
+
+1. Now, inside the function, isolate the lines of code that call the API and could
+   be displayed as a code example. Wrap the relevant lines in
+   `# {code_example|start} ... relevant lines go here ... # {code_example|end}`
+   comments. The lines inside these comments are what will be displayed as the
+   code example on our `/api` page.
+
+1. Finally, if the API docs page of the endpoint doesn't follow the
+   common API docs template in
+   `api_docs/api-doc-template.md`, then add its custom
+   Markdown file under `api_docs/`. However, it is a goal
+   to minimize the number of files that diverse from the common
+   template, so only do this if there's a good reason.
+
+1. Add the endpoint to the index in
+   `api_docs/include/rest-endpoints.md`. The URL should
+   match the `operationId` for the endpoint, and the link text should
+   match the title of the endpoint from the OpenAPI `summary` field.
+
+1. Test your endpoint, pretending to be a new user in a hurry, by
+   visiting it via the links on `http://localhost:9991/api/` (the API
+   docs are rendered from the Markdown source files on page load, so
+   just reload to see an updated version as you edit). You should
+   make sure that copy-pasting the code in your examples works, and
+   post an example of the output in the pull request.
+
+1. Run `./tools/create-api-changelog` which will create a new empty
+   markdown file in `api_docs/unmerged.d/` directory
+   (e.g., `api_docs/unmerged.d/ZF-1f4a39.md`). Open this
+   file and document the API changes, formatted as an unordered list
+   (`*` bullets). The raw content of this file will be merged into
+   `api_docs/changelog.md` when your commit is merged into the `main`
+   branch. Therefore, keep the formatting and line wrapping consistent
+   with the content in `api_docs/changelog.md`.
+
+1. Add a `**Changes**` note in the description of all updates and
+   additions to the API documentation (`zerver/openapi/zulip.yaml`),
+   and mention the name of the file generated in the previous step
+   (without the `.md` extension) in place of the API feature level,
+   for example:
+
+   ```yaml
+   **Changes**: New in Zulip 11.0 (feature level ZF-1f4a39).
+   ```
+
+   For a change to existing behavior, make sure the main description
+   describes the behavior of current servers, and write the note as
+   a diff going backward in time, describing only the old behavior
+   (see [above](#title-and-description)):
+
+   ```yaml
+   **Changes**: Before Zulip 11.0 (feature level ZF-1f4a39),
+   this parameter was ignored for direct messages.
+   ```
+
+1. Proofread your new documentation in its rendered HTML, including
+   all links! Unmerged changelog entries are conveniently previewed on
+   `/api/changelog`.
+
+## Redirecting an existing article
+
+From time to time, we might want to rename an article in the REST API
+documentation. This change will break incoming links, including links
+in published Zulip blog posts, links in other branches of the
+repository that haven't been rebased, and more importantly links from
+previous versions of Zulip.
+
+To fix these broken links, you can easily add a URL redirect in:
+`zerver/lib/url_redirects.py`.
+
+For REST API documentation, you will either need to rename the file,
+or you will need to update the endpoint's `operationId` in
+`zerver/openapi/zulip.yaml`. Then, you need to add a new `URLRedirect`
+to the `API_DOCUMENTATION_REDIRECTS` list in `url_redirects.py`:
+
+```python
+API_DOCUMENTATION_REDIRECTS: List[URLRedirect] = [
+    # Add URL redirects for REST API documentation here:
+    URLRedirect("/api/delete-stream", "/api/archive-stream"),
+    ...
+```
+
+You should still check for references to the old URL in your branch
+and replace those with the new URL (e.g., `git grep "/api/foo"`).
+One exception to this are links with the old URL that were included
+in the content of `zulip_update_announcements`, which can be found
+in `zerver/lib/zulip_update_announcements.py`. It's preferable to
+have the source code accurately reflect what was sent to users in
+those [Zulip update announcements][zulip-updates], so these should
+not be replaced with the new URL.
+
+If you have the Zulip development environment set up, you can manually
+test your changes by loading the old URL in your browser (e.g.,
+`http://localhost:9991/api/foo`), and confirming that it redirects to
+the new url (e.g., `http://localhost:9991/api`/bar`).
+
+There is also an automated test in `zerver/tests/test_urls.py` that
+checks all the URL redirects, which you can run from the command line:
+
+```console
+./tools/test-backend zerver.tests.test_urls.URLRedirectTest
+```
+
+[zulip-updates]: https://zulip.com/help/configure-automated-notices#zulip-update-announcements
+
+## Why a custom system?
+
+Given that our documentation is written in large part using the
+OpenAPI format, why maintain a custom Markdown system for displaying
+it? There's several major benefits to this system:
+
+- It is extremely common for API documentation to become out of date
+  as an API evolves; this automated testing system helps make it
+  possible for Zulip to maintain accurate documentation without a lot
+  of manual management.
+- Every Zulip server can host correct API documentation for its
+  version, with the key variables (like the Zulip server URL) already
+  pre-substituted for the user.
+- We're able to share implementation language and visual styling with
+  our help center, which is especially useful for the extensive
+  non-REST API documentation pages (e.g., our bot framework).
+
+Using the standard OpenAPI format gives us flexibility, though; if we
+later choose to migrate to third-party tools, we don't need to redo
+the actual documentation work in order to migrate tools.
+
+## Debugging schema validation errors
+
+A common function used to validate and test Zulip's REST API is
+`validate_against_openapi_schema`. It is used to verify that every
+successful API response returned in the backend and documentation test
+suites are a documented possibility in the API documentation.
+
+Therefore, when you add a new feature or setting to Zulip, you will most
+likely need to update the API documentation (`zerver/openapi/zulip.yaml`)
+in order to pass existing tests that use this function. Additionally, if
+you're writing documentation for a new or undocumented REST API endpoint,
+you'll want to use this function to validate and test your changes in
+`zerver/openapi/python_examples.py`.
+
+Below are some examples to help you when debugging the schema validation
+errors produced by `validate_against_openapi_schema`. Before reading
+through the examples, we recommend reviewing the
+[OpenAPI configuration](openapi.md) documentation if you're unfamiliar
+with the format.
+
+If you use Visual Studio Code, an OpenAPI extension can be very helpful in
+navigating Zulip's large and detailed OpenAPI file; see
+`.vscode/extensions.json`.
+
+### Deconstructing the error output
+
+To start with a clear example, let's imagine that we are writing the
+documentation for the REST API endpoint for uploading a file,
+[POST /api/v1/user_uploads](https://zulip.com/api/upload-file).
+
+There are no parameters for this endpoint, and only one return value
+specific to this endpoint, `uri`, which is the URL of the uploaded file.
+If we comment out that return value and example from the existing API
+documentation in `zerver/openapi/zulip.yaml`, for example:
+
+```yaml
+  /user_uploads:
+    post:
+      operationId: upload-file
+...
+      responses:
+        "200":
+          description: Success.
+          content:
+            application/json:
+              schema:
+                allOf:
+                  - $ref: "#/components/schemas/JsonSuccessBase"
+                  - additionalProperties: false
+                    properties:
+                      result: {}
+                      msg: {}
+                      # uri:
+                      #   type: string
+                      #   description: |
+                      #     The URI of the uploaded file.
+                    example:
+                      {
+                        "msg": "",
+                        "result": "success",
+                        # "uri": "/user_uploads/1/4e/m2A3MSqFnWRLUf9SaPzQ0Up_/zulip.txt",
+                      }
+```
+
+We will now get an error when we run the API documentation test suite
+in the development environment (`tools/test-api`):
+
+```console
+Running API tests...
+2022-12-19 15:05:42.347 WARN [django.server] "POST /api/v1/users HTTP/1.1" 400 88
+Waiting for children to stop...
+Traceback (most recent call last):
+  File "tools/test-api", line 93, in <module>
+    test_the_api(client, nonadmin_client, owner_client)
+  File "/srv/zulip/zerver/openapi/python_examples.py", line 1636, in test_the_api
+    test_users(client, owner_client)
+  File "/srv/zulip/zerver/openapi/python_examples.py", line 1555, in test_users
+    upload_file(client)
+  File "/srv/zulip/zerver/openapi/python_examples.py", line 52, in _record_calls_wrapper
+    return test_func(*args, **kwargs)
+  File "/srv/zulip/zerver/openapi/python_examples.py", line 1284, in upload_file
+    validate_against_openapi_schema(result, "/user_uploads", "post", "200")
+  File "/srv/zulip/zerver/openapi/openapi.py", line 489, in validate_against_openapi_schema
+    raise SchemaError(message) from None
+zerver.openapi.openapi.SchemaError: 1 response validation error(s) at post /api/v1/user_uploads (200):
+
+ValidationError: Additional properties are not allowed ('uri' was unexpected)
+
+Failed validating 'additionalProperties' in schema['allOf'][2]:
+    {'additionalProperties': False,
+     'example': {'msg': '',
+                 'result': 'success',
+     'properties': {'msg': {}, 'result': {}}}
+
+On instance:
+    {'msg': '',
+     'result': 'success',
+     'uri': '/user_uploads/2/85/XoqF0K7XEOLVGylgdpof80RB/img.jpg'}
+
+```
+
+We can see in the traceback that a `SchemaError` was raised in
+`validate_against_openapi_schema`:
+
+```console
+  File "/srv/zulip/zerver/openapi/openapi.py", line 478, in validate_against_openapi_schema
+    raise SchemaError(message) from None
+```
+
+The next line in the output, lets us know how many errors were found
+and for what endpoint.
+
+```console
+zerver.openapi.openapi.SchemaError: 1 response validation error(s) at post /api/v1/user_uploads (200):
+```
+
+As expected from commenting out the code above, there was one validation
+error for the `POST /api/v1/user_uploads` endpoint. The next line gives
+more information about that error.
+
+```console
+ValidationError: Additional properties are not allowed ('uri' was unexpected)
+```
+
+We see that there was a `uri` value returned by the endpoint that hasn't
+been documented. The next few lines of output, show us what return values
+are documented (again due to our changes) for this endpoint.
+
+```console
+Failed validating 'additionalProperties' in schema['allOf'][2]:
+    {'additionalProperties': False,
+     'example': {'msg': '',
+                 'result': 'success',
+     'properties': {'msg': {}, 'result': {}}}
+```
+
+And finally, we see the test instance that did not match our current
+documentation, which includes the `uri` return value.
+
+```console
+On instance:
+    {'msg': '',
+     'result': 'success',
+     'uri': '/user_uploads/2/85/XoqF0K7XEOLVGylgdpof80RB/img.jpg'}
+```
+
+This is a useful example because the endpoint's documentation is short
+and straightforward, helping to easily identify the parts of the
+error output that are useful in debugging these errors when testing the
+API documentation.
+
+### Adding a realm setting
+
+Building on [the new feature tutorial](../tutorials/new-feature-tutorial.md)
+example, if the realm setting for `mandatory_topics` was not documented
+in the `POST /api/v1/register` endpoint, running `tools/test-api` in the
+development environment would result in this error:
+
+```console
+...
+zerver.openapi.openapi.SchemaError: 1 response validation error(s) at post /api/v1/register (200):
+
+ValidationError: Additional properties are not allowed ('realm_mandatory_topics' was unexpected)
+
+Failed validating 'additionalProperties' in schema['allOf'][2]:
+    'OpenAPI schema omitted due to length of output.'
+
+On instance:
+    'Error instance omitted due to length of output.'
+```
+
+Because this endpoint is very long and descriptive, we do not print the
+entire documentation schema (or test instance, in this case) to the
+console. Doing so would print thousands of lines of output that are not
+useful for debugging what is missing from the API documentation.
+
+The key information for debugging this endpoint is in the line beginning
+with `ValidationError`. There we can see that the documentation does not
+include the new `realm_mandatory_topics` boolean that we added in the
+example feature tutorial, and we can look at other similar realm settings
+to add the documentation for that new feature.

docs/documentation/helpcenter.md

@@ -0,0 +1,653 @@
+# Writing help center articles
+
+Our goal is for Zulip to have complete, high-quality
+documentation about Zulip's features and key workflows, such
+as setting up an organization.
+
+There are two types of help center documents: articles about specific features,
+and a handful of longer guides. We aim to have comprehensive onboarding guides,
+plus a select handful of guides for administrators and power users. Our goal is
+to document all the features of the app and associated key workflows.
+
+The feature articles that make up the bulk of help center content serve many
+purposes:
+
+- Feature and workflow explanations for Zulip users and admins, including
+  specific instructions and tips on when the feature would be useful. For
+  complex settings (e.g., permissions), detailed explanations of how the feature
+  works.
+
+- Feature discovery, for someone browsing the help center, and looking at
+  the set of articles and guides.
+
+- Public documentation of our feature set for search engine and LLM users.
+
+- Quick responses to support questions; if someone emails a Zulip admin
+  asking "How do I change my name?", they can reply with a link to the doc.
+
+- A reference to link to any time we mention the feature (e.g., in a Zulip
+  update, on the Zulip website, in a blog post, etc.), for anyone who'd like
+  more detail.
+
+- Reference links from the Zulip app (e.g., for complex settings).
+
+It's important to keep the docs up to date. We should update the help center as
+needed whenever a feature is added or changed.
+
+## Getting started
+
+There are over 100 feature articles and longer guides in the
+[Zulip help center](https://zulip.com/help/), so make the most of
+the current documentation as a resource and guide as you begin.
+
+- Use the left sidebar in the help center documentation to find the
+  section of the docs (e.g., Preferences, Sending messages, Reading
+  messages, etc.) that relates to the new feature you're documenting.
+
+- Read through the existing articles in that section and pay attention
+  to the [writing style](#writing-style) used, as well as any
+  [features and components](#mdx-features-and-custom-zulip-components)
+  used to enhance the readability of the documentation.
+
+If you aren't sure how something should be documented, the
+[#documentation](https://chat.zulip.org/#narrow/channel/19-documentation)
+channel in the [Zulip development
+community](https://zulip.com/development-community/) is a great place to get
+help.
+
+## Structure of a feature article
+
+The general structure of a feature article is:
+
+1. [Introduction](#introduction)
+1. One or more [sections with instructions](#instructions-section)
+1. [Related articles](#related-articles)
+
+### Introduction
+
+This is the part of the article that requires the most thought and understanding
+of context. It should generally (often in roughly this order):
+
+1. Explain briefly **what the feature does** for the user (but not how to activate it).
+1. Give some **guidance** on when someone might want to use this feature, perhaps
+   with specific examples.
+1. **Explain more precisely** what the feature does, if there are important
+   points to note. Are there any details that the user may feel uncertainty
+   about that should be cleared up?
+1. Note any **limitations**, like organization permissions settings that could disable this feature.
+1. Briefly describe other features that are good **alternatives** for related workflows.
+
+For example, the intro to the [article on moving messages to another
+topic](https://github.com/zulip/zulip/blob/8ddc7bee008f7e8d13cbe0b5ec4d1e1eb2333a9d/starlight_help/src/content/docs/move-content-to-another-topic.mdx?plain=1)
+follows this structure:
+
+1. **What the feature does**:
+   > Zulip makes it possible to move messages between topics.
+1. **Guidance**:
+   > This is useful for keeping messages organized when there is a digression,
+   > or the discussion shifts from the original topic.
+1. **Alternatives**:
+   > You can also [rename a topic](https://zulip.com/help/rename-a-topic).
+1. **Explain more precisely**:
+   > When messages are moved, Zulip's [permanent links to messages in
+   > context](https://zulip.com/help/link-to-a-message-or-conversation#get-a-link-to-a-specific-message)
+   > will automatically redirect to the new location of the message. [Muted
+   > topics](https://zulip.com/help/mute-a-topic) are automatically migrated when
+   > an entire topic is moved.
+1. **Limitations**:
+   > Organization administrators can
+   > [configure](https://zulip.com/help/restrict-moving-messages) who is allowed to move
+   > messages.
+
+You can see that the structure has some flexibility, but all the key points are
+succinctly covered.
+
+To write a good introduction, you may need to reread the issue that was fixed
+and any user comments on the issue, and to skim any discussion around the
+feature for relevant context.
+
+### Instructions section
+
+If the context for the feature is fully covered in the intro, a simple
+subsection heading plus instructions works well. As described in detail at the
+bottom of this article, be sure to use the standard help center
+components/macros. Refer to other parts of the article you're working on, or
+related articles (e.g., another setting in the same settings section).
+
+Be sure to include a **Mobile** instructions tab if the feature is supported on
+mobile. You can use a stub **Mobile** tab if we have an issue for it, but it's
+not yet implemented.
+
+When there are multiple ways to access a feature that are worth noting, label
+tabs by the broad-level location of where the feature is accessed from (e.g.,
+"Left sidebar"). We always document how to get to a setting via the settings UI,
+even if there's a handy alternative. The most convenient method should be in the
+first tab.
+
+For a single set of desktop/web app instructions, use the "Desktop/Web" tab
+label. We generally add this tab label for user features even if there are no
+other tabs to add, but don't do so for admin features. The vast majority
+of settings aren't available on mobile, so admins can easily infer that we're
+referring to the desktop/web app.
+
+In longer articles, the introduction might not cover everything. In that case,
+we might add an introduction to an instructions section, with a similar
+structure to article intros. Some caveats and alternatives can appear after the
+instructions, but the most important content should precede them.
+
+Be sure to document any keyboard shortcuts (generally inside the instructions
+block).
+
+### Related articles
+
+What articles is the reader most likely to be interested in?
+
+- Articles already linked to above are good candidates, but often not all of
+  them will feel sufficiently relevant. For example, you don't need to link to
+  articles that were linked to earlier just as a reference on some view or part
+  of the UI that was mentioned.
+- Articles about features that support related workflows.
+- For a permission setting, the article that explains the feature (and conversely).
+- Articles that go deeper into the same part of the app.
+- For onboarding features, articles about features you might use around the same time.
+
+Consider the reader -- in the context of this article, are they an end user or
+an admin? Admin articles will primarily link to other admin articles, and
+end-user articles to other end-user articles.
+
+## What the help center _isn't_ for
+
+An anti-pattern is trying to make up for bad UX by adding help center
+documentation. When writing documentation, try to pay attention to whether
+something feels hard to explain. It's a sign that there may be a product UX
+problem to fix here. Are there ways to change UI strings or workflows that would
+make the feature easier to document? That could be a good product direction to
+explore.
+
+It's worth remembering that for most articles, almost 100% of
+the users of the feature will never read the article. Instructions for
+filling out forms, interacting with UI widgets (e.g., typeaheads), interacting
+with modals, etc. should never go in the help center documentation.
+In such cases, you may be able to fix the problem by adding text in-app,
+where the user will see it as they are interacting with the feature.
+
+## Adding and updating articles
+
+Should the feature you're documenting be added or merged into an existing
+article?
+
+Real estate in the left sidebar is somewhat precious.
+Minor features should rarely get their own article, and should instead
+be merged into the existing help center documentation where appropriate.
+
+If the new feature you're documenting is a refinement on,
+or related to, a feature that already has a dedicated help center
+article, the information will be more useful and discoverable for
+users as an addition to the existing article.
+
+Permissions settings have a different target audience (administrators) than the
+feature they control, and thus often get their own article (with cross-links).
+
+### Updating an existing article
+
+Here are some things to keep in mind when expanding and updating
+existing help center articles:
+
+- If you're repeating content, consider moving it to an /include file in a prep
+  commit, for maintainability.
+- While you're there, check for other updates: any new features the article
+  should link to? outdated writing patterns? icons we missed updating? mobile
+  instructions that need to be added? related articles you looked at that were
+  missing reference links to this one? etc.
+
+If your updates to the existing article will change the name of the
+markdown file, then see section below on [redirecting an existing
+article](#redirecting-an-existing-article). We're generally OK with the URL
+being a bit off from the current article title, as long as it still makes sense
+given the current content.
+
+### Adding a new article
+
+It's often easiest to get started by choosing an existing article to use as a
+template for your new article (usually a similar feature in the same sidebar
+section). You can match its format, wording, style, etc. If
+you decide _not_ to follow its pattern, consider whether the article you're
+using as a template should itself be updated.
+
+Consider what articles (feature articles and guides) might need to link to the
+article you're adding (see [related articles tips](#related-articles)). For
+major new features, take a look at `/features` and other parts of the website
+(e.g, `/for/business`) for potential updates to make.
+
+If the feature exists in other team chat products, consider checking their
+documentation for inspiration (but each app has its own documentation style).
+
+## Writing style
+
+Aim for a clear, specific, and succinct writing style. Remember that a wide
+variety of users may be reading the article, including non-native English
+speakers.
+
+Avoid technical vocabulary and jargon wherever possible. It might be fine and
+unavoidable in, e.g., explanations of linkifier details. But strongly avoid it
+in, e.g., an introduction to a general user article. Remember that some terms
+that feel familiar to you as perhaps an engineer, and someone who
+specifically develops Zulip at that, may not be in common use.
+
+### User interface
+
+When you refer to the features in the Zulip UI, you should **bold** the
+feature's name followed by the feature itself (e.g., **Settings** page,
+**Change password** button, **Email** field). No quotation marks should be
+used. Use **bold** for channel names, and quotation marks for topic names.
+
+It's important to use consistent terminology for each UI element — doing
+otherwise can be quite confusing to the reader. If you aren't quite sure what to
+call something, check by finding another article that you expect to mention the
+same part of the UI, or an analogous one.
+
+Keep in mind that the UI may change — don’t describe it in more detail than
+is needed. Beyond requiring more frequent updates, overly specific descriptions
+are hard to maintain because nobody is likely to realize that the help center
+needs to be updated when purely visual changes are made. In particular:
+
+- Do not specify what the default configuration is. This might change in the
+  future, or may even be different for different types of organizations.
+- Do not list out the options the user is choosing from. Once the user finds the
+  right menu in the UI, they'll be able to see the options. In some cases, we
+  may describe the options in more detail outside of the instructions block.
+- Never identify or refer to a button by its color. You _can_ describe its
+  location.
+- Use screenshots only when it's very difficult to get your point across without
+  them.
+
+We don't describe the UI just of the sake of describing it --- UI elements are
+mentioned in the service of helping the user figure out how to take the actions
+they need to take.
+
+### Voice
+
+Do not use `we` to refer to Zulip or its creators; for example, "Zulip also
+allows ...", rather than "we also allow ...". On the other hand, `you` is ok
+and used liberally.
+
+### Keyboard shortcuts
+
+Surround each keyboard key in the shortcut with `<kbd>` HTML element start and
+end tags (e.g., `<kbd>Enter</kbd>` or `<kbd>R</kbd>`).
+
+For shortcuts with more than one key, add a plus sign (+) surrounded by spaces
+in between the keys (e.g., `<kbd>Ctrl</kbd> + <kbd>K</kbd>`). Any shortcut for
+an arrow key (↑, ↓, ←, →) will also need the `"arrow-key"` CSS class included
+in the `<kbd>` start tag (e.g., ` <kbd class="arrow-key">↑</kbd>`).
+
+Use the labels one sees on the actual keyboard rather than the letter they
+produce when pressed (e.g., `R` and `Shift` + `R` rather than `r` and `R`).
+For symbols, such as `?` or `@`, that are produced through key combinations that
+change depending on the user's keyboard layout, you should use the symbol as it
+appears on a keyboard instead of any specific combination of keys.
+
+Use non-Mac keyboard keys; for example `Enter`, instead of `Return`. Zulip will
+automatically translate non-Mac keys to the Mac versions for users with a Mac
+user agent. If you want to confirm that your documentation is rendering Mac keys
+correctly when writing documentation in Windows or Linux, you can temporarily
+change `has_mac_keyboard` in `/web/src/common.ts` to always return `True`.
+Then when you view your documentation changes in the development environment,
+the keyboard shortcuts should be rendered with Mac keys where appropriate.
+
+If you're adding a tip to an article about a keyboard shortcut, you should
+use the more specific [keyboard tip](#asides) component. In general, all
+keyboard shortcuts should be documented on the [keyboard
+shortcuts](https://zulip.com/help/keyboard-shortcuts) help center
+article.
+
+### Images
+
+Images and screenshots should be included in help center documentation
+_only_ if they will help guide the user in how to do something (e.g., if
+the image will make it much clearer which element on the page the user
+should interact with). For instance, an image of an element should
+not be included if the element the user needs to interact with is the
+only thing on the page, but images can be included to show the end
+result of an interaction with the UI.
+
+Using too many screenshots creates maintainability problems (we have
+to update them every time the UI is changed) and also can make the
+instructions for something simple look long and complicated.
+
+When taking screenshots, the image should never include the whole
+Zulip browser window in a screenshot; instead, it should only show
+relevant parts of the app. In addition, the screenshot should always
+come _after_ the text that describes it, never before.
+
+Images used in the help center can be found at `starlight_help/src/images`.
+
+## Viewing and updating help center articles
+
+The help center is built with [@astro/starlight](https://starlight.astro.build/).
+Starlight is a full-featured documentation theme built on top of the
+[Astro](https://astro.build/) framework. Astro is a web framework designed
+for content driven websites. The content for the help center articles are
+[MDX](https://mdxjs.com/) files, which live at `starlight_help/src/content/docs`
+in the [main Zulip server repository](https://github.com/zulip/zulip).
+Images are usually linked from `starlight_help/src/images`.
+
+Zulip help center documentation is available under `/help/` on any Zulip server;
+(e.g., <https://zulip.com/help/> or `http://localhost:9991/help/` in
+the Zulip development environment). The help center documentation is not hosted
+on ReadTheDocs, since Zulip supports running a server completely disconnected
+from the Internet, and we'd like the documentation to be available in that
+environment.
+
+This means that you can contribute to the Zulip help center documentation by
+just adding to or editing the collection of MDX files under
+`starlight_help/src/content/docs`. To add a help center article, create a new
+file in `starlight_help/src/content/docs/`, and add a sidebar link to it in
+`starlight_help/astro.config.mjs`.
+
+If you have the Zulip development environment set up, you simply need to reload
+your browser on `http://localhost:9991/help/foo` to see the latest version of
+`foo.mdx` rendered.
+
+This system is designed to make writing and maintaining such documentation
+highly efficient.
+
+## MDX features and custom Zulip components
+
+MDX supports standard markdown syntax. Some useful markdown features to
+remember when writing help center documentation are:
+
+- Since raw HTML is supported in Markdown, you can include arbitrary
+  HTML/CSS in your documentation as needed.
+
+- Code blocks allow you to highlight syntax, similar to
+  [Zulip's own Markdown](https://zulip.com/help/format-your-message-using-markdown).
+
+- Anchor tags can be used to link to headers in other documents.
+
+Additionally, there are some useful MDX components implemented and used
+throughout the help center documentation:
+
+- [Icon](#icons) components allow documentation to use the exact icons
+  for a button or link that is used in the Zulip UI.
+
+- [Include files](#include-files) allow us to reuse repeated content
+  in the documentation.
+
+- Our custom [Aside](#asides) components create special highlighted
+  information and warning blocks for tips, keyboard shortcuts, and the
+  like.
+
+- Instructions can be formatted with [Tab](#tabs),
+  [Steps/FlattenSteps](#steps-and-flattenedsteps) and
+  [NavigationSteps](#navigationsteps) components.
+
+### Icons
+
+See [icons documentation](../subsystems/icons.md). Icons should always be
+referred to with their in-app tooltip or a brief action name, _not_ the
+name of the icon in the code.
+
+When using these icons in an MDX file, they act as any other component:
+
+```
+import SquarePlusIcon from "~icons/zulip-icon/square-plus"
+
+Click the **new topic** (<SquarePlusIcon />) button next to the name of the channel.
+```
+
+For the import statement, the icon component should be named as the camel
+case of the icon name, with any dashes removed, followed by `Icon`, e.g.,
+in the example above `square-plus` is imported as `SquarePlusIcon`.
+
+### Include files
+
+You can include any file inside another MDX file as a regular import, which
+helps to eliminate repeated content in our documentation.
+
+All our include files live at `starlight_help/src/content/docs/include`,
+and can be imported and used as a regular component:
+
+```
+import AdminOnly from "./include/_AdminOnly.mdx";
+
+<AdminOnly />
+```
+
+If you're adding a new include file, make sure to have an underscore at the
+beginning of the file name as that ensures the file won't be considered a
+standalone article in the help center.
+
+A lot of our include files are list macros, i.e., they are partial lists
+that are part of bigger lists. When the partial list is an ordered list,
+it needs to be wrapped in a [FlattenedSteps](#steps-and-flattenedsteps)
+component.
+
+We recommend avoiding having any h2 or h3 headers (`##`, `###`) in an
+include file because, when the file is imported into a help center article,
+those headings will not be rendered in the "On this page" outline in the
+right sidebar.
+
+If it is necessary to have headers in the include file content, then the
+workaround for having them rendered in the right sidebar is to insert the
+same headers into any file where you are importing and using that include
+file. In the example below, we add the two h3 headers from the include file so
+that they are rendered in the right sidebar:
+
+```
+import AutomaticallyFollowTopics from "./include/_AutomaticallyFollowTopics.mdx";
+
+<AutomaticallyFollowTopics>
+  ### Follow topics you start or participate in
+
+  ### Follow topics where you are mentioned
+</AutomaticallyFollowTopics>
+```
+
+### Asides
+
+We have customized aside components that are used for tips, warnings and
+keyboard tips in the help center.
+
+A **tip** is any suggestion for the user that is not part of the main set
+of instructions. For instance, it may address a common problem users may
+encounter while following the instructions, or point to an option for power
+users.
+
+```
+import ZulipTip from "../../components/ZulipTip.astro";
+
+<ZulipTip>
+  The app will update automatically to future versions.
+</ZulipTip>
+```
+
+A **keyboard tip** is a note for users to let them know that the same action
+can also be accomplished via a [keyboard shortcut](#keyboard-shortcuts).
+
+```
+import KeyboardTip from "../../components/KeyboardTip.astro";
+
+<KeyboardTip>
+  You can also use <kbd>?</kbd> to open the keyboard shortcuts reference.
+</KeyboardTip>
+```
+
+A **warning** is a note on what happens when there is some kind of problem.
+Tips are more common than warnings.
+
+```
+import ZulipNote from "../../components/ZulipNote.astro";
+
+<ZulipNote>
+  This feature is only available to organization owners and administrators.
+</ZulipNote>
+```
+
+There should be only one tip/warning inside each component. They usually
+should be formatted as a continuation of a numbered step, if they are in
+an ordered list.
+
+You can find the code for these custom aside components at
+`starlight_help/src/components`.
+
+### Tabs
+
+There are built-in [starlight/astro `Tab` and `TabItem`
+components](https://starlight.astro.build/components/tabs/) for creating
+tabbed instructions:
+
+```
+import {Steps, TabItem, Tabs} from "@astrojs/starlight/components";
+
+import FlattenedSteps from "../../components/FlattenedSteps.astro";
+
+import MobileSwitchAccount from "./include/_MobileSwitchAccount.mdx";
+
+<Tabs>
+  <TabItem label="Mobile">
+    <FlattenedSteps>
+      <MobileSwitchAccount />
+
+      1. Tap **Add new account**.
+      1. Enter the Zulip URL of the organization, and tap **Continue**.
+      1. Follow the on-screen instructions.
+    </FlattenedSteps>
+  </TabItem>
+
+  <TabItem label="Web">
+    <Steps>
+      1. Go to the Zulip URL of the organization.
+      1. Follow the on-screen instructions.
+    </Steps>
+  </TabItem>
+</Tabs>
+```
+
+The above example has instructions for logging in for the mobile app
+and the web app. Make sure you add a label for each `TabItem`.
+
+#### Steps and FlattenedSteps
+
+If you don't need multiple tabs, or a tabbed label, for your instructions,
+then you can just wrap the ordered list of instructions in a [`Steps`
+component](https://starlight.astro.build/components/steps/):
+
+```
+import {Steps} from "@astrojs/starlight/components";
+
+<Steps>
+  1. Go to the Zulip URL of the organization.
+  1. Follow the on-screen instructions.
+</Steps>
+```
+
+If you have an ordered list of instructions with a portion of the list in
+an [include file](#include-files), then you need to wrap the instructions
+in a `FlattenedSteps` component, so that the instructions are numbered
+correctly when rendered (e.g., `1.`, `2.`, `3.`, `4.`, etc.):
+
+```
+import FlattenedSteps from "../../components/FlattenedSteps.astro";
+
+import MobileSwitchAccount from "./include/_MobileSwitchAccount.mdx";
+
+<FlattenedSteps>
+  <MobileSwitchAccount />
+
+  1. Tap **Add new account**.
+  1. Enter the Zulip URL of the organization, and tap **Continue**.
+  1. Follow the on-screen instructions.
+</FlattenedSteps>
+```
+
+#### NavigationSteps
+
+Many instructions begin with the same set of steps, some of which can have
+relative links that directly go to the logged in session for the Zulip
+organization of the user reading the documentation. For these partial
+instruction lists, we use a custom `NavigationSteps` component, which needs
+to be wrapped in a `FlattenedSteps` component ([see above](#steps-and-flattenedsteps)).
+
+```
+import NavigationSteps from "../../components/NavigationSteps.astro";
+
+<FlattenedSteps>
+  <NavigationSteps target="settings/organization-permissions" />
+
+  1. Under **Joining the organization**, toggle **Invitations are required for
+     joining this organization**.
+
+  <SaveChanges />
+</FlattenedSteps>
+```
+
+Each `NavigationStep` has a `target`, which can be one of two forms:
+
+- Settings: `settings/{setting_key}`
+- Relative: `relative/{link_type}/{key}`
+
+The settings targets are for instructions that navigate to a section in
+either the personal or organization settings overlay in the web app.
+
+The relative targets are for instructions that navigate from the gear or
+help menus to a particular location in the web app.
+
+See `starlight_help/src/components/NavigationSteps.astro` for what values
+for settings and relative targets are currently implemented.
+
+You can find the code for any custom components used in the help center at:
+`starlight_help/src/components`.
+
+## Redirecting an existing article
+
+From time to time, we might want to rename an article in the help
+center. This change will break incoming links, including links in
+published Zulip blog posts, links in other branches of the repository
+that haven't been rebased, and more importantly links from previous
+versions of Zulip.
+
+To fix these broken links, you can easily add a URL redirect in:
+`starlight_help/astro.config.mjs`.
+
+For a help center article, once you've renamed the file in your
+branch (e.g., `git mv path/to/foo.mdx path/to/bar.mdx`), go to
+`astro.config.mjs` and add a new line to the `redirects` property:
+
+```
+    // Redirects in astro are just directories with index.html inside
+    // them doing the redirect we define in the value. The base of
+    // /help/ will apply to the keys in the list below but we will
+    // have to prepend /help/ in the redirect URL.
+    redirects: {
+        "pm-mention-alert-notifications": "/help/dm-mention-alert-notifications",
+    ...
+```
+
+Note that you will also need to add redirects when you're deleting
+a help center article and adding its content to an existing article
+as a section. In that case, the new URL will include the new section
+header:
+
+```
+"add-an-alert-word": "/help/dm-mention-alert-notifications#alert-words"
+```
+
+You should still check for references to the old URL in your branch
+and replace those with the new URL (e.g., `git grep "/help/foo"`).
+One exception to this are links with the old URL that were included
+in the content of `zulip_update_announcements`, which can be found
+in `zerver/lib/zulip_update_announcements.py`. It's preferable to
+have the source code accurately reflect what was sent to users in
+those [Zulip update announcements][help-zulip-updates], so these
+should not be replaced with the new URL.
+
+Updating section headers in existing help center articles does not
+require adding a URL redirect, but you will need to update any
+existing links to that article's section in your branch.
+
+You can manually test your changes in the dev environment by loading the old URL
+in your browser (e.g., `http://localhost:9991/help/foo`), and confirming that it
+redirects to the new URL (e.g., `http://localhost:9991/help/bar`).
+
+[help-zulip-updates]: https://zulip.com/help/configure-automated-notices#zulip-update-announcements

docs/documentation/index.md

@@ -0,0 +1,13 @@
+# Writing documentation
+
+```{toctree}
+---
+maxdepth: 3
+---
+
+overview
+helpcenter
+integrations
+api
+openapi
+```

docs/documentation/integrations.md

@@ -0,0 +1,363 @@
+# Documenting an integration
+
+In order for a [Zulip
+integration](https://zulip.com/api/integrations-overview) to be useful
+to users, it must be documented. Zulip's common system for documenting
+integrations involves writing Markdown files, either at
+`zerver/webhooks/{webhook_name}/doc.md` (for webhook integrations) or
+`templates/zerver/integrations/{integration_name}.md` (for other
+integrations).
+
+The [integrations][api-integrations] in
+[zulip/python-zulip-api][api-repo] have their corresponding Markdown files
+at `zulip/integrations/{integration_name}/doc.md`, which are imported into
+[zulip/zulip][zulip-repo] at
+`static/generated/integrations/{integration_name}/doc.md` using the
+`tools/setup/generate_bots_integrations_static_files.py` script.
+
+[api-repo]: https://github.com/zulip/python-zulip-api/
+[api-integrations]: https://github.com/zulip/python-zulip-api/tree/main/zulip/integrations
+[zulip-repo]: https://github.com/zulip/zulip
+
+Typically, the documentation process involves the following steps:
+
+- Add text explaining all of the steps required to set up the
+  integration, including what URLs to use, etc. See
+  [Writing guidelines](#writing-guidelines) for detailed writing guidelines.
+
+  Zulip's pre-defined Markdown macros can be used for some of these steps.
+  See [Markdown macros](#markdown-macros) for further details.
+
+- Make sure you've added your integration to `zerver/lib/integrations.py` in
+  the relevant `*_INTEGRATIONS` list, such as the
+  `INCOMING_WEBHOOK_INTEGRATIONS` list for incoming webhook integrations.
+
+  These lists get aggregated into the `INTEGRATIONS` registry which
+  configures your integration to appear on the `/integrations` page, and
+  makes it possible to automatically generate the screenshot of an example
+  message, which is important so that these screenshots can be easily
+  updated as Zulip's design changes.
+
+- You'll need to add an SVG graphic of your integration's logo under the
+  `static/images/integrations/logos/<name>.svg`, where `<name>` is the
+  name of the integration, all in lower case; you can usually find them in the
+  product branding or press page. Make sure to optimize the SVG graphic by
+  running `tools/setup/optimize-svg`. This will also run
+  `tools/setup/generate_integration_bots_avatars.py` automatically to generate
+  a smaller version of the image you just added and optimized. This smaller
+  image will be used as the bot avatar in the documentation screenshot that
+  will be generated in the next step.
+
+  If you cannot find an SVG graphic of the logo, please find and include a PNG
+  image of the logo instead.
+
+- Finally, you will need to generate a message sent by the integration, and
+  generate a screenshot of the message to provide an example message in the
+  integration's documentation.
+
+  If your new integration is not an incoming webhook and does not have fixtures, add a
+  message template and topic to `zerver/webhooks/fixtureless_integrations.py`.
+  Then, add your integration's name to `FIXTURELESS_INTEGRATIONS_WITH_SCREENSHOTS`
+  in `zerver/lib/integrations.py`. See [this commit][example-commit] for an
+  example.
+
+  Otherwise, you should have already added your integration along with its
+  screenshot config to `INCOMING_WEBHOOK_INTEGRATIONS`.
+
+  Generate the screenshot using `tools/screenshots/generate-integration-docs-screenshot`,
+  where `integrationname` is the name of the integration:
+
+  ```bash
+  ./tools/screenshots/generate-integration-docs-screenshot --integration integrationname
+  ```
+
+  If you have trouble using this tool, you can also manually generate the
+  screenshot using `manage.py send_webhook_fixture_message`. When generating the
+  screenshot of a sample message using this method, give your test bot a nice
+  name like "GitHub Bot", use the project's logo as the bot's avatar, and take
+  the screenshot showing the channel/topic bar for the message, not just the
+  message body.
+
+[example-commit]: https://github.com/zulip/zulip/commit/f6e847d026013097ecdc484ec6675680ce76aa84
+
+## Markdown macros
+
+**Macros** are elements in the format of `{!macro.md!}` that insert common
+phrases and steps at the location of the macros. Macros help eliminate
+repeated content in our documentation.
+
+The source for macros is the Markdown files under
+`templates/zerver/integrations/include/` in the
+[main Zulip server repository](https://github.com/zulip/zulip). If you find
+multiple instances of particular content in the documentation, you can
+always create a new macro by adding a new file to that folder.
+
+Here are a few common macros used to document Zulip's integrations:
+
+- `{!create-channel.md!}` macro - Recommends that users create a dedicated
+  channel for a given integration. Usually the first step is setting up an
+  integration or incoming webhook. For an example rendering, see **Step 1** of
+  [the docs for Zulip's Zendesk integration][zendesk].
+
+- `{!create-an-incoming-webhook.md!}` macro - Instructs users to create a bot
+  for a given integration and select **Incoming webhook** as the **Bot type**.
+  This macro is usually used right after `{!create-channel.md!}`. For an example
+  rendering, see **Step 2** of [the docs for Zulip's Zendesk integration][zendesk].
+
+- `{!create-a-generic-bot.md!}` macro - Instructs users to create a bot
+  for a given integration and select **Generic bot** as the **Bot type**. For an
+  example rendering, see [the docs for Zulip's Matrix integration][matrix].
+
+- `{!generate-webhook-url-basic.md!}` - Instructs user how to get the URL for a
+  bot for a given integration. Note that this macro should not be used with the
+  `{!create-channel.md!}` macro. For an example rendering, see **Step 2** of
+  [the docs for Zulip's GitHub integration][github-integration].
+
+- `{!generate-integration-url.md!}` - Instructs user how to get the URL for a
+  bot for a given integration. An example URL is generated automatically for
+  every incoming webhook by using attributes in the `IncomingWebhookIntegration` class
+  in [zerver/lib/integrations.py][integrations-file].
+
+  **Note:** If special configuration is required to set up the URL and you can't
+  use these macros, be sure to use the `{{ api_url }}` template variable, so
+  that your integration documentation will provide the correct URL for whatever
+  server it is deployed on.
+
+- `{!congrats.md!}` macro - Inserts congratulatory lines signifying the
+  successful setup of a given integration. This macro is usually used at
+  the end of the documentation, right before the sample message screenshot.
+  For an example rendering, see the end of
+  [the docs for Zulip's GitHub integration][github-integration].
+
+- `{!event-filtering-additional-feature.md!}` macro - If a webhook integration
+  supports event filtering, then this adds a section with the specific
+  events that can be filtered for the integration. Should be included in
+  the documentation if `all_event_types` is set in the webhook integration
+  view. For an example see, the **Filtering incoming events** section in
+  [Zulip's GitLab integration][gitlab].
+
+- `{!download-python-bindings.md!}` macro - Links to Zulip's
+  [API page](https://zulip.com/api/) to download and install Zulip's
+  API bindings. This macro is usually used in non-webhook integration docs under
+  `templates/zerver/integrations/<integration_name>.md`. For an example
+  rendering, see **Step 3** of
+  [the docs for Zulip's Codebase integration][codebase].
+
+- `{!change-zulip-config-file.md!}` macro - Instructs users to create a bot and
+  specify said bot's credentials in the config file for a given non-webhook
+  integration. This macro is usually used in non-webhook integration docs under
+  `templates/zerver/integrations/<integration_name>.md`. For an example
+  rendering, see **Step 4** of
+  [the docs for Zulip's Codebase integration][codebase].
+
+- `{!webhook-url-with-bot-email.md!}` - Used in certain non-webhook integrations
+  to generate URLs of the form:
+
+  ```text
+  https://bot_email:bot_api_key@your-org.zulipchat.com/api/v1/external/beanstalk
+  ```
+
+  For an example rendering, see
+  [Zulip's Beanstalk integration](https://zulip.com/integrations/beanstalk).
+
+[github-integration]: https://zulip.com/integrations/github
+[zendesk]: https://zulip.com/integrations/zendesk
+[matrix]: https://zulip.com/integrations/matrix#configure-the-bridge
+[codebase]: https://zulip.com/integrations/codebase
+[beanstalk]: https://zulip.com/integrations/beanstalk
+[front]: https://zulip.com/integrations/front
+[gitlab]: https://zulip.com/integrations/gitlab
+[integrations-file]: https://github.com/zulip/zulip/blob/main/zerver/lib/integrations.py
+
+## Writing guidelines
+
+For the vast majority of integrations, you should just copy the docs for a
+similar integration and edit it. [Basecamp][basecamp] is a good one to copy.
+
+[basecamp]: https://zulip.com/integrations/basecamp
+
+### General writing guidelines
+
+At a high level, the goals are for the instructions to feel simple, be easy to
+follow, and be easy to maintain. Easier said than done, but here are a few
+concrete guidelines.
+
+##### Feel simple
+
+- Use simple language.
+- Use the imperative tense.
+- Only describe things as much as necessary. For example: "Select **Project settings**."
+  is better than "Select **Project settings** from the dropdown."
+- Cut unnecessary words. For example, do not start steps with transition words
+  like "Next", "First", "Now", etc. Each step should be structurally
+  independent.
+
+##### Be easy to follow
+
+- Actions should appear in order, including implicit ones. So "Under
+  **Webhooks**, click **Add Webhook**." not "Click **Add Webhook** in the
+  **Webhooks** section." "Under **Webhooks**" is an action, since it’s basically
+  the same as "Find the Webhooks section".
+- UI elements in third-party websites should be **bolded**.
+- Trailing punctuation can be stripped from bolded elements. For example,
+  "**Enable this checkbox**" instead of "**Enable this checkbox?**".
+  Starting punctuation such as the "**+**" in "**+ New Notification**" should
+  be preserved.
+- You can use a screenshot if a step is particularly complicated
+  (see [Screenshots](#screenshots) below).
+
+##### Be easy to maintain
+
+- Follow the organization and wording of existing docs as much as possible.
+
+### Guidelines for specific steps
+
+Most doc files should start with a generic sentence about the
+integration, for example, "Get `webhook name` notifications in Zulip!"
+A typical doc will then have the following steps.
+
+##### "Create the channel" step
+
+- Use the `create-channel` macro. This step should be omitted if the
+  integration only supports notifications via direct messages.
+
+##### "Create the bot" step
+
+- Typically, use the `create-an-incoming-webhook` and
+  `generate-integration-url` macros.
+- [Existing macros](#markdown-macros) should be used for this if they exist,
+  but if the macro defaults don’t work, it may make sense to write something
+  custom for the integration in question. This step is mandatory for all
+  integrations.
+
+##### "Navigate to this screen" step
+
+- In general, this should be one step, even if it takes multiple clicks.
+- Begin by mentioning the third-party service being documented with language
+  such as "Go to your Taiga project", "Go to your GitHub repository",
+  "On your X project", etc. Assume the user is already logged in to their
+  account.
+- If a UI element is difficult to spot, you can use additional cues like
+  "Click on **Settings** in the top-right corner" or "On the left, click
+  **Webhooks**.". Only do this if the element is actually difficult to spot.
+- If this step includes more than 5 sentences, it may need to be split up
+  into multiple steps.
+
+##### "Fill out form and save" step
+
+- Filling out a form and clicking **Save** should generally be one step, even if
+  there are multiple fields to fill out.
+- It’s fine to say things like "Follow the on-screen instructions to create an
+  app" if they have a sequence of steps they guide you through that are pretty
+  clear.
+
+Lastly, end with the `congrats.md` macro and a screenshot of a sample message
+within Zulip.
+
+### Screenshots
+
+Screenshots are hard to maintain, so we generally err on the side of not
+including screenshots. That being said, screenshots may be used to aid the
+process if the third-party UI is confusing or a specific UI element is hard
+to find. A few things to keep in mind:
+
+- Screenshots should be small and should mainly include the third-party UI that
+  the text is referring to that is difficult to locate.
+- Screenshots should never include the entirety of a third-party website’s page.
+- Each screenshot should come after the step that refers to it.
+
+## Markdown features
+
+Zulip's Markdown processor allows you to include several special features in
+your documentation to help improve its readability:
+
+- Since raw HTML is supported in Markdown, you can include arbitrary
+  HTML/CSS in your documentation as needed.
+
+- Code blocks allow you to highlight syntax, similar to
+  [Zulip's own Markdown](https://zulip.com/help/format-your-message-using-markdown).
+
+- Anchor tags can be used to link to headers in other documents.
+
+- Inline [icons](#icons) are used to refer to features in the Zulip UI.
+
+- Utilize [macros](#markdown-macros) to limit repeated content in the documentation.
+
+- Create special highlight warning blocks using
+  [tips and warnings](#tips-and-warnings).
+
+- Format instructions with tabs using
+  [Markdown tab switcher](#tab-switcher).
+
+### Icons
+
+See [icons documentation](../subsystems/icons.md). Icons should always be
+referred to with their in-app tooltip or a brief action name, _not_ the
+name of the icon in the code.
+
+### Tips and warnings
+
+A **tip** is any suggestion for the user that is not part of the main set of
+instructions. For instance, it may address a common problem users may
+encounter while following the instructions, or point to an option for power
+users.
+
+```md
+!!! tip ""
+
+    If you want notifications for issues, as well as events, you can
+    scroll down to **Webhooks** on the same page, and toggle the
+    **issue** checkbox.
+```
+
+A **keyboard tip** is a note for users to let them know that the same action
+can also be accomplished via a [keyboard shortcut](https://zulip.com/help/keyboard-shortcuts).
+
+```md
+!!! keyboard_tip ""
+
+    Use <kbd>D</kbd> to bring up your list of drafts.
+```
+
+A **warning** is a note on what happens when there is some kind of problem.
+Tips are more common than warnings.
+
+```md
+!!! warn ""
+
+    **Note**: Zulip also supports configuring this integration as a
+    webhook in Sentry.
+```
+
+All tips/warnings should appear inside tip/warning blocks. There
+should be only one tip/warning inside each block, and they usually
+should be formatted as a continuation of a numbered step.
+
+### Tab switcher
+
+Our Markdown processor supports easily creating a tab switcher widget
+design to easily show the instructions for different languages in API
+docs, etc. To create a tab switcher, write:
+
+```md
+{start_tabs}
+{tab|python}
+# First tab's content
+{tab|js}
+# Second tab's content
+{tab|curl}
+# Third tab's content
+{end_tabs}
+```
+
+The tab identifiers (e.g., `python` above) and their mappings to
+the tabs' labels are declared in
+[zerver/lib/markdown/tabbed_sections.py][tabbed-sections-code].
+
+This widget can also be used just to create a nice box around a set of
+instructions
+([example](https://zulip.com/help/deactivate-your-account)) by
+only declaring a single tab, which is often used for the main set of
+instructions for setting up an integration.
+
+[tabbed-sections-code]: https://github.com/zulip/zulip/blob/main/zerver/lib/markdown/tabbed_sections.py

docs/documentation/openapi.md

@@ -0,0 +1,195 @@
+# OpenAPI configuration
+
+[OpenAPI][openapi-spec] is a popular format for describing an API. An
+OpenAPI file can be used by various tools to generate documentation
+for the API or even basic client-side bindings for dozens of
+programming languages.
+
+Zulip's API is described in `zerver/openapi/zulip.yaml`. Our aim is
+for that file to fully describe every endpoint in the Zulip API, and
+for the Zulip test suite to fail should the API every change without a
+corresponding adjustment to the documentation. In particular,
+essentially all content in Zulip's [REST API
+documentation](api.md) is generated from our OpenAPI
+file.
+
+In an OpenAPI Swagger file, every configuration section is an object.
+Objects may contain other objects, or reference objects defined
+elsewhere. Larger API specifications may be split into multiple
+files. See the [OpenAPI specification][openapi-spec].
+
+[openapi-spec]: https://swagger.io/docs/specification/about/
+
+This library isn't in production use yet, but it is our current plan
+for how Zulip's API documentation will work.
+
+## Working with the `zulip.yaml` file
+
+An OpenAPI specification file has three general parts: information and
+configuration, endpoint definitions, and object schemas referenced by
+other objects (as an alternative to defining everything inline.)
+References can either specify an individual object, using `$ref:`, or
+compose a larger definition from individual objects with `allOf:`
+(which may itself contain a `$ref`.)
+
+### Configuration
+
+These objects, at the top of `zulip.yaml`, identify the API, define
+the backend host for the working examples, list supported schemes and
+types of authentication, and configure other settings. Once defined,
+information in this section rarely changes.
+
+For example, the `swagger` and `info` objects look like this:
+
+```yaml
+# Basic Swagger UI info
+openapi: 3.0.1
+info:
+  version: 1.0.0
+  title: Zulip REST API
+  description: Powerful open source group chat.
+  contact:
+    url: https://zulip.com
+  license:
+    name: Apache 2.0
+    url: https://www.apache.org/licenses/LICENSE-2.0.html
+```
+
+### Endpoint definitions
+
+The [Paths Object](https://swagger.io/specification/#pathsObject)
+contains
+[Path Item Objects](https://swagger.io/specification/#pathItemObject)
+for each endpoint. It describes in detail the methods and parameters
+the endpoint accepts and responses it returns.
+
+There is one Path Item Object for each supported method, containing a
+[Parameters Definition Object](https://swagger.io/specification/#parametersDefinitionObject)
+describing the required and optional inputs. A
+[Response Object](https://swagger.io/specification/#responseObject)
+similarly specifies the content of the response. They may reference
+schemas from a global Definitions Object (see [Schemas](#schemas),
+below.)
+
+Example:
+
+The `/users/{user}/presence` endpoint (defined in a
+[Path Item Object](https://swagger.io/specification/#pathItemObject))
+expects a GET request with one
+[parameter](https://swagger.io/specification/#parameterObject), HTTP
+Basic authentication, and returns a JSON response containing `msg`,
+`result`, and `presence` values.
+
+```yaml
+/users/{user}/presence:
+  get:
+    description: Get presence data for another user.
+    operationId: getPresence
+    parameters:
+    - name: user
+      in: path
+      description: Enter email address
+      required: true
+      type: string
+    security:
+    - basicAuth: []
+    responses:
+      '200':
+        description: The response from a successful call
+        schema:
+          type: object
+          required:
+          - msg
+          - result
+          - presence
+          properties:
+            msg:
+              type: string
+            result:
+              type: string
+            presence:
+              type: array
+```
+
+### Schemas
+
+The
+[Definitions Object](https://swagger.io/specification/#definitionsObject)
+contains schemas referenced by other objects. For example,
+`MessageResponse`, the response from the `/messages` endpoint,
+contains three required parameters. Two are strings, and one is an
+integer.
+
+```yaml
+MessageResponse:
+  type: object
+  required:
+    - msg
+    - result
+    - id
+  properties:
+    msg:
+      type: string
+    result:
+      type: string
+    id:
+      type: integer
+      format: int64
+```
+
+You can find more examples, including GET requests and nested objects, in
+`zerver/openapi/zulip.yaml`.
+
+## Zulip Swagger YAML style:
+
+We're collecting decisions we've made on how our Swagger YAML files
+should be organized here:
+
+- Use shared definitions and YAML anchors to avoid duplicating content
+  where possible.
+
+## Tips for working with YAML:
+
+You can edit YAML files in any text editor. Indentation defines
+blocks, so whitespace is important (as it is in Python.) TAB
+characters are not permitted. If your editor has an option to replace
+tabs with spaces, this is helpful.
+
+You can also use the
+[Swagger Editor](https://swagger.io/swagger-editor), which validates
+YAML and understands the Swagger specification. Download and run it
+locally, or use the online version. If you aren't using a YAML-aware
+editor, make small changes and check your additions often.
+
+Note: if you are working with
+[Swagger UI](https://swagger.io/swagger-ui/) in a local development
+environment, it uses an online validator that must be able to access
+your file. You may see a red "ERROR" button at the bottom of your API
+docs page instead of the green "VALID" one even if your file is
+correct.
+
+### Formatting help:
+
+- Comments begin with a # character.
+
+- Descriptions do not need to be in quotes, and may use common
+  Markdown format options like inline code \` (backtick) and `#`
+  headings.
+
+- A single `|` (pipe) character begins a multi-line description on the
+  next line. Single spaced lines (one newline at the end of each) are
+  joined. Use an extra blank line for a paragraph break. We prefer
+  to use this format for all descriptions because it doesn't require
+  extra effort to expand.
+
+### Examples:
+
+```yaml
+Description: |
+             This description has multiple lines.
+             Sometimes descriptions can go on for
+             several sentences.
+
+             A description might have multiple paragraphs
+             as well.
+```

docs/documentation/overview.md

@@ -0,0 +1,166 @@
+# Documentation systems
+
+Zulip has three major documentation systems:
+
+- **Developer and sysadmin documentation**: Documentation for people
+  actually interacting with the Zulip codebase (either by developing
+  it or installing it), and written in Markdown.
+
+- **Core website documentation**: Complete webpages for complex topics,
+  written in HTML, JavaScript, and CSS (using the Django templating
+  system). These roughly correspond to the documentation someone
+  might look at when deciding whether to use Zulip. We don't expect
+  to ever have more than about 10 pages written using this system.
+
+- **User-facing documentation**: Zulip uses a scalable system for
+  documenting Zulip's integrations and REST API, without a lot of
+  overhead or duplicated code/syntax, written in Markdown. Zulip's
+  help center uses [@astro/starlight](https://starlight.astro.build/),
+  with most of the content written in MDX.
+  - [Help center documentation](#help-center-documentation)
+    (with a target audience of individual Zulip users)
+  - [Integrations documentation](#integrations-documentation)
+    (with a target audience of IT folks setting up integrations)
+  - [API documentation](#api-documentation) (with a target audience
+    of developers writing code to extend Zulip)
+
+These three systems are documented in detail.
+
+## Developer and sysadmin documentation
+
+What you are reading right now is part of the collection of
+documentation targeted at developers and people running their own
+Zulip servers. These docs are written in
+[CommonMark Markdown](https://commonmark.org/).
+We've chosen Markdown because it is
+[easy to write](https://commonmark.org/help/). The source for Zulip's
+developer documentation is at `docs/` in the Zulip Git repository, and
+they are served in production at
+[zulip.readthedocs.io](https://zulip.readthedocs.io/en/latest/).
+
+This documentation is hosted by the excellent [ReadTheDocs
+service](https://readthedocs.org/). ReadTheDocs automatically [builds
+a preview](https://docs.readthedocs.io/en/stable/pull-requests.html)
+for every pull request, accessible from a "Details" link in the
+"Checks" section of the pull request page. It's nonetheless valuable
+to submit a screenshot with any pull request modifying documentation
+to help make reviews efficient.
+
+If you want to build the developer documentation locally (e.g., to test
+your changes), the dependencies are automatically installed as part of
+Zulip development environment provisioning, and you can build the
+documentation using:
+
+```bash
+./tools/build-docs
+```
+
+and then opening `http://127.0.0.1:9991/docs/index.html` in your
+browser. The raw files are available at
+`file:///path/to/zulip/docs/_build/html/index.html` in your browser
+(so you can also use, for example, `firefox docs/_build/html/index.html`
+from the root of your Zulip checkout).
+
+If you are adding a new page to the table of contents, you will want
+to modify `docs/index.md` and run `make clean` before `make html`, so
+that other docs besides your new one also get the new entry in the
+table of contents.
+
+You can also usually test your changes by pushing a branch to GitHub
+and looking at the content on the GitHub web UI, since GitHub renders
+Markdown, though that won't be as faithful as the `make html`
+approach or the preview build.
+
+We manage Python requirements for the documentation build in the `docs` uv
+[group](https://docs.astral.sh/uv/concepts/projects/dependencies/#dependency-groups),
+which is used by our ReadTheDocs build configuration in
+[`.readthedocs.yaml`](https://docs.readthedocs.com/platform/stable/config-file/v2.html).
+
+## Core website documentation
+
+Zulip has around 10 HTML documentation pages under `templates/zerver`
+for specific major topics, like the features list, client apps,
+integrations, hotkeys, API bindings, etc. These documents often have
+somewhat complex HTML and JavaScript, without a great deal of common
+patterns between them other than inheriting from the `portico.html`
+template. We generally avoid adding new pages to this collection
+unless there's a good reason, but we don't intend to migrate them,
+either, since this system gives us the flexibility to express these
+important elements of the product clearly.
+
+## User-facing documentation
+
+Zulip's API and integrations documentation use a common Markdown-based
+framework with various extensions for macros and variable interpolation,
+(`render_markdown_path` in the code), designed to make it convenient
+to do the things one does a lot in each type of documentation.
+
+Zulip's help center is built with [@astro/starlight](https://starlight.astro.build/).
+Starlight is a full-featured documentation theme built on top of the
+[Astro](https://astro.build/) framework. Astro is a web framework designed
+for content driven websites.
+
+### Help center documentation
+
+Zulip's [help center](https://zulip.com/help/) documentation is
+designed to explain how the product works to end users. We aim for
+this to be clear, concise, correct, and readable to nontechnical
+audiences where possible.
+
+See our guide on [writing help center articles](helpcenter.md).
+
+### Integrations documentation
+
+Zulip's [integrations documentation](https://zulip.com/integrations/)
+is user-facing documentation explaining to end users how to set up each
+of Zulip's more than 100 integrations. There is a detailed [guide on
+documenting integrations](integrations.md), including style guidelines
+to ensure that the documentation is high quality and consistent.
+
+See also our broader [integrations developer
+guide](https://zulip.com/api/integrations-overview).
+
+### API documentation
+
+Zulip's [API documentation](https://zulip.com/api/) is intended to make
+it easy for a technical user to write automation tools that interact
+with Zulip. This documentation also serves as our main mechanism for
+Zulip server developers to communicate with client developers about
+how the Zulip API works.
+
+See the [API documentation tutorial](api.md) for
+details on how to contribute to this documentation.
+
+## Automated testing
+
+Zulip has several automated test suites that we run in CI and
+recommend running locally when making significant edits:
+
+- `tools/lint` catches a number of common mistakes, and we highly
+  recommend
+  [using our linter pre-commit hook](../git/zulip-tools.md#set-up-git-repo-script).
+  See the [main linter doc](../testing/linters.md) for more details.
+
+- The ReadTheDocs docs are built and the links tested by
+  `tools/test-documentation`, which runs `build-docs` and then checks
+  all the links.
+
+There's an exclude list for the link testing at this horrible path:
+`tools/documentation_crawler/documentation_crawler/spiders/common/spiders.py`,
+which is relevant for flaky links.
+
+- The API docs are tested by `tools/test-api`, which does some basic
+  payload verification. Note that this test does not check for broken
+  links (those are checked by `test-help-documentation`).
+
+- `tools/test-help-documentation` checks `/help/`, `/api/`,
+  `/integrations/`, and the core website ("portico") documentation for
+  broken links. Note that the "portico" documentation check has a
+  manually maintained whitelist of pages, so if you add a new page to
+  this site, you will need to edit `PorticoDocumentationSpider` to add it.
+
+- `tools/test-backend test_docs.py` tests various internal details of
+  the variable substitution logic, as well as rendering. It's
+  essential when editing the documentation framework, but not
+  something you'll usually need to interact with when editing
+  documentation.