upstream-zulip/docs/webhooks/incoming-webhooks-overview.md

# Incoming webhooks overview

An incoming webhook allows a third-party service to push data to Zulip
when something happens. There are several ways to set up an incoming
webhook in Zulip:

- Use our [REST API](https://zulip.com/api/rest) endpoint for [sending
  messages](https://zulip.com/api/send-message). This works great for
  internal tools or in cases where a third-party tool wants to control
  the formatting of the messages in Zulip.
- Use one of our supported [integration frameworks](https://zulip.com/integrations/category/meta-integration),
  such as the [Slack-compatible incoming webhook](https://zulip.com/integrations/slack_incoming),
  [Zapier integration](https://zulip.com/integrations/zapier), or
  [IFTTT integration](https://zulip.com/integrations/ifttt).
- Implement an incoming webhook integration, where all the logic for
  formatting the Zulip messages lives in the Zulip server. This process
  is explained in more detail below, and is how most of [Zulip's official
  integrations](https://zulip.com/integrations/) work, because they
  enable Zulip to support third-party services that have an "outgoing
  webhook" feature (without the third party needing to do any work
  specific to Zulip).

In an incoming webhook integration, the third-party service's "outgoing
webhook" feature sends an `HTTP POST` request to a special URL when
something happens, and then the Zulip "incoming webhook" integration
handles that incoming data in order to format and send a message in Zulip.

New official Zulip webhook integrations can take just a few hours to
write, including tests and documentation, if you use the right process.

## Quick guide

- Set up the [Zulip development environment](../development/overview.md).
- Use [Zulip's JSON integration](https://zulip.com/integrations/json),
  <https://webhook.site/>, or a similar site to capture an example
  "outgoing webhook" payload from the third-party service. Create a
  `zerver/webhooks/<mywebhook>/fixtures/` directory, and add the
  captured payload as a test fixture.
- Create an `IncomingWebhookIntegration` object, and add it to the
  `INCOMING_WEBHOOK_INTEGRATIONS` list in `zerver/lib/integrations.py`.
- Write a webhook handler in `zerver/webhooks/<mywebhook>/view.py`. There
  are a lot of examples in the `zerver/webhooks/` directory that you can
  use for reference.
- Write a test for your fixture in `zerver/webhooks/<mywebhook>/tests.py`.
  Run the test for your integration like this:
  ```
  tools/test-backend zerver/webhooks/<mywebhook>/
  ```
  Iterate on debugging the test and webhook handler until it all
  works.
- Capture payloads for the other common types of payloads the third-party
  service will make, and add tests for them.
- Document the integration in `zerver/webhooks/<mywebhook>/doc.md`. You
  can use existing documentation, like [this one](https://raw.githubusercontent.com/zulip/zulip/main/zerver/webhooks/github/doc.md),
  as a template. There is also a separate guide for [documenting an
  integration](../documentation/integrations.md).

## Hello world walkthrough

Check out [this detailed guide](incoming-webhooks-walkthrough.md) for
step-by-step instructions on developing an incoming webhook integration.

## Checklist

### Files that need to be created

Select a name for your incoming webhook and use it consistently. The
examples below are for an incoming webhook named `MyWebHook`.

- `zerver/webhooks/mywebhook/__init__.py`: An empty file that is an
  obligatory part of every python package. Remember to `git add` it.
- `zerver/webhooks/mywebhook/view.py`: This file will have the main
  webhook handler, called `api_mywebhook_webhook`, along with any
  necessary helper functions.
- `zerver/webhooks/mywebhook/fixtures/message_type.json`: A sample
  of payload data, from the third-party service, used by tests. Add
  fixtures covering the event types and conditions your integration
  supports.
- `zerver/webhooks/mywebhook/tests.py`: Tests for your webhook.
- `zerver/webhooks/mywebhook/doc.md`: End-user documentation explaining
  how to set up the integration.
- `static/images/integrations/logos/mywebhook.svg`: A square logo for the
  third-party service you are integrating. Used on the documentation
  pages.
- `static/images/integrations/mywebhook/001.png`: A screenshot of a
  message sent by the integration that's used on the documentation page.
  This can be generated by running
  `tools/screenshots/generate-integration-docs-screenshot --integration mywebhook`.
- `static/images/integrations/bot_avatars/mywebhook.png`: A square logo
  for the third-party service you are integrating, which is used to
  create the bot's avatar when generating example screenshots. This can
  be generated automatically from `static/images/integrations/logos/mywebhook.svg`
  by running `tools/setup/generate_integration_bots_avatars.py`.

### Files that need to be updated

- `zerver/lib/integrations.py`: Add your incoming webhook to
  `INCOMING_WEBHOOK_INTEGRATIONS`. This will automatically register a
  URL for the incoming webhook of the form `api/v1/external/mywebhook`
  and associate it with the function called `api_mywebhook_webhook` in
  `zerver/webhooks/mywebhook/view.py`.

## Common Helpers

If your integration will receive a test payload from the third-party
service, you can use `get_setup_webhook_message` to create a standard
message for a test payload. You can import this from `zerver/lib/webhooks/common.py`,
and it will generate a message like this: "GitHub webhook is successfully
configured! 🎉"

## General advice

- Consider using [Zulip's message formatting](https://zulip.com/help/format-your-message-using-markdown)
  to make the output from your integration visually attractive or useful
  (e.g., emoji, Markdown emphasis, or @-mentions).
- Use [topics](https://zulip.com/help/introduction-to-topics) effectively
  to ensure sequential messages about the same thing are threaded
  together; this makes for much better consumption by Zulip users. E.g.,
  for a bug tracker integration, put the bug number in the topic for all
  messages; for an integration like Nagios, put the service in the topic.
- If your integration references usernames from external accounts (such as
  GitHub usernames), consider using the `guess_zulip_user_from_external_account`
  helper function from `zerver/lib/webhooks/common.py`. This function automatically
  matches external accounts to Zulip users who have linked those accounts in their
  profile's custom fields. This enables converting external usernames to silent mentions,
  notifying the relevant Zulip users.
- Integrations that don't match a team's workflow can often be uselessly
  spammy. Give careful thought to providing options for triggering Zulip
  messages only for certain event types, certain projects, or sending
  different messages to different channels/topics, to make it easy for
  teams to configure the integration to support their workflow.
- Consistently capitalize the name of the integration in the documentation
  and the service the way the third-party vendor does. It's okay to use
  all lowercase in the implementation.
- Sometimes it can be helpful to contact the third-party service if it
  appears they don't have an API or outgoing webhook you can use.
  Sometimes the API you're looking for is just not properly documented.

## URL specification

The base URL for an incoming webhook integration bot, where
`INTEGRATION_NAME` is the name of the specific webhook integration and
`API_KEY` is the API key of the bot created by the user for the
integration, is:

```
https://your-org.zulipchat.com/v1/external/INTEGRATION_NAME?api_key=API_KEY
```

The list of existing incoming webhook integrations can be found by
browsing [Zulip's integrations documentation](https://zulip.com/integrations/)
or in `zerver/lib/integrations.py` at `INCOMING_WEBHOOK_INTEGRATIONS`.

Parameters accepted in the URL include:

### api_key (required)

The API key of [the bot](https://zulip.com/help/add-a-bot-or-integration)
created by the user for the integration. To get a bot's API key, see the
[API keys](https://zulip.com/api/api-keys) documentation.

### stream

The [channel](https://zulip.com/help/introduction-to-channels) for the
integration to send notifications to. This can be either the channel ID
or the [URL-encoded][url-encoder] channel name. By default, the
integration will send direct messages to the bot's owner.

:::{note}

A channel's ID can be found when [browsing
channels](https://zulip.com/help/introduction-to-channels#browse-and-subscribe-to-channels)
in the web or desktop apps.

:::

### topic

The [topic](https://zulip.com/help/introduction-to-topics) in the
specified channel for the integration to send notifications to. The topic
should also be [URL-encoded][url-encoder]. By default, the integration
will have a topic configured for channel messages.

### only_events, exclude_events

Some incoming webhook integrations support these parameters to filter
which events will trigger a notification. You can append either
`&only_events=["event_a","event_b"]` or
`&exclude_events=["event_a","event_b"]` (or both, with different events)
to the URL, with an arbitrary number of supported events.

You can use UNIX-style wildcards like `*` to include multiple events.
For example, `test*` matches every event that starts with `test`.

:::{note}

For a list of supported events, see a specific [integration's
documentation](https://zulip.com/integrations/) page.

:::

[url-encoder]: https://www.urlencoder.org/