snapshot of https://github.com/ThreeDotsLabs/watermill.git (master, 2026-09-15; blobs > 131072B omitted)
Some checks failed
master / ci (push) Failing after 0s
Some checks failed
master / ci (push) Failing after 0s
This commit is contained in:
upstream-mirror
176
README.md
Normal file
176
README.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# Watermill
|
||||
<img align="right" width="300" src="https://watermill.io/img/gopher.svg">
|
||||
|
||||
[](https://github.com/ThreeDotsLabs/watermill/actions/workflows/master.yml)
|
||||
[](https://pkg.go.dev/github.com/ThreeDotsLabs/watermill)
|
||||
[](https://goreportcard.com/report/github.com/ThreeDotsLabs/watermill)
|
||||
[](https://codecov.io/gh/ThreeDotsLabs/watermill)
|
||||
|
||||
Watermill is a Go library for working efficiently with message streams. It is intended
|
||||
for building event driven applications, enabling event sourcing, RPC over messages,
|
||||
sagas and basically whatever else comes to your mind. You can use conventional pub/sub
|
||||
implementations like Kafka or RabbitMQ, but also HTTP or PostgreSQL if that fits your use case.
|
||||
|
||||
## Goals
|
||||
|
||||
* **Easy** to understand.
|
||||
* **Universal** - event-driven architecture, messaging, stream processing, CQRS - use it for whatever you need.
|
||||
* **Fast** (see [Benchmarks](#benchmarks)).
|
||||
* **Flexible** with middlewares, plugins and Pub/Sub configurations.
|
||||
* **Resilient** - using proven technologies and passing stress tests (see [Stability](#stability)).
|
||||
|
||||
## Getting Started
|
||||
|
||||
Pick what you like the best or see in order:
|
||||
|
||||
1. [Quickstart](https://watermill.io/learn/quickstart/) — learn by coding!
|
||||
2. Follow the [Getting Started guide](https://watermill.io/learn/getting-started/).
|
||||
3. See examples below.
|
||||
4. Read the full documentation: https://watermill.io/
|
||||
|
||||
## Our online hands-on training
|
||||
|
||||
Go Event-Driven goes beyond Watermill Quickstart. You'll learn industry standard concepts and patterns like:
|
||||
|
||||
* Handling at-least-once delivery
|
||||
* Asynchronous read models
|
||||
* Events & Commands
|
||||
* Observability
|
||||
* Message ordering
|
||||
* Sagas
|
||||
|
||||
<a href="https://threedots.tech/event-driven/?utm_source=watermill-readme"><img align="center" width="400" src="https://threedots.tech/event-driven-banner.png"></a>
|
||||
|
||||
## Examples
|
||||
|
||||
* Basic
|
||||
* [Your first app](_examples/basic/1-your-first-app) - **start here!**
|
||||
* [Realtime feed](_examples/basic/2-realtime-feed)
|
||||
* [Router](_examples/basic/3-router)
|
||||
* [Metrics](_examples/basic/4-metrics)
|
||||
* [CQRS with protobuf](_examples/basic/5-cqrs-protobuf)
|
||||
* [Pub/Subs usage](_examples/pubsubs)
|
||||
* These examples are part of the [Getting started guide](https://watermill.io/learn/getting-started/) and show usage of a single Pub/Sub at a time.
|
||||
* Real-world examples
|
||||
* [Exactly-once delivery counter](_examples/real-world-examples/exactly-once-delivery-counter)
|
||||
* [Receiving webhooks](_examples/real-world-examples/receiving-webhooks)
|
||||
* [Sending webhooks](_examples/real-world-examples/sending-webhooks)
|
||||
* [Synchronizing Databases](_examples/real-world-examples/synchronizing-databases)
|
||||
* [Persistent Event Log](_examples/real-world-examples/persistent-event-log)
|
||||
* [Transactional Events](_examples/real-world-examples/transactional-events)
|
||||
* [Real-time HTTP updates with Server-Sent Events](_examples/real-world-examples/server-sent-events)
|
||||
* [Real-time HTTP updates with Server-Sent Events and htmx](_examples/real-world-examples/server-sent-events-htmx)
|
||||
* Complete projects
|
||||
* [NATS example with live code reloading](https://github.com/ThreeDotsLabs/nats-example)
|
||||
* [RabbitMQ, webhooks and Kafka integration](https://github.com/ThreeDotsLabs/event-driven-example)
|
||||
|
||||
## Background
|
||||
|
||||
Building distributed and scalable services is rarely as easy as some may suggest. There is a
|
||||
lot of hidden knowledge that comes with writing such systems. Just like you don't need to know the
|
||||
whole TCP stack to create a HTTP REST server, you shouldn't need to study all of this knowledge to
|
||||
start with building message-driven applications.
|
||||
|
||||
Watermill's goal is to make communication with messages as easy to use as HTTP routers. It provides
|
||||
the tools needed to begin working with event-driven architecture and allows you to learn the details
|
||||
on the go.
|
||||
|
||||
At the heart of Watermill there is one simple interface:
|
||||
```go
|
||||
func(*Message) ([]*Message, error)
|
||||
```
|
||||
|
||||
Your handler receives a message and decides whether to publish new message(s) or return
|
||||
an error. What happens next is up to the middlewares you've chosen.
|
||||
|
||||
You can find more about our motivations in our [*Introducing Watermill* blog post](https://threedots.tech/post/introducing-watermill/).
|
||||
|
||||
## Pub/Subs
|
||||
|
||||
All publishers and subscribers have to implement an interface:
|
||||
|
||||
```go
|
||||
type Publisher interface {
|
||||
Publish(topic string, messages ...*Message) error
|
||||
Close() error
|
||||
}
|
||||
|
||||
type Subscriber interface {
|
||||
Subscribe(ctx context.Context, topic string) (<-chan *Message, error)
|
||||
Close() error
|
||||
}
|
||||
```
|
||||
|
||||
Supported Pub/Subs:
|
||||
|
||||
- AMQP (RabbitMQ) Pub/Sub [(`github.com/ThreeDotsLabs/watermill-amqp/v3`)](https://github.com/ThreeDotsLabs/watermill-amqp/)
|
||||
- AWS SNS/SQS Pub/Sub [(`github.com/ThreeDotsLabs/watermill-aws`)](https://github.com/ThreeDotsLabs/watermill-aws/)
|
||||
- Bolt Pub/Sub [(`github.com/ThreeDotsLabs/watermill-bolt`)](https://github.com/ThreeDotsLabs/watermill-bolt/)
|
||||
- Firestore Pub/Sub [(`github.com/ThreeDotsLabs/watermill-firestore`)](https://github.com/ThreeDotsLabs/watermill-firestore/)
|
||||
- Google Cloud Pub/Sub [(`github.com/ThreeDotsLabs/watermill-googlecloud/v2`)](https://github.com/ThreeDotsLabs/watermill-googlecloud/)
|
||||
- HTTP Pub/Sub [(`github.com/ThreeDotsLabs/watermill-http/v2`)](https://github.com/ThreeDotsLabs/watermill-http/)
|
||||
- io.Reader/io.Writer Pub/Sub [(`github.com/ThreeDotsLabs/watermill-io`)](https://github.com/ThreeDotsLabs/watermill-io/)
|
||||
- Kafka Pub/Sub [(`github.com/ThreeDotsLabs/watermill-kafka/v3`)](https://github.com/ThreeDotsLabs/watermill-kafka/)
|
||||
- NATS Jetstream Pub/Sub [(`github.com/ThreeDotsLabs/watermill-nats/v2`)](https://github.com/ThreeDotsLabs/watermill-nats/)
|
||||
- Redis Stream Pub/Sub [(`github.com/ThreeDotsLabs/watermill-redisstream`)](https://github.com/ThreeDotsLabs/watermill-redisstream/)
|
||||
- SQL (MySQL / PostgreSQL) Pub/Sub [(`github.com/ThreeDotsLabs/watermill-sql/v4`)](https://github.com/ThreeDotsLabs/watermill-sql/)
|
||||
- SQLite Pub/Sub (Beta) [(`github.com/ThreeDotsLabs/watermill-sqlite/`)](https://github.com/ThreeDotsLabs/watermill-sqlite/)
|
||||
|
||||
All Pub/Subs implementation documentation can be found in the [documentation](https://watermill.io/pubsubs/).
|
||||
|
||||
## Unofficial libraries
|
||||
|
||||
Can't find your favorite Pub/Sub or library integration? Check [Awesome Watermill](https://watermill.io/docs/awesome/).
|
||||
|
||||
If you know another library or are an author of one, please [add it to the list](https://github.com/ThreeDotsLabs/watermill/edit/master/docs/content/docs/awesome.md).
|
||||
|
||||
## Contributing
|
||||
|
||||
Please check our [contributing guide](CONTRIBUTING.md).
|
||||
|
||||
## Stability
|
||||
|
||||
Watermill v1.0.0 has been released and is production-ready. The public API is stable and will not change without changing the major version.
|
||||
|
||||
To ensure that all Pub/Subs are stable and safe to use in production, we created a [set of tests](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L34) that need to pass for each of the implementations before merging to master.
|
||||
All tests are also executed in [*stress*](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L171) mode - that means that we are running all the tests **20x** in parallel.
|
||||
|
||||
All tests are run with the race condition detector enabled (`-race` flag in tests).
|
||||
|
||||
For more information about debugging tests, you should check [tests troubleshooting guide](http://watermill.io/docs/troubleshooting/#debugging-pubsub-tests).
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Initial tools for benchmarking Pub/Subs can be found in [watermill-benchmark](https://github.com/ThreeDotsLabs/watermill-benchmark).
|
||||
|
||||
All benchmarks are being done on a single 16 CPU VM instance, running one binary and dependencies in Docker Compose.
|
||||
|
||||
These numbers are meant to serve as a rough estimate of how fast messages can be processed by different Pub/Subs.
|
||||
Keep in mind that the results can be vastly different, depending on the setup and configuration (both much lower and higher).
|
||||
|
||||
Here's the short version for message size of 16 bytes.
|
||||
|
||||
| Pub/Sub | Publish (messages / s) | Subscribe (messages / s) |
|
||||
|---------------------------------|------------------------|--------------------------|
|
||||
| GoChannel | 315,776 | 138,743 |
|
||||
| Redis Streams | 59,158 | 12,134 |
|
||||
| NATS Jetstream (16 Subscribers) | 50,668 | 34,713 |
|
||||
| Kafka (one node) | 41,492 | 101,669 |
|
||||
| SQL (MySQL, batch size=100) | 6,371 | 2,794 |
|
||||
| SQL (PostgreSQL, batch size=1) | 2,831 | 9,460 |
|
||||
| Google Cloud Pub/Sub | 3,027 | 28,589 |
|
||||
| AMQP (RabbitMQ) | 2,770 | 14,604 |
|
||||
|
||||
## Support
|
||||
|
||||
If you didn't find the answer to your question in [the documentation](https://watermill.io/), feel free to ask us directly!
|
||||
|
||||
Please join us on the `#watermill` channel on the [Three Dots Labs Discord](https://discord.gg/QV6VFg4YQE).
|
||||
|
||||
## Why the name?
|
||||
|
||||
It processes streams!
|
||||
|
||||
## License
|
||||
|
||||
[MIT License](./LICENSE)
|
||||
Reference in New Issue
Block a user