snapshot of https://github.com/zulip/zulip.git @ 48c69911c7dd4e03b20552eb3bd77ad8151c1357 (2026-09-15)
Some checks failed
Code scanning / CodeQL (push) Failing after 15s
Zulip production suite / Ubuntu 22.04 production build (push) Failing after 26s
Zulip production suite / Debian 12 production install with custom db name and user (push) Has been skipped
Zulip production suite / Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga (push) Has been skipped
Zulip production suite / Ubuntu 24.04 production install (push) Has been skipped
Zulip production suite / Ubuntu 26.04 production install (push) Has been skipped
Zulip production suite / Debian 13 production install (push) Has been skipped
Zulip production suite / 7.0 Version Upgrade (push) Has been skipped
Zulip production suite / 8.0 Version Upgrade (push) Has been skipped
Zulip production suite / 6.0 Version Upgrade (push) Has been skipped
Zulip production suite / 10.0 Version Upgrade (push) Has been skipped
Zulip production suite / 9.0 Version Upgrade (push) Has been skipped
Zulip production suite / 12.0 Version Upgrade (push) Has been skipped
Zulip production suite / 11.0 Version Upgrade (push) Has been skipped
GitHub Actions Security Analysis with zizmor / zizmor (push) Failing after 3s
Zulip CI / Debian 12 (Python 3.11, backend + documentation) (push) Failing after 26s
Zulip CI / Ubuntu 22.04 (Python 3.10, backend + frontend) (push) Failing after 3s
Zulip CI / Ubuntu 24.04 (Python 3.12, backend) (push) Failing after 25s
Zulip CI / Ubuntu 26.04 (Python 3.14, backend) (push) Failing after 25s
Zulip CI / Debian 13 (Python 3.13, backend) (push) Failing after 28s
Zulip production suite / Required jobs (push) Has been cancelled
Zulip CI / Required jobs (push) Has been cancelled
API Documentation Update Check / check-feature-level-updated (push) Has been cancelled
API Documentation Update Check / notify-if-api-docs-changed (push) Has been cancelled
Some checks failed
Code scanning / CodeQL (push) Failing after 15s
Zulip production suite / Ubuntu 22.04 production build (push) Failing after 26s
Zulip production suite / Debian 12 production install with custom db name and user (push) Has been skipped
Zulip production suite / Ubuntu 22.04 production install and PostgreSQL upgrade with pgroonga (push) Has been skipped
Zulip production suite / Ubuntu 24.04 production install (push) Has been skipped
Zulip production suite / Ubuntu 26.04 production install (push) Has been skipped
Zulip production suite / Debian 13 production install (push) Has been skipped
Zulip production suite / 7.0 Version Upgrade (push) Has been skipped
Zulip production suite / 8.0 Version Upgrade (push) Has been skipped
Zulip production suite / 6.0 Version Upgrade (push) Has been skipped
Zulip production suite / 10.0 Version Upgrade (push) Has been skipped
Zulip production suite / 9.0 Version Upgrade (push) Has been skipped
Zulip production suite / 12.0 Version Upgrade (push) Has been skipped
Zulip production suite / 11.0 Version Upgrade (push) Has been skipped
GitHub Actions Security Analysis with zizmor / zizmor (push) Failing after 3s
Zulip CI / Debian 12 (Python 3.11, backend + documentation) (push) Failing after 26s
Zulip CI / Ubuntu 22.04 (Python 3.10, backend + frontend) (push) Failing after 3s
Zulip CI / Ubuntu 24.04 (Python 3.12, backend) (push) Failing after 25s
Zulip CI / Ubuntu 26.04 (Python 3.14, backend) (push) Failing after 25s
Zulip CI / Debian 13 (Python 3.13, backend) (push) Failing after 28s
Zulip production suite / Required jobs (push) Has been cancelled
Zulip CI / Required jobs (push) Has been cancelled
API Documentation Update Check / check-feature-level-updated (push) Has been cancelled
API Documentation Update Check / notify-if-api-docs-changed (push) Has been cancelled
This commit is contained in:
89
puppet/zulip/files/postgresql/centos_pg_hba.conf
Normal file
89
puppet/zulip/files/postgresql/centos_pg_hba.conf
Normal file
@@ -0,0 +1,89 @@
|
||||
# PostgreSQL Client Authentication Configuration File
|
||||
# ===================================================
|
||||
#
|
||||
# Refer to the "Client Authentication" section in the PostgreSQL
|
||||
# documentation for a complete description of this file. A short
|
||||
# synopsis follows.
|
||||
#
|
||||
# This file controls: which hosts are allowed to connect, how clients
|
||||
# are authenticated, which PostgreSQL user names they can use, which
|
||||
# databases they can access. Records take one of these forms:
|
||||
#
|
||||
# local DATABASE USER METHOD [OPTIONS]
|
||||
# host DATABASE USER ADDRESS METHOD [OPTIONS]
|
||||
# hostssl DATABASE USER ADDRESS METHOD [OPTIONS]
|
||||
# hostnossl DATABASE USER ADDRESS METHOD [OPTIONS]
|
||||
#
|
||||
# (The uppercase items must be replaced by actual values.)
|
||||
#
|
||||
# The first field is the connection type: "local" is a Unix-domain
|
||||
# socket, "host" is either a plain or SSL-encrypted TCP/IP socket,
|
||||
# "hostssl" is an SSL-encrypted TCP/IP socket, and "hostnossl" is a
|
||||
# plain TCP/IP socket.
|
||||
#
|
||||
# DATABASE can be "all", "sameuser", "samerole", "replication", a
|
||||
# database name, or a comma-separated list thereof. The "all"
|
||||
# keyword does not match "replication". Access to replication
|
||||
# must be enabled in a separate record (see example below).
|
||||
#
|
||||
# USER can be "all", a user name, a group name prefixed with "+", or a
|
||||
# comma-separated list thereof. In both the DATABASE and USER fields
|
||||
# you can also write a file name prefixed with "@" to include names
|
||||
# from a separate file.
|
||||
#
|
||||
# ADDRESS specifies the set of hosts the record matches. It can be a
|
||||
# host name, or it is made up of an IP address and a CIDR mask that is
|
||||
# an integer (between 0 and 32 (IPv4) or 128 (IPv6) inclusive) that
|
||||
# specifies the number of significant bits in the mask. A host name
|
||||
# that starts with a dot (.) matches a suffix of the actual host name.
|
||||
# Alternatively, you can write an IP address and netmask in separate
|
||||
# columns to specify the set of hosts. Instead of a CIDR-address, you
|
||||
# can write "samehost" to match any of the server's own IP addresses,
|
||||
# or "samenet" to match any address in any subnet that the server is
|
||||
# directly connected to.
|
||||
#
|
||||
# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
|
||||
# "gss", "sspi", "ident", "peer", "pam", "ldap", "radius" or "cert".
|
||||
# Note that "password" sends passwords in clear text; "md5" or
|
||||
# "scram-sha-256" are preferred since they send encrypted passwords.
|
||||
#
|
||||
# OPTIONS are a set of options for the authentication in the format
|
||||
# NAME=VALUE. The available options depend on the different
|
||||
# authentication methods -- refer to the "Client Authentication"
|
||||
# section in the documentation for a list of which options are
|
||||
# available for which authentication methods.
|
||||
#
|
||||
# Database and user names containing spaces, commas, quotes and other
|
||||
# special characters must be quoted. Quoting one of the keywords
|
||||
# "all", "sameuser", "samerole" or "replication" makes the name lose
|
||||
# its special character, and just match a database or username with
|
||||
# that name.
|
||||
#
|
||||
# This file is read on server startup and when the server receives a
|
||||
# SIGHUP signal. If you edit the file on a running system, you have to
|
||||
# SIGHUP the server for the changes to take effect, run "pg_ctl reload",
|
||||
# or execute "SELECT pg_reload_conf()".
|
||||
#
|
||||
# Put your actual configuration here
|
||||
# ----------------------------------
|
||||
#
|
||||
# If you want to allow non-local connections, you need to add more
|
||||
# "host" records. In that case you will also need to make PostgreSQL
|
||||
# listen on a non-local interface via the listen_addresses
|
||||
# configuration parameter, or via the -i or -h command line switches.
|
||||
|
||||
|
||||
|
||||
# TYPE DATABASE USER ADDRESS METHOD
|
||||
|
||||
# "local" is for Unix domain socket connections only
|
||||
local all all peer
|
||||
# IPv4 local connections:
|
||||
host all all 127.0.0.1/32 md5
|
||||
# IPv6 local connections:
|
||||
host all all ::1/128 md5
|
||||
# Allow replication connections from localhost, by a user with the
|
||||
# replication privilege.
|
||||
local replication all peer
|
||||
host replication all 127.0.0.1/32 ident
|
||||
host replication all ::1/128 ident
|
||||
51
puppet/zulip/files/postgresql/env-wal-g
Executable file
51
puppet/zulip/files/postgresql/env-wal-g
Executable file
@@ -0,0 +1,51 @@
|
||||
#!/bin/sh
|
||||
if [ -z "$ZULIP_SECRETS_CONF" ]; then
|
||||
ZULIP_SECRETS_CONF=/etc/zulip/zulip-secrets.conf
|
||||
fi
|
||||
|
||||
export PGHOST=/var/run/postgresql/
|
||||
|
||||
WALG_DELTA_MAX_STEPS=$(crudini --get /etc/zulip/zulip.conf postgresql backups_incremental 2>/dev/null)
|
||||
export WALG_DELTA_MAX_STEPS
|
||||
|
||||
WALG_COMPRESSION_METHOD=$(crudini --get /etc/zulip/zulip.conf postgresql backups_compression_method 2>/dev/null)
|
||||
export WALG_COMPRESSION_METHOD
|
||||
|
||||
s3_backups_bucket=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_backups_bucket 2>/dev/null)
|
||||
|
||||
if [ "$s3_backups_bucket" != "" ] || [ -n "$WALG_S3_PREFIX" ]; then
|
||||
AWS_REGION=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_region 2>/dev/null)
|
||||
if [ "$AWS_REGION" = "" ]; then
|
||||
# Fall back to the current region, if possible
|
||||
AZ=$(ec2metadata --availability-zone || true)
|
||||
if [ -n "$AZ" ] && [ "$AZ" != "unavailable" ]; then
|
||||
AWS_REGION=$(echo "$AZ" | sed 's/.$//')
|
||||
fi
|
||||
fi
|
||||
export AWS_REGION
|
||||
AWS_ACCESS_KEY_ID=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_backups_key 2>/dev/null)
|
||||
export AWS_ACCESS_KEY_ID
|
||||
AWS_SECRET_ACCESS_KEY=$(crudini --get "$ZULIP_SECRETS_CONF" secrets s3_backups_secret_key 2>/dev/null)
|
||||
export AWS_SECRET_ACCESS_KEY
|
||||
: "${WALG_S3_PREFIX:=s3://$s3_backups_bucket}"
|
||||
export WALG_S3_PREFIX
|
||||
|
||||
if storage_class=$(crudini --get /etc/zulip/zulip.conf postgresql backups_storage_class 2>/dev/null); then
|
||||
export WALG_S3_STORAGE_CLASS="$storage_class"
|
||||
fi
|
||||
else
|
||||
WALG_FILE_PREFIX=$(crudini --get /etc/zulip/zulip.conf postgresql backups_directory 2>/dev/null)
|
||||
if [ "$WALG_FILE_PREFIX" != "" ]; then
|
||||
export WALG_FILE_PREFIX
|
||||
if [ "$WALG_DELTA_MAX_STEPS" = "" ]; then
|
||||
# Default to only taking a full backup every week
|
||||
export WALG_DELTA_MAX_STEPS=6
|
||||
fi
|
||||
|
||||
else
|
||||
echo "Could not determine where to back up data to!"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
exec /usr/local/bin/wal-g "$@"
|
||||
81
puppet/zulip/files/postgresql/pg_backup_and_purge
Executable file
81
puppet/zulip/files/postgresql/pg_backup_and_purge
Executable file
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
import configparser
|
||||
import glob
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
logging.Formatter.converter = time.gmtime
|
||||
logging.basicConfig(format="%(asctime)s %(levelname)s: %(message)s")
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
config_file = configparser.RawConfigParser()
|
||||
config_file.read("/etc/zulip/zulip.conf")
|
||||
|
||||
|
||||
def get_config(
|
||||
section: str,
|
||||
key: str,
|
||||
default_value: str = "",
|
||||
) -> str:
|
||||
if config_file.has_option(section, key):
|
||||
return config_file.get(section, key)
|
||||
return default_value
|
||||
|
||||
|
||||
replicas = subprocess.check_output(
|
||||
["psql", "-v", "ON_ERROR_STOP=1", "-t", "-c", "SELECT COUNT(*) FROM pg_stat_replication"],
|
||||
stdin=subprocess.DEVNULL,
|
||||
text=True,
|
||||
).strip()
|
||||
if int(replicas) > 0:
|
||||
# We are the primary and we have replicas; we expect that backups
|
||||
# will be taken on one of them.
|
||||
sys.exit(0)
|
||||
|
||||
skip_backups = get_config("postgresql", "skip_backups", "")
|
||||
if skip_backups.lower() in [1, "y", "t", "yes", "true", "enable", "enabled"]:
|
||||
sys.exit(0)
|
||||
|
||||
is_rhel_based = os.path.exists("/etc/redhat-release")
|
||||
if is_rhel_based:
|
||||
pg_data_paths = glob.glob("/var/lib/pgsql/*/data")
|
||||
else:
|
||||
pg_data_paths = glob.glob("/var/lib/postgresql/*/main")
|
||||
if len(pg_data_paths) != 1:
|
||||
print(f"PostgreSQL installation is not unique: {pg_data_paths}")
|
||||
sys.exit(1)
|
||||
pg_data_path = pg_data_paths[0]
|
||||
|
||||
disk_concurrency = get_config("postgresql", "backups_disk_concurrency", "1")
|
||||
env = os.environ.copy()
|
||||
env["WALG_UPLOAD_DISK_CONCURRENCY"] = disk_concurrency
|
||||
subprocess.check_call(["env-wal-g", "backup-push", pg_data_path], env=env)
|
||||
|
||||
data = json.loads(subprocess.check_output(["env-wal-g", "backup-list", "--json"]))
|
||||
# Use strptime for Python < 3.11 compatibility; we might not have
|
||||
# backports-datetime-fromisoformat here
|
||||
backups = {
|
||||
datetime.strptime(backup["time"], "%Y-%m-%dT%H:%M:%S%z"): backup["backup_name"]
|
||||
for backup in data
|
||||
}
|
||||
|
||||
one_month_ago = datetime.now(tz=timezone.utc) - timedelta(days=30)
|
||||
for date in sorted(backups.keys(), reverse=True):
|
||||
if date < one_month_ago:
|
||||
# We pass `FIND_FULL` such that if delta backups are being
|
||||
# used, we keep the prior FULL backup and all of the deltas
|
||||
# that we need. In practice, this means that if we're doing
|
||||
# weekly full backups (`backups_incremental = 6`), we only
|
||||
# delete any data once a week.
|
||||
subprocess.check_call(
|
||||
["env-wal-g", "delete", "--confirm", "before", "FIND_FULL", backups[date]]
|
||||
)
|
||||
# Because we're going from most recent to least recent, we
|
||||
# only have to do one delete operation
|
||||
break
|
||||
@@ -0,0 +1,8 @@
|
||||
# This file is managed by Zulip's Puppet configuration.
|
||||
#
|
||||
# Linux 6.14+ charges (per-user) the rings from io_uring against to
|
||||
# RLIMIT_MEMLOCK, which is by default limited to 8MiB; the default
|
||||
# Zulip PostgreSQL configuration needs ~13MiB. Set a limit sizably
|
||||
# above that.
|
||||
[Service]
|
||||
LimitMEMLOCK=256M
|
||||
276
puppet/zulip/files/postgresql/process_fts_updates
Executable file
276
puppet/zulip/files/postgresql/process_fts_updates
Executable file
@@ -0,0 +1,276 @@
|
||||
#!/usr/bin/env python3
|
||||
# Processes updates to PostgreSQL full-text search for new/edited messages.
|
||||
#
|
||||
# Zulip manages its PostgreSQL full-text search as follows. When the
|
||||
# content of a message is modified, a PostgreSQL trigger logs the
|
||||
# message ID to the `fts_update_log` table. In the background, this
|
||||
# program processes `fts_update_log`, updating the PostgreSQL full-text
|
||||
# search column search_tsvector in the main zerver_message.
|
||||
#
|
||||
# There are four cases this has to cover:
|
||||
#
|
||||
# 1. Running in development, with a venv but no
|
||||
# /home/zulip/deployments/current nor /etc/zulip/zulip.conf
|
||||
#
|
||||
# 2. Running in production, with postgresql on the same machine as the
|
||||
# frontend, with a venv and both /home/zulip/deployments/current
|
||||
# and /etc/zulip/zulip.conf
|
||||
#
|
||||
# 3. Running in production, on a postgresql machine which is not the
|
||||
# frontend, with a /etc/zulip/zulip.conf but no venv nor
|
||||
# /home/zulip/deployments/current
|
||||
#
|
||||
# 4. Running in production, on an application frontend server connected
|
||||
# to a remote postgresql server, because we cannot run code _on_ the
|
||||
# PostgreSQL server, such as in docker-zulip.
|
||||
#
|
||||
# Because of case (3), we cannot rely on functions from outside this
|
||||
# file (e.g. provided by scripts.lib.zulip_tools). For case (1),
|
||||
# however, we wish to import `zerver.settings` since we have no
|
||||
# /etc/zulip/zulip.conf to read, and that case _requires_ loading the
|
||||
# venv.
|
||||
#
|
||||
# We also must handle the cases where we are run as the `nagios` user,
|
||||
# which may not have permission to read all configuration files, and
|
||||
# thus (2) will look like (3).
|
||||
|
||||
import argparse
|
||||
import configparser
|
||||
import logging
|
||||
import os
|
||||
import select
|
||||
import sys
|
||||
import time
|
||||
from collections.abc import Sequence
|
||||
|
||||
import psycopg2
|
||||
import psycopg2.extensions
|
||||
from psycopg2.sql import SQL
|
||||
|
||||
BATCH_SIZE = 1000
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--quiet", action="store_true")
|
||||
parser.add_argument("--nagios-check", action="store_true")
|
||||
parser.add_argument("--nagios-user")
|
||||
options = parser.parse_args()
|
||||
|
||||
logging.Formatter.converter = time.gmtime
|
||||
logging.basicConfig(format="%(asctime)s %(levelname)s: %(message)s")
|
||||
logger = logging.getLogger("process_fts_updates")
|
||||
if options.quiet:
|
||||
logger.setLevel(logging.INFO)
|
||||
else:
|
||||
logger.setLevel(logging.DEBUG)
|
||||
|
||||
|
||||
def update_fts_columns(conn: psycopg2.extensions.connection) -> int:
|
||||
with conn.cursor() as cursor:
|
||||
cursor.execute(
|
||||
"SELECT id, message_id FROM fts_update_log ORDER BY id LIMIT %s FOR UPDATE SKIP LOCKED",
|
||||
(BATCH_SIZE,),
|
||||
)
|
||||
cursor.arraysize = BATCH_SIZE
|
||||
parts = list(zip(*cursor.fetchmany(), strict=True))
|
||||
if not parts:
|
||||
row_ids: Sequence[int] = []
|
||||
message_ids: Sequence[int] = []
|
||||
else:
|
||||
row_ids, message_ids = parts[0], parts[1]
|
||||
|
||||
if message_ids:
|
||||
if USING_PGROONGA:
|
||||
update_sql = SQL(
|
||||
"search_pgroonga = escape_html(subject) || ' ' || rendered_content"
|
||||
)
|
||||
else:
|
||||
update_sql = SQL(
|
||||
"search_tsvector = to_tsvector('zulip.english_us_search', subject || rendered_content)"
|
||||
)
|
||||
cursor.execute(
|
||||
SQL(
|
||||
"UPDATE zerver_message SET {update_sql} "
|
||||
"WHERE ctid IN ("
|
||||
" SELECT ctid FROM zerver_message"
|
||||
" WHERE id IN %s"
|
||||
" ORDER BY id FOR UPDATE"
|
||||
")"
|
||||
).format(update_sql=update_sql),
|
||||
(message_ids,),
|
||||
)
|
||||
if row_ids:
|
||||
cursor.execute("DELETE FROM fts_update_log WHERE id IN %s", (row_ids,))
|
||||
conn.commit()
|
||||
return len(row_ids)
|
||||
|
||||
|
||||
def update_all_rows(msg: str, conn: psycopg2.extensions.connection) -> None:
|
||||
while True:
|
||||
start_time = time.perf_counter()
|
||||
rows_updated = update_fts_columns(conn)
|
||||
if rows_updated:
|
||||
logger.log(
|
||||
logging.INFO,
|
||||
"process_fts_updates: %s %d rows, %d rows/sec",
|
||||
msg,
|
||||
rows_updated,
|
||||
rows_updated / (time.perf_counter() - start_time),
|
||||
)
|
||||
|
||||
if rows_updated != BATCH_SIZE:
|
||||
return
|
||||
|
||||
|
||||
def am_master(conn: psycopg2.extensions.connection) -> bool:
|
||||
with conn.cursor() as cursor:
|
||||
cursor.execute("SELECT pg_is_in_recovery()")
|
||||
return not cursor.fetchall()[0][0]
|
||||
|
||||
|
||||
def get_config(
|
||||
config_file: configparser.RawConfigParser,
|
||||
section: str,
|
||||
key: str,
|
||||
default_value: str = "",
|
||||
) -> str:
|
||||
if config_file.has_option(section, key):
|
||||
return config_file.get(section, key)
|
||||
return default_value
|
||||
|
||||
|
||||
pg_args = {}
|
||||
|
||||
USING_PGROONGA = False
|
||||
try:
|
||||
# Case (1); we insert the path to the development root.
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../..")))
|
||||
|
||||
# Cases (2) and (4); we insert the path to the production root.
|
||||
# This likely works out the same as the above path.
|
||||
#
|
||||
# We insert this path after the above, so that if running this
|
||||
# command from a specific non-current Zulip deployment, we prefer
|
||||
# that deployment's libraries.
|
||||
sys.path.insert(1, "/home/zulip/deployments/current")
|
||||
|
||||
# For cases (2) and (4), we also need to set up the virtualenv, so we
|
||||
# can read the Django settings.
|
||||
from scripts.lib.setup_path import setup_path
|
||||
|
||||
setup_path()
|
||||
|
||||
os.environ["DJANGO_SETTINGS_MODULE"] = "zproject.settings"
|
||||
from django.conf import settings
|
||||
|
||||
pg_args["host"] = settings.DATABASES["default"]["HOST"]
|
||||
pg_args["port"] = settings.DATABASES["default"].get("PORT")
|
||||
pg_args["password"] = settings.DATABASES["default"].get("PASSWORD")
|
||||
pg_args["user"] = settings.DATABASES["default"]["USER"]
|
||||
pg_args["dbname"] = settings.DATABASES["default"]["NAME"]
|
||||
pg_args["sslmode"] = settings.DATABASES["default"]["OPTIONS"].get("sslmode")
|
||||
pg_args["connect_timeout"] = "600"
|
||||
USING_PGROONGA = settings.USING_PGROONGA
|
||||
except (ImportError, PermissionError):
|
||||
# Case (3): we know that the PostgreSQL server is on this host.
|
||||
pg_args["user"] = "zulip"
|
||||
|
||||
config_file = configparser.RawConfigParser()
|
||||
config_file.read("/etc/zulip/zulip.conf")
|
||||
|
||||
if get_config(config_file, "machine", "pgroonga", "false").lower() in [
|
||||
"1",
|
||||
"y",
|
||||
"t",
|
||||
"yes",
|
||||
"true",
|
||||
"enable",
|
||||
"enabled",
|
||||
]:
|
||||
USING_PGROONGA = True
|
||||
|
||||
pg_args["user"] = get_config(config_file, "postgresql", "database_user", "zulip")
|
||||
if pg_args["user"] != "zulip":
|
||||
secrets_file = configparser.RawConfigParser()
|
||||
secrets_file.read("/etc/zulip/zulip-secrets.conf")
|
||||
pg_args["password"] = get_config(secrets_file, "secrets", "postgres_password")
|
||||
pg_args["host"] = "localhost"
|
||||
pg_args["dbname"] = get_config(config_file, "postgresql", "database_name", "zulip")
|
||||
|
||||
conn: psycopg2.extensions.connection | None
|
||||
|
||||
if options.nagios_check:
|
||||
# Nagios connects as itself, unless you specify otherwise
|
||||
if options.nagios_user:
|
||||
pg_args["user"] = options.nagios_user
|
||||
else:
|
||||
del pg_args["user"]
|
||||
# connection_factory=None lets mypy understand the return type
|
||||
conn = psycopg2.connect(connection_factory=None, **pg_args)
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("SELECT count(*) FROM fts_update_log")
|
||||
num = cursor.fetchall()[0][0]
|
||||
|
||||
# nagios exit codes
|
||||
states = {
|
||||
"OK": 0,
|
||||
"WARNING": 1,
|
||||
"CRITICAL": 2,
|
||||
"UNKNOWN": 3,
|
||||
}
|
||||
|
||||
state = "OK"
|
||||
if num > 5:
|
||||
state = "CRITICAL"
|
||||
print(f"{state}: {num} rows in fts_update_log table")
|
||||
sys.exit(states[state])
|
||||
|
||||
|
||||
conn = None
|
||||
|
||||
retries = 1
|
||||
|
||||
while True:
|
||||
try:
|
||||
if conn is None:
|
||||
# connection_factory=None lets mypy understand the return type
|
||||
conn = psycopg2.connect(connection_factory=None, **pg_args)
|
||||
retries = 30
|
||||
|
||||
conn.autocommit = False
|
||||
conn.isolation_level = psycopg2.extensions.ISOLATION_LEVEL_READ_COMMITTED
|
||||
|
||||
first_check = True
|
||||
while not am_master(conn):
|
||||
if first_check:
|
||||
first_check = False
|
||||
logger.warning("In recovery; sleeping")
|
||||
time.sleep(5)
|
||||
|
||||
logger.debug("process_fts_updates: listening for search index updates")
|
||||
|
||||
with conn.cursor() as cursor:
|
||||
cursor.execute("LISTEN fts_update_log;")
|
||||
conn.commit()
|
||||
|
||||
# Catch up on any historical columns
|
||||
update_all_rows("Caught up", conn)
|
||||
|
||||
if select.select([conn], [], [], 30) != ([], [], []):
|
||||
conn.poll()
|
||||
conn.notifies.clear()
|
||||
update_all_rows("Updated", conn)
|
||||
|
||||
except psycopg2.OperationalError as e:
|
||||
# We will end up here if the database is restarted, or becomes read-only.
|
||||
retries -= 1
|
||||
if retries <= 0:
|
||||
raise
|
||||
logger.info(e.pgerror, exc_info=True)
|
||||
logger.info("Sleeping and reconnecting")
|
||||
time.sleep(5)
|
||||
if conn is not None:
|
||||
conn.close()
|
||||
conn = None
|
||||
except KeyboardInterrupt:
|
||||
print(sys.argv[0], "exited after receiving KeyboardInterrupt")
|
||||
break
|
||||
162
puppet/zulip/files/postgresql/wal-g-exporter
Executable file
162
puppet/zulip/files/postgresql/wal-g-exporter
Executable file
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env python3
|
||||
import configparser
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from collections.abc import Mapping
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from typing import Protocol
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
|
||||
class GaugeMetric(Protocol):
|
||||
def __call__(self, value: float, labels: Mapping[str, str] | None = None, /) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class WalGPrometheusServer(BaseHTTPRequestHandler):
|
||||
METRIC_PREFIX = "wal_g_backup_"
|
||||
|
||||
metrics: dict[str, list[str]] = {}
|
||||
metric_values: dict[str, dict[str, str]] = defaultdict(dict)
|
||||
|
||||
server_version = "wal-g-prometheus-server/1.0"
|
||||
|
||||
def gauge(
|
||||
self, name: str, description: str | None = None, default_value: float | None = None
|
||||
) -> GaugeMetric:
|
||||
if name in self.metrics:
|
||||
raise ValueError(f"Redefinition of {name} metric")
|
||||
self.metrics[name] = [f"# TYPE {self.METRIC_PREFIX}{name} gauge"]
|
||||
if description is not None:
|
||||
self.metrics[name].append(f"# HELP {self.METRIC_PREFIX}{name} {description}")
|
||||
|
||||
def inner(value: float, labels: Mapping[str, str] | None = None) -> None:
|
||||
label_str = ""
|
||||
if labels:
|
||||
label_str = "{" + ",".join(f'{k}="{v}"' for k, v in labels.items()) + "}"
|
||||
self.metric_values[name][label_str] = f"{self.METRIC_PREFIX}{name}{label_str} {value}"
|
||||
|
||||
if default_value is not None:
|
||||
inner(default_value)
|
||||
return inner
|
||||
|
||||
def print_metrics(self) -> None:
|
||||
lines = []
|
||||
for metric_name in self.metrics:
|
||||
if self.metric_values[metric_name]:
|
||||
# Print preamble
|
||||
lines += self.metrics[metric_name]
|
||||
lines += self.metric_values[metric_name].values()
|
||||
lines.append("")
|
||||
self.wfile.write("\n".join(lines).encode())
|
||||
|
||||
def do_GET(self) -> None:
|
||||
if urlsplit(self.path).path != "/metrics":
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
sys.stderr.flush()
|
||||
return
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header("Content-type", "text/plain; version=0.0.4")
|
||||
self.end_headers()
|
||||
|
||||
self.metrics = {}
|
||||
self.metric_values = defaultdict(dict)
|
||||
|
||||
backup_ok = self.gauge("ok", "If wal-g backup-list was OK", 0)
|
||||
backup_count = self.gauge("count", "Number of backups")
|
||||
backup_earliest_age_seconds = self.gauge("earliest_age_seconds", "Age of the oldest backup")
|
||||
backup_latest_age_seconds = self.gauge(
|
||||
"latest_age_seconds", "Age of the most recent backup"
|
||||
)
|
||||
backup_latest_duration_seconds = self.gauge(
|
||||
"latest_duration_seconds", "Duration the most recent backup took, in seconds"
|
||||
)
|
||||
backup_latest_compressed_size_bytes = self.gauge(
|
||||
"latest_compressed_size_bytes", "Size of the most recent backup, in bytes"
|
||||
)
|
||||
backup_latest_uncompressed_size_bytes = self.gauge(
|
||||
"latest_uncompressed_size_bytes",
|
||||
"Uncompressed size of the most recent backup, in bytes",
|
||||
)
|
||||
backup_total_compressed_size_bytes = self.gauge(
|
||||
"total_compressed_size_bytes", "Total compressed size of all backups, in bytes"
|
||||
)
|
||||
|
||||
now = datetime.now(tz=timezone.utc)
|
||||
try:
|
||||
query = parse_qs(urlsplit(self.path).query)
|
||||
modified_env = os.environ.copy()
|
||||
if query.get("bucket") and len(query["bucket"]) == 1:
|
||||
bucket = query["bucket"][0]
|
||||
modified_env["WALG_S3_PREFIX"] = f"s3://{bucket}"
|
||||
else:
|
||||
config_file = configparser.RawConfigParser()
|
||||
config_file.read("/etc/zulip/zulip-secrets.conf")
|
||||
bucket = config_file["secrets"]["s3_backups_bucket"]
|
||||
|
||||
backup_list_output = subprocess.check_output(
|
||||
["env-wal-g", "backup-list", "--detail", "--json"],
|
||||
text=True,
|
||||
env=modified_env,
|
||||
)
|
||||
data = json.loads(backup_list_output) if backup_list_output else []
|
||||
backup_count(len(data), {"bucket": bucket})
|
||||
|
||||
backup_total_compressed_size_bytes(
|
||||
sum(e["compressed_size"] for e in data), {"bucket": bucket}
|
||||
)
|
||||
|
||||
if len(data) > 0:
|
||||
data.sort(key=lambda e: e["start_time"], reverse=True)
|
||||
latest = data[0]
|
||||
labels = {
|
||||
"host": latest["hostname"],
|
||||
"pg_version": str(latest["pg_version"]),
|
||||
"bucket": bucket,
|
||||
}
|
||||
backup_latest_compressed_size_bytes(latest["compressed_size"], labels)
|
||||
backup_latest_uncompressed_size_bytes(latest["uncompressed_size"], labels)
|
||||
|
||||
def t(key: str, e: dict[str, str] = latest) -> datetime:
|
||||
return datetime.strptime(e[key], e["date_fmt"]).replace(tzinfo=timezone.utc)
|
||||
|
||||
backup_earliest_age_seconds(
|
||||
(now - t("start_time", data[-1])) / timedelta(seconds=1),
|
||||
{
|
||||
"host": data[-1]["hostname"],
|
||||
"pg_version": data[-1]["pg_version"],
|
||||
"bucket": bucket,
|
||||
},
|
||||
)
|
||||
backup_latest_age_seconds((now - t("start_time")) / timedelta(seconds=1), labels)
|
||||
backup_latest_duration_seconds(
|
||||
(t("finish_time") - t("start_time")) / timedelta(seconds=1), labels
|
||||
)
|
||||
backup_ok(1)
|
||||
except Exception:
|
||||
logging.exception("Error while getting backup information")
|
||||
finally:
|
||||
self.print_metrics()
|
||||
self.log_message(
|
||||
"Served in %.2f seconds",
|
||||
(datetime.now(tz=timezone.utc) - now) / timedelta(seconds=1),
|
||||
)
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s: %(message)s")
|
||||
server = HTTPServer(("127.0.0.1", 9188), WalGPrometheusServer)
|
||||
logging.info("Starting server...")
|
||||
with contextlib.suppress(KeyboardInterrupt):
|
||||
server.serve_forever()
|
||||
|
||||
server.server_close()
|
||||
logging.info("Stopping server...")
|
||||
149
puppet/zulip/files/postgresql/zulip_english.stop
Normal file
149
puppet/zulip/files/postgresql/zulip_english.stop
Normal file
@@ -0,0 +1,149 @@
|
||||
i
|
||||
me
|
||||
my
|
||||
myself
|
||||
we
|
||||
our
|
||||
ours
|
||||
ourselves
|
||||
you
|
||||
your
|
||||
yours
|
||||
yourself
|
||||
yourselves
|
||||
he
|
||||
him
|
||||
his
|
||||
himself
|
||||
she
|
||||
her
|
||||
hers
|
||||
herself
|
||||
it
|
||||
its
|
||||
itself
|
||||
they
|
||||
them
|
||||
their
|
||||
theirs
|
||||
themselves
|
||||
what
|
||||
which
|
||||
who
|
||||
whom
|
||||
this
|
||||
that
|
||||
these
|
||||
those
|
||||
am
|
||||
is
|
||||
are
|
||||
was
|
||||
were
|
||||
be
|
||||
been
|
||||
being
|
||||
have
|
||||
has
|
||||
had
|
||||
having
|
||||
do
|
||||
does
|
||||
did
|
||||
doing
|
||||
a
|
||||
an
|
||||
the
|
||||
and
|
||||
but
|
||||
if
|
||||
or
|
||||
because
|
||||
as
|
||||
until
|
||||
while
|
||||
of
|
||||
at
|
||||
by
|
||||
for
|
||||
with
|
||||
about
|
||||
against
|
||||
between
|
||||
into
|
||||
through
|
||||
during
|
||||
before
|
||||
after
|
||||
above
|
||||
below
|
||||
to
|
||||
from
|
||||
up
|
||||
down
|
||||
in
|
||||
out
|
||||
on
|
||||
off
|
||||
over
|
||||
under
|
||||
again
|
||||
further
|
||||
then
|
||||
once
|
||||
here
|
||||
there
|
||||
when
|
||||
where
|
||||
why
|
||||
how
|
||||
all
|
||||
any
|
||||
both
|
||||
each
|
||||
few
|
||||
more
|
||||
most
|
||||
other
|
||||
some
|
||||
such
|
||||
no
|
||||
nor
|
||||
not
|
||||
only
|
||||
own
|
||||
same
|
||||
so
|
||||
than
|
||||
too
|
||||
very
|
||||
s
|
||||
t
|
||||
can
|
||||
will
|
||||
just
|
||||
don
|
||||
should
|
||||
now
|
||||
b
|
||||
c
|
||||
d
|
||||
e
|
||||
f
|
||||
g
|
||||
h
|
||||
j
|
||||
k
|
||||
l
|
||||
m
|
||||
n
|
||||
o
|
||||
p
|
||||
q
|
||||
r
|
||||
u
|
||||
v
|
||||
w
|
||||
x
|
||||
y
|
||||
z
|
||||
Reference in New Issue
Block a user