archivebox.workers.supervisord_util

Module Contents

Functions

_shell_join

RUNSERVER_WORKER

is_port_in_use

Check if a port is already in use.

get_sock_file

Get the path to the supervisord socket file, symlinking to a shorter path if needed due to unix path length limits

follow

Yield each line from a file as they are written. sleep_sec is the time to sleep after empty reads.

create_supervisord_config

create_worker_config

Create a supervisord worker config file for a given daemon

get_existing_supervisord_process

stop_existing_supervisord_process

start_new_supervisord_process

wait_for_supervisord_ready

Poll for supervisord readiness without a fixed startup sleep.

get_or_create_supervisord_process

start_worker

get_worker

stop_worker

tail_worker_logs

tail_multiple_worker_logs

Tail multiple log files simultaneously, interleaving their output.

watch_worker

loop continuously and monitor worker’s health

start_server_workers

start_cli_workers

Data

LOG_FILE_NAME

CONFIG_FILE_NAME

PID_FILE_NAME

WORKERS_DIR_NAME

_supervisord_proc

RUNNER_WORKER

RUNNER_WATCH_WORKER

SERVER_WORKER

API

archivebox.workers.supervisord_util.LOG_FILE_NAME[source]

‘supervisord.log’

archivebox.workers.supervisord_util.CONFIG_FILE_NAME[source]

‘supervisord.conf’

archivebox.workers.supervisord_util.PID_FILE_NAME[source]

‘supervisord.pid’

archivebox.workers.supervisord_util.WORKERS_DIR_NAME[source]

‘workers’

archivebox.workers.supervisord_util._supervisord_proc[source]

None

archivebox.workers.supervisord_util._shell_join(args: list[str]) str[source]
archivebox.workers.supervisord_util.RUNNER_WORKER[source]

None

archivebox.workers.supervisord_util.RUNNER_WATCH_WORKER[source]

None

archivebox.workers.supervisord_util.SERVER_WORKER[source]

None

archivebox.workers.supervisord_util.RUNSERVER_WORKER(host: str, port: str, *, reload: bool, pidfile: str | None = None, nothreading: bool = False)[source]
archivebox.workers.supervisord_util.is_port_in_use(host: str, port: int) bool[source]

Check if a port is already in use.

archivebox.workers.supervisord_util.get_sock_file()[source]

Get the path to the supervisord socket file, symlinking to a shorter path if needed due to unix path length limits

archivebox.workers.supervisord_util.follow(file, sleep_sec=0.1) collections.abc.Iterator[str][source]

Yield each line from a file as they are written. sleep_sec is the time to sleep after empty reads.

archivebox.workers.supervisord_util.create_supervisord_config()[source]
archivebox.workers.supervisord_util.create_worker_config(daemon)[source]

Create a supervisord worker config file for a given daemon

archivebox.workers.supervisord_util.get_existing_supervisord_process()[source]
archivebox.workers.supervisord_util.stop_existing_supervisord_process()[source]
archivebox.workers.supervisord_util.start_new_supervisord_process(daemonize=False)[source]
archivebox.workers.supervisord_util.wait_for_supervisord_ready(max_wait_sec: float = 5.0, interval_sec: float = 0.1)[source]

Poll for supervisord readiness without a fixed startup sleep.

archivebox.workers.supervisord_util.get_or_create_supervisord_process(daemonize=False)[source]
archivebox.workers.supervisord_util.start_worker(supervisor, daemon, lazy=False)[source]
archivebox.workers.supervisord_util.get_worker(supervisor, daemon_name)[source]
archivebox.workers.supervisord_util.stop_worker(supervisor, daemon_name)[source]
archivebox.workers.supervisord_util.tail_worker_logs(log_path: str)[source]
archivebox.workers.supervisord_util.tail_multiple_worker_logs(log_files: list[str], follow=True, proc=None)[source]

Tail multiple log files simultaneously, interleaving their output.

Args: log_files: List of log file paths to tail follow: Whether to keep following (True) or just read existing content (False) proc: Optional subprocess.Popen object - stop tailing when this process exits

archivebox.workers.supervisord_util.watch_worker(supervisor, daemon_name, interval=5)[source]

loop continuously and monitor worker’s health

archivebox.workers.supervisord_util.start_server_workers(host='0.0.0.0', port='8000', daemonize=False, debug=False, reload=False, nothreading=False)[source]
archivebox.workers.supervisord_util.start_cli_workers(watch=False)[source]