Workers

The workers module provides background task management with support for both thread-based and process-based concurrency strategies. It offers a consistent interface for running tasks in the background with proper lifecycle management, error handling, and graceful shutdown capabilities.

Overview

Workers are designed for running long-running background tasks with different concurrency models:

• ThreadWorker: Ideal for I/O-bound tasks that benefit from shared memory access with the main process
• ProcessWorker: Best for CPU-bound tasks that need to bypass Python's Global Interpreter Lock (GIL)
• Server: Manages multiple workers collectively with signal handling for graceful shutdowns

All workers share a common lifecycle with setup(), main(), and teardown() hooks, supporting both synchronous and asynchronous implementations. The module also includes utilities for working with async code and optionally integrating uvloop for improved async performance.

API Reference

spoc.workers

Worker module providing different concurrency strategies for background tasks.

This module implements various worker classes that can run tasks in different concurrency contexts (threads, processes) with consistent lifecycle management and error handling. It also provides a server to manage multiple workers.

AbstractWorker(name)

Abstract base class for all worker implementations.

Provides a consistent interface for different worker types (thread, process) with unified lifecycle management and error handling.

is_running property

Return True if the worker is running.

stop()

Signal the worker to stop.

start()

Start the worker.

join(timeout=None)

Join the worker, with optional timeout.

run()

Main run method that handles the worker lifecycle.

setup()

Setup method called before main. Override in subclasses.

main() abstractmethod

Main worker method. Must be implemented by subclasses.

teardown()

Teardown method called after main. Override in subclasses.

lifecycle(event_type, **data)

Lifecycle event handler. Override in subclasses.

ThreadWorker(name='ThreadWorker', daemon=True)

Worker implementation that runs in a thread.

Useful for I/O-bound tasks or tasks that need to share memory with the main process.

ProcessWorker(name='ProcessWorker', daemon=True)

Worker implementation that runs in a separate process.

Useful for CPU-bound tasks that benefit from bypassing the GIL.

Server(name='Server')

Server to manage multiple workers.

Provides methods to start, stop, and manage the lifecycle of workers collectively, with proper signal handling for graceful shutdowns.

add_worker(worker)

Add a worker to be managed by this server.

start()

Start all managed workers.

stop()

Stop all managed workers.

join_all(timeout=5)

Join all workers with a shared timeout.

run_forever()

Run the server until stopped by signal or exception.

set_uvloop_if_available()

Set uvloop as the event loop policy if installed.

run_async_safely(coro)

Run coroutine safely whether in an existing event loop or not.