Integration

Python job monitoring: Celery and APScheduler

In a Python application, scheduled tasks multiply and nobody keeps the list up to date. The SilenceWatch library keeps it for you: it reads your Celery beat schedule or your APScheduler jobs, declares them with their real schedule, and reports every run.

One package, one key

pip install "silencewatch[celery]"       # or "silencewatch[apscheduler]"
export SILENCEWATCH_API_KEY=sw_...

from silencewatch.integrations.celery import instrument

instrument(app)

from silencewatch.integrations.apscheduler import instrument

instrument(scheduler)

Restart the application. Every scheduled task appears in SilenceWatch with its real schedule, and sends a heartbeat around every run, with its duration and its failures. No check to create by hand, no ping URL to copy, nothing to keep in sync when the code changes.

What sets it apart

It reads the real schedules. A crontab(hour=2, minute=0) is declared as 0 2 * * *, in the time zone of your Celery app; an APScheduler CronTrigger or IntervalTrigger is read the same way.
It gives each job a stable identity (celery:shop.tasks.backup, or the job id for APScheduler): history survives restarts, redeployments and renames.
It never deletes anything. A job that disappears from the code is flagged orphaned; the history is kept until you decide to delete it.
It tells environments apart: the same application declared in staging and in production yields two separate checks.
It is built to take more schedulers. Celery and APScheduler are two modules of one small package; the next one is another module, with the same declaration and the same rules.

It cannot hurt you

A monitoring tool that disturbs production is worse than none. The library follows strict rules:

it never fails your job: every error is swallowed and logged at WARNING, and your task runs once, its result and its exception passing through unchanged;
it never blocks your job: sending goes through a bounded queue and a daemon thread, and survives the fork of Celery’s workers;
it degrades silently: with no key, no network or no server, the application runs exactly as without it;
SILENCEWATCH_ENABLED=false removes the whole mechanism;
it has no dependency: the standard library only.

All settings and limits are in the library documentation.

If your job is not Celery or APScheduler

Nothing forces you to use the library: any job that can make an HTTP request can be monitored with the ping API. See cron job monitoring and the examples.

Library documentation Library source

Frequently asked questions

How do I monitor Celery beat tasks?

Install the silencewatch package with the celery extra, set an API key and call instrument(app). When a worker starts it declares the beat schedule, and each scheduled task sends a heartbeat around every run.

Does it also monitor APScheduler?

Yes, APScheduler 3: cron and interval triggers are read from the scheduler, and every run is reported. Give each job an id so its identity does not change from one start to the next.

Can the library slow down or crash my application?

No. Network errors are swallowed, sending goes through a bounded queue handled in a background thread, and with no key or no network the application runs as without the library. It can be disabled entirely.

What about Django, RQ or Dramatiq?

Periodic tasks kept in the database by django-celery-beat are read by the Celery integration, with nothing to configure. RQ and Dramatiq are not read yet. The library is organised as one module per scheduler, so more can be added; until then, any job can be monitored by calling the ping API.