[Python Tips] Apscheduler

11 comments

themarkymark
73
20 days agoBusy4 min read

Introducing Apscheduler

Apscheduler is a job manager (similar to cron) to run sections of code on a schedule. I use it frequently to manage jobs that need to run at a set interval.

Apscheduler can be used to schedule code execution in three different ways:

  • Cron style scheduling
  • Interval system scheduling
  • One-off execution

Job Persistence

If you want to use a database to store jobs, you can use any of these supported databases:

Apscheduler can be used to save jobs in a database to survive restarts and reboots. The job scheduler can persist as well as run jobs that should have run when it was offline.

  • Any supported SQLAlchemy DB
  • MongoDB
  • Redis
  • RethinkDB
  • ZooKeeper

Installing Apscheduler

I recommend using pip or better yet pipenv

pip install apscheduler

How to use Apscheduler

I am not going to cover all use cases as Apscheduler is a very powerful module. In most cases, I use the BackgroundScheduler but there are seven different schedulers depending on your needs. For example, there is a blocking scheduler if you want to force jobs to be executed in a single thread. There are customized schedulers for Qt, Twisted, Tornado, and Async.

Setup

You have import the correct scheduler you want to use.

from apscheduler.schedulers.background import BackgroundScheduler

You also have to instantiate the scheduler.

scheduler = BackgroundScheduler()

This setup will store the job queue in memory and will not persist across restarts and reboots. You can use a database if you need this functionality.

One of the great things about Apscheduler is that it allows you to easily multi-thread your application without dealing with threads directly.

Adding a job

Once you have the boilerplate finished, you need to add a job. You have two choices here, use add_job() or decorate a function with scheduled_job()

I am going to use add_job() as it is the most straightforward to use.

scheduler.add_job(load_blacklist, 'interval', minutes=10, id='load_blacklist')

In some of my scripts that don't yet support the Global Blacklist API, I load a local copy of my blacklist. I do this using an interval based job schedule every 10 minutes.

Starting the scheduler

Once you have added your jobs, you can start the scheduler.

scheduler.start()

You can also stop and pause the scheduler. You can and list, modify, and remove jobs.

Advanced Usage

There is a lot of advanced functionality in Apscheduler that allows for a ton of customization.

Examples
  • max_instances - By default, Apscheduler supports 10 threads but there is a limit of one instance of any specific job. This allows you to run multiple instances of a specific job at once.
  • misfire_grace_time - This is used to run jobs while the scheduler was unable to run a job because it was offline or busy.

Logging

Apscheduler supports the logging module and has its own logger named apscheduler.

Example logging boiler plate

import logging

logging.basicConfig()
logging.getLogger('apscheduler').setLevel(logging.INFO)

This should give you enough information to see the value of Apscheduler and how to start using it for your Python projects. More information can be found in the Online Documentation and you can find examples in the examples folder.

Apscheduler Github Repo

https://github.com/agronholm/apscheduler

My Python Tips Series

Comments

Sort byBest