[Python Tips] Apscheduler
Apscheduler is a job manager (similar to
cron) to run sections of code on a schedule. I use it frequently to manage jobs that need to run at a set interval.
Apscheduler can be used to schedule code execution in three different ways:
- Cron style scheduling
- Interval system scheduling
- One-off execution
If you want to use a database to store jobs, you can use any of these supported databases:
Apscheduler can be used to save jobs in a database to survive restarts and reboots. The job scheduler can persist as well as run jobs that should have run when it was offline.
- Any supported SQLAlchemy DB
I recommend using
pip or better yet
pip install apscheduler
How to use Apscheduler
I am not going to cover all use cases as Apscheduler is a very powerful module. In most cases, I use the
BackgroundScheduler but there are seven different schedulers depending on your needs. For example, there is a blocking scheduler if you want to force jobs to be executed in a single thread. There are customized schedulers for Qt, Twisted, Tornado, and Async.
You have import the correct scheduler you want to use.
from apscheduler.schedulers.background import BackgroundScheduler
You also have to instantiate the scheduler.
scheduler = BackgroundScheduler()
This setup will store the job queue in memory and will not persist across restarts and reboots. You can use a database if you need this functionality.
One of the great things about Apscheduler is that it allows you to easily multi-thread your application without dealing with threads directly.
Adding a job
Once you have the boilerplate finished, you need to add a job. You have two choices here, use
add_job() or decorate a function with
I am going to use
add_job() as it is the most straightforward to use.
scheduler.add_job(load_blacklist, 'interval', minutes=10, id='load_blacklist')
In some of my scripts that don't yet support the Global Blacklist API, I load a local copy of my blacklist. I do this using an interval based job schedule every 10 minutes.
Starting the scheduler
Once you have added your jobs, you can start the scheduler.
You can also stop and pause the scheduler. You can and list, modify, and remove jobs.
There is a lot of advanced functionality in Apscheduler that allows for a ton of customization.
max_instances- By default, Apscheduler supports 10 threads but there is a limit of one instance of any specific job. This allows you to run multiple instances of a specific job at once.
misfire_grace_time- This is used to run jobs while the scheduler was unable to run a job because it was offline or busy.
Apscheduler supports the logging module and has its own logger named
Example logging boiler plate
import logging logging.basicConfig() logging.getLogger('apscheduler').setLevel(logging.INFO)
This should give you enough information to see the value of
Apscheduler and how to start using it for your Python projects. More information can be found in the Online Documentation and you can find examples in the