Fork me on GitHub

Ivan-Site.com

Autoreload Code in Python

While you're developing and debugging your WSGI application there's lots of ways to automatically reload your code on change out of the box. For example, if you're using werkzeug you can just pass the use_reloader flag:

run_sumple('127.0.0.1', 5000, app, use_reloader=True)

For Flask, which actually uses werzeug internally, setting debug=True is all you need:

app.run(debug=True)

Django will automatically do it for you when you use:

manage.py runserver

All of these examples work great while developing locally, however, they are greatly discouraged against being used in production. So the question arises, what do you do to automatically reload your code in production?

uWSGI

if you're using uWSGI and Django you can actually just hook into Django's own autoreloading:

import uwsgi
from uwsgidecorators import timer
from django.utils import autoreload

@timer(3)
def change_code_gracefull_reload(sig):
    if autoreload.code_changed():
        uwsgi.reload()

If you're using any other framework, or no framework at all then you will probably have to include code monitoring inside of your application. Here's an example that was kindly borrowed and modified from CherryPy:

import os, sys

_mtimes = {}
_win = (sys.platform == "win32")

_error_files = []

def code_changed():
    filenames = []
    for m in sys.modules.values():
        try:
            filenames.append(m.__file__)
        except AttributeError:
            pass
    for filename in filenames + _error_files:
        if not filename:
            continue
        if filename.endswith(".pyc") or filename.endswith(".pyo"):
            filename = filename[:-1]
        if filename.endswith("$py.class"):
            filename = filename[:-9] + ".py"
        if not os.path.exists(filename):
            continue # File might be in an egg, so it can't be reloaded.
        stat = os.stat(filename)
        mtime = stat.st_mtime
        if _win:
            mtime -= stat.st_ctime
        if filename not in _mtimes:
            _mtimes[filename] = mtime
            continue
        if mtime != _mtimes[filename]:
            _mtimes.clear()
            try:
                del _error_files[_error_files.index(filename)]
            except ValueError:
                pass
            return True
    return False

you can save the above in autoreload.py inside your project and then we'll be able to use it like so (similar to the django example):

import uwsgi
from uwsgidecorators import timer
import autoreload

@timer(3)
def change_code_gracefull_reload(sig):
    if autoreload.code_changed():
        uwsgi.reload()

gunicorn

For gunicorn, we'll have to use one of the hooks in your gunicorn config:

import threading
import time
try:
    from django.utils import autoreload
except ImportError:
    import autoreload # we're not using django

def reloader(server):
    while True:
        if autoreload.code_changed():
            server.reload()
        time.sleep(3)

def when_ready(server):
    t = threading.Thread(target=reloader, args=(server, ))
    t.daemon = True # needed to be able to stop gunicorn properly
    t.start()

You'll need to save the above into a file, lets say config.py, and pass it to gunicorn like so:

gunicorn -c config.py application

external solution

You can also use an external solution outside of WSGI server you're using that just signals it to restart on a code change like watchdog. For example:

watchmedo shell-command --patterns="*.py" --recursive --command='kill -HUP `cat /tmp/gunicorn.pid`' /path/to/project/

Posted Sun 07 April 2013 by Ivan Dyedov in Python (Python, uWSGI, gunicorn, WSGI)