Handling Django Settings Files

Jan 10

Handling Django Settings Files

I have seen a lot of talk over the past couple years about how to handle different settings files and databases, synced between production and development. I have happened onto a way of doing it that makes me happy, and figured I would share it with the world.

File structure

I use a file structure that looks like this:

project/
    settings/
        __init__.py (empty)
        base.py
        sqlite.py
        postgres.py

The base.py contains all of the configuration options that are shared among the databases. INSTALLED_APPS, etc. All of the DATABASE settings should be specified in the more-specific files. As well as things that differ by environment, like remote servers, cache settings, cookie domains, and other things.

This allows you to run the sqlite settings file, and have it be set to localhost, or whatever your development settings are. Then in production you just run against the postgres settings.

A good example of this being used in practice is on Read the Docs.

But wait, there's more!

manage.py for dev

./manage.py is great for development. It is the easiest way to get started, and it automatically sets up your paths and stuff. With my setup, I actually explicitly set manage.py's settings file to the sqlite file.

This means that whenever you are using manage.py, you are in a development context. So, what do you do about production?

django-admin.py is for production

In production, you set your DJANGO_SETTINGS_MODULE to the postgres settings file. So whenever you use django-admin.py, you will be running against the production database.

I really like this scheme, because it gives you a logical distinction between production and development in your code, and in your interface on the CLI. When you are developing, you are using manage.py and editing the sqlite settings file. The reverse for production.

""" The HOSTMAP variable is a dictionary of lists, the keys representing roles of a server, the values representing the hostnames of machines that fill those roles. You can get your hostname by typing `hostname` into a terminal. """ import socket, warnings, sys from django.utils.importlib import import_module HOSTMAP = { 'development': [ 'iRule', 'ubuntu', ], 'staging': [ 'staging.brokenseal', ], 'production': [ 'brokenseal', ], } def update_current_settings(file_name): """ Given a filename, this function will insert all variables and functions in ALL_CAPS into the global scope. """ new_settings = import_module(file_name) for k, v in new_settings.__dict__.items(): if k.upper() == k: globals().update({k:v}) to_load = [] for k, v in HOSTMAP.items(): if socket.gethostname() in v: to_load.append(k) for x in to_load: new_settings_path= 'mmds.conf.%s' % x try: update_current_settings(new_settings_path) except ImportError: warnings.warn("Failed to import %s" % new_settings_path)

Comments

1 Josh Smeaton says...

Agreed. We're successfully using a similar scheme at our work. Each developer also has their own settings file for extreme customizations where required.

Posted at 11:08 p.m. on January 10, 2011

2 Flaviu Simihaian says...

Great suggestions. I started trying this and it makes deployment easier to keep straight in my head.

Posted at 11:22 p.m. on January 10, 2011

3 Alon Swartz says...

Nice post Eric, we use a similar scheme described here:

http://www.turnkeylinux.org/blog/django-settings

Posted at 1:53 a.m. on January 11, 2011

4 Chris Forrette says...

Hey Eric,

This is pretty rad -- I like what you said about the distinction of using manage.py for dev and django-admin.py for production -- but why not have, say, 'development.py' and 'production.py' over 'sqlite.py' and 'postgres.py'? It seems like it would make the configuration a little more obvious to developers coming into a project who are unfamiliar with the convention. Plus I use MySQL quite a bit and I haven't worked with Django a ton yet -- is using sqlite for dev and postgres for production a common practice?

Posted at 2:11 a.m. on January 11, 2011

5 Davide Callegari says...

Hi, I have to agree with Chris Forrette, development.py, staging.py and production.py are much clearer names.

I'm using a similar way of handling settings file, even though I have a slightly different approach:

project/ settings.py conf/ default.py development.py staging.py production.py

Every file inside 'conf' import everything from default.py, just like your base.py, and settings.py looks like this (inspired by https://gist.github.com/217198):

&quot;&quot;&quot;
The HOSTMAP variable is a dictionary of lists, the keys representing
roles of a server, the values representing the hostnames of machines that
fill those roles.

You can get your hostname by typing `hostname` into a terminal.
&quot;&quot;&quot;

import socket, warnings, sys
from django.utils.importlib import import_module

HOSTMAP = {
    &#39;development&#39;: [
        &#39;iRule&#39;,
        &#39;ubuntu&#39;,
    ],
    &#39;staging&#39;: [
        &#39;staging.brokenseal&#39;,
    ],
    &#39;production&#39;: [
        &#39;brokenseal&#39;,
    ],
}

def update_current_settings(file_name):
    &quot;&quot;&quot;
    Given a filename, this function will insert all variables and
    functions in ALL_CAPS into the global scope.
    &quot;&quot;&quot;
    new_settings = import_module(file_name)

    for k, v in new_settings.__dict__.items():
        if k.upper() == k:
            globals().update({k:v})

to_load = []

for k, v in HOSTMAP.items():
    if socket.gethostname() in v:
        to_load.append(k)

for x in to_load:
    new_settings_path= &#39;mmds.conf.%s&#39; % x
    try:
        update_current_settings(new_settings_path)
    except ImportError:
        warnings.warn(&quot;Failed to import %s&quot; % new_settings_path)

Posted at 3:24 a.m. on January 11, 2011

6 Konrad Delong says...

I use a collection of settings files (devel, production, staging, fasttesting) and have a root-level shell scripts for running them, like:

./devel-admin [command]
./production-admin [command]
./staging-admin [command]
./testing-admin [command]

Posted at 3:47 a.m. on January 11, 2011

7 Harro says...

We kinda use the same scheme. Except that the init.py does a: try: from settings.local_settings import * except ImportError: from settings.default_settings import *

Then we have a bunch of {dev,staging,live}_settings.py files which all import from the default_settings.

So for local development you create a local_settings which specifies which database you want to use and some other stuff. On the live server we don't really use the mangement commands directly, everything goes through a fabric file which handles using the right settings file. And with the new fabric 1.0 interactive commands that works just great.

Posted at 4:45 a.m. on January 11, 2011

8 Issac Kelly says...

The one convention that I've seen the most is having a local_settings.py that does not exist in version control.

I think that coupled with some sort of build script to deploy the site into production/create the production file works pretty well.

Posted at 9:53 a.m. on January 11, 2011

9 codekoala says...

Neat. Thanks for sharing!

Posted at 10:44 a.m. on January 11, 2011

10 Ben Keating says...

Maybe Im doin it wrong, but I have only one settings.py file. I use the same DB locally as I do in production/staging, etc. I consider the choice of DB part of the project, not the environment.

There is a catch tho --I'm a one-man-developer. So I don't have to worry about REAL data being copied off everywhere.

Whats more is I use Fabric and in my fabfile.py I wrote a "getdb" command that connects to a production and grabs a copy, so Im constantly developing off the most current db, locally.

Has been working for me for a few years now. I don't worry about DEBUG being on as I import socket and check the hostname, if it's not that of the production, DEBUG = True. I never have to worry unless hostnames change, but thats very rare.

Posted at 2:12 p.m. on January 13, 2011

11 Eric says...

We do something very simple for settings.

In each project, there is a default_settings.py that holds defaults -- things like INSTALLED_APPS, any custom globals, etc.

To use the project, a developer must create a settings.py (which is ignored via our source control ignore configuration) that does from default_settings import * and customize the 3 or 4 settings needed for either development or a particular production setting.

This seems very simple to our dev team, requires that they read documentation instead of haphazardly guessing for a project's setup requirements, and makes it dead simple to trace where a particular configuration value is coming from. It may be less elegant than some of the solutions in the post/comments, but it Just Works(tm).

Posted at 4:28 p.m. on January 16, 2011

12 Vincent Hillenbrink says...

Our team believes that "settings management" is everything when developing Django apps (well, maybe not everything, but it makes us smile almost once every day). We do everything based on DJANGO_SETTINGS_MODULE, always. We completely ditched manage.py and settings.py as well.

Instead, we use a straightforward convention for managing django settings using generalization via imports, much like Eric Holscher's example.

See https://bitbucket.org/goeiejongens/django-environments/wi... and check out the example settings for development & production in the env directory. (There also some bash script code for increased command line productivity, all based on the settings convention. This way, you can start streamlining Django settings, virtualenv and shell scripts - check out commands like get_django_setting, djbrowse and pipup.)

Oh, and from experience we think the "from local_settings import *" is absolutely evil! When you catch ImportError things can be hard to figure out, but in the postgres.py of readthedocs there's even a bare "except all"... ouch! :^)

Posted at 7:41 a.m. on January 21, 2011

Surfing in Kansas