src directory layout

March 24, 2010 python buildout pylons distutils

If you want to have your egg’s package contents inside a ‘src’ folder, you’ll need to tweak the configuration files a bit. Here’s what I’ve learned…

I’ve been spending a lot of time trying to figure out something quite simple, so I’ll write a blog post about it to hopefully save someone else some time with this.

I’m in the process of creating a Pylons egg. For illustration purposes, I’ll call it MyEgg.

The egg is created as follows:

paster create --template=pylons MyEgg

The default egg directory looks like this:

MyEgg 
  docs 
  myegg
    config
    controllers
    lib
    model
    public
    templates
    tests
MyEgg.egg-info

I prefer however to put the package contents myegg inside a ./src directory. That way, I can more easily use zc.buildout to manage MyEgg and its dependencies. I can then use buildout to manage and develop this egg’s dependencies by also putting them in ./src.

After almost 3 years of using buildout on a daily basis, this layout just fits my brain better.

The new folder structure would now look like this:

MyEgg 
  docs 
  src   
    myegg     
      config     
      controllers     
      lib     
      model     
      public     
      templates     
      tests 
MyEgg.egg-info

Updating your setup.py script:

To make Distutils aware of the new directory layout, you also need to add the following two lines to the setup method call in your setup.py script:

packages=find_packages('src', exclude=['ez_setup']),
package_dir={'': 'src'},

Extracting translatable strings from your python files:

To generate a *.pot file that contains all the translateable strings in your code, you run the following command:

python setup.py extract_messages

To have the new directory structure (with ./src) and still be able to use extract_messages to internationalize your egg takes a bit more tweaking.

Firstly, in the setup.cfg file, you need to append src to the paths in the different sections. For example, in theextract_messages section, it would look like this:

[extract_messages]
add_comments = TRANSLATORS:
output_file = src/myegg/i18n/myegg.pot
width = 80

NOTE: there are other commands (such as compile_catalog) that also need to be run later as part of enabling I18N support for your package. For the purpose of this blog post, I’m concentrating on extract_messages, but the above change in the setup.cfg file applies to the other sections as well. For more info on I18N in Pylons, refer to the Pylons Book.

And now for the problem I was struggling with ;)

The problem I encountered with this layout was that the extract_messages command (courtesy of Babel) that is used to extract translatable strings for I18N support, now didn’t find my python files anymore.

python setup.py extract_messagesrunning extract_messageswriting PO
template src/myegg/i18n/myegg.pot

If you look at a Pylon project’s setup.py files (if you created it with the paster’s pylons template), you will see the following lines commented out:

##message_extractors = {'myegg': [
## ('**.py', 'python', None),
## ('templates/**.mako', 'mako', None),
## ('public/**', 'ignore', None)]},

The solution to this problem was to uncomment the message_extractors parameter in setup.py, and to tell it to look in the src directory:

message_extractors = {'src': [
   ('**.py', 'python', None), ('templates/**.mako', 'mako',
    None), ('public/**', 'ignore', None)
]},

Now the extract_messages command works and creates a myegg.pot file that contains all the translatable strings in my app.

Here’s the output (shortened for brevity):

$ python setup.py extract_messages running extract_messages
extracting messages from src/__init__.py extracting messages from
src/myegg/__init__.py .... extracting messages from
src/myegg/tests/functional/__init__.py writing PO template file to
src/myegg/i18n/myegg.pot

Hope this helps another Pylons, Distutils or I18N newbie to save some time.

Using extract_messages in your Pylons/Python egg with a ./src directory layout

Updating your setup.py script:

Extracting translatable strings from your python files:

And now for the problem I was struggling with ;)

Further reading: