Using extract_messages in your Pylons/Python egg with a ./src directory layout

I've been spending a lot of time trying to figure out something quite simple, so I'll write a blog post about it to hopefully save someone else some time with this.

I'm in the process of creating a Pylons egg. For illustration purposes, I'll call it MyEgg.

The egg is created as follows:

paster create --template=pylons MyEgg

The default egg directory looks like this:

MyEgg
 docs
 myegg
   config
   controllers
   lib
   model
   public
   templates
   tests
 MyEgg.egg-info

I prefer however to put the package contents myegg inside a ./src directory. That way, I can more easily use zc.buildout to manage MyEgg and its dependencies. I can then use buildout to manage and develop this egg's dependencies by also putting them in ./src.

After almost 3 years of using buildout on a daily basis, this layout just fits my brain better.

The new folder structure would now look like this:

MyEgg
 docs
 src
   myegg
     config
     controllers
     lib
     model
     public
     templates
     tests
 MyEgg.egg-info

Updating your setup.py script:

To make Distutils aware of the new directory layout, you also need to add the following two lines to the setup method call in your setup.py script:

packages=find_packages('src', exclude=['ez_setup']),
package_dir={'': 'src'},

Extracting translatable strings from your python files:

To generate a *.pot file that contains all the translateable strings in your code, you run the following command:

python setup.py extract_messages

To have the new directory structure (with ./src) and still be able to use extract_messages to internationalize your egg takes a bit more tweaking.

Firstly, in the setup.cfg file, you need to append src to the paths in the different sections. For example, in the extract_messages section, it would look like this:

[extract_messages]
add_comments = TRANSLATORS:
output_file = src/myegg/i18n/myegg.pot
width = 80

NOTE: there are other commands (such as compile_catalog) that also need to be run later as part of enabling I18N support for your package. For the purpose of this blog post, I'm concentrating on extract_messages, but the above change in the setup.cfg file applies to the other sections as well. For more info on I18N in Pylons, refer to the Pylons Book.

And now for the problem I was struggling with ;)

The problem I encountered with this layout was that the extract_messages command (courtesy of Babel) that is used to extract translatable strings for I18N support, now didn't find my python files anymore.

python setup.py extract_messages
running extract_messages
writing PO template src/myegg/i18n/myegg.pot

If you look at a Pylon project’s setup.py files (if you created it with the paster's pylons template), you will see the following lines commented out:

#message_extractors = {'myegg': [
#        ('**.py', 'python', None),
#        ('templates/**.mako', 'mako', None),
#        ('public/**', 'ignore', None)]},

The solution to this problem was to uncomment the message_extractors parameter in setup.py, and to tell it to look in the src directory:

message_extractors = {'src': [
 ('**.py', 'python', None),
 ('templates/**.mako', 'mako', None),
 ('public/**', 'ignore', None)]},

Now the extract_messages command works and creates a myegg.pot file that contains all the translatable strings in my app.

Here's the output (shortened for brevity):

$ python setup.py extract_messages 
running extract_messages
extracting messages from src/__init__.py
extracting messages from src/myegg/__init__.py
....
extracting messages from src/myegg/tests/functional/__init__.py
writing PO template file to src/myegg/i18n/myegg.pot

Hope this helps another Pylons, Distutils or I18N newbie to save some time. :D

Further reading:

For more information on using zc.buildout with Pylons, read this community HowTo and also this informative blogpost by Marius Gedminas.