Navigation

Translating text strings

Description

Translating Python and TAL template source code text strings using the gettext framework and other Plone/Zope i18n facilities.

Introduction

Internationalization is a process to make your code locale- and language-aware. Usually this means supplying translation files for text strings used in the code.

Plone internally uses the UNIX standard term:gettext tool to perform i18n.

There are two separate gettext systems. Both use the .po file format to describe translations.

Note that this chapter concerns only code-level translations. Content translations are managed by the Products.LinguaPlone add-on product.

zope.i18n

  • Follows gettext best practices
  • Translations are stored in locales folder of your application. Example: locales/fi/LC_MESSAGES/your.app.po
  • Has zope.i18nmessageid package which is string-like class allowing storing translation domain with translatable text strings easily.
  • .po files must be manually converted to .mo binary files every time the translations are updated. See i18ndude.

Plone (at least 3.3) uses only filename and path to search the translation files. Information in the .po file headers is ignored.

Generating a .pot template file for your package(s)

infrae.i18nextract can be used in your buildout to create a script which searches particular packages for translation strings. This can be particularly useful for creating a single translations package which contains the translations for the set of packages which make up your application.

Add the following to your buildout.cfg:

[translation]
recipe = infrae.i18nextract
packages =
   myapplication.policy
   myapplication .theme
output = ${buildout:directory}/src/myapplication.translation/myapplication/translation/locales
output-package = myapplication.translations
domain = mypackage

Running the ./bin/translation-extract script will produce a .pot file in the specified output directory which can then be used to create the .po files for each translation:

msginit --locale=fr --input=locales/mypackage.pot --output=locales/fr/LC_MESSAGES/mypackage.po

The locales directory should contain a directory for each language, and a directory called LC_MESSAGES within each of these, followed by the corresponding .po files containing the translation strings:

./locales/en/LC_MESSAGES/mypackage.po
./locales/fi/LC_MESSAGES/mypackage.po
./locales/ga/LC_MESSAGES/mypackage.po

Marking translatable strings

Each module declares its own MessageFactory which is a callable and marks strings with translation domain. MessageFactory is declared in the main __init__.py file of your package.

from zope.i18nmessageid import MessageFactory

# your.app.package must match domain declaration in .po files
MessageFactory = MessageFactory('youpackage.name')

You also need to have the following ZCML entry:

<configure xmlns:i18n="http://namespaces.zope.org/i18n">
    <i18n:registerTranslations directory="locales" />
</configure>

After the setup above you can use message factory to mark strings with translation domains. i18ndude translation utilities use underscore _ to mark translatable strings (gettext message ids). Message ids must be unicode strings.

from your.app.package import yourAppMessageFactory as _
my_translatable_text = _(u"My text")

The object will still look like a string:

>>> my_translatable_text
u'My text'

But in reality it is a zope.i18nmessageid.message.Message object:

>>> my_translatable_text.__class__
<type 'zope.i18nmessageid.message.Message'>

>>> my_translatable_text.domain
'your.app.package'

To see the translation:

>>> from zope.i18n import translate
>>> translate(my_translatable_text)
u"The text of the translation." # This is the corresponding msgstr from the .po file

For more information see:

Automatically translated message ids

Plone will automatically perform translation for message ids which are output in page templates.

The following code would translate my_translateable_text to the native language activated for the current page.

<span tal:content="view/my_translateable_text">

Note that since my_translateable_text is a zope.i18nmessageid.message.Message instance containing its own gettext domain information, the i18n:domain attribute in page templates does not affect message ids declared through message factories.

Manually translated message ids

If you need to manipulate translated text outside page templates, you need to perform the final translation manually.

Translation always needs context (i.e. under which site the translation happens), as the active language and other preferences are read from the HTTP request object and site object settings.

Translation can be performed using the context.translate() method:

# Translate some text
msgid = _(u"My text") # my_text is zope.

# Use inherited translate() function to get the final text string
translated = self.context.translate(msgid)

# translated is now u"Käännetty teksti" (in Finnish)

context.translate() uses the translate.py Python script from LanguageTool.

It has the signature:

def translate(self, domain, msgid, mapping=None, context=None,
      target_language=None, default=None):

and does the trick:

from Products.CMFCore.utils import getToolByName

# get tool
tool = getToolByName(context, 'translation_service')

# this returns type unicode
value = tool.translate(msgid,
                        domain,
                        mapping,
                        context=context,
                        target_language=target_language,
                        default=default)

Note

Translation needs HTTP request object and thus may not work correctly from command-line scripts.

Non-python message ids

There are also other message id markers in code outside the Python domain, that have their own mechanisms:

  • ZCML entries
  • GenericSetup XML
  • TAL page templates

Testing translations

Here is a simple way to check if your gettext domains are correctly loaded.

Plone 4

You can start the Plone debug shell and manually check if translations can be performed.

First start Plone in debug shell:

bin/instance debug

and then call translation service, in your site, manually:

>>> site = app.yoursiteid
>>> translation_service = site.translation_service
>>> translation_service.translate("Add Events Portlet", domain="plone", target_language="fi")
u'Lis\xe4\xe4 Tapahtumasovelma'

Plone 3

You can find PlacelessTranslationService in the ZMI root/control panel (not site root).

Translation string substitution

Translation string substitutions must be used when the final translated message contains variable strings.

Plone content classes inherit the translate() function which can be used to get the final translated string. It will use the currently activate language. Translation domain will be taken from the msgid object itself, which is a string-like zope.i18nmessageid instance.

Message ids are immutable (read-only) objects so you need to always create a new message id if you use different variable substitution mappings.

Python code:

from saariselka.app import appMessageFactory as _

class SomeView(BrowserView):

    def do_stuff(self):

        msgid = _(u"search_results_found_msg", default=u"Found ${results} results", mapping={ u"results" : len(self.contents)})

        # Use inherited translate() function to get the final text string
        translated = self.context.translate(msgid)

        # Show the final result count to the user as a portal status message
        messages = IStatusMessage(self.request)
        messages.addStatusMessage(translated, type="info")

Corresponding .po file entry:

#. Default: "Found ${results} results"
#: ./browser/accommondationsummaryview.py:429
msgid "search_results_found_msg"
msgstr "Löytyi ${results} majoituskohdetta"

For more information, see

PlacessTranslationService

  • Historic, being phased out
  • Stores .po files in i18n folder of your add-on product
  • Used for main "plone" translation catalog (until Plone 3.3.x)
  • Translation files are processed when Plone is restarted. Example: i18n/yourapp-fi.po.

i18ndude

i18ndude is a developer-oriented command-line utility to manage .po and .mo files.

Usually you build our own shell script wrapper around i18ndude to automate generation of .mo files of your product .po files.

Note

Plone 3.3 and onwards do not need manual .po -> .mo compilation. It is done on start up. Plone 4 has a special switch for this: in your buildout.cfg in the part using plone.recipe.zope2instance you can set an environment variable for this:

environment-vars =
    zope_i18n_compile_mo_files true

Note that the value does not matter: the code in zope.i18n simply looks for the existence of the variable and does not care what its value is.

See:

Examples:

Installing i18ndude

The recommended method is to have i18ndude installed via your buildout.

Add the following to your buildout.cfg:

parts =
    ...
    i18ndude

[i18ndude]
unzip = true
recipe = zc.recipe.egg
eggs = i18ndude

After this i18ndude is available in your buildout/bin folder

bin/i18ndude -h
Usage: i18ndude command [options] [path | file1 file2 ...]]

You can also call it relative to your current package source folder

huiske-imac:twinapex moo$  cd src/mfabrik.plonezohointegration/
huiske-imac:mfabrik.plonezohointegration moo$ ../../bin/i18ndude

Warning

Do not easy_install i18ndude. i18ndude depends on various Zope packages and pulling them to your system-wide Python configuration could be dangerous, due to potential conflicts with corresponding, but different versions, of the same packages used with Plone.

More information

Setting up folder structure for Finnish and English

Example:

mkdir locales
mkdir locales/fi
mkdir locales/en
mkdir locales/fi/LC_MESSAGES
mkdir locales/en/LC_MESSAGES

Creating .pot base file

Example:

i18ndude rebuild-pot --pot locales/mydomain.pot --create your.app.package .

Manual .po entries

i18ndude scans source .py and .pt files for translatable text strings. On some occassions this is not enough - for example if you dynamically generate message ids in your code. Entries which cannot be detected by automatic code scan are called manual po entries. They are managed in locales/manual.pot which is merged to generated locales/yournamespace.app.pot file.

Here is a sample manual.pot file:

msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=1; plural=0\n"
"Preferred-Encodings: utf-8 latin1\n"
"Domain: mfabrik.app\n"

# This entry is used in gomobiletheme.mfabrik  templates for the campaign page header
# It is not automatically picked, since it is referred from external package
#. Default: "Watch video"
msgid "watch_video"
msgstr ""

Managing .po files

Example shell script to manage i18n files. Change CATALOGNAME to reflect the actual package of your product:

The script will:

  • pick up all changes to i18n strings in code and reflect them back to the translation catalog of each language;
  • pick up changes in manual.pot file and reflect them back to the translation catalog of each language.
#!/bin/sh
#
# Shell script to manage .po files.
#
# Run this file in the folder main __init__.py of product
#
# E.g. if your product is yourproduct.name
# you run this file in yourproduct.name/yourproduct/name
#
#
# Copyright 2010 mFabrik http://mfabrik.com
#
# http://plone.org/documentation/manual/plone-community-developer-documentation/i18n/localization
#

# Assume the product name is the current folder name
CURRENT_PATH=`pwd`
CATALOGNAME="yourproduct.app"

# List of languages
LANGUAGES="en fi de"

# Create locales folder structure for languages
install -d locales
for lang in $LANGUAGES; do
    install -d locales/$lang/LC_MESSAGES
done

# Assume i18ndude is installed with buildout
# and this script is run under src/ folder with two nested namespaces in the package name (like mfabrik.plonezohointegration)
I18NDUDE=../../../../bin/i18ndude

if test ! -e $I18NDUDE; then
        echo "You must install i18ndude with buildout"
        echo "See http://svn.plone.org/svn/collective/collective.developermanual/trunk/source/i18n/localization.txt"
        exit
fi

#
# Do we need to merge manual PO entries from a file called manual.pot.
# this option is later passed to i18ndude
#
if test -e locales/manual.pot; then
        echo "Manual PO entries detected"
        MERGE="--merge locales/manual.pot"
else
        echo "No manual PO entries detected"
        MERGE=""
fi

# Rebuild .pot
$I18NDUDE rebuild-pot --pot locales/$CATALOGNAME.pot $MERGE --create $CATALOGNAME .


# Compile po files
for lang in $(find locales -mindepth 1 -maxdepth 1 -type d); do

    if test -d $lang/LC_MESSAGES; then

        PO=$lang/LC_MESSAGES/${CATALOGNAME}.po

        # Create po file if not exists
        touch $PO

        # Sync po file
        echo "Syncing $PO"
        $I18NDUDE sync --pot locales/$CATALOGNAME.pot $PO


        # Plone 3.3 and onwards do not need manual .po -> .mo compilation,
        # but it will happen on start up if you have
        # registered the locales directory in ZCML
        # For more info see http://vincentfretin.ecreall.com/articles/my-translation-doesnt-show-up-in-plone-4

        # Compile .po to .mo
        # MO=$lang/LC_MESSAGES/${CATALOGNAME}.mo
        # echo "Compiling $MO"
        # msgfmt -o $MO $lang/LC_MESSAGES/${CATALOGNAME}.po
    fi
done

Note

Remember to register the locales directory in configure.zcml for automatic .mo compilation as instructed above.

More information

Dynamic content

If your HTML template contains dynamic content such as

<h1 i18n:translate="search_form_heading">Search from <span tal:content="context/@@plone_portal_state/portal_title" /></h1>

it will produce .po entry:

msgstr "Hae sivustolta <span>${DYNAMIC_CONTENT}</span>"

You need to give the name to the dynamic part

<h1 i18n:translate="search_form_heading">Search from <span i18n:name="site_title" tal:content="context/@@plone_portal_state/portal_title" /></h1>

... and then you can refer the dynamic part by a name:

#. Default: "Search from <span>${DYNAMIC_CONTENT}</span>"
#: ./skins/gomobiletheme_basic/search.pt:46
#: ./skins/gomobiletheme_plone3/search.pt:46
msgid "search_form_heading"
msgstr "Hae sivustolta ${site_title}

More info

Table Of Contents

About Plone

This is documentation for Plone®. Plone is a popular, open source, content management system written in Python programming language.

Navigation

© Copyright Plone community and other individual contributors. Created using Sphinx 1.1.2.



TEST Brought to you by Read the Docs
Test

Edit this document

The source code of this file is hosted on GitHub. Everyone can update and fix errors in this document with few clicks - no downloads needed.

  1. Go to Translating text strings on GitHub.
  2. Press Fork and edit this file button.
  3. Edit file contents using GitHub's text editor in your web browserm
  4. Fill in the Commit message text box at the end of the page telling why you did the changes. Press Propose file change button next to it when done.
  5. On Send a pull request page you don't need to fill in text anymore. Just press Send pull request button.
  6. Your changes are now queued for review under project's Pull requests tab on Github.

For basic information about updating this manual and Sphinx format please see Writing and updating the manual guide.