Difference between revisions of "Development/Tutorials/Localization/i18n Build Systems"

Jump to: navigation, search
(Mention https://api.kde.org/frameworks/ki18n/html/prg_guide.html since it's awesome)
 
(71 intermediate revisions by 24 users not shown)
Line 1: Line 1:
 +
 
{{TutorialBrowser|
 
{{TutorialBrowser|
  
Line 13: Line 14:
 
Now that your application is ready to be localized, we next look at how to incorporate the necessary mechanisms into the CMake build system of your application.
 
Now that your application is ready to be localized, we next look at how to incorporate the necessary mechanisms into the CMake build system of your application.
  
First we'll explain the "theory" of the steps that need to happen from extracting message strings to installing the generated .po files. After that we'll look at how to implement those steps ([[# Theory: The xgettext toolchain ]]). If your application is developed in KDE's subversion repository, many of those steps will happen (almost) automatically (see [[# handling i18n in KDE's subversion repository ]]). Else you will want to read on in the [[# handling i18n in third party applications ]] section.
+
First we'll explain the "theory" of the steps that need to happen from extracting message strings to installing the generated .po files. After that we'll look at how to implement those steps ([[#Theory: The xgettext toolchain]]). If your application is developed in KDE's repositories, many of those steps will happen automatically (see [[#Handling i18n in KDE's repositories]]). Else you will want to read on in the [[#Handling i18n outside KDE's repositories]] section.
  
 
== Theory: The xgettext toolchain ==
 
== Theory: The xgettext toolchain ==
  
 
Making translations work consists of the following steps:
 
Making translations work consists of the following steps:
  - Extract the translatable strings
+
# Extract the translatable strings
  - Merge the new or changed strings with existing translations
+
# Merge the new or changed strings with existing translations
  - Compile the translations into message catalogs
+
# Compile the translations into message catalogs
  - Install the message catalogs
+
# Install the message catalogs
  - Use the message catalogs in the application
+
# Use the message catalogs in the application
  
 
=== Extracting the strings ===
 
=== Extracting the strings ===
Line 28: Line 29:
 
In this step, all the strings you marked as i18n()/ki18n()/etc. in your sources need to be collected into a translation template (.pot) file. Some translateable strings are also contained in .ui, .rc, or .kcfg files. Also tips-of-the-day need to be collected into the .pot file.
 
In this step, all the strings you marked as i18n()/ki18n()/etc. in your sources need to be collected into a translation template (.pot) file. Some translateable strings are also contained in .ui, .rc, or .kcfg files. Also tips-of-the-day need to be collected into the .pot file.
  
These steps are handled by xgettext, extractrc, and preparetips programs, respectively. In some cases you may also need extractattr.
+
These steps are handled by {{path|xgettext}}, {{path|extractrc}}, and {{path|preparetips}} programs, respectively. In some cases you may also need {{path|extractattr}}.
  
 
=== Merging translations ===
 
=== Merging translations ===
  
Generally, only a few translatable string will change at a time, in an application. Some will be removed, some will be added, some will be changed into different strings, and often many will be moved around in the source files. These changes need to be reflected in the translations, but of course it would be a huge effort to redo the entire translation every time a string was changed. Rather the changes should be merged with the existing translations. This is the job of the msgmerge tool.
+
Generally, only a few translatable string will change at a time, in an application. Some will be removed, some will be added, some will be changed into different strings, and some will be moved around in the source files. These changes need to be reflected in the translations, but of course it would be a huge effort to redo the entire translation every time a string was changed. Rather the changes should be merged with the existing translations. This is the job of the {{path|msgmerge}} tool.
  
 
=== Compiling the translations ===
 
=== Compiling the translations ===
  
In order to make message lookup fast, the .po files need to be compiled into so-called "message catalogs" (.mo / .gmo). This is done using the msgfmt tool.
+
In order to make message lookup fast, the .po files need to be compiled into so-called "message catalogs" (.mo / .gmo). This is done using the {{path|msgfmt}} tool.
  
 
=== Installing the message catalogs ===
 
=== Installing the message catalogs ===
  
The compiled message catalogs need to be installed alongside the application. In KDE, the standard location for message catalogs is {{path|$KDEDIR/share/locale/xx/LC_MESSAGES/}}.
+
The compiled message catalogs need to be installed alongside the application. In KDE, the standard location for message catalogs is {{path|$DATAROOTDIR/locale/xx/LC_MESSAGES/}}.
  
 
=== Using the message catalogs ===
 
=== Using the message catalogs ===
Line 46: Line 47:
 
Finally, when the application is run, it needs to load the appropriate message catalog in order to be able to look up and show translated strings. In KDE applications, this is the job of the {{class|KLocale}} class, and in the great majority of cases happens automatically.
 
Finally, when the application is run, it needs to load the appropriate message catalog in order to be able to look up and show translated strings. In KDE applications, this is the job of the {{class|KLocale}} class, and in the great majority of cases happens automatically.
  
For some special cases such as plugins, look at [[# Runtime Loading Of Catalogs ]].
+
For some special cases, such as plugins, look at [[#Runtime Loading Of Catalogs]].
  
== handling i18n in KDE's subversion repository ==
+
== Handling i18n in KDE's repositories ==
  
If your application is developed inside KDE's subversion repository, most of the steps outlined above are mostly automated. In this case, generally, all you will need to do is provide a simple script called {{path|Messages.sh}}, which we will look a below.
+
If your application is developed inside KDE's repositories, most of the steps outlined above are automated. In this case, generally, all you will need to do is provide a simple script called {{path|Messages.sh}}, which we will look at below.
  
Of course, for the curious, a more detailed account of what happens behind the scences is also provided.
+
Of course, for the curious, a more detailed account of what happens behind the scenes is also provided.
  
 
=== Writing a Messages.sh script ===
 
=== Writing a Messages.sh script ===
  
Basically, the only thing that is necesarry to prepare and install translations for applications in KDE's subversion repository, is to provide information, which sources, ui-files or tips need to be translated. For this purpose, you write a small script called {{path|Messages.sh}} and place it in your sources. Here is an example with inline comments:
+
Basically, the only thing that is necessary to prepare and install translations for applications in KDE's repositories, is to provide information, which sources, ui-files or tips need to be translated. For this purpose, you write a small script called {{path|Messages.sh}} and place it in your sources. Here is an example with inline comments:
  
<code cppqt n>
+
<syntaxhighlight lang="bash">
#!bin/sh
+
#!/bin/sh
  
# invoke the extractrc script on all .ui, .rc, and .kcgf files in the sources
+
# invoke the extractrc script on all .ui, .rc, and .kcfg files in the sources
# the results are stored in a pseudo .cpp file.
+
# the results are stored in a pseudo .cpp file to be picked up by xgettext.
$EXTRACTRC `find . -name \*.rc -o -name \*.ui -o -name \*.kcfg` > rc.cpp || exit 11
+
$EXTRACTRC `find . -name \*.rc -o -name \*.ui -o -name \*.kcfg` >> rc.cpp
# if your application contains tips-of-the-day, call preparetips.
+
# invoke the grantlee extract script for translatable string from Grantlee themes
 +
$EXTRACT_GRANTLEE_TEMPLATE_STRINGS `find . -name \*.html` >> html.cpp
 +
# if your application contains tips-of-the-day, call preparetips as well.
 
$PREPARETIPS > tips.cpp
 
$PREPARETIPS > tips.cpp
# call xgettext on all source files. If your sources have other filename extensions
+
# call xgettext on all source files. If your sources have other filename
# besides .cc, .cpp, and .h, just add them in the find call.
+
# extensions besides .cc, .cpp, and .h, just add them in the find call.
# replace APPNAME with the name of your application (see [[#Naming .pot Files]] for exceptions)
+
$XGETTEXT `find . -name \*.cc -o -name \*.cpp -o -name \*.h -name \*.qml` -o $podir/APPNAME.pot
$XGETTEXT `find . -name \*.cc -o -name \*.cpp -o -name \*.h` -o $podir/APPNAME.pot
+
</syntaxhighlight>
</code>
+
  
As you can see, this script contains only actual three lines of code, and not all may even be needed. The $XGETTEXT, $PREPARETIPS, $EXTRACTRC, and $podir environment variables are predefined, you do not need to worry about setting these.
+
As you can see, this script contains only four actual lines of code, and not all may even be needed. The $XGETTEXT, $PREPARETIPS, $EXTRACTRC, $EXTRACT_GRANTLEE_TEMPLATE_STRINGS and $podir environment variables are predefined, you do not need to worry about setting these. The only thing that you will need to do is to replace "APPNAME" with the name of your application (but see [[#Naming .pot Files]] for exceptions).
  
=== What's happening behind the scenes ===
+
* $XGETTEXT - Extract i18n translatable strings from C++ files
 +
* $EXTRACT_TR_STRINGS - Extract Qt tr translatable strings from C++ files for Qt5-based projects
 +
* $EXTRACTRC - Extract translatable strings from xml configuration and ui files.
 +
* $EXTRACT_GRANTLEE_TEMPLATE_STRINGS - Extract translatable strings from Grantlee template files.
  
 +
Try to make sure your Messages.sh script collects only those messages that are really needed. If in doubt, look at other Messages.sh files in the KDE repositories for inspiration, or -- of course -- [https://mail.kde.org/mailman/listinfo/kde-i18n-doc ask].
  
== handling i18n in third party applications ==
+
=== Testing your Messages.sh script ===
  
=== General considerations ===
+
To test your Messages.sh script, you need a checkout of the l10n scripts. To get one:
  
=== Extracting and merging messages ===
+
    svn checkout svn://anonsvn.kde.org/home/kde/trunk/l10n-kf5/scripts
  
=== Compiling and installing message catalogs ===
+
You can then go to your project dir and run:
  
 +
    mkdir po
 +
    mkdir enpo # only needed if $EXTRACT_TR_STRINGS is used in any Messages.sh file
 +
    PATH=/path/to/scripts:$PATH bash /path/to/scripts/extract-messages.sh
  
== CMake and i18n ==
+
When it is done, the po/ dir should contain the generated .pot files.
  
Creating message catalogs has been dramatically simplified for KDE4. In the {{path|l10n-kde4/scripts}} module in KDE's subversion repository you will find a file called {{path|extract-messages.sh}}. When this script is run it will create all the necessary files to begin the translation process.
+
=== Translating .desktop Files ===
 +
You don't need to do anything to get the common fields (See [[#What's happening behind the scenes]] for a list) of .desktop or .desktop.cmake files translated.
  
{{improve|Add in a CMakeLists.txt recipe.}}
+
If you have a file with the same syntax than a .desktop file but it has a different extension, you have to create an {{path|ExtraDesktop.sh}} file that outputs the path of those files
 +
<syntaxhighlight lang="bash">
 +
#! /bin/sh
 +
find -name *desktop.in -print
 +
</syntaxhighlight>
  
== The Translation Process ==
+
=== Translating Mimetype descriptions ===
 +
Add a file named <code>XmlMessages.sh</code> to the folder with your mimetype XML  foo.xml with this content:
 +
<syntaxhighlight lang="bash">
 +
function get_files
 +
{
 +
    echo foo.xml
 +
}
 +
function po_for_file
 +
{
 +
    case "$1" in
 +
      foo.xml)
 +
          echo foo_xml_mimetypes.po
 +
      ;;
 +
    esac
 +
}
 +
function tags_for_file
 +
{
 +
    case "$1" in
 +
      foo.xml)
 +
          echo comment
 +
      ;;
 +
    esac
 +
}
 +
</syntaxhighlight>
  
Assuming the build system is set up properly, the following outlines the process that occurs to translate your application.
+
=== What's happening behind the scenes ===
 
+
{{tip|If your application is in the KDE code repository, all this happens automatically. If your code is not in the KDE repository, you must perform these steps yourself. See the section below on distributing message catalogs.}}
+
  
 +
{{tip|If your application is in the KDE code repository, all this happens automatically. If your code is not in the KDE repository, you must perform these steps yourself. See [[#Handling i18n outside KDE's repositories]].}}
  
Periodically, the script {{path|extract-messages.sh}} (a.k.a. scripty) runs on the KDE server. This program extracts all the i18n messages from your application per the messages targets in your Makefile.am's. The extracted messages are stored in template (.pot) files in the templates folder of the l10n module. See What Is Scripty for more information.  
+
Periodically, the script {{path|extract-messages.sh}} (a.k.a. scripty) runs on the KDE server. This program basically calls all Messages.sh scripts in the repository with the appropriate parameters. The extracted messages are stored in template (.pot) files in the templates folder of the l10n module. See [[What Is Scripty]] for more information.  
  
The KDE translation teams translate the messages and commit them into a messages folder corresponding to the language, which is further broken down by module. For example, German translated messages for konqueror are committed to {{path|l10n/de/messages/kdebase/konqueror.po}}.  
+
The KDE translation teams translate the messages and commit them into a messages folder corresponding to the language, which is further broken down by module. For example, German translated messages for konqueror are committed to {{path|l10n/de/messages/applications/konqueror.po}}.  
  
When the l10n module is built, the .po files are compiled into a binary format for fast lookup and installed as .mo files to {{path|$KDEDIR/share/locale/xx/LC_MESSAGES/}}, where xx is the two-letter ISO 639 code for the language. These are called the message catalogs.  
+
When the l10n module is built, the .po files are compiled into a binary format for fast lookup and installed as .mo files to {{path|$DATAROOTDIR/locale/xx/LC_MESSAGES/}}, where xx is the two-letter ISO 639 code for the language. These are called the message catalogs.  
  
 
At runtime, the <tt>i18n(...)</tt> function, using the original string you coded, looks up the string in the message catalog of the user's desktop language and returns the translated string. If the message catalog is missing, or the specific string is not found, <tt>i18n(...)</tt> falls back to the original string in your code.  
 
At runtime, the <tt>i18n(...)</tt> function, using the original string you coded, looks up the string in the message catalog of the user's desktop language and returns the translated string. If the message catalog is missing, or the specific string is not found, <tt>i18n(...)</tt> falls back to the original string in your code.  
Line 109: Line 145:
 
.desktop files in your project are handled separately. {{path|makemessages}} extracts strings, such as Name and Comment from the .desktop files and places them into a file named {{path|desktop_mmmm.pot}}, where mmmm is the module name, in the templates folder. Once translators have translated this file, makemessages inserts the translated strings back into the .desktop files. The list of strings extracted is in {{path|l10n/scripts/apply.cc}}. Here's the code that checks for them:
 
.desktop files in your project are handled separately. {{path|makemessages}} extracts strings, such as Name and Comment from the .desktop files and places them into a file named {{path|desktop_mmmm.pot}}, where mmmm is the module name, in the templates folder. Once translators have translated this file, makemessages inserts the translated strings back into the .desktop files. The list of strings extracted is in {{path|l10n/scripts/apply.cc}}. Here's the code that checks for them:
  
<code cppqt n>if (checkTag("Name", in, argc, argv, newFile))
+
<syntaxhighlight lang="cpp-qt">
 +
if (checkTag("Name", in, argc, argv, newFile))
 
     continue;
 
     continue;
 
if (checkTag("Comment", in, argc, argv, newFile))
 
if (checkTag("Comment", in, argc, argv, newFile))
Line 126: Line 163:
 
     continue;
 
     continue;
 
if (checkTag("ExtraNames", in, argc, argv, newFile))
 
if (checkTag("ExtraNames", in, argc, argv, newFile))
     continue;</code>
+
     continue;
 +
</syntaxhighlight>
  
== Runtime Loading Of Catalogs ==
+
== KI18n==
 +
KI18n is the framework most used in applications made by KDE (you can also use the Qt translation system though it's not recommented, see [[#Qt5-only: Code using Qt translation system]]
  
To have translations show up properly in an application, the name of the .pot file must match the name of the message catalog (.mo) file that your application will read at runtime. But what name will be used at runtime?
+
It has a very comprehensive documentation that you can read at [https://api.kde.org/frameworks/ki18n/html/prg_guide.html https://api.kde.org/frameworks/ki18n/html/prg_guide.html]
  
In general, it will be the value returned by <tt>{{class|KGlobal}}::instance()->instanceName()</tt>. But what will that be? The general rules for standalone applications are:
+
== Special cases (plugins, multiple catalogs, etc.) ==
* the message catalog will default to the name of the application passed as the first argument to {{class|KAboutData}}
+
* if your code calls <tt>{{class|KLocale}}::setMainCatalog()</tt>, that name will be used instead
+
* if your code calls <tt>{{class|KLocale}}::insertCatalog(const QString&)</tt>, the specified catalog will also be searched in addition to the main catalog
+
  
If your code does not call either of the {{class|KLocale}} methods mentioned above and your application is a plugin, it gets a little more complicated. If your code exports your plugin using the <tt>K_EXPORT_COMPONENT_FACTORY</tt> macro, like this:
+
==== Declarative plasmoids ====
 +
For plasmoids written in pure qml, the .pot file must be named the same as its plugin name (with '''plasma_applet_''' prefix), specified in its desktop file as '''X-KDE-PluginInfo-Name'''. For instance if we have '''X-KDE-PluginInfo-Name=org.kde.active.activityscreen''' the .pot file will be called '''plasma_applet_org.kde.active.activityscreen.pot'''.
  
<code cppqt>K_EXPORT_COMPONENT_FACTORY( libkhtmlkttsdplugin,
+
If the qml files are developed in the form of package instead of an autonomous plasmoid (for instance a package loaded by a C++ plasmoid) the .pot file will have the prefix '''plasma_package_''' instead.
  KGenericFactory<KHTMLPluginKTTSD>("khtmlkttsd") )</code>
+
  
then the catalog name will be the name passed in the {{class|KGenericFactory}} constructor - in the example above that woudl be <tt>khtmlkttsd</tt>. This is because the macro creates a {{class|Kinstance}} for you with the specified name.
+
==== Akonadi Agents ====
 +
Akonadi agents get the catalog loaded based on the name of the binary, so your .pot file should be named the same way your binary adding .pot at the end
 +
You can check that in AgentBase::parseArguments in kdepimlibs
  
However, some classes, such as <tt>KTextEditor</tt> create a <tt>KPart</tt> containing your component and the name passed in the macro is not used. In this case, you should call <tt>{{class|KLocale}}::insertCatalog(const QString&)</tt> in the component's constructor.
+
==== Qt5-only: Code using Qt translation system ====
 +
Some Qt5-based applications and libraries use the Qt translation system instead of KI18n. Examples of such code are:
  
If in doubt, the safest thing to do is to call <tt>{{class|KLocale}}::insertCatalog(const QString&)</tt>.  
+
* Tier 1 KF5 frameworks because they cannot depend on KI18n.
 +
* Applications which provide a Qt-only version.
  
{{tip|If you're not using the export macro for your plugin, you probably should be. See the API documentation of KGenericFactory for more information.}}
+
To make translators aware of this distinction, your .pot file should follow these naming schemes:
  
Calling <tt>{{class|KLocale}}::insertCatalog(const QString&)</tt> if the catalog has already been inserted does nothing. However, calling <tt>{{class|KLocale}}::removeCatalog(const QString&)</tt> removes the catalog no matter how many times it has been inserted. Therefore, take care that you do not inadvertently remove the catalog when multiple components are sharing a single catalog. For example, the following sequence will break translations:  
+
* '''*.po''': code using gettext with KDE translation extensions. .po files are compiled into .mo files (most common case).
 +
* '''*_qt.po''': code using Qt translation system. The .po files are compiled into .qm files (some KF5 frameworks, applications providing a Qt-only version).
 +
* '''*_qt-mo.po''': code using pure gettext. The .po files are compiled into .mo files (legacy).
  
<code cppqt n>// Component A loads and uses catalog xyz by calling
 
insertCatalogue("xyz");
 
// Component B loads and also uses the same catalog.
 
insertCatalogue("xyz");
 
// component B unloads and calls
 
removeCatalogue("xyz")
 
// Component A now doesn't translate properly.</code>
 
  
Notice that a sequence such as above will occur automatically if both components A and B are loaded using the <tt>K_EXPORT_COMPONENT_FACTORY</tt> macro and pass the same name argument to the KGenericFactory constructor.
+
If you are using the Qt translation system you have to make sure the translations are loaded properly in your application at runtime. For that you can use [https://api.kde.org/ecm/module/ECMPoQmTools.html ECMPoQmTools] by adding this to your CMakeLists.txt:
  
== Naming .pot Files ==
+
<pre>
 +
include(ECMPoQmTools)
  
The name that you settle on for both the .pot file and catalog should be governed by where your code resides. If it is a component of another application and resides in that application's source tree, you'll want to share the main catalog of the application.
+
ecm_create_qm_loader(mybinaryname_QM_LOADER mypotfilename_qt)
  
If is a component of another application but resides elsewhere in the code repository, you will probably have to create a separate .pot, but keep in mind that .pot filenames must be unique across all of KDE.
+
set(my_project_SRCS
 +
    ...
 +
    ${mybinaryname_QM_LOADER})
 +
</pre>
  
If it is a component of another application, but resides in the source tree of a second application, you should share with the second application. For example, the KDE text-to-speach daemon (KTTSD) includes a plugin for embedded Kate in {{path|kdeaccessibility/kttsd}} source tree. Since the plugin is not installed unless kttsd is installed, it shares its catalog with kttsd and calls <tt>{{class|KLocale}}::insertCatalog("kttsd")</tt> to do this.
+
== Handbooks ==
  
== Distributing Message Catalogs ==
+
Handbooks, which are written in docbook format, are handled different from applications. The translated Handbooks are committed into the l10n module in the docs folder under each language. In addition, the docs folder is broken down by module and application. For example, the German Kate Handbook is committed to the {{path|l10n-kf5/de/docs/applications/kate/}} folder. When compiled, the German Kate Handbook is installed to {{path|$DATAROOTDIR/doc/HTML/de/kate/index.cache.bz2}}.
  
{{note|The following improvements are needed in this section:
+
Note that it is up to each translation team to generate the translated Handbook when they feel it is complete.
How can messages for .desktop files be extracted?
+
How can translated .desktop messages be inserted back into .desktop files?
+
"make -f admin/Makefile.common package-messages" is obviously no longer correct}}
+
  
If your application is in the KDE code repository, you don't have to do anything to distribute message catalogs. Users will either download the l10n source and build it or install binaries prepared by packagers.
+
== Handling i18n outside KDE's repositories ==
  
In addition, if your top-level folder and .pot file have the same name, the svn2dist script will automatically include .po files when you use it to make a source tarball for your app.
+
There's a whole page dedicated to that in [[/Outside_KDE_repositories]]
 
+
If your application is not in the KDE code repository, you have to extract the messages yourself and provide the translated PO files within your distribution. To extract the messages you need to have xgettext installed and extractrc from kdesdk has to be in your path.
+
 
+
 
+
{{improve|The following paragraph needs to be updated to reflect the new work going on in branches/work/kde-l10n/scripts}}
+
 
+
Then do the following commands in the base folder of your KDE application's source:
+
 
+
<code>
+
mkdir po
+
make -f admin/Makefile.common package-messages
+
echo po>>SUBDIRS
+
echo "POFILES = AUTO">po/Makefile.am
+
</code>
+
 
+
After that you will find a file <appname>.pot, which contains all messages of your application in the subfolder po.
+
 
+
Now you just have to find translators to translate the extracted messages into the various languages. As your application gets used by more people, you will find that translators will volunteer to do this. Translated PO files then have to be stored in the po folder with the naming scheme <languagecode>.po.
+
 
+
=== Technical details  ===
+
 
+
The actual translation is done with the gettext package. It contains tools for extracting messages from source files and to handle changed messages, so that translators do not have to start over and over again. The extracted messages and the translations are stored in so called ``PO files'' using different encodings. These files are then compiled into a binary format (MO files) which then get installed.
+
 
+
All translations of a language have to be stored in the same encoding which is defined in the charset file. KLocale reads this file when constructed and uses this information to decode the translations.
+
 
+
Nowadays, UTF-8 is required as the PO files encoding in KDE code repository.
+
 
+
== Handbooks ==
+
 
+
Handbooks, which are written in docbook format, are handled different from applications. The translated Handbooks are committed into the l10n module in the docs folder under each language. In addition, the docs folder is broken down by module and application. For example, the German Kate Handbook is committed to the {{path|l10n/de/docs/kdebase/kate/}} folder. When compiled, the German Kate Handbook is installed to {{path|$KDEDIR/share/doc/HTML/de/kate/index.cache.bz2}}.
+
 
+
Note that it is up to each translation team to generate the translated Handbook when they feel it is complete.
+
  
 
[[Category:Programming]]
 
[[Category:Programming]]

Latest revision as of 21:37, 29 May 2018

Building KDE's l10n Module
Tutorial Series   Localization
Previous   Writing Applications With Localization in Mind
What's Next   Dealing with language changes
Further Reading   n/a

Abstract

Now that your application is ready to be localized, we next look at how to incorporate the necessary mechanisms into the CMake build system of your application.

First we'll explain the "theory" of the steps that need to happen from extracting message strings to installing the generated .po files. After that we'll look at how to implement those steps (#Theory: The xgettext toolchain). If your application is developed in KDE's repositories, many of those steps will happen automatically (see #Handling i18n in KDE's repositories). Else you will want to read on in the #Handling i18n outside KDE's repositories section.

Theory: The xgettext toolchain

Making translations work consists of the following steps:

  1. Extract the translatable strings
  2. Merge the new or changed strings with existing translations
  3. Compile the translations into message catalogs
  4. Install the message catalogs
  5. Use the message catalogs in the application

Extracting the strings

In this step, all the strings you marked as i18n()/ki18n()/etc. in your sources need to be collected into a translation template (.pot) file. Some translateable strings are also contained in .ui, .rc, or .kcfg files. Also tips-of-the-day need to be collected into the .pot file.

These steps are handled by xgettext, extractrc, and preparetips programs, respectively. In some cases you may also need extractattr.

Merging translations

Generally, only a few translatable string will change at a time, in an application. Some will be removed, some will be added, some will be changed into different strings, and some will be moved around in the source files. These changes need to be reflected in the translations, but of course it would be a huge effort to redo the entire translation every time a string was changed. Rather the changes should be merged with the existing translations. This is the job of the msgmerge tool.

Compiling the translations

In order to make message lookup fast, the .po files need to be compiled into so-called "message catalogs" (.mo / .gmo). This is done using the msgfmt tool.

Installing the message catalogs

The compiled message catalogs need to be installed alongside the application. In KDE, the standard location for message catalogs is $DATAROOTDIR/locale/xx/LC_MESSAGES/.

Using the message catalogs

Finally, when the application is run, it needs to load the appropriate message catalog in order to be able to look up and show translated strings. In KDE applications, this is the job of the KLocale class, and in the great majority of cases happens automatically.

For some special cases, such as plugins, look at #Runtime Loading Of Catalogs.

Handling i18n in KDE's repositories

If your application is developed inside KDE's repositories, most of the steps outlined above are automated. In this case, generally, all you will need to do is provide a simple script called Messages.sh, which we will look at below.

Of course, for the curious, a more detailed account of what happens behind the scenes is also provided.

Writing a Messages.sh script

Basically, the only thing that is necessary to prepare and install translations for applications in KDE's repositories, is to provide information, which sources, ui-files or tips need to be translated. For this purpose, you write a small script called Messages.sh and place it in your sources. Here is an example with inline comments:

#!/bin/sh

# invoke the extractrc script on all .ui, .rc, and .kcfg files in the sources
# the results are stored in a pseudo .cpp file to be picked up by xgettext.
$EXTRACTRC `find . -name \*.rc -o -name \*.ui -o -name \*.kcfg` >> rc.cpp
# invoke the grantlee extract script for translatable string from Grantlee themes
$EXTRACT_GRANTLEE_TEMPLATE_STRINGS `find . -name \*.html` >> html.cpp
# if your application contains tips-of-the-day, call preparetips as well.
$PREPARETIPS > tips.cpp
# call xgettext on all source files. If your sources have other filename
# extensions besides .cc, .cpp, and .h, just add them in the find call.
$XGETTEXT `find . -name \*.cc -o -name \*.cpp -o -name \*.h -name \*.qml` -o $podir/APPNAME.pot

As you can see, this script contains only four actual lines of code, and not all may even be needed. The $XGETTEXT, $PREPARETIPS, $EXTRACTRC, $EXTRACT_GRANTLEE_TEMPLATE_STRINGS and $podir environment variables are predefined, you do not need to worry about setting these. The only thing that you will need to do is to replace "APPNAME" with the name of your application (but see #Naming .pot Files for exceptions).

  • $XGETTEXT - Extract i18n translatable strings from C++ files
  • $EXTRACT_TR_STRINGS - Extract Qt tr translatable strings from C++ files for Qt5-based projects
  • $EXTRACTRC - Extract translatable strings from xml configuration and ui files.
  • $EXTRACT_GRANTLEE_TEMPLATE_STRINGS - Extract translatable strings from Grantlee template files.

Try to make sure your Messages.sh script collects only those messages that are really needed. If in doubt, look at other Messages.sh files in the KDE repositories for inspiration, or -- of course -- ask.

Testing your Messages.sh script

To test your Messages.sh script, you need a checkout of the l10n scripts. To get one:

   svn checkout svn://anonsvn.kde.org/home/kde/trunk/l10n-kf5/scripts

You can then go to your project dir and run:

   mkdir po
   mkdir enpo # only needed if $EXTRACT_TR_STRINGS is used in any Messages.sh file
   PATH=/path/to/scripts:$PATH bash /path/to/scripts/extract-messages.sh

When it is done, the po/ dir should contain the generated .pot files.

Translating .desktop Files

You don't need to do anything to get the common fields (See #What's happening behind the scenes for a list) of .desktop or .desktop.cmake files translated.

If you have a file with the same syntax than a .desktop file but it has a different extension, you have to create an ExtraDesktop.sh file that outputs the path of those files

#! /bin/sh
find -name *desktop.in -print

Translating Mimetype descriptions

Add a file named XmlMessages.sh to the folder with your mimetype XML foo.xml with this content:

function get_files
{
    echo foo.xml
}
function po_for_file
{
    case "$1" in
       foo.xml)
           echo foo_xml_mimetypes.po
       ;;
    esac
}
function tags_for_file
{
    case "$1" in
      foo.xml)
           echo comment
       ;;
    esac
}

What's happening behind the scenes

Ktip.png
 
Tip
If your application is in the KDE code repository, all this happens automatically. If your code is not in the KDE repository, you must perform these steps yourself. See #Handling i18n outside KDE's repositories.


Periodically, the script extract-messages.sh (a.k.a. scripty) runs on the KDE server. This program basically calls all Messages.sh scripts in the repository with the appropriate parameters. The extracted messages are stored in template (.pot) files in the templates folder of the l10n module. See What Is Scripty for more information.

The KDE translation teams translate the messages and commit them into a messages folder corresponding to the language, which is further broken down by module. For example, German translated messages for konqueror are committed to l10n/de/messages/applications/konqueror.po.

When the l10n module is built, the .po files are compiled into a binary format for fast lookup and installed as .mo files to $DATAROOTDIR/locale/xx/LC_MESSAGES/, where xx is the two-letter ISO 639 code for the language. These are called the message catalogs.

At runtime, the i18n(...) function, using the original string you coded, looks up the string in the message catalog of the user's desktop language and returns the translated string. If the message catalog is missing, or the specific string is not found, i18n(...) falls back to the original string in your code.

.desktop files in your project are handled separately. makemessages extracts strings, such as Name and Comment from the .desktop files and places them into a file named desktop_mmmm.pot, where mmmm is the module name, in the templates folder. Once translators have translated this file, makemessages inserts the translated strings back into the .desktop files. The list of strings extracted is in l10n/scripts/apply.cc. Here's the code that checks for them:

if (checkTag("Name", in, argc, argv, newFile))
    continue;
if (checkTag("Comment", in, argc, argv, newFile))
    continue;
if (checkTag("Language", in, argc, argv, newFile))
    continue;
if (checkTag("Keywords", in, argc, argv, newFile))
    continue;
if (checkTag("About", in, argc, argv, newFile))
    continue;
if (checkTag("Description", in, argc, argv, newFile))
    continue;
if (checkTag("GenericName", in, argc, argv, newFile))
    continue;
if (checkTag("Query", in, argc, argv, newFile))
    continue;
if (checkTag("ExtraNames", in, argc, argv, newFile))
    continue;

KI18n

KI18n is the framework most used in applications made by KDE (you can also use the Qt translation system though it's not recommented, see #Qt5-only: Code using Qt translation system

It has a very comprehensive documentation that you can read at https://api.kde.org/frameworks/ki18n/html/prg_guide.html

Special cases (plugins, multiple catalogs, etc.)

Declarative plasmoids

For plasmoids written in pure qml, the .pot file must be named the same as its plugin name (with plasma_applet_ prefix), specified in its desktop file as X-KDE-PluginInfo-Name. For instance if we have X-KDE-PluginInfo-Name=org.kde.active.activityscreen the .pot file will be called plasma_applet_org.kde.active.activityscreen.pot.

If the qml files are developed in the form of package instead of an autonomous plasmoid (for instance a package loaded by a C++ plasmoid) the .pot file will have the prefix plasma_package_ instead.

Akonadi Agents

Akonadi agents get the catalog loaded based on the name of the binary, so your .pot file should be named the same way your binary adding .pot at the end You can check that in AgentBase::parseArguments in kdepimlibs

Qt5-only: Code using Qt translation system

Some Qt5-based applications and libraries use the Qt translation system instead of KI18n. Examples of such code are:

  • Tier 1 KF5 frameworks because they cannot depend on KI18n.
  • Applications which provide a Qt-only version.

To make translators aware of this distinction, your .pot file should follow these naming schemes:

  • *.po: code using gettext with KDE translation extensions. .po files are compiled into .mo files (most common case).
  • *_qt.po: code using Qt translation system. The .po files are compiled into .qm files (some KF5 frameworks, applications providing a Qt-only version).
  • *_qt-mo.po: code using pure gettext. The .po files are compiled into .mo files (legacy).


If you are using the Qt translation system you have to make sure the translations are loaded properly in your application at runtime. For that you can use ECMPoQmTools by adding this to your CMakeLists.txt:

include(ECMPoQmTools)

ecm_create_qm_loader(mybinaryname_QM_LOADER mypotfilename_qt)

set(my_project_SRCS
    ...
    ${mybinaryname_QM_LOADER})

Handbooks

Handbooks, which are written in docbook format, are handled different from applications. The translated Handbooks are committed into the l10n module in the docs folder under each language. In addition, the docs folder is broken down by module and application. For example, the German Kate Handbook is committed to the l10n-kf5/de/docs/applications/kate/ folder. When compiled, the German Kate Handbook is installed to $DATAROOTDIR/doc/HTML/de/kate/index.cache.bz2.

Note that it is up to each translation team to generate the translated Handbook when they feel it is complete.

Handling i18n outside KDE's repositories

There's a whole page dedicated to that in /Outside_KDE_repositories


This page was last modified on 29 May 2018, at 21:37. Content is available under Creative Commons License SA 4.0 unless otherwise noted.