Development/Tutorials/Localization/i18n Semantics

Note

Ki18n maintains it's own Programmers Guide as part of it's documentation

Abstract

Typical way of formatting user visible strings in application interfaces, for a long time has been that of plain text or at most visual markup like HTML tags. In most textual content environments, shift to semantic markup has been recognized as superior to visual (for example, the Docbook XML for documentation). Why not go down the same road for UI strings?

Note

A handy cheat sheet for the KUIT semantic markup language has been prepared by apachelogger.

Semantic Markup by Examples

In the semantic model, user interface strings are marked for their context, and text elements within for their meaning rather than visual appearance. Consider few i18n examples of usual, non-semantic format:

i18n("Move");

i18n("Descending");

i18n("<qt><b>%1</b> does not exist</qt>", fname);

i18n("<h1>History Sidebar</h1> You can configure the history sidebar here.");

Using KDE UI Text (KUIT for short) semantic markup, these strings would be formated like this:

i18nc("@action:button", "Move");

i18nc("@item:inmenu", "Descending");

i18nc("@info", "<filename>%1</filename> does not exist", fname);

i18nc("@info:whatsthis",
      "<title>History Sidebar</title>"
      "<para>You can configure the history sidebar here.</para>");

Two distinct differences between ordinary and KUIT markup can be observed.

The first is the use of context i18n calls, the i18nc(), to convey the usage context of the string by means of the context marker. The first message above, "Move", has been assigned the @action:button marker, where @action is the semantic role which describes the text as an action to be taken (e.g. operation on data or opening of a new dialog), and :button is the interface subcue saying that this text is displayed on a pushbutton widget. The second message, "Descending", has been marked as semantically a list item (@item), displayed in a menu (:inmenu). The interface subcue can be left out if none is appropriate, as has been done in the third message.

The other difference is the use of the semantic tags, which convey the meaning of a word or phrase within the text. The <filename>%1</filename> part of the third message tells that the substituted text is the name of a file. The <title> and <para> tags in the last message clearly lay out structure of a longer informational text.

Note

The context marker can be added when within some of the standard XML sources too. In Qt Designer forms (.ui files), each text label to a widget has comment attribute (presented as "disambiguation" property within Designer, or "comment" prior to Qt 4.5), which can be used in the same manner as context argument of i18nc() call. Similarly, in the KXmlGui (.rc) and KConfigXT (.kcfg) files, tags <text>, label, and whatsthis can have a context attribute. For example:

<label context="@label">Hide trivial details</label>
<whatsthis context="@info:whatsthis">Option to hide drivel</whatsthis>
<text context="@item:inmenu">&amp;New...</text>

Even when a context marker is present, sometimes the programmer may want to provide an additional "free-form" description to translators, in order to shed more light on particularly ambiguous strings. The free-form description is just separated by a whitespace from the context marker proper, like this:

i18nc("@item:inmenu Sorting order", "Descending");

One particularly useful and yet general piece of free-form description, is the title under which the message is grouped -- the menu title for actions in menu, the group title of set of radio-buttons, the listbox label for items in list, etc. For example:

iconSizeBox = new QGroupBox(i18nc("@title:group", "Icon Size"), this);
//...
iconSize1 = new QRadioButton(i18nc("@option:radio Icon Size", "Small"), this);
iconSize2 = new QRadioButton(i18nc("@option:radio Icon Size", "Medium"), this);
iconSize3 = new QRadioButton(i18nc("@option:radio Icon Size", "Large"), this);
//...

Advantages of Semantic Markup

KUIT markup has advantages both to users and to translators of applications that make use of it.

For the users, the use of semantic tags means consistent formatting of same kinds of text. A notorious example of inconsistent visual formatting are filenames and paths, which are sometimes put in as is, sometimes in quotes (and ordinary quotes at that, rather than proper English fancy quotes), and sometimes in bold tags. Furthermore, the text within the tag may be modified when semantically marked; for example, the standard "/" path delimiters in a <filename> text will be substituted for platform specific ones.

Translators will benefit from both context markers and tags. For the @action role of the "Move" string in the example above, the translator may use command form of the verb, while gerund form (like "Moving") may be more appropriate for the @title role, which would be used if the string was title of the menu, window, etc. The interface subcue, like :button above, if present, additionally enables the translator to mentally picture the actual runtime GUI. Tags within the text will also benefit translators, as they may clarify the structure of the sentence, especially in presence of placeholder substitutions.

Context markers also serve a technical purpose: they decide whether what form of visual formatting is used. For example, any @title role will use plain text, whereas @info will frequently produce rich text, depending on the subcue.

None the least, semantic markup removes the burden from programmers of thinking about the visual formatting to apply, like "Should I put the path in quotes or <b>?", or "Should the title be <h2> or <h3>?", and so on.

Context Markers

Context marker consist of the semantic role and the interface subcue, in the form of @role:subcue. Each message should be given a role, but the subcue may be left out. The subcue should be given only when the string clearly maps to the user interface element that it describes.

Every role/subcue combination determines the default visual formatting of resulting string, whether it comes out as plain or rich text. See the section on limitations for a way to override the default formatting when necessary.

Warning

Roles and subcues in the context marker cannot be specified at will, but must be drawn from the sets defined below. This is important for several reasons, one being that translators have deliberated about and agreed upon the meanings of presented sets, rather than having to second-guess arbitrary combinations.

KUIT defines the following roles and subcues (with notes on default visual formatting):

@action

Text to all clickable widgets that cause some action to be performed, like an operation on the data, view restructuring, or opening a dialog. The button texts and menu entries (except submenu titles) all fall into this category.

:button - pushbuttons in windows and dialogs

:inmenu - menu entries that perform an action

:intoolbar - toolbar buttons

All @action markers are formatted as plain text by default.

@title

Text that is semantically a title in the interface. These would include window titles, menu titles, tab names, option group names in configuration dialogs, and column and row names in list views.

:window - window title (also dock name)

:menu - menu name

:tab - tab name

:group - option group

:column - column name

:row - row name

All @title markers are formatted as plain text by default.

@option

Text to options which user can turn on and off, or choose between. These are the labels to checkboxes (either in dialogs or in menus) and radio buttons.

:check - checkbox label

:radio - radio-button label

All @option markers are formatted as plain text by default.

@label

Text labels to various widgets in the interface, which are none of @action, @title, @option. These include labels to sliders, spinboxes, combo, list and text boxes, font and color choosers.

:slider - slider labels (but end-ranges are @item:inrange!)

:spinbox - spinbox labels

:listbox - list and combo boxes

:textbox - text and edit boxes

:chooser - chooser widgets (fonts, colors, etc.)

All @label markers are formatted as plain text by default.

@item

Strings that can be considered one from a range of possibilities or properties. Entries in listings, dropdown and combo boxes are obvious, but also some menu items (e.g. encoding selection, sort orderings), end-labels to ranges (e.g. high/low, more/less), inserts into longer texts, and properites (e.g. file types, permissions) frequently displayed in tabular form (e.g. column views, property dialogs).

:inmenu - items presented as menu entries

:inlistbox - items in list and combo boxes

:intable - items presented in table-like forms

:inrange - range labels to sliders, etc.

:intext - words and phrases inserted into other messages

All @item markers are formatted as plain text by default.

@info

General texts for user's information, that do not fall under any of the previous roles. These are for example tooltip and "What's This?" texts, text in message boxes, fields in status bar, and strings in progress dialogs.

:tooltip - hovering tooltips

:whatsthis - "What's This?" explanations of widgets

:status - texts in status displays (e.g. in status bar)

:progress - the current state of ongoing process

:tipoftheday - introductory tips on application startup

:credit - contributor names and their contributions

:shell - info output to the terminal, rather than to GUI

Standalone @info, without a subcue, is formatted as rich text by default; the same holds for :tooltip, :whatsthis, and :tipoftheday subcues. With :status, :progress, and :credit, plain text is produced. :shell produces terminal text (like plain, but with possible shell escape sequences).

Semantic Tags

KUIT semantic tags come in several logical groups:

phrase tags - those that ascribe meaning to certain phrases and inserts
sentence tags - which describe the purpose of a complete sentence in text
structure tags - used to order longer text into paragraphs, titles, etc.

Phrase tags

Phrase tags will by default not admit any subtags; where some subtags can be used, it is so indicated. KUIT defines the folowing phrase tags:

<application>: Name of an application.

i18nc("@action:inmenu",
      "Open with <application>%1</application>", appName);

<bcode>: Line-breaking body of code, for short listings.

i18nc("@info:whatsthis",
      "You can try the following snippet:<bcode>"
      "\\begin{equation}"
      "  C_{x_i} = \\frac{C_z^2}{e \\pi \\lambda}"
      "\\end{equation}"
      "</bcode>");

<command>: Name of shell command or system call. Its man section can be provided via section attribute.

i18nc("@info",
      "This will call <command>%1</command> internally.", cmdName);

i18nc("@info",
      "Consult man entry for <command section='1'>%1</command>", cmdName);

<email>: Email address. Without attributes, the tag text is the address. Address can also be given with address attribute, in which case the tag text is the name or description attached to the address.

i18nc("@info",
      "Send bug reports to <email>%1</email>.", emailNull);

i18nc("@info",
      "Send praises to <email address='%1'>the author</email>.", emailMy);

The construct will be hyperlinked in rich text format.

<emphasis>: Emphasize a word or phrase in the text.

i18nc("@info:progress",
      "Checking <emphasis>feedback</emphasis> circuits...");

For strong emphasis, attribute strong (since KDE 4.3) may be used, with value 1, yes, or true.

<envar>: Environment variable. The $ sign will be prepended automatically in formatted text.

i18nc("@info",
      "Assure that your <envar>PATH</envar> is properly set.");

<filename>: File or folder name or path. The path separators will be transformed into what is native to the platform.

i18nc("@info", "Cannot read <filename>%1</filename>.", filename);

i18nc("@info",
      "<filename><envar>HOME</envar>/.foorc</filename> does not exist.");

The <envar> can be used as subtag.

<icode>: Inline code, like shell command lines.

i18nc("@info:tooltip",
      "Execute <icode>svn merge</icode> on selected revisions.");

The <placeholder> can be used as subtag.

<interface>: Path to GUI interface element. If there is more than one element in the path, use "|" or "->" to delimit elements, which will be converted into canonical delimiter.

i18nc("@info:whatsthis",
      "If you make a mistake, click "
      "<interface>Reset</interface> to start again.");
i18nc("@info:whatsthis",
      "The line colors can be changed under "
      "<interface>Settings->Visuals</interface>.");

<link>: Link to a URL-addressable resource. Without attributes, the tag text is the URL; alternatively, the URL can be given by url attribute, and then the tag text serves as description.

i18nc("@info:tooltip",
      "Go to <link>%1</link> website.", urlKDE);

i18nc("@info:tooltip",
      "Go to <link url='%1'>the KDE website</link>.", urlKDE);

The variant with URL/description separation is preferred when applicable. The construct will be hyperlinked in rich text format.

<message>: An external message to be reported to the user.

i18nc("@info",
      "The fortune cookie says: <message>%1</message>", trouble);

<nl>: Line break, counterpart to HTML's <br/>. In plain text it will format as a newline character, and in rich text as <br/>.

i18nc("@info",
      "Do you really want to delete:<nl/>"
      "<filename>%1</filename>", fileName);

<numid>: By default, numbers supplied as arguments to i18n calls are formatted into localized form. If the number is supposed to be a numeric identifier instead, like a port number, use this tag to signal numeric-id environment.

i18nc("@info:progress",
      "Connecting to <numid>%1</numid>...", portNo);

<placeholder>: A placeholder text, either something to be replaced by the user, or a generic item in a list.

i18nc("@info",
      "Replace <placeholder>name</placeholder> with your name.");

i18nc("@item:inlistbox",
      "<placeholder>All images</placeholder>");

<resource>: General named resource. Names of documents, sessions, projects, toolbars, plugins, schemes and themes, accounts, etc.

i18nc("@info", "Apply color scheme <resource>%1</resource>?", colScheme);

<shortcut>: Combination of keys to press. Separate the keys by "+" or "-", and the shortcut will be converted into canonical form.

i18nc("@info:whatsthis",
      "Cycle through layouts using <shortcut>Alt+Space</shortcut>.");

Sentence tags

Sentence tags mark complete sentences in text, and will admit any phrase tags as subtags. The following are defined:

<note>: The sentence is a side note of significance to the topic.

i18nc("@info",
      "Probably the best known of all duck species is the Mallard. "
      "It breeds throughout the temperate areas around the world. "
      "<note>Most domestic ducks are derived from Mallard.</note>");

Do not explicitly add "Note:", it will be added automatically. If you really need other label than "Note", use attribute label, e.g. "<note label='Trivia'>...</note>".

<warning>: The sentence is a warning.

i18nc("@info",
      "Really delete this key?"
      "<warning>This cannot be undone.</warning>");

Do not explicitly add "Warning:", it will be added automatically. If you really need other label than "Warning", use attribute label, e.g. "<warning label='Danger'>...</warning>".

Structure tags

Structure tags are used to split longer texts into titles, paragraphs, and lists. By default they can contain any phrase or sentence tags, unless indicated otherwise.

<para>: Text paragraph.

<title>: The title of the text. Must be the first tag if present, but can be omitted.

<subtitle>: Subtitle in the text. Must be followed by at least one <para>.

<list>: List of items. Can contain only <item> as subtags. List is considered an element of the paragraph, so the <list> must be found inside <para>.

<item>: List item.

If any of the structure tags is present, then there must be no text outside of structure tags. The following is not valid KUIT markup:

// invalid markup
i18nc("@info",
      "<title>History Sidebar</title>"
      "You can configure the history sidebar here."); // <para> missing

Limitations to Semantic Markup

Semantic markup cannot be used in "dumb" strings, which do not pass through KDE's i18n subsystem. These would be, for example, strings in .desktop format files. But not the strings in UI files, as in Qt Designer they can be equipped with both context markers (via comment field to text properties) and semantic tags.

Qt's rich text HTML tags can be used concurrently with KUIT tags, but this is not advised unless necessary. They may be needed, for example, to create tables or insert images, as KUIT does not implement this functionality at the moment.

Sometimes, the visual formatting may not be quite appropriate for the output device; every role/subcue combination has a preset default formatting. For example, if the @info role is applied to a string which is used in a widget that does not handle rich text, it will come out with HTML tags displayed verbatim. To handle this, visual formatting can be explicitly signaled by /format modifier appended to context marker:

i18nc("@info/plain",
      "<filename>%1</filename> does not exist", fname);

Presently, the possible format modifiers are /plain, /rich and /term (for terminal format, possible use of escape sequences).

To specify context in kpartgui rc file use context attribute (<text context="@title:menu">File</text>). A context-aware version of I18N_NOOP is I18N_NOOP2.

Should I Go For Semantic Markup?

Admittedly, KUIT markup is an additional thing to be learned and applied throughout the course of development. By now you may be wondering if it is worthwile to invest time into that, particularly in view of two cases:

starting work on a new application, and
porting messages in existing applications.

You are strongly advised to use KUIT for new code. Compared to the total time spent on code, writing UI messages is only a small fraction. Context markers will help translators a lot, and message tags will provide consistent visual text formatting to your application.

When modifying existing code, there are two issues. First, obviously it is a daunting task to go through hundreds (or worse) of messages and equip them with semantic markup. Second, by changing the messages, the translators too will have to review their existing translations; however, it is not expected that the porting will take so "epic" proportions that the translators cannot keep up. Sumarily, feel free to do as you see fit.

Additionally, for porting, keep in mind that it is not all-or-nothing proposal. Any amount of semantic messages are useful to translators, and users can only see the difference for the better. Thus, for example, deciding to make all new messages semantic and slowly over time fix old messages, is a perfectly fine strategy.

To make your job easier, there is an i18n-checker script that will report the problems in KUIT markup, as well as check some other i18n nuances. It is run daily on KDE code repository-wide as part Krazy-framework, but you can also run it locally on your code. It can be found in trunk/quality/krazy2/plugins/general/i18ncheckarg. Furthermore, as of this moment Krazy checks only C++ sources, while when you run the script locally, some of the XML sources containing translatable strings will be checked as well. i18ncheckarg requires Perl libxml bindings, which are probably already packaged for your distribution (Debian package is libxml-libxml-perl).

Note

By default i18ncheckarg takes a single filename to check, but using --allsources option you can provide as many file or directory paths as you wish; typically you would give top directory path to your sources. Also, to report all missing context markers use --ctxmark option, or otherwise missing markers will be reported only if some threshold of marked-to-total number of messages in a file is reached.

Last but not the least, there is also a chic-effect to the KUIT. Its wide use, together with some under-the-hood elements on translators' disposal, will make KDE4's i18n layer without peer in free or proprietary software world. Insofar as you consider localization excellence an important part of the overall KDE excellence, this is something that may also tip your decision :) -- Your Friendly Translator.