Localization/Tools/Lbundle Check

Lbundle Checker

On Localization	Tools
Prerequisites	Subversion Ops, Localizing Non-Text Resources
Related Articles	n/a
External Reading	n/a

About

The lbundle_check.py script checks and records the state of localized non-text resources organized in lbundles, allowing translators to track their relation to original resources. It lives in l10n-kde4/scripts/ directory in the appropriate branch in the KDE repository.

lbundle_check.py can be used to track both out-of-source and in-source lbundles, but the setup needed for these two modes is somewhat different. First the setup is explained for each type of lbundle, followed by the details of operation.

Setup

Before going through the setup examples, refresh the examples on organization of lbundles in the repository (localizing the splash screen of the imaginary KDingus application).

As will become apparent below, to be able to track lbundle states, the translator (one of the coordinators) needs to have all original counterparts to localized resources checked out from the KDE repository, and in the same relative positions. E.g. for KDingus example as kdeutils application, checked out files could have the following structure: $HOME/kde-svn

   trunk/
       l10n-kde4/
           aa/
       KDE/
           kdeutils/
               kdingus/

To have it like this without checking out full parent directories, one can use -N option to svn checkout, which avoids recursion into subdirectories.

Out-of-Source Lbundles

When it was assumed that KDingus is living in kdeutils, a core KDE module, its out-of-source lbundle for the language aa was organized like this: /trunk/l10n-kde4/aa/data/kdeutils/

   CMakeLists.txt
   kdingus/
       CMakeLists.txt
       pics/
           CMakeLists.txt
           l10n-spec
           l10n/
               aa/
                   kdingus-splash.png

except that this listing includes one new file, the l10n-spec. This is the setup file for tracking the lbundle, or rather, all lbundles on its level and beneath.

The l10n-spec files are composed of key = value fields per line, and for the example present example the content of l10n-spec would be:

l10n-spec for KDingus' out-of-source bundle.

source-root = trunk/KDE/kdeutils/kdingus source-vcs = svn bundle-vcs = svn languages = aa track-unbundled = CMakeLists.txt <

The fields are as follows:

source-root: the root of the KDingus' sources in the KDE repository. This is needed to link the original counterparts to localized files. The path of the original file in the repository will be composed of this, plus the relative path of the localized file in the lbundle (relative to the l10n-spec file), and stripped of the l10n/aa/ subpath.

source-vcs, bundle-vcs: the version control systems used by the source code and the lbundle, respectively. These are needed so that the checker script knows how to update the sources (if requested), and to avoid trying to track version control bookkeeping files in the lbundle (e.g. .svn subdirs for SVN). Currently the only known VCS is svn, but the script abstracts this internally, so that other may be added in the future.

languages: space-delimited list of languages expected in l10n subdirs. This is an optional field, which serves to check for naming errors with language subdirs. It may be left out, preventing such checks, but there is no reason to do so (especially for in-source bundles described below).

track-unbundled: space-delimited list of pairs of file paths, where < (less-than) sign for the second states it to be same as the first. The pairs define files which are not really localized resources, but should be tracked as such nevertheless; the first of each pair is the relative path in the lbundle, and the second in the source (so they are allowed to differ). In this example, it is important to track CMakeLists.txt file too, as the installation instruction for the lbundle may need to change if those for the original resources change.

ignore-unbundled: (not used in the example) space-delimited list of relative paths of files which are to be completely ignored by the lbundle checker. For an out-of-source bundle, a file which is not a localized resource should be either tracked too (field track-unbundled), or ignored by this field, otherwise the checker will complain about it.

The usual #-comments are allowed in l10n-spec files, as well as line continuation by trailing backslashes (e.g. when several filename pairs are needed in track-unbundled field).

In-Source Lbundles

For KDingus living as an extragear app, its directory structure with the in-source lbundle having aa and bb languages, and an l10n-spec file, would be this: /trunk/extragear/utils/

   kdingus/
       CMakeLists.txt
       pics/
           CMakeLists.txt
           kdingus-splash.png
           l10n-spec
           l10n/
               aa/
                   kdingus-splash.png
               bb/
                   kdingus-splash.png

and the contents of l10n-spec somewhat simpler than for out-of-source lbundles:

l10n-spec for KDingus' in-source bundle.

source-vcs = svn languages = aa bb

Since the lbundle is kept together with the application, sharing same root directory and version control system, there is no need for the source-root and bundle-vcs fields. In fact, presence or lack of source-root field identifies the bundle as out-of-source or in-source to the checker script.

Fields track-unbundled and ignore-unbundled are not present either. All files which are not localized resources are silently ignored. E.g. it is assumed that the KDingus' maintainer will modify and test install instructions so to not break installation of localized resources.

languages field could be omitted here as well, but for in-source lbundles it is even more important to keep tight check of wrongly named language subdirs.

Operation

If the repository is checked out as exemplified above, to check all lbundles for the language aa in trunk, lbundle-check.py can be run like this: $ cd $HOME/kde-svn/ $ lbundle-check.py -s $HOME/kde-svn -l aa trunk/ This will process all lbundles found in trunk/, whether out-of-source or in-source, containing the language aa. (If -l option weren't specified, it would check all languages in trunk/.) Option -s states the top local directory of the repository, which is needed to resolve original resource paths for out-of-source bundles (path in field source-root from l10n-spec files is appended to it).

Or, to check only out-of-source bundles in the trunk/l10n-kde4/aa/data, one would use: $ cd $HOME/kde-svn/trunk/l10n-kde4 $ lbundle-check.py -s $HOME/kde-svn aa/data where -l aa is omitted since out-of-source bundles will contain only their own language.

So, what does lbundle-check.py actually do, and what does it report? For brevity, let's limit to the out-of-source lbundle for the KDingus as kdeutils app. After setting up the l10n-spec for that case (as detailed above), running lbundle-check.py for the first time would output: $ cd $HOME/kde-svn/trunk/l10n-kde4 $ lbundle-check.py -s $HOME/kde-svn aa/data ! aa/data/kdeutils/kdingus/pics/l10n-track

Added to tracking: 2

 aa/data/kdeutils/kdingus/pics/CMakeLists.txt
 aa/data/kdeutils/kdingus/pics/l10n/aa/kdingus-splash.png

$ creating the file aa/data/kdeutils/kdingus/pics/l10n-track with the following content:

Do not edit manually, except to remove complete lines.-

ok        ¦CMakeLists.txt¦  865f7...e926d  755260

aa

ok ¦l10n/aa/kdingus-splash.png¦ 37c4a7...8fe3a 755647 (For a check, executing the same command line immediately for the second time would produce no output, nor change any files.)

For each l10n-spec file encountered, lbundle-check.py will create one of these l10n-track files. l10n-track file, in each non-comment, non-empty line, contains four fields, in order: the state of the localized resource against the original, the relative path to the localized resource or unbundled-but-tracked file, the checksum of the original resource, and the VCS revision string to the original file which has this checksum.

At this point, when l10n-spec file has been manually written, and l10n-track created by running the checker, both should be added and committed to version control.

So long as the original resource does not change, rerunning lbundle-check.py in the same way will do nothing, as everything is in sync. Obviously, the original resources should be updated to the latest version prior to checking, and lbundle-check.py will do it itself if started with -u option: $ lbundle-check.py -s $HOME/kde-svn -u aa/data svn up /home/.../kdeutils/kdingus/pics/CMakeLists.txt At revision 762512. svn up /home/.../kdeutils/kdingus/pics/kdingus-splash.png At revision 762512. $

Once the original resource is modified, e.g. KDingus' splash screen is souped up, lbundle-check.py will report the following: $ lbundle-check.py -s $HOME/kde-svn aa/data ! aa/data/kdeutils/kdingus/pics/l10n-track

Newly fuzzied: 1

 aa/data/kdeutils/kdingus/pics/l10n/aa/kdingus-splash.png

and its entry in l10n-track will state: fuzzy ¦l10n/aa/kdingus-splash.png¦ 37c4a7...8fe3a 755647 i.e. only the status will have changed from ok to fuzzy. Now the translator has the information needed to compare what has changed in the original splash image: this entry still states the revision of the previous original splash, on which the localized one was based.

If the original resource is renamed, moved, or deleted, the report will be slightly different: $ lbundle-check.py -s $HOME/kde-svn aa/data ! aa/data/kdeutils/kdingus/pics/l10n-track

Newly obsoleted: 1

 aa/data/kdeutils/kdingus/pics/l10n/aa/kdingus-splash.png

and the entry in l10n-track: obsolete ¦l10n/aa/kdingus-splash.png¦ 37c4a7...8fe3a 755647 Now the translator must find out, using the old revision string, what exactly happened to the original resource, and do the same for the localized one too.

When the fuzzy or obsolete state has been resolved, localized resource updated to reflect changes in the original, it should be recorded as such. To do this, the entry for the resource is first manually deleted from l10n-track file, and then lbundle-check.py is rerun. This will recreate the entry with a fresh ok state.