Localization/Tools/Lbundle Check - Revision history

Ilic: Removed extra empty lines.

2013-02-02T16:05:04Z

Removed extra empty lines.

Ilic: Updated to new Git/Subversion state of affairs.

2013-02-02T16:04:05Z

Updated to new Git/Subversion state of affairs.

Neverendingo: Text replace - "`" to ""`


2011-06-29T21:12:39Z
Text replace - "<code text>" to "<syntaxhighlight lang="text">"
Show changes

`Neverendingo: Text replace - "`" to ""

2011-06-29T20:57:23Z

Text replace - "</code>" to "</syntaxhighlight>"

Show changes

Ilic: Update repository location of checker script.

2008-08-03T14:54:28Z

Update repository location of checker script.

Ilic: Compact specification of external paths.

2008-06-08T17:08:51Z

Compact specification of external paths.

Ilic: Instructions for tracking documentation.

2008-06-07T17:04:22Z

Instructions for tracking documentation.

Ilic: New field, track-by-subdir

2008-06-06T22:07:39Z

New field, track-by-subdir

Ilic: Describe greedy bundling.

2008-06-05T22:13:05Z

Describe greedy bundling.

Ilic: New features. Change in mapping format for track-unbundled.

2008-06-01T02:45:41Z

New features. Change in mapping format for track-unbundled.

@@ Line 16: / Line 16: @@
 <tt>lbundle_check.py</tt> is in no way tied to KDE Translation Project, but can be used in any other environment. Only the instructions in this article are specifically about using <tt>lbundle_check.py</tt> in context of KDE TP.
 == Checking Out and Updating Sources ==
@@ Line 86: / Line 85: @@
 </syntaxhighlight>
 <tt>lbundle_check.py</tt> is capable of gracefully handling this case as well.
 == Setup ==
@@ Line 198: / Line 196: @@
 ; greedy-only-started: normally all original resources missing in localization are reported as such (except the ignored ones), no matter where they reside in the original subdirectory tree covered by the lbundle. This field (a boolean) is used to limit greedy bundling only to those subdirectories already existing in the lbundle. Useful in several scenarios, e.g. when starting the resource localization to avoid being engulfed in listings of missing localized resources. Automatically enabled when <tt>track-by-subdir</tt> is in effect.
 == Operation ==
@@ Line 281: / Line 278: @@
 When [[#Greedy Bundling|greedy bundling]] is engaged, each original file which doesn't have a localized counterpart will be reported in the track file (either as bundled or unbundled, depending on <tt>greedy-monolingual</tt> option), with it's state set to <tt>missing</tt> (except for the ignored files, given by the <tt>ignore-unbundled</tt> option). When the localized file is placed at the appropriate location, next run of <tt>lbundle-check.py</tt> will change the state to <tt>ok</tt>.
 == Special Case: Tracking KDE Documentation Resources ==

@@ Line 13: / Line 13: @@
 == About ==
-The <tt>lbundle_check.py</tt> script checks and records the state of localized non-text resources organized in lbundles, allowing translators to track their relation to original resources. It lives in <tt>l10n-kde4/scripts/</tt> directory in the appropriate branch in the KDE repository.
+The <tt>lbundle_check.py</tt> script checks and records the state of localized non-text resources organized in lbundles, allowing translators to track their relation to original resources. Its latest version can be found in <tt>trunk/l10n-support/scripts/</tt> directory in the KDE repository.
 <tt>lbundle_check.py</tt> can be used to track both out-of-source and in-source lbundles, but the setup needed for these two modes is somewhat different. First the setup is explained for each type of lbundle, followed by the details of operation.

@@ Line 71: / Line 71: @@
 Several other fields, not used in this example, are available:
 ; ignore-unbundled: space-delimited list of relative file paths which are to be completely ignored by the lbundle checker. For an out-of-source bundle, a file which is unbundled should be either tracked too (using <tt>track-unbundled</tt>) or ignored using this field, otherwise the checker will complain about it. Shell globs also possible.
@@ Line 83: / Line 79: @@
 ; track-by-subdir: by default there is only one tracking file per <tt>l10n-spec</tt> file (see [[#Operation|operation]] for tracking files); when this option is enabled (a boolean), there will be instead one tracking file per subdirectory (excluding <tt>l10n/</tt> directories) starting from the level of <tt>l10n-spec</tt> file and below. Usefull when frequent moving of subdirectories within the repository is expected, and there are a lot of localized resources to track.
 The usual #-comments are allowed in <tt>l10n-spec</tt> files, as well as line continuation by trailing backslashes (e.g. when several file paths are needed in <tt>track-unbundled</tt> field).
@@ Line 139: / Line 143: @@
 $ lbundle-check.py -s $HOME/kde-svn -l aa trunk/
 </code>
-This will process all lbundles found in <tt>trunk/</tt>, whether out-of-source or in-source, containing the language <tt>aa</tt>. (If <tt>-l</tt> option weren't specified, it would check ''all'' languages in <tt>trunk/</tt>.) Option <tt>-s</tt> states the top local directory of the repository, which is needed to resolve original resource paths for out-of-source bundles (it is prepended to paths in <tt>source-root</tt> fields from <tt>l10n-spec</tt> files; where instead <tt>source-root-rel</tt> or <tt>source-root-repl</tt> fields are given, the top local directory is not used).
+This will process all lbundles found in <tt>trunk/</tt>, whether out-of-source or in-source, containing the language <tt>aa</tt>. (If <tt>-l</tt> option weren't specified, it would check ''all'' languages in <tt>trunk/</tt>.) Option <tt>-s</tt> states the top local directory of the repository, which is needed to resolve original resource paths for out-of-source bundles; it is prepended to paths of external resources (e.g. <tt>source-root</tt> field) in <tt>l10n-spec</tt> files, where they were specified as relative to repository top (where specified relative to the <tt>l10n-spec</tt> file itself, or by a path replacement, the top local directory is not used).
 Or, to check only out-of-source bundles in the <tt>trunk/l10n-kde4/aa/data</tt>, one would use:
@@ Line 255: / Line 259: @@
 both with the following, branch-independent content:
 <code text>
-source-root-repl = ll/docs:documentation
+source-root = ll/docs:documentation
 source-vcs = svn
 bundle-vcs = svn
@@ Line 314: / Line 318: @@
 Note that version control operations are done automatically too.
-Assume now that, in the above example, after some time original <tt>configure.png</tt> is modified, and original <tt>embedded-files-bar.png</tt> is removed. Then, running the tracker produces:
+Assume now that, in the above example, after some time the original <tt>configure.png</tt> is modified, and the original <tt>embedded-files-bar.png</tt> is removed. Then, running the tracker produces:
 <code text>
 $ cd $KDEREPO/trunk/l10n-kde4/

@@ Line 23: / Line 23: @@
 As it will become apparent below, to be able to track lbundle states, the translator (one of the coordinators) needs to have all the original counterparts to localized resources checked out from the KDE repository, and in the same relative positions. E.g. for KDingus example as kdeutils application, checked out files could have the following structure:
 <code text>
-$HOME/kde-svn
+$KDEREPO/
      trunk/
          l10n-kde4/
@@ Line 220: / Line 220: @@
 When [[#Greedy Bundling|greedy bundling]] is engaged, each original file which doesn't have a localized counterpart will be reported in the track file (either as bundled or unbundled, depending on <tt>greedy-monolingual</tt> option), with it's state set to <tt>missing</tt> (except for the ignored files, given by the <tt>ignore-unbundled</tt> option). When the localized file is placed at the appropriate location, next run of <tt>lbundle-check.py</tt> will change the state to <tt>ok</tt>.

@@ Line 81: / Line 81: @@
 ; ignore-substr: space-delimited list of substrings to ignore in names of bundled files when determining their original counterparts. This is never needed in KDE, but in other environments some localized files may be handled differently than a plain substitute for the original file, based on a substring in their name (e.g. a localized image may be overlayed over the original, rather than replacing it).
 The usual #-comments are allowed in <tt>l10n-spec</tt> files, as well as line continuation by trailing backslashes (e.g. when several file paths are needed in <tt>track-unbundled</tt> field).
@@ Line 128: / Line 130: @@
 ; greedy-from-level: if the lbundle covers a tree of original subdirectories, this field can be used to limit the greedy collection and reporting of missing files only to subdirectories at this level and below. 0 is the level of the <tt>l10n-spec</tt> file itself, 1 is one level below, etc.
-; greedy-only-started: normally all original resources missing in localization are reported as such (except the ignored ones), no matter where in the original  subdirectory tree covered by the lbundle they reside. This field (a boolean) is used to limit greedy bundling only to those subdirectories already existing in the lbundle. Useful in several scenarios, e.g. when starting the resource localization to avoid being engulfed in listings of missing localized resources.
+; greedy-only-started: normally all original resources missing in localization are reported as such (except the ignored ones), no matter where in the original  subdirectory tree covered by the lbundle they reside. This field (a boolean) is used to limit greedy bundling only to those subdirectories already existing in the lbundle. Useful in several scenarios, e.g. when starting the resource localization to avoid being engulfed in listings of missing localized resources. Automatically enabled when <tt>track-by-subdir</tt> is in effect.
 == Operation ==
@@ Line 137: / Line 139: @@
 $ lbundle-check.py -s $HOME/kde-svn -l aa trunk/
 </code>
-This will process all lbundles found in <tt>trunk/</tt>, whether out-of-source or in-source, containing the language <tt>aa</tt>. (If <tt>-l</tt> option weren't specified, it would check ''all'' languages in <tt>trunk/</tt>.) Option <tt>-s</tt> states the top local directory of the repository, which is needed to resolve original resource paths for out-of-source bundles (path in field <tt>source-root</tt> from <tt>l10n-spec</tt> files is appended to it).
+This will process all lbundles found in <tt>trunk/</tt>, whether out-of-source or in-source, containing the language <tt>aa</tt>. (If <tt>-l</tt> option weren't specified, it would check ''all'' languages in <tt>trunk/</tt>.) Option <tt>-s</tt> states the top local directory of the repository, which is needed to resolve original resource paths for out-of-source bundles (it is prepended to paths in <tt>source-root</tt> fields from <tt>l10n-spec</tt> files; where instead <tt>source-root-rel</tt> or <tt>source-root-repl</tt> fields are given, the top local directory is not used).
 Or, to check only out-of-source bundles in the <tt>trunk/l10n-kde4/aa/data</tt>, one would use:

@@ Line 62: / Line 62: @@
 The fields used in the example are:
-; source-root: the root of the KDingus' sources in the KDE repository. This is needed to link the original counterparts to localized files. The path of the original file in the repository will be composed of this, plus the relative path of the localized file in the lbundle (relative to the <tt>l10n-spec</tt> file), and stripped of the <tt>l10n/aa/</tt> subpath.
+; source-root: the root of the KDingus' sources in the KDE repository. This is needed to link the original counterparts to localized files. The path of the original file relative to the repository top will be composed of this, plus the relative path of the localized file in the lbundle (relative to the <tt>l10n-spec</tt> file), and stripped of the <tt>l10n/aa/</tt> subpath.
 ; source-vcs, bundle-vcs: the version control systems used by the source code and the lbundle, respectively. These are needed so that the checker script knows how to update the sources (if requested), and to avoid trying to track version control bookkeeping files in the lbundle (e.g. <tt>.svn</tt> subdirs for SVN). Currently the only known VCS is <tt>svn</tt>, but the script abstracts this internally, so that other may be added in the future.
@@ Line 71: / Line 71: @@
 Several other fields, not used in this example, are available:
 ; ignore-unbundled: space-delimited list of relative file paths which are to be completely ignored by the lbundle checker. For an out-of-source bundle, a file which is unbundled should be either tracked too (using <tt>track-unbundled</tt>) or ignored using this field, otherwise the checker will complain about it. Shell globs also possible.
@@ Line 109: / Line 113: @@
 <tt>languages</tt> field could be omitted here as well, but for in-source lbundles it is even more important to keep tight check of wrongly named language subdirs.
 == Operation ==
@@ Line 196: / Line 216: @@
 To resolve the fuzzy or obsolete state in strict mode, it is not necessary to manually remove the entry from <tt>l10n-track</tt>. Instead, it is enough to put an updated version of the file (without the state maker in its name) next to the out-of-sync version, and rerun <tt>lbundle-check.py</tt>. It will remove the old version and update the entry in <tt>l10n-track</tt>. (The renaming/removal are version control aware, i.e. proper version control commands will be used for these operations if <tt>*-vcs</tt> fields have been set in the spec file.)

@@ Line 1: / Line 1: @@
 {{LocalizationBrowser|
@@ Line 19: / Line 21: @@
 Before going through the setup examples, refresh the examples on organization of [[Localization/Concepts/Non_Text_Resources#Inside The Repository|lbundles in the repository]] (localizing the splash screen of the imaginary KDingus application).
-As will become apparent below, to be able to track lbundle states, the translator (one of the coordinators) needs to have all original counterparts to localized resources checked out from the KDE repository, and in the same relative positions. E.g. for KDingus example as kdeutils application, checked out files could have the following structure:
+As it will become apparent below, to be able to track lbundle states, the translator (one of the coordinators) needs to have all the original counterparts to localized resources checked out from the KDE repository, and in the same relative positions. E.g. for KDingus example as kdeutils application, checked out files could have the following structure:
 <code text>
 $HOME/kde-svn
@@ Line 46: / Line 48: @@
                      kdingus-splash.png
 </code>
-except that this listing includes one new file, the <tt>l10n-spec</tt>. This is the setup file for tracking the lbundle, or rather, all lbundles on its level and beneath.
+except that this listing includes one new file, the <tt>l10n-spec</tt>. This is the setup file for tracking the lbundle, or rather, all lbundles on its level and beneath. The files outside of <tt>l10n/</tt> subdirs are going to be called ''unbundled'', as they are not considered a part of lbundle proper.
-The <tt>l10n-spec</tt> files are composed of <tt>key = value</tt> fields per line, and for the example present example the content of <tt>l10n-spec</tt> would be:
+The <tt>l10n-spec</tt> files are composed of <tt>key = value</tt> fields per line. For the example above the content of <tt>l10n-spec</tt> would be:
 <code text>
 # l10n-spec for KDingus' out-of-source bundle.
@@ Line 55: / Line 57: @@
 bundle-vcs = svn
 languages = aa
-track-unbundled = CMakeLists.txt <
+track-unbundled = CMakeLists.txt
 </code>
-The fields are as follows:
+The fields used in the example are:
 ; source-root: the root of the KDingus' sources in the KDE repository. This is needed to link the original counterparts to localized files. The path of the original file in the repository will be composed of this, plus the relative path of the localized file in the lbundle (relative to the <tt>l10n-spec</tt> file), and stripped of the <tt>l10n/aa/</tt> subpath.
@@ Line 66: / Line 68: @@
 ; languages: space-delimited list of languages expected in <tt>l10n</tt> subdirs. This is an optional field, which serves to check for naming errors with language subdirs. It may be left out, preventing such checks, but there is no reason to do so (especially for in-source bundles described below).
-; track-unbundled: space-delimited list of pairs of file paths, where <tt>&lt;</tt> (less-than) sign for the second states it to be same as the first. The pairs define files which are not really localized resources, but should be tracked as such nevertheless; the first of each pair is the relative path in the lbundle, and the second in the source (so they are allowed to differ). In this example, it is important to track <tt>CMakeLists.txt</tt> file too, as the installation instruction for the lbundle may need to change if those for the original resources change.
+; track-unbundled: space-delimited list of relative file paths, or pairs of file paths, of unbundled files which should be tracked nevertheless. In this example, <tt>CMakeLists.txt</tt> should be tracked because the installation instructions for the lbundle may need to change when those for the original resources change. A pair of file paths, as two space-separated paths within parenthesis (e.g. <tt>(foo.txt bar.txt)</tt>), can be given instead of a single file path when the local relative path needs to be different from that of the tracked original. A shell glob can be specified instead of a particular path (e.g. <tt>*.txt</tt>), but only for single paths, not in pairs. Backslash serves as escape character, e.g. when the file name contains a space.
-; ignore-unbundled: (not used in the example) space-delimited list of relative paths of files which are to be completely ignored by the lbundle checker. For an out-of-source bundle, a file which is not a localized resource should be either tracked too (field <tt>track-unbundled</tt>), or ignored by this field, otherwise the checker will complain about it.
+Several other fields, not used in this example, are available:
-The usual #-comments are allowed in <tt>l10n-spec</tt> files, as well as line continuation by trailing backslashes (e.g. when several filename pairs are needed in <tt>track-unbundled</tt> field).
+; ignore-unbundled: space-delimited list of relative file paths which are to be completely ignored by the lbundle checker. For an out-of-source bundle, a file which is unbundled should be either tracked too (using <tt>track-unbundled</tt>) or ignored using this field, otherwise the checker will complain about it. Shell globs also possible.
 === In-Source Lbundles ===
@@ Line 98: / Line 106: @@
 Since the lbundle is kept together with the application, sharing same root directory and version control system, there is no need for the <tt>source-root</tt> and <tt>bundle-vcs</tt> fields. In fact, presence or lack of <tt>source-root</tt> field identifies the bundle as out-of-source or in-source to the checker script.
-Fields <tt>track-unbundled</tt> and <tt>ignore-unbundled</tt> are not present either. All files which are not localized resources are silently ignored. E.g. it is assumed that the KDingus' maintainer will modify and test install instructions so to not break installation of localized resources.
+Field <tt>track-unbundled</tt> (or <tt>ignore-unbundled</tt>) are not present either. All files which are unbundled are silently ignored. E.g. it is assumed that the KDingus' maintainer will modify and test install instructions so as to not break installation of localized resources.
 <tt>languages</tt> field could be omitted here as well, but for in-source lbundles it is even more important to keep tight check of wrongly named language subdirs.
@@ Line 118: / Line 126: @@
 where <tt>-l aa</tt> is omitted since out-of-source bundles will contain only their own language.
-So, what does <tt>lbundle-check.py</tt> actually do, and what does it report? For brevity, let's limit to the out-of-source lbundle for the KDingus as kdeutils app. After setting up the <tt>l10n-spec</tt> for that case (as detailed above), running <tt>lbundle-check.py</tt> for the first time would output:
+So, what does <tt>lbundle-check.py</tt> actually do, and what does it report? For brevity, let's limit to the out-of-source lbundle for the KDingus as kdeutils app. After setting up the <tt>l10n-spec</tt> for that case (as detailed [[#Out-of-Source Lbundles|above]]), running <tt>lbundle-check.py</tt> for the first time would output:
 <code text>
 $ cd $HOME/kde-svn/trunk/l10n-kde4
@@ Line 129: / Line 137: @@
+$
 </code>
-creating the file <tt>aa/data/kdeutils/kdingus/pics/l10n-track</tt> with the following content:
+creating the file <tt>aa/data/kdeutils/kdingus/pics/l10n-track</tt> in the process, with the following content:
 <code text>
 # Do not edit manually, except to remove complete lines.
@@ Line 141: / Line 149: @@
 (For a check, executing the same command line immediately for the second time would produce no output, nor change any files.)
-For each <tt>l10n-spec</tt> file encountered, <tt>lbundle-check.py</tt> will create one of these <tt>l10n-track</tt> files. <tt>l10n-track</tt> file, in each non-comment, non-empty line, contains four fields, in order: the state of the localized resource against the original, the relative path to the localized resource or unbundled-but-tracked file, the checksum of the ''original'' resource, and the VCS revision string to the original file which has this checksum.
+For each <tt>l10n-spec</tt> file encountered, <tt>lbundle-check.py</tt> will create one of these <tt>l10n-track</tt> files. <tt>l10n-track</tt> file, in each non-comment, non-empty line, contains four fields, in order: the state of the localized resource against the original, the relative path to the bundled resource, the checksum of the ''original'' resource, and the VCS revision string of the original file which has this checksum.
 At this point, when <tt>l10n-spec</tt> file has been manually written, and <tt>l10n-track</tt> created by running the checker, both should be added and committed to version control.
-So long as the original resource does not change, rerunning <tt>lbundle-check.py</tt> in the same way will do nothing, as everything is in sync. Obviously, the original resources should be updated to the latest version prior to checking, and <tt>lbundle-check.py</tt> will do it itself if started with <tt>-u</tt> option:
+So long as the original resource does not change, rerunning <tt>lbundle-check.py</tt> in the same way will do nothing, as everything is in sync. Obviously, the original resources should be updated to the latest repository version prior to checking, and <tt>lbundle-check.py</tt> can do that itself if started with <tt>-u</tt> option:
 <code text>
 $ lbundle-check.py -s $HOME/kde-svn -u aa/data
@@ Line 183: / Line 191: @@
 Now the translator must find out, using the old revision string, what exactly happened to the original resource, and do the same for the localized one too.
-When the fuzzy or obsolete state has been resolved, localized resource updated to reflect changes in the original, it should be recorded as such. To do this, the entry for the resource is first manually deleted from <tt>l10n-track</tt> file, and then <tt>lbundle-check.py</tt> is rerun. This will recreate the entry with a fresh <tt>ok</tt> state.
+When the fuzzy or obsolete state has been resolved, localized resource updated to reflect changes in the original, it should be recorded as such. To do this, the entry for the resource is first manually deleted from <tt>l10n-track</tt> file (the whole line is removed), and then <tt>lbundle-check.py</tt> is rerun. This will recreate the entry with a fresh <tt>ok</tt> state.