Difference between revisions of "Development/Tools/Using kconf update"

m (Announce my upcoming work on this page, link to GitLab issue.)
m (Insert link to KDED)
(One intermediate revision by the same user not shown)
Line 1: Line 1:
{{Review|
 
* Update to KF5 (Mostly done)
 
* Double check info and paths (Indeed, most given paths are wrong. See https://invent.kde.org/frameworks/kconfig/-/issues/1)
 
* Rewrite and so on...
 
}}
 
 
 
{{Being_Edited}}
 
{{Being_Edited}}
  
==What it does==
+
== Introduction ==
<tt>kconf_update</tt> is a tool designed to update configuration files. Over time applications sometimes need to rearrange the way configuration options are stored. Since such an update shouldn't influence the configuration options that the user has selected, the application must take care that the options stored in the old way will still be honored.
+
'''kconf_update''' is a tool designed to update KConfig configuration files.
 
What used to happen is that the application looks up both the old and the new configuration option and then decides which one to use. This method has several drawbacks:
 
 
* The application may need to read more configuration files than strictly needed, resulting in a slower startup
 
* The application becomes bigger with code that will only be used once
 
  
kconf_update addresses these problems by offering a framework to update configuration files without adding code to the application itself.
+
__TOC__
 
==How it works==
 
Applications can install so called ''update files'' under {{path|$KDEDIR/share/apps/kconf_update}}. An update file has ''.upd'' as extension and contains instructions for transferring and converting configuration information from one place to another.
 
 
Updating the configuration happens automatically, either when KDE gets started or when <tt>kded</tt> detects a new update file in the above mentioned location.
 
 
Update files are separated into sections. Each section has an unique ID. When a section describing a configuration change has been applied, the ID will be stored in the file {{path|$KDEHOME/share/config/kconf_updaterc}}. This information is used to make sure that a configuration update is only performed once.
 
 
If you overwrite an existing update file with a new version that contains a new section, only the update instructions from this extra section will be performed.
 
  
==File format of the update file==
+
=== What it does ===
Empty lines or lines that start with '#' are considered comments.
+
Applications sometimes need to rearrange the way configuration options are stored.
Commas are used to seperate fields and may not occur as part of any field.
 
All of the keywords are case-sensitive.
 
 
The remainder of the file is parsed and executed sequentially from top to bottom. Each line can contain one entry. The following entries are recognized:
 
 
 
* Version=5
 
  
:Specifies which KDE Frameworks version this update file targets. At the time of writing the only valid value is 5, meaning modern (KDE Frameworks 5) kconf_update binaries will process the file. A file without Version value is skipped by kconf_update. A file with Version=5 may still be run on older kconf_updates!
+
To avoid losing the old configuration, the application can look up both the old and the new configuration, and then decide which one to use.
 +
But this means that the application contains additional code, which deals only with configuration of the past, and is only used once.
  
* Id=<id>
+
kconf_update addresses this problem by offering a framework to update configuration files, without adding complexity to the application itself.
  
:With <id> identifying the group of update entries that follows. Once a group of entries have been applied, their ID is stored and this group of entries will not be applied again. This ID has to be unique.
+
kconf_update is primarily used to move configuration entries from one place to another, but can also be used to modify the configuration data itself.
 
* File=<oldfile>,<newfile>
 
* File=<oldfile>
 
  
:Specifies that configuration information is read from <oldfile> and written to <newfile>.
+
=== How it works ===
 +
Applications can install so called ''update files'' in {{path|${KDE_INSTALL_KCONFUPDATEDIR}}}.
 +
An update file has {{path|.upd}} as extension, and contains instructions for transferring and converting configuration information.
 +
Update files are separated into sections, and each section has an unique ID.
  
:If you only specify <oldfile>, the information is read from as well as written to <oldfile>.
+
kconf_update is started automatically when [https://api.kde.org/frameworks/kded/html/index.html '''KDED'''] detects a new update file in the above mentioned location.
:Note that if the file does not exist at the time kconf_update first checks, no related update will be performed.
+
kconf_update will first check in its own configuration file ({{path|kconf_updaterc}}, usually located in {{path|~/.config/}}) which sections of the update file haven’t been processed yet.
+
When a section has been processed, its ID is stored in {{path|kconf_updaterc}} and the affected configuration files, to make sure it will not be processed again.
* Script=<_script>[,<interpreter>]
 
  
:All entries from <oldfile> are piped into <_script>.
+
If you overwrite an existing update file with a new version that contains a new section, only the update instructions from this new section will be processed.
  
:The output of <_script> is used as new entries for <newfile>. Existing entries can be deleted by adding lines with
+
If your application does not depend explicitely on KDED (e. g. if your target platform isn’t Linux or FreeBSD), kconf_update may not be started automatically.
# DELETE [group]key
+
You can start it manually using <code>KConfig::checkUpdate()</code>.
:in the output of the script. To delete a whole group use
+
Note that kconf_update might process your update multiple times simultaneously in this case.
# DELETEGROUP [group]
+
This can cause problems if your update file uses the <code>Script=</code> command.
+
See the [[#Note on side effects]] section for more details.
:<_script> should be installed into {{path|$(kde_datadir)/kconf_update}}, or kconf_update will not be able to find it.  
 
  
:It is not portable to install binary applications in {{path|$kde_datadir}}, so you have to stick with interpreted scripts like shell or perl. To make your scripts compatible with Windows, you should not use shell scripts.
+
=== How to use ===
 +
To use kconf_update in your application, you only need to write an update file and install it in the right place.
 +
An example update file is in the section [[#Minimal example update file]].
  
:It is also possible to install kconf_update applications in {{path|$(kde_bindir)/kconf_update_bin}}, which opens the door to kconf_update applications that are written in C++ and use Qt's powerful string API instead.
+
Add the following lines to your {{path|CMakeLists.txt}}:
  
:If the ''Script'' command was issued after a ''Group'' command the behavior is slightly different: All entries from <oldfile>/<oldgroup> are piped into <_script>. The output of script is used as new entries for <newfile>/<newgroup>, unless a different group is specified with ''[group]''. Existing entries can be deleted from <oldgroup> by adding lines with
+
<syntaxhighlight lang="cmake">
# DELETE key
+
# Extra CMake Modules (ECM) defines variables like KDE_INSTALL_KCONFUPDATEDIR:
:in the output of the script. To delete <oldgroup> use
+
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)
# DELETEGROUP
 
 
:<interpreter> can be something like "perl".
 
 
:It is also possible to have a script without specifying <oldfile> or <newfile>. In that case the script is run but it will not be fed any input and its output will simply be discarded.
 
 
 
*ScriptArguments=<arguments>
 
  
:If specified, the arguments will be passed to <_script>. IMPORTANT: Specify the ScriptArguments '''before''' the ''Script'' command.
+
# Makes your application compile with KConfig:
 
+
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
* Group=<oldgroup>,<newgroup>
+
    Config
* Group=<oldgroup>
+
    [...]
 +
)
  
:Specifies that configuration information is read from the group <oldgroup> and written to <newgroup>. If you only specify <oldgroup>, the information is read from as well as written to <oldgroup>. You can use <default> to specify keys that are not under any group.
+
# Installs the update file in the right place:
 
+
install(FILES okular.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
* RemoveGroup=<oldgroup>  
+
</syntaxhighlight>
  
:Specifies that <oldgroup> is removed entirely. This can be used to remove obsolete entries or to force a revert to default values.
+
{{Warning|
 
+
Before you deploy update files to your users, you should verify that they do what you want. Otherwise you will probably get bug reports, but you already missed your change to fix them.
* Options=<option1>, <option2>, ....
+
See the [[#Testing update files]] section.
 +
}}
  
:With this entry you can specify options that apply to the next ''Script'' command, ''Key'' or ''AllKeys'' entry (only to the first!). Possible options are:
+
=== Minimal example update file ===
+
Imagine your need to change the name of one entry in your configuration file.
::- "copy" Copy the configuration item instead of moving it. This means that the configuration item will not be deleted from <oldfile>/<oldgroup>.
+
This happened in '''Okular''', and this is the content of {{path|${KDE_INSTALL_KCONFUPDATEDIR}/okular.upd}}:
 
::- "overwrite" Normally, a configuration item is not moved if an item with the new name already exists. When this option is specified the old configuration item will overwrite any existing item.
 
 
 
* Key=<oldkey>,<newkey>
 
* Key=<oldkey>
 
  
:Specifies that configuration information is read from the key <oldkey> and written to <newkey>. If you only specify <oldkey>, the information is read from as well as written to <oldkey>.
+
<syntaxhighlight lang="ini">
 
+
#Configuration update for Okular
* AllKeys
+
Version=5
  
:Specifies that all configuration information in the selected group should be moved (All keys).
+
#Convert user-defined annotation tools to quick annotation tools
 
+
Id=annotation-toolbar
* AllGroups
+
File=okularpartrc
 +
Group=Reviews
 +
Key=AnnotationTools,QuickAnnotationTools
 +
</syntaxhighlight>
  
:Specifies that all configuration information from all keys in '''all''' groups should be moved.  
+
It will change the contents of {{path|okularpartrc}} from e. g.:
 
* RemoveKey=<oldkey>
 
  
:Specifies that <oldkey> is removed from the selected group. This can be used to remove obsolete entries or to force a revert to default values.
+
<syntaxhighlight lang="ini">
 +
[Reviews]
 +
AnnotationTools=highlight,underline
 +
</syntaxhighlight>
  
== Example update file ==
+
to:
  
 
<syntaxhighlight lang="ini">
 
<syntaxhighlight lang="ini">
# This is comment
+
[$Version]
 +
update_info=okular.upd:annotation-toolbar
  
Version=5
+
[Reviews]
Id=kde2.2
+
QuickAnnotationTools=highlight,underline
File=kioslaverc,kio_httprc
 
Group=Proxy Settings
 
Key=NoProxyFor
 
Key=UseProxy
 
Key=httpProxy,Proxy
 
Group=Cache Settings,Cache
 
Key=MaxCacheSize
 
Key=UseCache
 
Group=UserAgent
 
AllKeys
 
RemoveGroup=KDE
 
# End of file
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
The above update file extracts config information from the file {{path|kioslaverc}} and stores it into the file {{path|kio_httprc}}.
+
Note that it changed the name of the <code>AnnotationTools</code> entry in the <code>Reviews</code> group, and added a new <code>$Version</code> group to mark the <code>annotation-toolbar</code> update as done.
+
 
It reads the keys "NoProxyFor", "UseProxy" and "httpProxy" from the group "Proxy Settings" in the {{path|kioslaverc}} file. If any of these options are present they are written to the keys "NoProxyFor", "UseProxy" and "Proxy" (!) in the group "Proxy Settings" in the {{path|kio_httprc}} file.
+
To try this example, see the [[#Testing update files]] section.
+
 
It also reads the keys "MaxCacheSize" and "UseCache" from the group "Cache Settings" in the {{path|kioslaverc}} file and writes this information to the keys "MaxCacheSize" and "UseCache" in the group "Cache" (!) in the {{path|kio_httprc}} file.
+
== Testing update files ==
+
You can run kconf_update manually on a specific update file, to see how it will process it.
Then it takes all keys in the "UserAgent" group of the file "kioslaverc" and moves then to the "UserAgent" group in the {{path|kio_httprc}} file.
+
 
+
First, you need to locate your kconf_update executable.
Finally it removes the entire "KDE" group in the {{path|kioslaverc}} file.
+
(Try e. g. <code>locate kconf_update</code>.)
 +
If you can’t locate it, you can also compile it from the KConfig repository.
 +
Let’s assume your kconf_update executable is located at {{path|/usr/lib/kf5/kconf_update}}.
  
== Further Examples ==
+
kconf_update offers the command line options <code>--debug</code> and <code>--testmode</code>.
 +
<br/><code>--debug</code> enables helpful debugging output on the command line.
 +
<br/><code>--testmode</code> activates <code>QStandardPaths::setTestModeEnabled()</code>, so kconf_update will not update your actual configuration files, but files you have placed in a certain test directory.
 +
See the <code>QStandardPaths::GenericConfigLocation</code> documentation for your platform.
 +
Let’s assume this test directory is {{path|~/.qttest/config/}}.
  
The best way to learn how to write scripts and update files is to look at existing ones, for example the ones of [http://websvn.kde.org/trunk/KDE/kdepim/kmail/kconf_update/ KMail].
+
Now place your update file and a test configuration file in your working directory.
 +
Assuming you are testing the file from [[#Minimal example update file]], you run:
  
==Debugging and testing==
+
{{Input|1=<nowiki>
===Testing update entries===
+
mkdir -p ~/.qttest/config/
To test an <code>*.upd</code> file it may be convenient to manually run kconf_update, usually located at <code>/usr/lib/kf5/kconf_update</code>. This is particularly relevant if you are testing an update file for a program installed in a custom prefix that is not the same as the one of kded, because kde will not run kconf_update automatically.
+
cp okularpartrc ~/.qttest/config/
 +
/usr/lib/kf5/kconf_update --debug --testmode ./okular.upd
 +
</nowiki>}}
  
For example if you want to test the Okular update file okular.upd
+
Then you should get an output like:
  
 
{{Output|1=<nowiki>
 
{{Output|1=<nowiki>
#Configuration update for Okular
+
Automatically enabled the debug logging category kf.config.kconf_update
Version=5
+
kf5.kconfig.update: Checking update-file "./okular.upd" for new updates
 +
kf5.kconfig.update: "okular.upd" : Found new update "annotation-toolbar"
 +
kf5.kconfig.update: "okular.upd" : Updating "okularpartrc" : "Reviews" : "QuickAnnotationTools" to "highlight,underline"
 +
kf5.kconfig.update: "okular.upd" : Removing "okularpartrc" : "Reviews" : "AnnotationTools" , moved.
 +
</nowiki>}}
 +
 
 +
Now you should open {{path|~/.qttest/config/okularpartrc}} to verify that the update was performed correctly.
 +
If you see the new <code>$Version</code> group there, but <code>AnnotationTools</code> was not renamed, the update was performed, but something is wrong with your update instructions.
 +
If you don’t see the new <code>$Version</code> group, the update didn’t affect the correct configuration file.
 +
 
 +
{{Tip|
 +
For the first tests, it is convenient to write a configuration file that contains only the entries you are interested in (like in [[#Minimal example update file]]).
 +
But you should also perform a test with a configuration file from an actual installation of your application.
 +
}}
 +
 
 +
{{Tip|
 +
If you want to test the update file again, you need to restore an old version of the configuration file first.
 +
Additionally, you need to open {{path|kconf_updaterc}}, and remove the <code>okular.upd</code> group, otherwise kconf_update will skip your update file.
 +
 
 +
Since Frameworks version KF 5.76, kconf_update will ignore {{path|kconf_updaterc}} while in <code>--testmode</code>
 +
}}
 +
 
 +
=== Testing installed update files ===
 +
When you think your update file works fine, you may try to completely compile and install the new version of your application.
 +
After you have run e. g. <code>make install</code>, your update file should show up in the correct directory.
 +
KDED will detect this, and run kconf_update.
 +
Your installed configuration file should now be updated, and you can open it to verify the update.
 +
 
 +
However, if you find problems now, you probably want to let the update run again.
 +
If you install a new update file, KDED will run kconf_update again, but kconf_update will not perform the update.
 +
What you need to do is:
 +
 
 +
* Restore an old version of the configuration file, so there is something to update again. The old version shouldn’t contain the <code>$Version</code> group.
 +
* Locate {{path|kconf_updaterc}} (e. g. with the command <code>locate kconf_updaterc</code>), and remove the group named by your update file.
 +
 
 +
Now, kconf_update will process the update again, as soon as you overwrite the update file.
 +
Since <code>make install</code> won’t overwrite the file with itself, you need to manually delete it first.
 +
 
 +
You can also try to run kconf_update manually, e. g. with the command <code>/usr/lib/kf5/kconf_update --debug</code> (without other arguments).
  
#Convert user-defined annotation tools to quick annotation tools
+
==== Debug output while running in the background ====
Id=annotation-toolbar
+
Because kconf_update will now run in the background, you will not directly get debug output.
File=okularpartrc
+
After KDE Frameworks 5.57, debug output goes through the category <code>kf.config.kconf_update</code>.
Group=Reviews
+
If this category is enabled, debug output will be printed to ''stderr'' and consequently end up in {{path|xsession-errors}} or {{path|wayland-errors}}.
Key=AnnotationTools,QuickAnnotationTools
+
See the <code>QLoggingCategory</code> documentation for how to enable this logging category.
</nowiki>}}
+
Probably you can add the following group to a configuration file like {{path|/usr/share/qt5/qtlogging.ini}}:
  
you can first make a copy of the config file you want to target to the test path {{Path | .qttest/}}
 
 
{{Input|1=<nowiki>
 
{{Input|1=<nowiki>
mkdir -p .qttest/config/
+
[Rules]
cp ~/.config/okularpartrc ~/.qttest/config/
+
kf.config.kconf_update=true
 
</nowiki>}}
 
</nowiki>}}
  
then you can manually run kconf_update from the folder where okular.upd is located
+
If Qt was built with support for logging to system log facilities (such as systemd-journald), the output may be found there instead.
 +
Before 5.57, the output was always sent to ''stderr'' and may be found in the aforementioned errors files.
 +
 
 +
== File format of the update file ==
 +
Each line of the update file can contain a command (like <code>File=okularpartrc</code>) or a comment (like <code># Hello!</code>).
 +
Commands are processed sequentially from top to bottom.
 +
 
 +
=== Description of the minimal example update file ===
 +
In the section [[#Minimal example update file]], we see the following lines:
 +
 
 +
; <code>#Configuration update for Okular</code>
 +
: Empty lines and lines that start with <code>#</code> are considered comments.
 +
 
 +
; <code>Version=5</code>
 +
: Specifies that this update file is written for the KF5 version of kconf_update.
 +
: Every update file must start with this command.
 +
 
 +
; <code>Id=annotation-toolbar</code>
 +
: Starts an update section called ''annotation-toolbar''.
 +
 
 +
; <code>File=okularpartrc</code>
 +
: Tells kconf_update which configuration file shall be affected.
 +
 
 +
; <code>Group=Reviews</code>
 +
: Tells kconf_update that it shall operate on the ''Reviews'' group of the configuration file.
 +
 
 +
; <code>Key=AnnotationTools,QuickAnnotationTools</code>
 +
: Tells kconf_update that it shall read from the entry ''AnnotationTools'' and write to the entry ''QuickAnnotationTools''.
 +
: In this case, this causes the entry to be renamed.
 +
 
 +
=== Update file structure ===
 +
Update files are structured as follows:
 +
 
 +
* ''Version=5'' (required)
 +
* any number of update sections, each started by an ''Id='' command
 +
** any number of ''Script='' and ''ScriptArguments='' commands
 +
** or any number of ''File='' commands, followed by:
 +
*** any number of ''Group='', ''Key='', ''AllKeys'', ''RemoveGroup='', ''RemoveKey='', ''Options='', ''Script='', ''ScriptArguments='' commands
 +
*** ''AllGroups'' (optional)
 +
 
 +
=== Update file command reference ===
 +
All commands are case-sensitive.
 +
Commas are used to separate parameters, and may not occur as part of any parameter.
 +
Parameters in square brackets are optional.
 +
Parameters are referenced in angle brackets, sometimes parameters of preceeding commands are referenced.
 +
(Square brackets and angle brackets are not written in the actual update file.)
 +
 
 +
The following commands are available:
 +
 
 +
; Version=5
 +
: This must be the first command in any update file.
 +
: Specifies which KDE Frameworks version this update file targets. At the time of writing the only valid value is 5, meaning modern (KDE Frameworks 5) kconf_update binaries will process the file. A file with ''Version=5'' may still be run on older kconf_updates!
 +
 
 +
; Id=<id>
 +
: Starts a new update section. All commands until the next ''Id='' command are processed as part of this section. If ''<id>'' is stored in {{path|kconf_updaterc}}, this section will be skipped. When this section has been processed, ''<id>'' will be stored in {{path|kconf_updaterc}}.
 +
: ''<id>'' has to be unique. You should give your section a name that describes why this update necessary. You can also include the year of your changes, like in ''Id=annotation-toolbar-2020''.
 +
 
 +
; File=<oldfile>[,<newfile>]
 +
: Specifies that configuration information is read from ''<oldfile>'' and written to ''<newfile>''. If you only specify ''<oldfile>'', the information is read from as well as written to <oldfile>.
 +
: If ''<oldfile>'' contains the ''<id>'' of the current section, the current section will be skipped. Otherwise, ''<id>'' is stored in ''<oldfile>'' and ''<newfile>''.
 +
: If ''<oldfile>'' does not exist at the time kconf_update first checks, the current section will be skipped.
 +
: Also note the [[#Updating multiple files per section]] section.
 +
 
 +
; Group=<oldgroup>[,<newgroup>]
 +
: Specifies that configuration information is read from the group ''<oldgroup>'' and written to ''<newgroup>''. If you only specify ''<oldgroup>'', the information is read from as well as written to ''<oldgroup>''.
 +
: You can use <code><default></code> to select the top-level group.
 +
 
 +
; Key=<oldkey>[,<newkey>]
 +
: Moves the configuration entry ''<oldkey>'' from ''<oldfile>/<oldgroup>'' to ''<newfile>/<newgroup>''.
 +
: If you  specify ''<newkey>'', the entry is additionally renamed.
 +
: If you did not specify ''<newfile>'' nor ''<newgroup>'' nor ''Options=copy'', the entry is only renamed.
 +
 
 +
; AllGroups
 +
: Moves all groups (including all entries) from ''<oldfile>'' to ''<newfile>''.
 +
: Note that the top-level group is not included. To include the top-level group, you need to use the ''AllKeys'' command additionally.
 +
 
 +
; AllKeys
 +
: Moves all entries from ''<oldgroup>'' to ''<newgroup>''.
 +
 
 +
; RemoveGroup=<oldgroup>
 +
: Removes ''<oldgroup>'' from ''<oldfile>''.
 +
: This can be used to remove obsolete entries or to force a revert to default values.
 +
 
 +
; RemoveKey=<oldkey>
 +
: Removes ''<oldkey>'' from ''<oldfile>/<oldgroup>''.
 +
: This can be used to remove an obsolete entriy or to force a revert to default values.
 +
 
 +
; Options=<option1>, <option2>, ...
 +
: With this entry you can specify options that apply to the next ''Script='', ''Key='', ''AllKeys'', or ''AllGroups'' command (only to the first). Note that ''Script='' probably ignores the options, see the [[#Script output format]] section.
 +
: Possible options are:
 +
:; copy
 +
:: Copy the configuration item instead of moving it. This means that the configuration item will not be deleted from ''<oldfile>'' or ''<oldgroup>''.
 +
:; overwrite
 +
:: Normally, configuration entries are not moved if an entry with the new name already exists. When this option is specified, any existing entries are overwritten.
 +
 
 +
; Script=<_script>[,<interpreter>]
 +
; ScriptArguments=<arguments>
 +
: See the [[#Update script commands]] section.
 +
 
 +
== Using external update scripts ==
 +
kconf_update provides commands to move configuration information around, but it can not manipulate the configuration data itself.
 +
Using external, so called ''update scripts'', configuration data can be manipulated through kconf_update.
 +
 
 +
The concept is that old configuration entries are piped into a script.
 +
The output of the script is then interpreted as new configuration entries, which are merged to ''<newfile>'' or ''<newgroup>''.
 +
 
 +
It is also possible to use scripts outside the context of a ''File='' command, then it will not get any input, and the output is ignored.
 +
See the section [[#Note on side effects]] about this.
 +
 
 +
=== Update script commands ===
 +
The following two commands are available to run update scripts:
 +
 
 +
; Script=<_script>[,<interpreter>]
 +
: Executes the script file ''<_script>'' using the command ''<interpreter>''. If ''<interpreter>'' is not given, executes the compiled script file ''<_script>'' directly. You can not pass arguments with this command.
 +
: If in the context of a ''File='' command, all groups of ''<oldfile>'' (excluding the top-level group) will be piped into the script. If in the context of a ''Group='' command, all entries of ''<oldgroup>'' will be piped into the script. You can use <code>Group=<default></code> to process top-level entries.
 +
: The output of <_script> is merged into ''<newfile>'' or ''<newgroup>''.
 +
 
 +
; ScriptArguments=<arguments>
 +
: ''<arguments>'' is passed to ''<_script>'' of the next ''Script='' command (only to the first).
 +
: ''<arguments>'' is everything from the <code>=</code> sign to the end of the line. Escape sequences are not possible.
 +
: Note that you must specify ''ScriptArguments='' before ''Script=''.
 +
 
 +
=== Script output format ===
 +
The output of the script is parsed as configuration file, and merged to ''<newfile>'' or ''<newgroup>''. If in the context of ''File='', script output is merged to the top-level group of ''<newfile>''. If in the context of ''Group='', the group context is initialized to ''<newgroup>''.
 +
 
 +
These lines in the script output are recognized by kconf_update:
 +
 
 +
; [<groupname>]
 +
: Sets the group context to ''<groupname>''. This will not create a subgroup, even in the context of ''Group=''.
 +
: You can use <code>[<default>]</code> to select the top-level group. However, this will not work in context of ''Group=''.
 +
 
 +
; <key>=<value>
 +
: Creates a new entry in the current group context.
 +
: Note that existing entries will probably not be overridden, even with ''Options=override'' specified.
 +
 
 +
; # DELETE <key>
 +
: Deletes the entry ''<key>'' in the current group context, but from ''<oldfile>''.
 +
 
 +
; # DELETE [<group>]<key>
 +
: Deletes the entry ''<key>'' from the group ''<oldfile>/<group>''.
 +
: Note that this can change the group context, you should explicitely set the group context afterwards.
 +
 
 +
; # DELETEGROUP <group>
 +
: Deletes the whole group ''<group>'' (not a subgroup) from ''<oldfile>''.
 +
: Note that this can change the group context, you should explicitely set the group context afterwards.
 +
 
 +
=== Providing the script files ===
 +
Script files should be installed in {{path|${KDE_INSTALL_KCONFUPDATEDIR}}}. You should install only actual scripts here, not binary executables.
 +
 
 +
You should choose a script language whose interpreter will be available on your target platform.
 +
E. g. if your target platforms include Windows, you should not use shell scripts.
 +
 
 +
Alternatively, you can use “compiled scripts”, which are executable binaries installed in {{path|${KDE_INSTALL_LIBDIR}/kconf_update_bin/}}. You can e. g. write your update script in C++ and use all the Qt API and other libraries you need, and compile and install the binary together with your application.
 +
 
 +
=== Example interpreted update script ===
 +
Imagine you want to switch from a ''DoNotUseTabs'' configuration entry to a ''UseTabs'' configuration entry.
 +
This can be easily done using this small Python script:
 +
 
 +
<syntaxhighlight lang="python">
 +
import fileinput
 +
 
 +
for line in fileinput.input():
 +
    if line.startswith("DoNotUseTabs=false"):
 +
        print("UseTabs=true")
 +
    if line.startswith("DoNotUseTabs=true"):
 +
        print("UseTabs=false")
 +
 
 +
print("# DELETE DoNotUseTabs")
 +
</syntaxhighlight>
 +
 
 +
Your update file could look like this:
 +
 
 +
<syntaxhighlight lang="ini">
 +
Version=5
 +
 
 +
Id=no-more-double-negation-2020
 +
File=myApprc
 +
Group=UserInterface
 +
Script=myApp_no_more_double_negation_2020.py,python3
 +
</syntaxhighlight>
 +
 
 +
Your CMakeLists.txt could look like this:
 +
 
 +
<syntaxhighlight lang="cmake">
 +
[...]
 +
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)
 +
[...]
 +
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
 +
    Config
 +
    [...]
 +
)
 +
[...]
 +
install(FILES conf/update/myApp_configFiles.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
 +
install(FILES conf/update/myApp_no_more_double_negation_2020.py DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
 +
</syntaxhighlight>
 +
 
 +
Don’t forget that your appliation now depends on python3.
 +
 
 +
=== Example compiled update script ===
 +
Imagine your application made the state of a KActionMenu user configurable.
 +
Because you changed to the new QToolButton::ToolButtonPopupMode API, you need to migrate from the old ''delayed'' and ''stickyMenu'' entries to the new ''popupMode'' entry.
 +
You can do that with the following compiled update script, which uses the KConfig framework directly:
 +
 
 +
<syntaxhighlight lang="cpp">
 +
#include <KConfig>
 +
#include <KConfigGroup>
 +
 
 +
#include <QTemporaryFile>
 +
#include <QTextStream>
 +
 
 +
int main(int argc, char **argv)
 +
{
 +
    // Create a KConfig object from stdin.
 +
    QFile inputFile;
 +
    inputFile.open(stdin, QIODevice::ReadOnly);
 +
    QTemporaryFile inputFileForKConfig("popupmode_api_tmp");
 +
    inputFileForKConfig.open();
 +
    inputFileForKConfig.write(inputFile.readAll());
 +
    inputFileForKConfig.close();
 +
    KConfig inputConfig(inputFileForKConfig.fileName(), KConfig::SimpleConfig);
 +
 
 +
    // Create a text stream to stdout.
 +
    QTextStream outputStream(stdout, QIODevice::WriteOnly);
 +
 
 +
    // Unify delayed and stickyMenu.
 +
    // delayed => 0 (DelayedPopup)
 +
    // !delayed && stickyMenu => 2 (InstantPopup)
 +
    // !delayed && !stickyMenu => 1 (MenuButtonPopup) (defined as default with KConfigXT)
 +
    KConfigGroup rootGroup = inputConfig.group("");
 +
    bool old_delayed = rootGroup.readEntry<bool>("delayed", true);
 +
    bool old_stickyMenu = rootGroup.readEntry<bool>("stickyMenu", true);
 +
    int new_popupMode = (old_delayed) ? 0 : (old_stickyMenu) ? 2 : 1;
 +
    if (new_popupMode != 1) {
 +
        outputStream << "popupMode=" << new_popupMode << Qt::endl;
 +
    }
 +
    outputStream << "# DELETE delayed" << Qt::endl;
 +
    outputStream << "# DELETE stickyMenu" << Qt::endl;
 +
 
 +
    return 0;
 +
}
 +
</syntaxhighlight>
 +
 
 +
Your update file could look like this:
 +
 
 +
<syntaxhighlight lang="ini">
 +
Version=5
 +
 
 +
Id=toolbuttonpopupmode-api-2020
 +
File=myApprc
 +
Group=ActionMenuState
 +
Script=myApp-toolbuttonpopupmode-api-2020
 +
</syntaxhighlight>
 +
 
 +
Your CMakeLists.txt could look like this:
 +
 
 +
<syntaxhighlight lang="cmake">
 +
[...]
 +
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)
 +
[...]
 +
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
 +
    Config
 +
    [...]
 +
)
 +
[...]
 +
install(FILES myApp_configFiles.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
 +
 
 +
add_executable(myApp-toolbuttonpopupmode-api-2020 conf/update/scripts/popupmode.cpp)
 +
target_link_libraries(myApp-toolbuttonpopupmode-api-2020
 +
    KF5::ConfigCore
 +
)
 +
install(TARGETS myApp-toolbuttonpopupmode-api-2020 DESTINATION ${KDE_INSTALL_LIBDIR}/kconf_update_bin/)
 +
</syntaxhighlight>
 +
 
 +
Because you don’t need an interpreter, you didn’t add any dependency to your application.
 +
 
 +
=== Testing external update scripts ===
 +
The first step to test external update scripts is to call them directly, and feed them with your configuration files.
 +
 
 +
For example, you can test the script from [[#Example interpreted update script]] with this command:
  
 
{{Input|1=<nowiki>
 
{{Input|1=<nowiki>
/usr/lib/kf5/kconf_update ./okular.upd --debug --testmode
+
cat myApprc_UserInterface | python3 myApp_no_more_double_negation_2020.py
 
</nowiki>}}
 
</nowiki>}}
  
The <code>--testmode</code> flag allows you to use test directories under {{Path | ~/.qttest }} to stay away the user's real files, while the <code>--debug</code> flag enables useful debugging messages displaying what kconf_update is doing.
+
If the script works correctly, you should get output like this:
  
If the rule is applied correctly, the command displays the following output
 
 
{{Output|1=<nowiki>
 
{{Output|1=<nowiki>
Automatically enabled the debug logging category kf5.kconfig.update
+
UseTabs=true
kf5.kconfig.update: Checking update-file "/opt/install_prefix/share/kconf_update/okular.upd" for new updates
+
# DELETE DoNotUseTabs
kf5.kconfig.update: "okular.upd" : Found new update "annotation-toolbar"
+
</nowiki>}}
kf5.kconfig.update: "okular.upd" : Updating "okularpartrc" : "Reviews" : "QuickAnnotationTools" to "<tool type=\"note-linked\" id=\"1\">..."
+
 
kf5.kconfig.update: "okular.upd" : Removing "okularpartrc" : "Reviews" : "AnnotationTools" , moved.
+
The second step is to test your update file together with the script.
 +
This means, you need to place the script file in a place where kconf_update will find it while in <code>--testmode</code>.
 +
kconf_update will look for the script in <code>QStandardPaths::GenericDataLocation</code>.
 +
Let’s assume this path is {{path|~/.qttest/share/}}, then you need to place your script in {{path|~/.qttest/share/kconf_update/}}.
 +
 
 +
Place your update file, your update script, and your configuration file in your working directory, and run:
 +
 
 +
{{Input|1=<nowiki>
 +
mkdir -p ~/.qttest/config/
 +
cp myApprc ~/.qttest/config/
 +
mkdir -p ~/.qttest/share/kconf_update/
 +
cp myApp_no_more_double_negation_2020.py ~/.qttest/share/kconf_update/
 +
/usr/lib/kf5/kconf_update --debug --testmode ./myApp_configFiles.upd
 
</nowiki>}}
 
</nowiki>}}
  
The configuration file {{Path | ~/.qttest/config/okularpartrc}} should now have the following key indicating that it has been updated using the update rule okular.upd:annotation-toolbar
+
You should get an output like:
  
 
{{Output|1=<nowiki>
 
{{Output|1=<nowiki>
[$Version]
+
Automatically enabled the debug logging category kf.config.kconf_update
update_info=okular.upd:annotation-toolbar
+
kf.config.kconf_update: Checking update-file "./myApp_configFiles.upd" for new updates
 +
kf.config.kconf_update: "demo9.upd" : Found new update "no-more-double-negation-2020"
 +
kf.config.kconf_update: "demo9.upd" : Running script "myApp_no_more_double_negation_2020.py"
 +
kf.config.kconf_update: Script input stored in "/tmp/kconf_update.lKlJpe"
 +
kf.config.kconf_update: About to run "/usr/bin/python3"
 +
kf.config.kconf_update: Script contents is:
 +
"[...]"
 +
kf.config.kconf_update: Successfully ran "/usr/bin/python3"
 +
kf.config.kconf_update: Script output stored in "/tmp/kconf_update.CCeaBP"
 +
kf.config.kconf_update: Script output is:
 +
"UseTabs=true\n# DELETE DoNotUseTabs\n"
 +
kf.config.kconf_update: "demo9.upd" : Script removes "myApprc" : ("UserInterface") : "DoNotUseTabs"
 
</nowiki>}}
 
</nowiki>}}
  
If you now run kconf_update, the update rule will not be applied again because kconf_update registered that it has already been applied. To test the update rule again you have to:
+
If you want to test compiled update scripts, the workflow is the same, except that you may place your compiled update script in one of your {{path|$PATH}} directories instead.
 +
 
 +
=== Note on side effects ===
 +
External update scripts can cause side effects.
 +
(E. g. rebuilding the database of another service.)
 +
This is an intended use case; you can use ''Script='' in the context of ''Id='', then no script input or output is processed.
 +
 
 +
Nevertheless, you should be careful if you use update scripts for side effects.
 +
If your update scripts cause side effects, you probably want these side effects to happen at most once.
 +
KDED will run the update only once, but if <code>KConfig::checkUpdate()</code> is involved, this is not guaranteed.
  
# copy the old configuration file {{Path |~/.config/okularpartrc}} over the updated one {{Path |~/.qttest/config/okularpartrc}}
+
Consider separating updates into two update files, where one update file performs only deterministic configuration file updates, and the other update file causes only side effects.
# remove the <code>[$Version]</code> settings group from the configuration file
+
This way, you can savely call <code>KConfig::checkUpdate()</code> to guarantee configuration file updates, while the other update file is processed at most once by KDED.
# remove the okular:annotation-toolbar entry from {{Path | ~/.config/kconf_updater}}
 
  
 +
== Updating multiple files per section ==
 +
You can use multiple ''File='' commands in the context of one ''Id='' command, then this update section will sequentially update multiple files.
 +
This can be used to split a single configuration file into multiple configuration files, or to join multiple configuration files into a single configuration file.
  
===Debugging update scripts===
+
However, in the latter case you will have different ''<oldfile>'' parameters.
If you are developing a kconf_update script and want to test or debug it you need to make sure kconf_update runs again after each of your changes. There are a number of ways to achieve this.
+
You should be aware that kconf_update will stop processing a section, when it encounters an ''<oldfile>'' which already has this update applied.
+
In most cases, this shouldn’t be a problem.
The easiest is to not install the kconf_update script in the first place, but manually call it through a pipe. If you want to test the update script for your application KHello's config file {{path|khellorc}}, you can test by using
+
If it is a problem, you should consider to use multiple sections with only one ''File='' command each.
 +
 
 +
== Advanced example update file ==
 +
<syntaxhighlight lang="ini">
 +
Version=5
  
<syntaxhighlight lang="bash">
+
Id=kde2.2
cat ~/.local/share/config/khellorc | khello_conf_update.sh
+
File=kioslaverc,kio_httprc
 +
 
 +
Group=Proxy Settings
 +
Key=NoProxyFor
 +
Key=UseProxy
 +
Key=httpProxy,Proxy
 +
 
 +
Group=Cache Settings,Cache
 +
Key=MaxCacheSize
 +
Key=UseCache
 +
 
 +
Group=UserAgent
 +
AllKeys
 +
 
 +
RemoveGroup=KDE
 
</syntaxhighlight>
 
</syntaxhighlight>
 
(assuming {{path|khello_conf_update.sh}} is the kconf_update script and {{path|~/.local/share}} is your $XDG_DATA_HOME). This is easier than installing every time, but has the obvious downside that you need to 'parse' your script's output yourself instead of letting kconf_update do it and check the resulting output file.
 
 
After 'make install' the kconf_update script is run by kded, but it does so only once. This is of course the idea behind it, but while developing it can be a problem. You can increase the revision number for each subsequent run of 'make install' to force a new kconf_update run, but there's a better approach that doesn't skyrocket the version number for a mediocre debug session.
 
 
kded doesn't really ignore scripts that it has already run right away. Instead it checks the affected config file every time a .upd file is added or changed. The reason it still doesn't run again on your config file lies in the traces kconf_update leaves behind: it adds a special config group '[$Version]' with a key 'update_info'. This key lists all kconf_update scripts that have already been run on this config file. Just remove your file's entry, 'make install', and kconf_update will happily run your script again, without you having to increase the version number.
 
 
After KDE Frameworks 5.57 status updates of kconf_update are run through QDebug and may be controlled in any way other QDebugs may be controlled. Notably, by default, debug output is not generated unless the category <code>kf5.kconfig.update</code> is explicitly enabled and will be printed to stderr and consequently end up in {{path|xsession-errors}} or {{path|wayland-errors}}. If Qt was built with support for logging to system log facilities (such as  systemd-journald) the output may be found there instead.
 
Before 5.57 the output was always sent to stderr and may be found in the aforementioned errors files.
 
  
==Common Problems==
+
The above update file extracts config information from the file {{path|kioslaverc}} and stores it into the file {{path|kio_httprc}}.
 +
 
 +
It reads the keys ''NoProxyFor'', ''UseProxy'' and ''httpProxy'' from the group ''Proxy Settings'' in the {{path|kioslaverc}} file.
 +
If any of these options are present they are written to the keys ''NoProxyFor'', ''UseProxy'' and ''Proxy'' (!) in the group ''Proxy Settings'' in the {{path|kio_httprc}} file.
 +
 
 +
It also reads the keys ''MaxCacheSize'' and ''UseCache'' from the group ''Cache Settings'' in the {{path|kioslaverc}} file and writes this information to the group ''Cache'' (!) in the {{path|kio_httprc}} file.
 +
 
 +
Then it takes all keys in the ''UserAgent'' group of the file {{path|kioslaverc}} and moves then to the ''UserAgent'' group in the {{path|kio_httprc}} file.
 +
 
 +
Finally it removes the entire ''KDE'' group from the {{path|kioslaverc}} file.
 +
 
 +
== Further example update files ==
 +
Another way to learn how to write scripts and update files is to look at existing ones.
 +
 
 +
With the following command, you can look which applications on your system have installed update files:
 +
 
 +
{{Input|1=<nowiki>
 +
locate kconf_update
 +
</nowiki>}}
 +
 
 +
You can also search through various KDE projects with this link:
 +
 
 +
https://lxr.kde.org/search?_filestring=.%2B%5C.upd%5Cz&_advanced=1
  
'''kconf_update refuses to update an entry'''
+
== Common Problems ==
  
If you change the value of an entry without changing the key or file, make sure to tell kconf_update that it should overwrite the old entry by adding "Options=overwrite".
+
; kconf_update refuses to update an entry
 +
: If you change the value of an entry without changing the key or file, make sure to tell kconf_update that it should overwrite the old entry by adding ''Options=overwrite''.

Revision as of 20:47, 26 October 2020

Being edited.png
 
Currently Being Edited
This page is currently being edited.
If this notice persists for an unreasonable time, please either notify irc.freenode.org #kde-www or report on Annewilson's Talk page.


Introduction

kconf_update is a tool designed to update KConfig configuration files.

What it does

Applications sometimes need to rearrange the way configuration options are stored.

To avoid losing the old configuration, the application can look up both the old and the new configuration, and then decide which one to use. But this means that the application contains additional code, which deals only with configuration of the past, and is only used once.

kconf_update addresses this problem by offering a framework to update configuration files, without adding complexity to the application itself.

kconf_update is primarily used to move configuration entries from one place to another, but can also be used to modify the configuration data itself.

How it works

Applications can install so called update files in ${KDE_INSTALL_KCONFUPDATEDIR}. An update file has .upd as extension, and contains instructions for transferring and converting configuration information. Update files are separated into sections, and each section has an unique ID.

kconf_update is started automatically when KDED detects a new update file in the above mentioned location. kconf_update will first check in its own configuration file (kconf_updaterc, usually located in ~/.config/) which sections of the update file haven’t been processed yet. When a section has been processed, its ID is stored in kconf_updaterc and the affected configuration files, to make sure it will not be processed again.

If you overwrite an existing update file with a new version that contains a new section, only the update instructions from this new section will be processed.

If your application does not depend explicitely on KDED (e. g. if your target platform isn’t Linux or FreeBSD), kconf_update may not be started automatically. You can start it manually using KConfig::checkUpdate(). Note that kconf_update might process your update multiple times simultaneously in this case. This can cause problems if your update file uses the Script= command. See the #Note on side effects section for more details.

How to use

To use kconf_update in your application, you only need to write an update file and install it in the right place. An example update file is in the section #Minimal example update file.

Add the following lines to your CMakeLists.txt:

# Extra CMake Modules (ECM) defines variables like KDE_INSTALL_KCONFUPDATEDIR:
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)

# Makes your application compile with KConfig:
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
    Config
    [...]
)

# Installs the update file in the right place:
install(FILES okular.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
Warning
Before you deploy update files to your users, you should verify that they do what you want. Otherwise you will probably get bug reports, but you already missed your change to fix them. See the #Testing update files section.


Minimal example update file

Imagine your need to change the name of one entry in your configuration file. This happened in Okular, and this is the content of ${KDE_INSTALL_KCONFUPDATEDIR}/okular.upd:

#Configuration update for Okular
Version=5

#Convert user-defined annotation tools to quick annotation tools
Id=annotation-toolbar
File=okularpartrc
Group=Reviews
Key=AnnotationTools,QuickAnnotationTools

It will change the contents of okularpartrc from e. g.:

[Reviews]
AnnotationTools=highlight,underline

to:

[$Version]
update_info=okular.upd:annotation-toolbar

[Reviews]
QuickAnnotationTools=highlight,underline

Note that it changed the name of the AnnotationTools entry in the Reviews group, and added a new $Version group to mark the annotation-toolbar update as done.

To try this example, see the #Testing update files section.

Testing update files

You can run kconf_update manually on a specific update file, to see how it will process it.

First, you need to locate your kconf_update executable. (Try e. g. locate kconf_update.) If you can’t locate it, you can also compile it from the KConfig repository. Let’s assume your kconf_update executable is located at /usr/lib/kf5/kconf_update.

kconf_update offers the command line options --debug and --testmode.
--debug enables helpful debugging output on the command line.
--testmode activates QStandardPaths::setTestModeEnabled(), so kconf_update will not update your actual configuration files, but files you have placed in a certain test directory. See the QStandardPaths::GenericConfigLocation documentation for your platform. Let’s assume this test directory is ~/.qttest/config/.

Now place your update file and a test configuration file in your working directory. Assuming you are testing the file from #Minimal example update file, you run:

mkdir -p ~/.qttest/config/
cp okularpartrc ~/.qttest/config/
/usr/lib/kf5/kconf_update --debug --testmode ./okular.upd

Then you should get an output like:

Automatically enabled the debug logging category kf.config.kconf_update
kf5.kconfig.update: Checking update-file "./okular.upd" for new updates
kf5.kconfig.update: "okular.upd" : Found new update "annotation-toolbar"
kf5.kconfig.update: "okular.upd" : Updating "okularpartrc" : "Reviews" : "QuickAnnotationTools" to "highlight,underline"
kf5.kconfig.update: "okular.upd" : Removing "okularpartrc" : "Reviews" : "AnnotationTools" , moved.

Now you should open ~/.qttest/config/okularpartrc to verify that the update was performed correctly. If you see the new $Version group there, but AnnotationTools was not renamed, the update was performed, but something is wrong with your update instructions. If you don’t see the new $Version group, the update didn’t affect the correct configuration file.

Tip
For the first tests, it is convenient to write a configuration file that contains only the entries you are interested in (like in #Minimal example update file). But you should also perform a test with a configuration file from an actual installation of your application.


Tip
If you want to test the update file again, you need to restore an old version of the configuration file first.

Additionally, you need to open kconf_updaterc, and remove the okular.upd group, otherwise kconf_update will skip your update file.

Since Frameworks version KF 5.76, kconf_update will ignore kconf_updaterc while in --testmode


Testing installed update files

When you think your update file works fine, you may try to completely compile and install the new version of your application. After you have run e. g. make install, your update file should show up in the correct directory. KDED will detect this, and run kconf_update. Your installed configuration file should now be updated, and you can open it to verify the update.

However, if you find problems now, you probably want to let the update run again. If you install a new update file, KDED will run kconf_update again, but kconf_update will not perform the update. What you need to do is:

  • Restore an old version of the configuration file, so there is something to update again. The old version shouldn’t contain the $Version group.
  • Locate kconf_updaterc (e. g. with the command locate kconf_updaterc), and remove the group named by your update file.

Now, kconf_update will process the update again, as soon as you overwrite the update file. Since make install won’t overwrite the file with itself, you need to manually delete it first.

You can also try to run kconf_update manually, e. g. with the command /usr/lib/kf5/kconf_update --debug (without other arguments).

Debug output while running in the background

Because kconf_update will now run in the background, you will not directly get debug output. After KDE Frameworks 5.57, debug output goes through the category kf.config.kconf_update. If this category is enabled, debug output will be printed to stderr and consequently end up in xsession-errors or wayland-errors. See the QLoggingCategory documentation for how to enable this logging category. Probably you can add the following group to a configuration file like /usr/share/qt5/qtlogging.ini:

[Rules]
kf.config.kconf_update=true

If Qt was built with support for logging to system log facilities (such as systemd-journald), the output may be found there instead. Before 5.57, the output was always sent to stderr and may be found in the aforementioned errors files.

File format of the update file

Each line of the update file can contain a command (like File=okularpartrc) or a comment (like # Hello!). Commands are processed sequentially from top to bottom.

Description of the minimal example update file

In the section #Minimal example update file, we see the following lines:

#Configuration update for Okular
Empty lines and lines that start with # are considered comments.
Version=5
Specifies that this update file is written for the KF5 version of kconf_update.
Every update file must start with this command.
Id=annotation-toolbar
Starts an update section called annotation-toolbar.
File=okularpartrc
Tells kconf_update which configuration file shall be affected.
Group=Reviews
Tells kconf_update that it shall operate on the Reviews group of the configuration file.
Key=AnnotationTools,QuickAnnotationTools
Tells kconf_update that it shall read from the entry AnnotationTools and write to the entry QuickAnnotationTools.
In this case, this causes the entry to be renamed.

Update file structure

Update files are structured as follows:

  • Version=5 (required)
  • any number of update sections, each started by an Id= command
    • any number of Script= and ScriptArguments= commands
    • or any number of File= commands, followed by:
      • any number of Group=, Key=, AllKeys, RemoveGroup=, RemoveKey=, Options=, Script=, ScriptArguments= commands
      • AllGroups (optional)

Update file command reference

All commands are case-sensitive. Commas are used to separate parameters, and may not occur as part of any parameter. Parameters in square brackets are optional. Parameters are referenced in angle brackets, sometimes parameters of preceeding commands are referenced. (Square brackets and angle brackets are not written in the actual update file.)

The following commands are available:

Version=5
This must be the first command in any update file.
Specifies which KDE Frameworks version this update file targets. At the time of writing the only valid value is 5, meaning modern (KDE Frameworks 5) kconf_update binaries will process the file. A file with Version=5 may still be run on older kconf_updates!
Id=<id>
Starts a new update section. All commands until the next Id= command are processed as part of this section. If <id> is stored in kconf_updaterc, this section will be skipped. When this section has been processed, <id> will be stored in kconf_updaterc.
<id> has to be unique. You should give your section a name that describes why this update necessary. You can also include the year of your changes, like in Id=annotation-toolbar-2020.
File=<oldfile>[,<newfile>]
Specifies that configuration information is read from <oldfile> and written to <newfile>. If you only specify <oldfile>, the information is read from as well as written to <oldfile>.
If <oldfile> contains the <id> of the current section, the current section will be skipped. Otherwise, <id> is stored in <oldfile> and <newfile>.
If <oldfile> does not exist at the time kconf_update first checks, the current section will be skipped.
Also note the #Updating multiple files per section section.
Group=<oldgroup>[,<newgroup>]
Specifies that configuration information is read from the group <oldgroup> and written to <newgroup>. If you only specify <oldgroup>, the information is read from as well as written to <oldgroup>.
You can use <default> to select the top-level group.
Key=<oldkey>[,<newkey>]
Moves the configuration entry <oldkey> from <oldfile>/<oldgroup> to <newfile>/<newgroup>.
If you specify <newkey>, the entry is additionally renamed.
If you did not specify <newfile> nor <newgroup> nor Options=copy, the entry is only renamed.
AllGroups
Moves all groups (including all entries) from <oldfile> to <newfile>.
Note that the top-level group is not included. To include the top-level group, you need to use the AllKeys command additionally.
AllKeys
Moves all entries from <oldgroup> to <newgroup>.
RemoveGroup=<oldgroup>
Removes <oldgroup> from <oldfile>.
This can be used to remove obsolete entries or to force a revert to default values.
RemoveKey=<oldkey>
Removes <oldkey> from <oldfile>/<oldgroup>.
This can be used to remove an obsolete entriy or to force a revert to default values.
Options=<option1>, <option2>, ...
With this entry you can specify options that apply to the next Script=, Key=, AllKeys, or AllGroups command (only to the first). Note that Script= probably ignores the options, see the #Script output format section.
Possible options are:
copy
Copy the configuration item instead of moving it. This means that the configuration item will not be deleted from <oldfile> or <oldgroup>.
overwrite
Normally, configuration entries are not moved if an entry with the new name already exists. When this option is specified, any existing entries are overwritten.
Script=<_script>[,<interpreter>]
ScriptArguments=<arguments>
See the #Update script commands section.

Using external update scripts

kconf_update provides commands to move configuration information around, but it can not manipulate the configuration data itself. Using external, so called update scripts, configuration data can be manipulated through kconf_update.

The concept is that old configuration entries are piped into a script. The output of the script is then interpreted as new configuration entries, which are merged to <newfile> or <newgroup>.

It is also possible to use scripts outside the context of a File= command, then it will not get any input, and the output is ignored. See the section #Note on side effects about this.

Update script commands

The following two commands are available to run update scripts:

Script=<_script>[,<interpreter>]
Executes the script file <_script> using the command <interpreter>. If <interpreter> is not given, executes the compiled script file <_script> directly. You can not pass arguments with this command.
If in the context of a File= command, all groups of <oldfile> (excluding the top-level group) will be piped into the script. If in the context of a Group= command, all entries of <oldgroup> will be piped into the script. You can use Group=<default> to process top-level entries.
The output of <_script> is merged into <newfile> or <newgroup>.
ScriptArguments=<arguments>
<arguments> is passed to <_script> of the next Script= command (only to the first).
<arguments> is everything from the = sign to the end of the line. Escape sequences are not possible.
Note that you must specify ScriptArguments= before Script=.

Script output format

The output of the script is parsed as configuration file, and merged to <newfile> or <newgroup>. If in the context of File=, script output is merged to the top-level group of <newfile>. If in the context of Group=, the group context is initialized to <newgroup>.

These lines in the script output are recognized by kconf_update:

[<groupname>]
Sets the group context to <groupname>. This will not create a subgroup, even in the context of Group=.
You can use [<default>] to select the top-level group. However, this will not work in context of Group=.
<key>=<value>
Creates a new entry in the current group context.
Note that existing entries will probably not be overridden, even with Options=override specified.
# DELETE <key>
Deletes the entry <key> in the current group context, but from <oldfile>.
# DELETE [<group>]<key>
Deletes the entry <key> from the group <oldfile>/<group>.
Note that this can change the group context, you should explicitely set the group context afterwards.
# DELETEGROUP <group>
Deletes the whole group <group> (not a subgroup) from <oldfile>.
Note that this can change the group context, you should explicitely set the group context afterwards.

Providing the script files

Script files should be installed in ${KDE_INSTALL_KCONFUPDATEDIR}. You should install only actual scripts here, not binary executables.

You should choose a script language whose interpreter will be available on your target platform. E. g. if your target platforms include Windows, you should not use shell scripts.

Alternatively, you can use “compiled scripts”, which are executable binaries installed in ${KDE_INSTALL_LIBDIR}/kconf_update_bin/. You can e. g. write your update script in C++ and use all the Qt API and other libraries you need, and compile and install the binary together with your application.

Example interpreted update script

Imagine you want to switch from a DoNotUseTabs configuration entry to a UseTabs configuration entry. This can be easily done using this small Python script:

import fileinput

for line in fileinput.input():
    if line.startswith("DoNotUseTabs=false"):
        print("UseTabs=true")
    if line.startswith("DoNotUseTabs=true"):
        print("UseTabs=false")

print("# DELETE DoNotUseTabs")

Your update file could look like this:

Version=5

Id=no-more-double-negation-2020
File=myApprc
Group=UserInterface
Script=myApp_no_more_double_negation_2020.py,python3

Your CMakeLists.txt could look like this:

[...]
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)
[...]
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
    Config
    [...]
)
[...]
install(FILES conf/update/myApp_configFiles.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})
install(FILES conf/update/myApp_no_more_double_negation_2020.py DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})

Don’t forget that your appliation now depends on python3.

Example compiled update script

Imagine your application made the state of a KActionMenu user configurable. Because you changed to the new QToolButton::ToolButtonPopupMode API, you need to migrate from the old delayed and stickyMenu entries to the new popupMode entry. You can do that with the following compiled update script, which uses the KConfig framework directly:

#include <KConfig>
#include <KConfigGroup>

#include <QTemporaryFile>
#include <QTextStream>

int main(int argc, char **argv)
{
    // Create a KConfig object from stdin.
    QFile inputFile;
    inputFile.open(stdin, QIODevice::ReadOnly);
    QTemporaryFile inputFileForKConfig("popupmode_api_tmp");
    inputFileForKConfig.open();
    inputFileForKConfig.write(inputFile.readAll());
    inputFileForKConfig.close();
    KConfig inputConfig(inputFileForKConfig.fileName(), KConfig::SimpleConfig);

    // Create a text stream to stdout.
    QTextStream outputStream(stdout, QIODevice::WriteOnly);

    // Unify delayed and stickyMenu.
    // delayed => 0 (DelayedPopup)
    // !delayed && stickyMenu => 2 (InstantPopup)
    // !delayed && !stickyMenu => 1 (MenuButtonPopup) (defined as default with KConfigXT)
    KConfigGroup rootGroup = inputConfig.group("");
    bool old_delayed = rootGroup.readEntry<bool>("delayed", true);
    bool old_stickyMenu = rootGroup.readEntry<bool>("stickyMenu", true);
    int new_popupMode = (old_delayed) ? 0 : (old_stickyMenu) ? 2 : 1;
    if (new_popupMode != 1) {
        outputStream << "popupMode=" << new_popupMode << Qt::endl;
    }
    outputStream << "# DELETE delayed" << Qt::endl;
    outputStream << "# DELETE stickyMenu" << Qt::endl;

    return 0;
}

Your update file could look like this:

Version=5

Id=toolbuttonpopupmode-api-2020
File=myApprc
Group=ActionMenuState
Script=myApp-toolbuttonpopupmode-api-2020

Your CMakeLists.txt could look like this:

[...]
find_package(ECM ${KF5_REQUIRED_VERSION} CONFIG REQUIRED)
[...]
find_package(KF5 ${KF5_REQUIRED_VERSION} REQUIRED COMPONENTS
    Config
    [...]
)
[...]
install(FILES myApp_configFiles.upd DESTINATION ${KDE_INSTALL_KCONFUPDATEDIR})

add_executable(myApp-toolbuttonpopupmode-api-2020 conf/update/scripts/popupmode.cpp)
target_link_libraries(myApp-toolbuttonpopupmode-api-2020
    KF5::ConfigCore
)
install(TARGETS myApp-toolbuttonpopupmode-api-2020 DESTINATION ${KDE_INSTALL_LIBDIR}/kconf_update_bin/)

Because you don’t need an interpreter, you didn’t add any dependency to your application.

Testing external update scripts

The first step to test external update scripts is to call them directly, and feed them with your configuration files.

For example, you can test the script from #Example interpreted update script with this command:

cat myApprc_UserInterface | python3 myApp_no_more_double_negation_2020.py

If the script works correctly, you should get output like this:

UseTabs=true
# DELETE DoNotUseTabs

The second step is to test your update file together with the script. This means, you need to place the script file in a place where kconf_update will find it while in --testmode. kconf_update will look for the script in QStandardPaths::GenericDataLocation. Let’s assume this path is ~/.qttest/share/, then you need to place your script in ~/.qttest/share/kconf_update/.

Place your update file, your update script, and your configuration file in your working directory, and run:

mkdir -p ~/.qttest/config/
cp myApprc ~/.qttest/config/
mkdir -p ~/.qttest/share/kconf_update/
cp myApp_no_more_double_negation_2020.py ~/.qttest/share/kconf_update/
/usr/lib/kf5/kconf_update --debug --testmode ./myApp_configFiles.upd

You should get an output like:

Automatically enabled the debug logging category kf.config.kconf_update
kf.config.kconf_update: Checking update-file "./myApp_configFiles.upd" for new updates
kf.config.kconf_update: "demo9.upd" : Found new update "no-more-double-negation-2020"
kf.config.kconf_update: "demo9.upd" : Running script "myApp_no_more_double_negation_2020.py"
kf.config.kconf_update: Script input stored in "/tmp/kconf_update.lKlJpe"
kf.config.kconf_update: About to run "/usr/bin/python3"
kf.config.kconf_update: Script contents is:
 "[...]"
kf.config.kconf_update: Successfully ran "/usr/bin/python3"
kf.config.kconf_update: Script output stored in "/tmp/kconf_update.CCeaBP"
kf.config.kconf_update: Script output is:
 "UseTabs=true\n# DELETE DoNotUseTabs\n"
kf.config.kconf_update: "demo9.upd" : Script removes "myApprc" : ("UserInterface") : "DoNotUseTabs"

If you want to test compiled update scripts, the workflow is the same, except that you may place your compiled update script in one of your $PATH directories instead.

Note on side effects

External update scripts can cause side effects. (E. g. rebuilding the database of another service.) This is an intended use case; you can use Script= in the context of Id=, then no script input or output is processed.

Nevertheless, you should be careful if you use update scripts for side effects. If your update scripts cause side effects, you probably want these side effects to happen at most once. KDED will run the update only once, but if KConfig::checkUpdate() is involved, this is not guaranteed.

Consider separating updates into two update files, where one update file performs only deterministic configuration file updates, and the other update file causes only side effects. This way, you can savely call KConfig::checkUpdate() to guarantee configuration file updates, while the other update file is processed at most once by KDED.

Updating multiple files per section

You can use multiple File= commands in the context of one Id= command, then this update section will sequentially update multiple files. This can be used to split a single configuration file into multiple configuration files, or to join multiple configuration files into a single configuration file.

However, in the latter case you will have different <oldfile> parameters. You should be aware that kconf_update will stop processing a section, when it encounters an <oldfile> which already has this update applied. In most cases, this shouldn’t be a problem. If it is a problem, you should consider to use multiple sections with only one File= command each.

Advanced example update file

Version=5

Id=kde2.2
File=kioslaverc,kio_httprc

Group=Proxy Settings
Key=NoProxyFor
Key=UseProxy
Key=httpProxy,Proxy

Group=Cache Settings,Cache
Key=MaxCacheSize
Key=UseCache

Group=UserAgent
AllKeys

RemoveGroup=KDE

The above update file extracts config information from the file kioslaverc and stores it into the file kio_httprc.

It reads the keys NoProxyFor, UseProxy and httpProxy from the group Proxy Settings in the kioslaverc file. If any of these options are present they are written to the keys NoProxyFor, UseProxy and Proxy (!) in the group Proxy Settings in the kio_httprc file.

It also reads the keys MaxCacheSize and UseCache from the group Cache Settings in the kioslaverc file and writes this information to the group Cache (!) in the kio_httprc file.

Then it takes all keys in the UserAgent group of the file kioslaverc and moves then to the UserAgent group in the kio_httprc file.

Finally it removes the entire KDE group from the kioslaverc file.

Further example update files

Another way to learn how to write scripts and update files is to look at existing ones.

With the following command, you can look which applications on your system have installed update files:

locate kconf_update

You can also search through various KDE projects with this link:

https://lxr.kde.org/search?_filestring=.%2B%5C.upd%5Cz&_advanced=1

Common Problems

kconf_update refuses to update an entry
If you change the value of an entry without changing the key or file, make sure to tell kconf_update that it should overwrite the old entry by adding Options=overwrite.

Content is available under Creative Commons License SA 4.0 unless otherwise noted.