< Development‎ | Tools
Revision as of 14:03, 9 April 2009 by TMG (Talk | contribs) (inital kdepim policy section)

Jump to: navigation, search

What is svnmerge and why use it?

When you have multiple branches of the same project in SVN, you probably want to keep these branches in sync, at least to a certain extend.

One example of this would be feature branches, i.e. a branch were a bigger feature is developed. You usually want to keep that feature branch in sync with trunk by merging all commits from trunk into that feature branch. Once the feature branch is ready to be merged back to trunk, you want to do just that with as little work as possible.

Another example are the KDEPIM enterprise branches. All changes from the enterprise branches are merged to trunk, but only some changes from trunk are merged back to the enterprise branches.

In both situation, will help you by keeping track of the merges and assisting you in the process of merging.

It has the following useful features:

  • Keeping a list of merged and unmerged commits, so you never forget to merge a single commit
  • Ability to block commits from merging
  • Ability to automatically generate commit messages
  • Support for merging many commits at once
  • Support for multiple branches as merge source
  • Support for bidirectional merges

See also the svnmerge wiki page.

Note that should not be confused with svn merge. The later is a normal SVN command, which is used, among others, by When refering to svnmerge on this page, I always mean Also, when referring to svn merge, I mean the built-in subversion command.

How does it work internally?

svnmerge somehow needs to keep track of which commits are merged and which are not. For this, it uses the SVN properties of the top-level project directory. The properties contain a list with revisions, and that list is updated each time you merge something with svnmerge.

You can view that list with

svn propedit svnmerge-blocked .

and with

svn propedit svnmerge-integrated .

Never change those properties manually!

Installing svnmerge

Download from the svnmerge home page. The trunk version should be fine.

Then, put it into /usr/local/bin and make sure it has executable permissions by issuing

chmod +x /usr/local/bin/

You need to have Python installed, probably with some add-on modules.

Initializing merge tracking for a branch

After creating a new work branch, you need to tell svnmerge about this, so it can set up the initial revision list in the properties. You can skip this section if somebody else already did set up merge tracking for your branches.

To get help in doing so, type help init

After you executed the correct init command, make sure you commit your changes. Only the SVN properties should have changed, type svn status to confirm this.

Afterwards, commit it with

svn ci -F svnmerge-commit-message.txt

Merging Changes

All the svnmerge commands below should be issued in the top-level project working directory, i.e. the directory init was run in. This directory is what I mean with working directory below.

Getting the list of available merges

First of all, you probably want a list of revisions that are not yet merged. This list can be retrieved with svnmerge avail. The avail command supports many options, to get help, type: help avail

Sometimes you already know the revision numbers of the commits you want to merge, in this case you don't need to run svnmerge avail. But be careful, it is easy to forget commits when you don't run this command.

A good practice is get the list of revisions to be merged together with their log message, which often comes in handy. Run the following to store that list in a file: avail --log -b > avail.txt &

Note that the command can take a long time, so we run it in the background with &. Also, svnmerge issues a lot of SVN command in the background, so be sure your passphrase is cached when using svn+ssh, by using ssh-add and ssh-agent.

When you have got multiple branches to merge from, you need to specify the target branch with the -S option. This is the case in the enterprise4 branch, which can merge from the enterprise35 branch or the KDE 4.2.x branch. avail --log -b -S enterprise > avail.txt &

Now, open avail.txt in a text editor. Your goal is now to deal with all revisions in this list, either by merging them, blocking them or marking them as already merged.

Merging commits

Merging a commit is simple: merge [-S enterprise] -r900000

Here, 900000 is a revision number you got from avail.txt, and the name after -S is the branch you want to merge from. You can omit -S if you only have a single branch with merge tracking.

Before typing that command, make sure your working directory is clean, neither the SVN properties nor any files in sub-directories should have any pending changes. Normally, svnmerge will abort if the directories are not clean. Also, your working branch should be up-to-date.

After svnmerge is finished, it will have modified the properties of the working directory, because the properties contain the list of merged revisions, and the revision 900000 was just added to that list. Also, the command has modified your source files, as it merged all changes from the original revision. Nothing is committed yet.

You should now test that the changes actually compile and work as expected, and fix any problems. Sometimes, you'll get conflicts, which you have to resolve manually and then mark as resolved by using svn resolved. After that is done, you can check your changes in.

svnmerge has helpfully created a file named svnmerge-commit-message.txt. You can use that file directly as commit message, or modify its contents. Check in your changes with

svn ci -F svnmerge-commit-message.txt

Congratulations, you have just merged your first commit!

You can also merge multiple revisions at once. But use this with care, it will appear as a single commit, and therefore make svn annotate less useful. Only use this for closely related changes. For this, you can specify a revision list after -r, like: merge [-S enterprise] -r900000,9000002,900004-900010

Blocking commits

Sometimes, you don't want to merge a certain revision, and want that revision to not appear in the avail list at all, for example if the original revision is not useful for the current branch. This can be done by blocking a commit.

Blocking commits has the same syntax as merging: block [-S enterprise] -r900099,9000101,900110-900120

Unlike merging, you should block all revisions you want to block in one go, so you don't create many unnecessary commits.

You should edit svnmerge-commit-message.txt before committing, adding a very brief explanation why you block the commits. Also, add the SVN_SILENT keyword.

Now you can commit your changes with

svn ci -F svnmerge-commit-message.txt

Blocking a commit only changes the SVN properties, no source files are changed (since the goal of blocking the revision is to not merge the source file changes of that revision).

The next time you run avail, the revisions you blocked will not show up there.

Recording merges

Sometimes, it happens that a revision is already merged, but svnmerge doesn't know this. One situation where this happens is if somebody else, who didn't use svnmerge, merged that revision manually. Because the revision was merged manually, the list of merged revisions in the SVN properties was not updated. This means that the commit will show up in avail, although it was already merged.

To fix this situation, you can mark a revision as merged by adding -M to the merge command: merge -M [-S enterprise] -r900099,9000201,900210-900220

This command will add the given revisions to the list of merged commits in the list of the SVN properties.

As with blocking commits, try to record as many merges in a single go as possible. Also, add a short explanation to the commit message, as well as the SVN_SILENT keyword.

To check in your changes, type

svn ci -F svnmerge-commit-message.txt

Commit and merge policy for the KDEPIM branches

This section is only about the commits to the KDEPIM enterprise branches and for those who work on it.

There are 4 branches in total which need to be kept in sync: enterprise3, enterprise4, trunk and 4.2.

Merge tracking is available in the following directions:

(3.5 -> enterprise3)
enterprise3 -> enterprise4
enterprise4 -> trunk
4.2 -> enterprise4

It is important to use as much as possible. Avoid manual merging and crossporting! Manual merges and backports should be avoided because it doesn't update the merge tracking information in the SVN properties. This means that the merge guy later has to find out which commits were merged to which places, which usually involves digging through mail archives and, which is very time consuming.

The ideal lifecycle of a commit is the following:

  • The inital commit is done into the enterprise3 branch
  • The commit is merged with from enterprise3 to enterprise4
  • The commit is merged with from enterprise4 to trunk
  • The commit is manually backported from trunk into the 4.2 branch, if it is an important bugfix and does not contain string changes or new features. While manually backporting, the original svn log message (which includes the auto-generated svnmerge log messages) should be included, so that it is still known where the commit originated from.
  • If the commit is backported to KDE 4.2, it needs to be blocked with in the 4.2 -> enterprise4 merge direction, since it is already in enterprise4 and therefore a cyclic merge.

Try to stick to this ideal lifecycle, it makes the life of the merge guy much easier! This ideal lifecycle also means that commits should never be started in trunk or the KDE 4.2 branch, since then you need to do manual merging, for example to the enterprise3 branch.

One exception to this is when the commit is needed only in enterprise3 or only in the KDE4 branches (enterprise4, trunk, 4.2).

The lifecycle for a commit that is only needed in the enterprise3 branch is:

  • The inital commit is done into the enterprise3 branch
  • The commit is blocked with in the enterprise3 -> enterprise4 direction

The lifecycle for a commit that is only needed in the KDE 4 branches is:

  • The inital commit is done into the enterprise4 branch
  • The commit is merged with from enterprise4 to trunk
  • The commit is manually backported from trunk into the 4.2 branch. See notes abvoe.
  • If the commit is backported to KDE 4.2, it needs to be blocked with in the 4.2 -> enterprise4 merge direction

If you encounter a situation where no of the above 3 lifecycles would apply, please ask Thomas McGuire.

Problems & Solutions

Common Error Messages

  • "xyz" is neither a valid URL, nor an unambiguous substring of a repository path, nor a working directory
This can happen when using the -S parameter. Make sure you run svnmerge on the top-level project directory, not a subdirectory. Also make sure you didn't misspell the branch name after the -S parameter.
  • svnmerge: no integration info available
Run svnmerge in the top-level project directory, not a subdirectory. Also, someone should have run init on the top-level project directory once.

Files that have been moved will be skipped

One problem of svnmerge is that it can't deal well with moved files. As an example, take merging from the KDE 4.2 branch to the enterprise4 branch. In KDE 4.2, some files of KOrganizer have been moved to the views sub-directory, while in the enterprise4 branch, they are still in the top-level KOrganizer folder.

When this happens, svnmerge will say something like Skipped missing target xyz.cpp. It is now your job to merge the changes to the moved files manually.

This can be done by running svn merge. Example: cd korganizer svn merge $SVNBASE/branches/KDE/4.2/kdepim/korganizer/views/agendaview/ -c934407 In this example, the exact directory where the moved files, in the branch to merge from, are, is specified (korganizer/views/agendaview). If this command is run in the directory (korganizer) where the files are, in the branch to merge to, the changes will be applied.

After you merged the changes to the skipped and moved files manually, you can commit like you normally would do.

Files have been moved to a different project

The above case deals with files that have been moved within the project. But what to do if the files have been moved to a totally different project, which is not covered by the merge tracking?

One example where this happens is when merging from the enterprise35 branch to the enterprise4 branch. Say there was a commit in enterprise35, which changed both the KCal library libkcal and KOrganizer. The enterprise 35 branch is KDE 3 based. In KDE 3, the kdepim directory had both the libkcal and the korganizer sub-directories. In the KDE 4 based enterprise4 branch, the KCal library has been moved to a new module, kdepimlibs, and was renamed from libkcal to kcal. But kdepimlibs is not covered by merge tracking, only the two kdepim directories are. What now?

First, you start by normally merging the revision, but not yet committing it. merge -S enterprise -r678876

You'll see some messages about skipped files, since we know they have been moved. In our example, it would be something like Skipped missing target libkcal/incidence.h. Only the KOrganizer changes are not skipped.

You need to merge the moved files manually with svn merge. In our example above, you would first change into the directory where the new files are, in this case the KCal library that has been moved to kdepimlibs:

cd /path/to/kdepimlibs/kcal

Then, use svn merge, like so:

svn merge $SVNBASE/branches/kdepim/enterprise/kdepim/libkcal -c678876

Now, you can commit the changes to kdepimlibs. The easiest way is to reuse the commit message svnmerge created in the kdepim directory:

svn ci -F /path/to/kdepim/svnmerge-commit-message.txt

You can also copy the commit message file and add something like First part of the merge to it, so others will know that there will be more commits for this merge.

So now, you have done the first part of the merge. You have merged the changes of the files that have been moved outside of the project, and you have not changed any SVN properties yet that keep track of the merges.

Time for the second part of the merge. Change back into the original project directory where merge tracking information is available, in our case the kdepim directory:

cd /path/to/kdepim

Then simply commit the changes svnmerge has made in the first step. Those are the changes to the files that have not been moved.

svn ci -F svnmerge-commit-message.txt

Now, both the changes to moved files and the changes to the non-moved files have been made. Also, the SVN properties have been updated in the second step to record the merge. The merge is now complete.

As always, you should make sure your changes compile and work before checking anything in.

Conflicts on .

When you have a merge chain, it can sometimes happen that there are conflicts on the SVN properties, which will show up as conflicts on .. One place where this can happen is when merging a commit that was itself a merge of another commit, for example when merging from enterprise35 to enterprise4, and than merging that commit to trunk. This also happens when backporting a merge.

This conflict normally is bogus. To resolve it, type

svn resolved .

Afterwards, double-check that the SVN properties changes are indeed correct, by using the mergediff script from below. You do not want your SVN properties to become corrupted!

Helpful Scripts

Place those functions into your ~/.bashrc.


The mergediff function shows the changes on the SVN properties in a nice way, because the normal output of svn diff is unreadable.

function mergediff {

 svn diff -N | {    
   read; read; read; read # skip headers}
   read line;                            
   echo $line | tr ' ' '\n' | tr ',:' '\n' | grep -v "^-$" > /tmp/svnmerge.pre
   while read line; do                                                        
     if ! [ -z "$line" ]; then break; fi
   echo $line | tr ' ' '\n' | tr ',:' '\n' | grep -v "^+$" > /tmp/
 diff -u /tmp/svnmerge.pre /tmp/
 rm /tmp/svnmerge.pre /tmp/



The automerge function is helpful if you want to automatically merge a list of revisions, but do not want to use merge because it wouldn't preserve history for svn annotate correctly.

Normally, you would do merge -r1,2,3,4,5

However, that places all 5 revisions into a single commit, so you'll loose history information.

Instead, you can do

automerge_e4 1,2,3,4,5

This will use svnmerge to automatically merge all 5 revisions, but makes sure it uses a separate commit for each revision, thus producing 5 distinct commits. This preserves history much better. The _e4 suffix is just from a custom one-liner function to merge from the enterprise4 branch, it is easy to create your own functions. See the code below for more details. Also notice the lack of -r.

Warning: This automatically commits stuff, without any opportunity for reviews. The script will only stop in case of conflicts, even compile errors are not tested! Only use this when you know what you are doing!! A good way is to first use merge for your revisions, test if that compiles and works, then use svn revert -R . * to revert those changes again. Afterwards, you can use automerge, because you already know that those revisions work correctly, since you just tried that. For a large list of commits that can be merged without conflicts, this is still faster than manually merging every single commit.

You need to export $SVNBASE before using this function.

The ssh-add is only needed when using svn+ssh, remove that is you use https or have found a better way to cache the key.

function revlist_sort {

 sed 's/,/\n/g' | sort


function automerge_general {

 ssh-add -t 99999999
 while read line
   rm -f svnmerge-commit-message.txt
   echo "Merging revision $line from $REPO"
   echo "  Running svnmerge..."
   echo "    command: merge -S $REPO -r$line" merge -S $REPO -r$line
   if test -e svnmerge-commit-message.txt; then
     echo "  Committing..."
     echo "    command: svn ci -F svnmerge-commit-message.txt"
     svn ci -F svnmerge-commit-message.txt
     echo "    Commit file missing! Something went wrong! Aborting!"
     return 1
 return 0


function automerge_e4 {

 echo $1 | revlist_sort | automerge_general $SVNBASE/branches/kdepim/enterprise4/kdepim


Further Reading