Difference between revisions of "Development/Tools/svnmerge.py"

Jump to: navigation, search
(Helpful Scripts: add one script)
(add link to homepage)
Line 172: Line 172:
 
  }
 
  }
  
 +
== Further Reading ==
 +
 +
* [http://www.orcaware.com/svn/wiki/Svnmerge.py The project page of svnmerge]
 
[[Category:PIM]]
 
[[Category:PIM]]
 
[[Category:Tutorial]]
 
[[Category:Tutorial]]

Revision as of 20:50, 17 March 2009

Contents

What is svnmerge and why use it?

When you have multiple branches of the same project in SVN, you probably want to keep these branches in sync, at least to a certain extend.

One example of this would be feature branches, i.e. a branch were a bigger feature is developed. You usually want to keep that feature branch in sync with trunk by merging all commits from trunk into that feature branch. Once the feature branch is ready to be merged back to trunk, you want to do just that with as little work as possible.

Another example are the KDEPIM enterprise branches. All changes from the enterprise branches are merged to trunk, but only some changes from trunk are merged back to the enterprise branches.

In both situation, svnmerge.py will help you by keeping track of the merges and assisting you in the process of merging.

It has the following useful features:

  • Keeping a list of merged and unmerged commits, so you never forget to merge a single commit
  • Ability to block commits from merging
  • Ability to automatically generate commit messages
  • Support for merging many commits at once
  • Support for multiple branches as merge source
  • Support for bidirectional merges

See also the svnmerge wiki page.

Note that svnmerge.py should not be confused with svn merge. The later is a normal SVN command, which is used, among others, by svnmerge.py. When refering to svnmerge on this page, I always mean svnmerge.py. Also, when referring to svn merge, I mean the built-in subversion command.

How does it work internally?

svnmerge somehow needs to keep track of which commits are merged and which are not. For this, it uses the SVN properties of the top-level project directory. The properties contain a list with revisions, and that list is updated each time you merge something with svnmerge.

You can view that list with

svn propedit svnmerge-blocked .

and with

svn propedit svnmerge-integrated .

Never change those properties manually!

Installing svnmerge

Download svnmerge.py from the svnmerge home page. The trunk version should be fine.

Then, put it into /usr/local/bin and make sure it has executable permissions by issuing

chmod +x /usr/local/bin/svnmerge.py

You need to have Python installed, probably with some add-on modules.

Initializing merge tracking for a branch

After creating a new work branch, you need to tell svnmerge about this, so it can set up the initial revision list in the properties. You can skip this section if somebody else already did set up merge tracking for your branches.

To get help in doing so, type

svnmerge.py help init

After you executed the correct svnmerge.py init command, make sure you commit your changes. Only the SVN properties should have changed, type svn status to confirm this.

Afterwards, commit it with

svn ci -F svnmerge-commit-message

Merging Changes

All the svnmerge commands below should be issued in the top-level project working directory, i.e. the directory svnmerge.py init was run in. This directory is what I mean with working directory below.

Getting the list of available merges

First of all, you probably want a list of revisions that are not yet merged. This list can be retrieved with svnmerge avail. The avail command supports many options, to get help, type:

svnmerge.py help avail

Sometimes you already know the revision numbers of the commits you want to merge, in this case you don't need to run svnmerge avail. But be careful, it is easy to forget commits when you don't run this command.

A good practice is get the list of revisions to be merged together with their log message, which often comes in handy. Run the following to store that list in a file:

svnmerge.py avail --log -b > avail.txt &

Note that the command can take a long time, so we run it in the background with &. Also, svnmerge issues a lot of SVN command in the background, so be sure your passphrase is cached when using svn+ssh, by using ssh-add and ssh-agent.

When you have got multiple branches to merge from, you need to specify the target branch with the -S option. This is the case in the enterprise4 branch, which can merge from the enterprise35 branch or the KDE 4.2.x branch.

svnmerge.py avail --log -b -S enterprise > avail.txt &

Now, open avail.txt in a text editor. Your goal is now to deal with all revisions in this list, either by merging them, blocking them or marking them as already merged.

Merging commits

Merging a commit is simple:

svnmerge.py merge [-S enterprise] -r900000

Here, 900000 is a revision number you got from avail.txt, and the name after -S is the branch you want to merge from. You can omit -S if you only have a single branch with merge tracking.

Before typing that command, make sure your working directory is clean, neither the SVN properties nor any files in sub-directories should have any pending changes. Normally, svnmerge will abort if the directories are not clean. Also, your working branch should be up-to-date.

After svnmerge is finished, it will have modified the properties of the working directory, because the properties contain the list of merged revisions, and the revision 900000 was just added to that list. Also, the command has modified your source files, as it merged all changes from the original revision. Nothing is committed yet.

You should now test that the changes actually compile and work as expected, and fix any problems. Sometimes, you'll get conflicts, which you have to resolve manually and then mark as resolved by using svn resolved. After that is done, you can check your changes in.

svnmerge has helpfully created a file named svnmerge-commit-message. You can use that file directly as commit message, or modify its contents. Check in your changes with

svn ci -F svnmerge-commit-message

Congratulations, you have just merged your first commit!

You can also merge multiple revisions at once. But use this with care, it will appear as a single commit, and therefore make svn annotate less useful. Only use this for closely related changes. For this, you can specify a revision list after -r, like:

svnmerge.py merge [-S enterprise] -r900000,9000002,900004-900010

Blocking commits

Sometimes, you don't want to merge a certain revision, and want that revision to not appear in the avail list at all, for example if the original revision is not useful for the current branch. This can be done by blocking a commit.

Blocking commits has the same syntax as merging:

svnmerge.py block [-S enterprise] -r900099,9000101,900110-900120

Unlike merging, you should block all revisions you want to block in one go, so you don't create many unnecessary commits.

You should edit svnmerge-commit-message before committing, adding a very brief explanation why you block the commits. Also, add the SVN_SILENT keyword.

Now you can commit your changes with

svn ci -F svnmerge-commit-message

Blocking a commit only changes the SVN properties, no source files are changed (since the goal of blocking the revision is to not merge the source file changes of that revision).

The next time you run svnmerge.py avail, the revisions you blocked will not show up there.

Recording merges

Sometimes, it happens that a revision is already merged, but svnmerge doesn't know this. One situation where this happens is if somebody else, who didn't use svnmerge, merged that revision manually. Because the revision was merged manually, the list of merged revisions in the SVN properties was not updated. This means that the commit will show up in svnmerge.py avail, although it was already merged.

To fix this situation, you can mark a revision as merged by adding -M to the svnmerge.py merge command:

svnmerge.py merge -M [-S enterprise] -r900099,9000201,900210-900220

This command will add the given revisions to the list of merged commits in the list of the SVN properties.

As with blocking commits, try to record as many merges in a single go as possible. Also, add a short explanation to the commit message, as well as the SVN_SILENT keyword.

To check in your changes, type

svn ci -F svnmerge-commit-message

Problems & Solutions

Files that have been moved will be skipped

One problem of svnmerge is that it can't deal well with moved files. As an example, take merging from the KDE 4.2 branch to the enterprise4 branch. In KDE 4.2, some files of KOrganizer have been moved to the views sub-directory, while in the enterprise4 branch, they are still in the top-level KOrganizer folder.

When this happens, svnmerge will say something like Skipped missing target xyz.cpp. It is now your job to merge the changes to the moved files manually.

This can be done by running svn merge. Example:

svn merge $SVNBASE/branches/KDE/4.2/kdepim/korganizer/views/agendaview/ -c934407

In this example, the exact directory where the moved files are is specified. If this command is run in the directory where the not moved files are, the changes will be applied.

After you merged the changes to the skipped and moved files manually, you can commit like you normally would do.

Conflicts on .

When you have a merge chain, it can sometimes happen that there are conflicts on the SVN properties, which will show up as conflicts on .. One place where this can happen is when merging a commit that was itself a merge of another commit, for example when merging from enterprise35 to enterprise4, and than merging that commit to trunk. This also happens when backporting a merge.

This conflict normally is bogus. To resolve it, type

svn resolved .

Afterwards, double-check that the SVN properties changes are indeed correct, by using the mergediff script from below. You do _not_ want your SVN properties to become corrupted!

Helpful Scripts

Place those functions into your .bashrc.

The mergediff function shows the changes on the SVN properties in a nice way, because the normal output of svn diff is unreadable.

function mergediff { 
  svn diff -N | {    
    read; read; read; read # skip headers}
    read line;                            
    echo $line | tr ' ' '\n' | tr ',:' '\n' | grep -v "^-$" > /tmp/svnmerge.pre
    while read line; do                                                        
      if ! [ -z "$line" ]; then break; fi
    done
    echo $line | tr ' ' '\n' | tr ',:' '\n' | grep -v "^+$" > /tmp/svnmerge.post
  }
  diff -u /tmp/svnmerge.pre /tmp/svnmerge.post
  rm /tmp/svnmerge.pre /tmp/svnmerge.post
}

Further Reading


KDE® and the K Desktop Environment® logo are registered trademarks of KDE e.V.Legal