Projects/MovetoGit

This is the page for co-ordinating KDE's move to Git.

If you're interested in helping, you should join the kde-scm-interest@kde.org mailinglist and #kde-git on freenode.

Meetings are wednesdays, 19:30 UTC, in #kde-git.

The Plan

KDE is, eventually, moving to Git. We will be using gitorious.org servers, with funding from Nokia. We will also have our own mirrors using existing KDE servers.

We are working with the Gitorious people to ensure their server will meet all our needs as well as everyone's privacy requirements. The distributed nature of Git will make it easy for us to migrate off gitorious.org at any time should the need arise (but that's unlikely :).

In the summer of 2009, Amarok moved to Gitorious to test the waters and find problems that would affect KDE. Once those problems have been solved, all of KDE will be able to switch.

Why?

Git offers many advantages over svn, including offline commits and much easier to keep a feature branch up-to-date. Many KDE developers are already using git-svn, but this tool has its limitations. We want to have the full power of Git available, and we have people willing to do the work necessary to migrate.

How?

When we move, KDE's svn repository will be migrated into several Git repos, all on gitorious.org. Main modules such as kdelibs and kdebase will each become one repository. Projects in extragear will each have their own repository. The kde.gitorious.org site will have a list (lists?) of all these repositories using the builtin project wiki. Scripts will be provided for downloading, say, all of extragear, so "moving" a project from kdereview to extragear would simply involve editing a file kept online that defined the location of projects.

A few things will stay in subversion - currently websites, translations and manuals. It's possible they could move to Git later, but they won't be part of the mass migration.

On Gitorious, all KDE developers will be part of the kde-developers group. Developers in this group are required to set their "full name" for their Gitorious account to their real name. If you want to be in this group, file a bugreport for sysadmin on bugs.kde.org. :)

The procedure to move a project from svn to gitorious.org can be found in Steps to follow for Moving.

Blockers

Tasks that need to get done before we can migrate

SLA for gitorious.org

30% completed (estimate)

  

Owner: aseigo, frank

Status: Progressing well; will take a while

The SLA terms need to be documented as well as who will be footing the bill, if any. TZander has talked to Shortcut AS and sent the relevant information (eg cost) to the KDE eV board.

aseigo and Frank (and someone?) had a meeting with Shortcut people during tokamak. They asked for a rather large sum of money; negotiations are continuing, but we might want to consider hosting our own server...

Write / update importing rules for svn2git

0% completed (estimate)

  

Owner: see below - volunteers needed!

Status: sandsmark: stuck on a svn2git bug. sho: ???, tumaix:started to read the docs, cryos: getting started [2010-01-06]

The importer is on gitorious.org as svn2git we have a set of rules to tell the importer what svn dirs turn into which git repos and those need constant updating whenever a new branch or tag or project is created. Currently the rules are mostly a rough draft, as seen by the large amount of rule-editing that had to be done for Konversation and Amarok. This has not been done for quite some time and so someone should rsync the svn repo run svn2git and fix the rules and importer whenever the import stops.

This is a very big task, too big for one person; it's probably best to tackle it one module at a time

To get started on a module, read Using Svn2Git

TZander has done the koffice ruleset as of 2009-01-06

Jpwhiting has finished (more or less) the kdeaccessibility ruleset 2010-01-24.

aavci has done the k3b ruleset as of 2010-01-27

progress details:

repo	owner	%	comments
SC/kdeaccessibility	jpwhiting	99	"more or less"?
SC/kdeadmin		0
SC/kdeartwork		0
SC/kdebase		0
SC/kdebindings		0
SC/kdeedu	cryos?	?	update me!
SC/kdeexamples		0
SC/kdegames	tumaix?	?	update me!
SC/kdegraphics		0
SC/kdelibs		0
SC/kdemultimedia	eean	0
SC/kdenetwork	grundleborg	0
SC/kdepim	tnyblom	0	Just getting started Help needed.
SC/kdepimlibs	tnyblom	0	Just getting started Help needed.
SC/kdeplasma-addons		0
SC/kdesdk	mattr	0	just starting
SC/kdetoys	mattr	0	just starting
SC/kdeutils		0
SC/kdewebdev		0
extragear/sdk/kdevelop	apaku	85	Misses history from KDE3.3 and older (KDevelop3.2), probably something with layout-changes, might need recurse-action but don't know how to do it.
extragear/sdk/kdevplatform	apaku	100	done, all tags seem fine all branches are there
extragear/*/*		xx	expand the *'s later (let's focus on the base modules first)
kde-common	mattr	0	just starting
kdesupport		0
koffice	tzander	100
promo		0
quality		0
tests		0

Nice to have before the migration

Push log

0% completed (estimate)

  

Owner: nobody!!

Status: argonel had a hd failure :( [2010-01-06]

Gitorious records who pushed each commit. This is useful information because commits themselves can say anything, and there are legitimate reasons to push commits with another person's name.

Internally, Gitorious stores this information in an SQL database, and the information is viewable through the web interface. However we want a way to backup this information for the case that Gitorious suddenly go offline.

quotes from the mailing list:

> How about every repo has, by convention, a "commits" branch and > a post commit hook that ensures whatever meta info is required, > however it can be gleaned, is also checked into that commits branch. > A bit like how gitosis uses a repo to store auth/acl info to help > manage the other repos.

That's exactly my idea. And of course it won't be named commits, because we're not talking about commits.

It's a push log, similar to a local repository's reflog.

---

For every push, log:

- who pushed (not the Unix username, which will be "git")
- which branch heads changed (what from, what to)
- which tags were created
- the state of all other branches and tags

Just use git commit-tree with the empty tree and save everything in the commit message, one after the other.

---

This does not require help from gitorious.org until the script is actually written: it's just another pre-receive hook for them to install on their servers. it *might* also be possible to run it as a post-receive hook (in which case we can do the whole thing ourselves).

if we self-host, then we won't need this.

Script for downloading virtual KDE hierarchies

0% completed (estimate)

  

Owner:

Status:

Let's start over on this.

what we already have: two build scripts for kde; kdesvn-build and build-tool. what we want: an easy way for people to get large chunks of kde, without thinking about what urls the repos come from or having to look things up.

do kde-svnbuild and build-tool satisfy this well enough? or do we want a computer-readable file listing all the repos too?

btw, scripty has its own list of repos already. it's just in a rather weird bash file.

Links [1] Projects/MovetoGit/MassCloneScript [2]

pre-receive hooks

0% completed (estimate)

  

Owner: volunteers needed!!

• Line endings and encodings

Discussion: this got accidentally marked as done or something, but it's not.

Notes: > > As for line-endings, be careful because Git is different from Subversion. > > different how?

Just ensure that all files are stored as LF only, except if there's a .gitattributes file saying "-crlf" (i.e., allow it to have CRLF).

Snapshot to read-only svn

0% completed (estimate)

  

Owner:

It's work, but maybe some people would like it. NEEDED for documentation, in order to get it back into SVN for the translators/scripty/?

Discussion

Could be done with a git-svn gateway presumably? -Mike Arthur 19/10/2009 16:04

if we leave the docbook stuff in svn, we can avoid this a bit longer. --Chani 23:21, 12 November 2009 (UTC)

Techbase Documentation

Owner: Chani, greeneg, - please help out!

10% completed (estimate)

  

At least minimal documentation about how to checkout, how to get a Gitorious account, how to request a merge needed, other git documentation and links to other git information would be very useful also.

see the Git Tutorial Page. help wanted!!

Discussion

Setup git mirrors for cloning

0% completed (estimate)

  

Owner: No one (help!)

Re-purpose the anonsvn servers. This item might be a blocker.

Discussion

Local pre-commit hooks

0% completed (estimate)

  

Owner: argonel

A set of recommended local hooks that give useful warnings could be nice to have.

Discussion ...on the other hand, if we get a lot of bikeshedding about what hooks, then it won't be so nice. so I'd put this in the "very optional" pile. --Chani 19:10, 16 December 2009 (UTC)

Post-migration Issues

Website Branding

2% (initial ideas on the table)

  

Owner: ruphy

KDE Gitorious should be branded accordingly, and should be reachable from git.kde.org as well as kde.gitorious.org

Discussion

Unscheduled & Open

Allow tagging without involving sysadmins

0% completed (estimate)

  

Owner: johan

Pushing a tag currently requires permissions to do a force push, which is a repository-wide checkbox that can only be toggled by a kde-developers admin. Thus the workflow for a dev wanting to do a release tag for his app is to ask an admin to enable force pushing, then to push his tag, and then to tell the admin he can disable force pushing again. This doesn't scale, is insecure, and at odds with KDE's open access policy when it comes to managing the repos (right now in SVN, you need sysadmin to create an app dir in /tags for you, but don't have to ask permission for every individual tag). Johan has promised a solution for this.

Notable discussion points on kde-scm-interest:

http://mail.kde.org/pipermail/kde-scm-interest/2009-November/000782.html

http://mail.kde.org/pipermail/kde-scm-interest/2009-November/000784.html

Discussion

Account setup on Gitorious

0% completed (estimate)

  

Owner: eean, johan, boud, dario

Creating an account on Gitorious isn't hard, but asking to be added to the KDE group is inconvenient. For the migration we should set up a system (via email or wiki?) where developers can ask to have an account autocreated for them, or add their existing Gitorious account to a list to be added to the group. Once this is in place an announcement should be sent to all svn accounts explaining the process, and privacy information.

Basically the currently method of using Bugzilla works fine now and works fine in the longterm. But in the transition month when hundreds of accounts must be created or added, we need a better system. Its important to make it as easy as possible so that we don't lose anyone in the shuffle.

Discussion

I think most people have made their own accounts by now. this isn't really needed.

post-update hooks

90% completed (estimate)

  

Owner: morice Ian Monroe

• License checker

Discussion: We have a fairly complete set of post-update hooks now. See remotehook. However, it would be nice to have a system that lives on the Gitorious server and/or requires less manual maintenance. But its certainly workable and no longer a blocker.

Completed Tasks

Get rid of svn:externals

100% completed (estimate)

  

Owner: David Faure

Status: ???

not possible with git, broken by design.

Discussion

Exists, but ignorable:

• kdesupport shared-desktop-ontologies (temporary)
• playground/utils strigi-chemical/test/ctfr
• playground/devtools kdevelop4-extra-plugins/php/parser/generated/kdevelop-pg-qt
• playground/devtools kdevelop4-extra-plugins/python/parser/generated/kdevelop-pg-qt
• playground/devtools kdevelop4-extra-plugins/qmake/parser/generated/kdevelop-pg-qt
• playground/devtools kommander-plugins/database3/admin
• playground/devtools kommander-plugins/database/admin
• playground/devtools kommander-plugins/datetimefuncs/admin
• playground/devtools kommander-plugins/htmlpart/admin
• playground/devtools kommander-plugins/httpform/admin
• playground/devtools kommander-plugins/kparts/admin
• playground/devtools kommander-plugins/qtactionproxy/admin
• playground/devtools kommander-plugins/timewidget/admin
• playground/devtools kommander-plugins/webkit3/admin
• playground/devtools kpackagemaker/admin

EBN

95% completed (estimate)

  

Owner: drf

Status: Amarok has EBN checks

EBN's krazy checks currently run on kde's svn repo; it needs upgrading to download and check our git repos too.

This would be easier if there was a repo-list that EBN could parse, as it can no longer just svn up to get everything.

Talk to people using other distros about git

100% completed (estimate)

  

Owner: Sebas, Eike

Discussion

• Gentoo: They seem to be prepared for moving their live SVN packages to git; their package manager has easily-reusable classes to fetch from an SCM and moving the ebuilds to using the git class rather than the SVN class should be easy. Positive comments to that end from people in #gentoo-kde.
• Fedora: Some unhappyness about git because SVN allows them to remotely produce a diff between two SVN URLs (or two revisions of one and the same URL) without making a checkout first, while git requires making a clone. Kevin Kofler (IRC nick Kevin_Kofler, #fedora-kde) says this will make their packager work harder.
• Debian: Is indifferent about the SCM switch.

Post Update hooks

100% completed (estimate)

  

Owner: morice, johan, mattr

List of scripts needed:

• BUG/CCMAIL
• email/CIA

Gitorious needs to provide a way for hooks to be called; KDE needs to write said hooks.

Discussion

There is a branch of gitorious called web-hooks http://gitorious.org/gitorious/mainline/commits/web-hooks --Panagiotis Papadopoulos 1 November 2009

Same situation as commit emails. I can do it but it doesn't scale well and a Gitorious-supported solution would be nicer. --eean 16:07, 12 November 2009 (UTC)

Opt-in privacy exception required for kde-developers

0% completed (estimate)

  

Owner: Gitorious, KDE e.V. Board, eean

KDE sysadmins need access to some information that Shortcut could not give them due to their privacy policy. Examples include an email list of all the developers and SQL-level access to information about all the repos in KDE (since it stores who pushes what, information not stored in the git repo itself).

Such requirements will likely be put into the contract with Shortcut.

Discussion

So the e.V. Board is an owner since this is a legal/contract/money issue. Added myself only because I'm shepherding the issue. --eean 16:16, 12 November 2009 (UTC)

We will not get sql access to the information. For the alternative solution see the Push Log issue

Reviewboard

100% completed (estimate)

  

Owner: darktears

This should be easily done with Gitorious web interface and merge requests actually.

Discussion

but reviewboard has features gitorious (right now) doesn't, like commenting on specific lines and not having to set up a merge request. --chani

Also email notifications when someone reviews are needed --thomasz

We're working on this for someone else right now, so pretty soon --johan-s

I consider the latest changes to gitorious to finish this. If more reviewboard features are still needed, and git supports reviewboard, I think this is something we can look at doing post-conversion. --Ian Monroe

Gitorious Needs a feature to disable merge request emails for certain repos

100% completed (estimate)

  

Owner: Gitorious

Have a sensible system for merge request emails. This is now in place - you can join groups, chose whether to have emails on a per repo basis, etc.

SSH blocked in corporations and universities.

100% completed (estimate)

  

Owner: Unknown

Some universities tend to block the SSH port. There should be a workaround to use SSH on some different port. github.com already runs a SSH server on port 443. But that assumes you are using a proxy. It has been found that this hasn't worked with a lot of people, especially those who have a direct connection to the internet ( so some transparent blocking by the ISP ). It would be great if (almost) every KDE developer were to be asked to check if other ports work before KDE made the switch. Otherwise there could be an automated email where the git patches could be sent, and appropriately patched to the right location too.

Discussion

http://blog.gitorious.org/2009/10/20/stuck-behind-a-firewall/, and there's always been HTTP cloning (although the current impl. in Git is a bit on the slow side) --johan-s

Talk to windows guys about git.

100% completed (estimate)

  

Owner: aseigo

Discussion

They aren't huge fans of git, but are using it. They require a single mainline and can't cope with multiple branches. Otherwise, it's workable, even if it will take an adjustment period.

pre-commit hooks

100% completed (estimate)

  

Owner: (unknown)

acltest, docbook, EOL/UTF-8

A web hook isn't good enough for these because they have to run and return whether to allow the push, for every single push to every KDE repo.

Discussion

gitorious guys said they *might* be willing to allow a few scripts on their server for KDE as a special exception, iirc. --chani

Yes, at least for basic things, heavier things like doc building would probably have to be mirrored (goes for pre/post) --johan-s

It turns out that acl and docbook might not be needed so long as web and docs/ stuff stays in svn.

Here's where to find the current scripts - http://websvn.kde.org/trunk/kde-common/svn/hooks/ --Argonel 23:06, 11 November 2009 (UTC)

So: this is actually done because it needs no longer to be done? (boud)

Apparently, so; moving to complete. (aseigo)

other notes

kde-common/accounts

Someone said: KDE accounts file is no longer necessary---used for mapping svn ID -> email, but we have that now from Gitorious. Answer from David Faure: I strongly disagree. We still need a KDE accounts file. This is very useful for finding people's email addresses, and having an overview on the number of active kde contributors; and if we keep it we can even have a kdepim resource again for filling an addressbook from it, for completion in kmail's composer (so you can write to any other kde contributor by just typing his/her name). It's also used for populating automatically the kde-cvs-announce mailing-list, for announcements. kde-common/accounts is our family tree (well, list), let's not get rid of it.

Here's my proposal for a kde-common/accounts replacement for the git era: We write a post-receive hook that looks at every commit and records all known email addresses for a given real name as well as the commit hash and date of when an address was last encountered. We can then present that data in the form of a file like kde-common/accounts, or write a web interface to query it (with nice links to the commits on Gitorious, etc.) --Eike (Sho_ on IRC)

To clear up possible confusion: The author information for a given commit is baked into the commit object itself, and comes from the configuration of the git repository it was created in. It is unrelated to any Gitorious account. Due to the distributed nature of Git, the one who uses his Gitorious account to push a commit need not be the same who created it. If Developer A creates a commit in his local clone and Developer B fetches it into his local clone directly from Developer A's machine and then pushes it into the public repo, the repo will only show a commit from Developer A. The Gitorious website will show that Developer B has pushed up a commit from Developer A, but that data is not contained in the repository. Thus collecting only Gitorious accounts and their mail addresses is insufficient. --Eike

Random

http://mail.kde.org/pipermail/dot-stories/2005-May/000509.html might be a good guide on what docs we need.

some of this stuff was from the list from GCDS that was in this email http://markmail.org/message/u6eqfjece7fibfyo

IRC Meetings

• Minutes of meeting 11 November 2009
• Next meeting 18:00, 25 November 2009 (UTC)

jobs

TODO merge this with the todolists above

michael jansen: talking to kdesvn-build/mpyne

--Done? -> http://kdesvn-build.kde.org/releases/kdesvn-build-1.10.php -- Panagiotis Papadopoulos 1 November 2009

Yes, but the __kdesvn-build-remote used in the impl isn't pleasant for users already on git so it still needs more work for them. Mpyne 20:32, 11 November 2009 (UTC)

jonas: domain name

ML: convert to SSH

chani: techbase docs for scripty

sebas/lydia/leo: communication with teams! tell people! keeping track that everything is being done.