Projects/MoveToGit/UsingSvn2Git (pt BR)
Projects/MoveToGit/UsingSvn2Git
Languages: عربي | Asturianu | Català | Česky | Kaszëbsczi | Dansk | Deutsch | English | Esperanto | Español | Eesti | فارسی | Suomi | Français | Galego | Italiano | 日本語 | 한국어 | Norwegian | Polski | Português Brasileiro | Română | Русский | Svenska | Slovenčina | Slovenščina | српски | Türkçe | Tiếng Việt | Українська | 简体中文 | 繁體中文
This page documents how to go about getting a KDE module ready for the Great Git Migration of 2010.
Servidores para todos os desenvolvedores
KDE Sysadmin dispõe de 3 servidores que estão configurados para escrever regras. Você pode encontrar mais informações sobre isso em: http://community.kde.org/Sysadmin/DeveloperAccessForRuleWriting.
Todos os desenvolvedores do KDE registados têm acesso a essas máquinas. O restante deste documento assume uma configuração no seu computador local, você está livre para configurá-lo em seu computador local, mas não há nenhuma necessidade. Você pode usar os servidores fornecidos pelo KDE Sysadmin.
Obtendo as ferramentas
As ferramentas necessárias estão hospedados na http://www.gitorious.org/svn2git. Para começar, faça:
git clone git://gitorious.org/svn2git/svn2git.git
git clone git://git.kde.org/kde-ruleset.git
Então instale o pacote libsvn-dev.
Isso vai te dar o código-fonte para dar um build no svn2git e nos arquivos de conjunto de regras do KDE como ele estão atualmente. Dê um build na ferramenta svn2git antes de passar para a próxima etapa.
Building svn2git
Verifique se você tem Qt4 instalado, entao entre use os comandos qmake && make para dar um no build the executável chamado "svn-all-fast-export"
Como o conjuntos de regras funcionam
O formato das regras svn2git é bastante simples. Em primeiro lugar você tem que declarar alguns repositórios:
create repository kdelibs end repository
Isto diz svn2git que deve criar um repositório git chamado "kdelibs" que mais tarde, podemos usar para dar commits nele.
O resto do arquivo são regras de correspondência caminhos específicos no Subversion, cada regra especifica o que fazer com o commit que aparecer no path especificado. As possíveis ações possíveis são ignorá-los ou adicioná-las a uma determinada branch ou a um repositório específico. Nota: Ignorar é feito simplesmente por deixar de fora as informações sobre o repositório e da branch.
Como exemplos são mais explicativos, a regra a seguir coloca todos os commits do 123453 até 456789 do path / trunk / KDE / kdelibs para a branch master do kdelibs:
match /trunk/KDE/kdelibs/ min revision 123453 max revision 456789 repository kdelibs branch master end match
A revisão min e max são úteis nos casos em que o mesmo path no SVN contém código para diferentes branchs. Um exemplo seria KDevelop3, onde 3,3 KDevelop foi colocado com o KDE 3.5 até 3.5.7, 3.5.8 contendo KDevelop 3.4 e 3.5.9 contendo KDevelop 3.5 e todas as versões do kdevelop agora estão sob a / branches/KDE/3.5/kdevelop.
Os dois parâmetros de revisão não são obrigatórios, se eles são deixados de fora, então todas os commits para o caminho dado no SVN são retomadas na branch especificada.
Para gerar tags com o git você usa um formato especial para o parâmetro da branch: refs / tags / <tagname>. Então, para colocar todos os commits de /tags/KDE/4.4.0/kdelibs para a tag v4.4.0 no repositório git do kdelibs a regra seria a seguinte:
match /tags/KDE/4.4.0/kdelibs/ repository kdelibs branch refs/tags/v4.4.0 end match
Para mais exemplos ver o diretório svn2git/samples/ e as regras no repositório kde-ruleset. A ação é um hack para dizer svn2git recurse em um diretório que acabou copiado ou que existia, porque é de interesse.
Exemplo: Se estamos importando kdelibs, existe em trunk/KDE/kdelibs. Na branch, alguém fez: svn cp $SVNROOT/trunk/KDE $SVNROOT/branches/KDE/4.4
SVN gravado naquele commit na branches/KDE/4.4 que era o único caminho mudado.
Isso significa que a regra
branches/KDE/[^/]+/kdelibs/
não vai corresponder.
Nós precisamos dizer a ferramenta de que algo interessante aconteceu e deveria recurse. Em seguida, ele irá solicitar novamente todas as regras para os arquivos que existem nesse ponto, ponto em que as regras irão coincidir.
Detalhes Importantes
- Todas as regras de correspondência precisam terminar com uma '/', ou então a ferramenta irá falhar em algum ponto. Este é um bug conhecido. A única exceção são as regras Usando o recurso de ação.
- Regras de correspondência podem usar expressões regulares (de acordo com a sintaxe QRegExp) na linha de partida e pode usar backreferences nos parâmetros repositório e filiais usando \n (n = 1,2,3 ,...) para reduzir a quantidade de regras.
- As regras formam uma lista ordenada que a ferramenta percorre preenchendo os caminhos correspondentes em cada commit. Portanto, se duas regras correspondem ao mesmo caminho e nenhum dos dois tem mais critérios de correspondência, a regra que está escrita mais acima no arquivo tem prioridade. Isso é útil para excluir certos commits a partir do processo de extração, se você olhar para o conjunto de regras existentes você vai notar que no topo algumas revisões são ignorados.
Setting up your system
Você vai precisar de ~ 65GB de espaço em disco para começar, pois o processo requer uma cópia do banco de dados CVS do KDE. Existe um script que irá transferir este para você (e que pode ser usado para atualizá-lo periodicamente com o rsync) na kde-ruleset/bin/startSync.
more stuff goes here ...
Passo-a-passo para escrever regras para um módulo
Analisando o histórico do Subversion para escrever regras
Primeiro de tudo você deve verificar se já existem regras para este módulo no repositório kde-regras. Se existem regras já, por favor vá em "Running svn2git".
Se não há regras, no entanto, vamos começar com a brunch master (aka trunk). A maneira mais fácil de descobrir o histórico com o SVN é executando:
svn log -v --stop-on-copy file:///path/to/kde_svn/trunk/KDE/module
Please note that '/path/to/kde_svn/' is the path to the 65GB you downloaded, and the 'trunk/KDE/module' is the module you want to write rules for, but those two put together is *not* a path that physically exists on your disk. svn log is smart enough to do what you want.
This will give you a history of the given module in trunk, it'll stop on the first commit that copied the code from somewhere else. The verbose output will allow you to see where this copy came from.
Now we have a starting point to write a rule, we want all commits from this path in our module repository in the master branch:
match /trunk/KDE/module/ repository module branch master end match
If the log stops at a commit that copied the module from somewhere, we need to follow this to also get the history imported from the "old" place the module resided. The same svn command can be used with slightly different path argument:
svn log -v --stop-on-copy file:///path/to/kde_svn/some/other/path@revision
The @revision is important as the original path usually doesn't exist anymore. With this we can write the next rule to the rules file and repeat until we've finally reached the point where the code was initially imported into svn (or probably cvs in the old days)
Now we can take care of the branches, this is a bit more involved as there may be multiple branches scattered over the /branches directory in svn. You can use the same commands as before to find out the history of a branch if you know the path. This time however you can stop following the source of copy-operations once you've found a source that you've already matched in a rule. That way your branch will be connected to the branch it originated from (which is often trunk aka master) in git.
A useful help with finding branches is svn ls in combination with the path@revision syntax, that way you can view the content of a particular svn directory as it was in an older revision. With this you can even find branches that are not visible (have been deleted) in the current revision.
The rule for putting commits into a git branch in the final repository is only slightly different (the example is for a core module):
match /branches/KDE/4.4/module repository module branch 4.4 end match
And last are the tags, this works the same as branches and trunk, except for using branch refs/tags/v<tag-version> for the branch parameter.
Running svn2git
This is the easiest, but most time-consuming part. As example lets say that in our current working directory we have the kde rules repository in kde-ruleset subdir, the svn2git tool in the svn2git subdir and the KDE repository in the kde_svn subdir:
svn2git/svn-all-fast-export --identity-map kde-ruleset/account-map --rules kde-ruleset/module kde_svn
This will take a few hours usually, but it'll spit out the progress. The tool also writes a logfile to module.log, so in case something goes wrong you can find more details in there.
Once its done you should have a new "module" git repository in your current working directory.
Checking for proper history in the new git repository
A very easy way to check whether the history was imported properly is to use the gitk tool from git. It shows you a graphical representation of the history in the git repository which makes it easy to identify where something is wrong.
The tool should be run with the --all switch so it shows all branches.
You can now scroll through the history to check whether things have been imported correctly.
First and foremost there should be the master branch starting at the top with the most recent commit to trunk/ and ending in the oldest commit that imported the code into KDE's svn or cvs repository.
From the master branch there should be several branches going away for each branch you imported. And eventually also branches that start from another non-master branch.
Things that you should look out for are branches that start "nowhere", that is the first commit in the branch has no parent in another branch or master. This means that svn2git didn't see a commit that created this branch from another using a svn cp command. That can mean that you may have forgotton to add a match rule for some path or that the same path was used for different branches in different revisions. The same applies to tags which have a commit without any parent.
This can usually be fixed by using svn log and svn ls to follow the history of the branch. Eventually you might need to apply the min/max revision paramters.
You'll notice that some tags are looking like this:
| * * <v1.2.3> | | * * | / *
Thats normal for our tags even if a bit ugly. The reason is that often compile-fixes are done in trunk/ after the tag has been created and then the commit has been merged over to the tag.
Another thing however are tags that are named vx.y.z_124321. These are tags that have been deleted and re-created later. You can usually see that in the svn log history, these tags can either be manually deleted after the repository creation using git tag or you can add rules that ignore certain revisions of the tag-path before the one putting the commits into the git repository:
match /tags/KDE/3.3.2/kdelibs min revision 424234 max revision 424236 end match match /tags/KDE/3.3.2/kdelibs repository kdelibs branch refs/tags/3.3.2 end match
If you choose to delete them manually please make sure to document this with a textfile or inside the rule file so if someone else does the conversion later again he'll know what manual steps you did.
Before publishing the newly created git repository make sure to repack it. This can greatly reduce it's size (i.e. Phonon's git repository could be shrunken from 18 MB to 5.2 MB)
How to update the account-map file
Currently, account-map file is being generated with 'generateAccountMap'[1] script which parses kde-common/accounts[2] and kde-common/disabled-accounts[3] from SVN.
Once you have your git repository you should check if there are accounts not listed in account-map file (you can use checkMissingAccounts[4]), if that is the case, check if the missing accounts are listed in kde-common/accounts or kde-common/disabled-accounts, if it's not there file a sysadmin bug report[5] to get your missing account included in disabled-accounts. Once you get your missing accounts included in disabled accounts, you could generate the account-map file running 'bin/generateAccountMap', then run svn-all-fast-export again. Do not edit account-map file directly!
Troubleshooting
Recurse action doesn't work with cvs2svn tag commits
You may have to deal with a commit done by cvs2svn to create a tag, for example:
r386536 | (no author) | 2005-02-05 22:16:00 +0100 (Sat, 05 Feb 2005) | 2 lines Changed paths: A /branches/beta_0_7_branch (from /trunk:386535) D /branches/beta_0_7_branch/art-devel D /branches/beta_0_7_branch/arts D /branches/beta_0_7_branch/bugs D /branches/beta_0_7_branch/devel-home D /branches/beta_0_7_branch/developer.kde.org D /branches/beta_0_7_branch/enterprise.kde.org D /branches/beta_0_7_branch/events.kde.org D /branches/beta_0_7_branch/foundation D /branches/beta_0_7_branch/kckde D /branches/beta_0_7_branch/kde-common D /branches/beta_0_7_branch/kde-i18n D /branches/beta_0_7_branch/kde-qt-addon D /branches/beta_0_7_branch/kde-women.kde.org D /branches/beta_0_7_branch/kdeaccessibility D /branches/beta_0_7_branch/kdeaddons D /branches/beta_0_7_branch/kdeadmin D /branches/beta_0_7_branch/kdeartwork D /branches/beta_0_7_branch/kdebase D /branches/beta_0_7_branch/kdebindings D /branches/beta_0_7_branch/kdeedu D /branches/beta_0_7_branch/kdeextragear-1 D /branches/beta_0_7_branch/kdeextragear-2 M /branches/beta_0_7_branch/kdeextragear-3 D /branches/beta_0_7_branch/kdeextragear-3/Makefile.am.in D /branches/beta_0_7_branch/kdeextragear-3/Makefile.cvs D /branches/beta_0_7_branch/kdeextragear-3/README D /branches/beta_0_7_branch/kdeextragear-3/configure.in.bot D /branches/beta_0_7_branch/kdeextragear-3/configure.in.in D /branches/beta_0_7_branch/kdeextragear-3/digikam D /branches/beta_0_7_branch/kdeextragear-3/digikamimageplugins D /branches/beta_0_7_branch/kdeextragear-3/doc D /branches/beta_0_7_branch/kdeextragear-3/filelight D /branches/beta_0_7_branch/kdeextragear-3/kcfgcreator D /branches/beta_0_7_branch/kdeextragear-3/kconfigeditor D /branches/beta_0_7_branch/kdeextragear-3/kdebluetooth D /branches/beta_0_7_branch/kdeextragear-3/kdetv D /branches/beta_0_7_branch/kdeextragear-3/keurocalc D /branches/beta_0_7_branch/kdeextragear-3/kiosktool D /branches/beta_0_7_branch/kdeextragear-3/klicker D /branches/beta_0_7_branch/kdeextragear-3/kplayer D /branches/beta_0_7_branch/kdeextragear-3/pwmanager D /branches/beta_0_7_branch/kdeextragear-libs-1 D /branches/beta_0_7_branch/kdegames D /branches/beta_0_7_branch/kdegraphics D /branches/beta_0_7_branch/kdeinstaller D /branches/beta_0_7_branch/kdejava D /branches/beta_0_7_branch/kdekiosk D /branches/beta_0_7_branch/kdelibs D /branches/beta_0_7_branch/kdemultimedia D /branches/beta_0_7_branch/kdenetwork D /branches/beta_0_7_branch/kdenonbeta D /branches/beta_0_7_branch/kdenox D /branches/beta_0_7_branch/kdepim D /branches/beta_0_7_branch/kdeplayground-artwork D /branches/beta_0_7_branch/kdeplayground-base D /branches/beta_0_7_branch/kdeplayground-edu D /branches/beta_0_7_branch/kdeplayground-games D /branches/beta_0_7_branch/kdeplayground-ioslaves D /branches/beta_0_7_branch/kdeplayground-multimedia D /branches/beta_0_7_branch/kdeplayground-network D /branches/beta_0_7_branch/kdeplayground-pim D /branches/beta_0_7_branch/kdeplayground-utils D /branches/beta_0_7_branch/kdereview D /branches/beta_0_7_branch/kdesdk D /branches/beta_0_7_branch/kdesecurity D /branches/beta_0_7_branch/kdesupport D /branches/beta_0_7_branch/kdetoys D /branches/beta_0_7_branch/kdeutils D /branches/beta_0_7_branch/kdevelop D /branches/beta_0_7_branch/kdewebdev D /branches/beta_0_7_branch/kdoc D /branches/beta_0_7_branch/kfte D /branches/beta_0_7_branch/khtmltests D /branches/beta_0_7_branch/klyx D /branches/beta_0_7_branch/kmusic D /branches/beta_0_7_branch/koffice D /branches/beta_0_7_branch/kofficetests D /branches/beta_0_7_branch/konstruct D /branches/beta_0_7_branch/qt-copy D /branches/beta_0_7_branch/quanta D /branches/beta_0_7_branch/sysconfig D /branches/beta_0_7_branch/valgrind D /branches/beta_0_7_branch/www This commit was manufactured by cvs2svn to create branch 'beta_0_7_branch'.
If you do this
match /branches/beta_0_7_branch/kdeextragear-3/krecipes/ repository krecipes branch 0.7 end match match /branches/beta_0_7_branch/ min revision 386536 max revision 386536 action recurse end match
svn-all-fast-export will fail, you'll get an error sayining that '/foo/bar/path' was not found where '/foo/bar/path' is one of the deleted paths in the cvs2svn commit. This is because some paths were deleted in the same commit where you want to do an 'action recurse'. Therefore, to avoid matching the deleted paths you should do an action recurse on each intermediate directory from '/branches/beta_0_7_branch/' to '/branches/beta_0_7_branch/kdeextragear-3/krecipes/' and you should use a final '$' to make sure that the deleted paths will not be considered, thusly:
match /branches/beta_0_7_branch/kdeextragear-3/krecipes/ repository krecipes branch 0.7 end match match /branches/beta_0_7_branch/kdeextragear-3/$ min revision 386536 max revision 386536 action recurse end match match /branches/beta_0_7_branch/$ min revision 386536 max revision 386536 action recurse end match
Getting Help
If you run into strange things or can't find a rule for something you can reach the KDE Git migration team on IRC: irc.freenode.org, #kde-git or on the kde-scm-interest mailinglist