Projects/GitoriousKDE: Difference between revisions

    From KDE TechBase
    mNo edit summary
    No edit summary
    (15 intermediate revisions by 3 users not shown)
    Line 9: Line 9:
    =KDE repository structure=
    =KDE repository structure=


    First we need to study the KDE repository structure with Git.
    The goal is to have a git repository for each KDE module, (Thiago: "given the current level of cross-repository copy/move support in Git, I'd be tempted to lump kdelibs and kdebase together"). Only playground would make use of submodules.
    KDE is an umbrella of many big projects, each of them is made of several smaller parts that are projects of their own.
    What seems to be the current agreement is that KDE big projects would be git supermodules and KDE sub-projects would be git submodules.


    '''This is what it looks like''':
    '''This is what it looks like''':
    Line 21: Line 19:
       includes
       includes
       doc
       doc
      ...
       kdecore/
       kdecore.git    (module)
       kdeui/
       kdeui.git      (module)
       kio/
       kio.git        (module)
       knewstuff/
       knewstuff.git  (module)
      ...


    kdebase.git
    kdebase.git
    Line 33: Line 29:
       doc
       doc
       ...
       ...
       plasma.git    (module)
       plasma/
        ruphy-plasma.git
       kdm/
        notmart-plasma.git
        chani-plasma.git
        someone-plasma.git
        ...
       kdm.git        (module)
       ...
       ...


    * all repositories are readable (clone, fetch)
    * "private" repositories are only push-accessible by its owner
    * "project" repositories have ACLs that can be controlled by whoever created that project
    * "mainline" repositories are push-accessible by anyone with an account
    * "release" repositories are locked and are writable by the "release dude" (Admins do manual control on these).


    =KDE commit politic=
    =KDE commit politic=
    Line 53: Line 49:
    ==Type of users and their abilities==
    ==Type of users and their abilities==


    ===regular users===
    ===Developer rights===
    * ability to create clones and commit to their clones
    * create "project" or "private" repositories, either as clones or new repos
    * ability to make merge request to mainline
    * submit to any mainline repository
    * ability to apply for developer status
    * ability to request for the creation of a new supermodule/submodule
    * ability to make bug reports/whishes
    ===KDE developers===
    * ability to make commits to mainline
    * ability to accept or reject appliances for developer status (see question 2)
    * ability to request for the creation of a new supermodule/submodule
    ===Administrator===
    * ability to create (and remove?) supermodules and submodule
    * ability to move submodule from one supermodule to another supermodule (ex: kdeplayground=>kdeedu )
    * ability to accept creation of new projects inside of supermodules


    ===Admin rights===
    * anything on the server


    This is what we have so far.


    =We have a few questions too=
    =We have a few questions too=


    1) What's the use of private clones?
    1) The question is to know if each submodule of a supermodule will have one or several admin or if every developers of a submodule will have the ability to accept or reject new developers?
     
    2) If there are submodules admins, who does grant them there role? the supermodule admin?
     
    3) Gitorious would be a websvn replacement only or will it be a bugzilla replacement too? Gitorious devs are implementing ticket tracking as of this document writing.
     
    4) is it necesary to implement "applying for developer status" or will it continue to happen by mail as it does now?
     
    ==Answers to these questions we have so far==
    '''According to Mario Ðanić''':
    1) first question: any commiter to mainline can grant access. It is assumed that developers will discuss that between themselves anyway.


    2) The question is to know if each submodule of a supermodule will have one or several admin or if every developers of a submodule will have the ability to accept or reject new developers?
    2) If there are submodules admins, who does grant them there role? the supermodule admin?  --> Submodules admins are just commiters to mainline of the submodule.


    3) If there are submodules admins, who does grant them there role? the supermodule admin?
    3) Gitorious would be a websvn replacement only, and not a bugzilla replacement


    4) Gitorious would be a websvn replacement only or will it be a bugzilla replacement too? Gitorious devs are implementing ticket tracking as of this document writing.
    4) it is neccessary to implement applying for developer status


    5) is it necesary to implement "applying for developer status" or will it continue to happen by mail as it does now?
    '''ruphy thinks that...'''
    1) Several admins, for sure... but... do we really need that? I'd say that anyone should be able to send the occasional bugfix upstream. (at least in a mob-branch-fashon way?). However, this could be avoided if we make forking extremely trivial (as in pusing a button and waiting <5sec), and similarly also pull requests.


    2) I really think so....
    3) I know bugzilla sucks, but we're going to stick with it for a while still... handling bugs for KDE is just extremely complicated. :\
    4) I... guess so? however, the simpler it is, the better.


    =Suggestions=
    =Suggestions=
    Line 109: Line 111:
       ...
       ...
    3) knewstuff2 server (git repo => knewstuff2 => user )
    3) knewstuff2 server (git repo => knewstuff2 => user )
    4) If KDE is going to keep bugzilla, we can still hook up gitorious with bugzilla in order to make them work together. As an example when a commit message has "closes #123" it will close the bugzilla bug #123, that's just an example of interaction. To have this kind of interaction, we'd need bugzilla xml-rpc web service API turned on, is that the case?
    5) Automatically generate [[Schedules/KDE4/4.1_Feature_Plan | Feature Plans]] by mapping branches to feature, when a branch is created its status is marked as "TODO", when it starts getting some commits, its status is marked as "IN PROGRESS", when the branch is merged back in mainline its status is marked as "DONE".


    =Sum up of the evaluation=
    =Sum up of the evaluation=
    Line 123: Line 130:


    * adapt gitorious design to KDE web style
    * adapt gitorious design to KDE web style
    * CIA integration...
    =initial IRC conversation=
    trapni:  thiago: I was looking for a git hosting service as my local host can't be up as good as a server, so I had to push the repo (including history, ideally) to such a service (I decided to use gitorious - hopefully it's good though :)
    thiago:  ok
    trapni:  thiago: finally, is kde moving from svn to git anytime soon? do you have an idea? (just to be curious)
    thiago:  hasn't had the time to do that
    thiago:  which means it hasn't moved, since I am the only person trying to do it
    trapni:  i know tha lag of time, yep
    trapni:  what are the issues?
    trapni:  are there some?
    thiago:  submodule support isn't good enough, we haven't finished writing the import rules, we don't know the layout of the repositories yet, we need some server software (web service) for creating and cloning repositories, setting ACLs; getting rid of svn:externals
    trapni:  might be interesting to know wether there actually is an interest in moving from svn to git for the majority of kde devs or not
    thiago:  ah, and support for copy-with-history
    trapni:  ah, yeah
    trapni:  one big lag of all those git hosting services is, that they don't provide a homepage for your project - to sad :)
    pygi:  yet ;)
    thiago:  we decided that all our repositories will have a mandatory wiki homepage
    trapni:  a wiki is nice, yeah
    thiago:  if you request a repo on the server, you have to write a wiki page saying what your repo is for
    trapni:  on *what* server are you talking about?
    trapni:  you mean the kde repository?
    thiago:  git.kde.org
    trapni:  oh, you really have one?
    thiago:  no
    trapni:  :(
    pygi:  could volunteer during SoC to work with thiago to improve gitorious for kde needs ;)
    trapni:  i also thought about having one of my libraries to be hosted at kde, in case you'd have a homepage *and* git for me (it's the capseo/libcaptury video capturing framework used by kwin) ;)
    trapni:  pygi++
    thiago:  I don't see why not
    thiago:  we also need a central DNS registry for us because of D-Bus names
    trapni:  yep
    trapni:  thiago: what do you need me to do got get my git repositories (and wiki pages) hosted their?
    trapni:  i'd be glad in fullfilling the requirements though :)
    thiago:  1) get the server to exist
    thiago:  2) get the software running on the server that doesn't exist yet
    thiago:  3) get kde to convert
    trapni:  aha, you mentioned the 'server' word - which i can't provide currently :D
    thiago:  we have the machine already
    trapni:  oh
    pygi:  trapni, would be nice to have ability to request project addition, admins allow it, and then the space for website and git are provided for you
    pygi:  (eventually even mailman auto-create magic even ;))
    trapni:  pygi: exactly
    thiago:  in half the cases, the new repositories are project branches
    trapni:  while i noticed very very poor feedpack performance on savannah.[non]gnu.org where I first tried to host my capturing libraries on
    pygi:  thiago, branches can be done without approval
    thiago:  so the software should clone and set up the alternates link
    trapni:  they do require admin acceptance, gitorious didn't - which i quited liked
    pygi:  thiago, yup, one would be able to see a network of folks who cloned certain repos/branches
    pygi:  only addition of new modules (repositories) would require admin approval
    pygi:  cloned/forked branches dont need web-space, or mailing list, or anything, anyway
    thiago:  pygi: right
    thiago:  but we still want to enforce a wiki page describing the project
    trapni:  forked branches maybe do, but general purpose branches don't (they're most likely to be merged back)
    thiago:  so that we can have a proper list of repositories with their purpose
    pygi:  thiago, well, yes ... wiki page per project, not per branch :)
    thiago:  instead of a "braindump" like http://git.kernel.org/
    thiago:  no, per *repository*
    trapni:  thiago: i highly encourage the wiki-requirement, i guess nobody will have a problem with it, at least I were explicitely looking for a git hosting service with homepage feature (which I failed in finding yet anyways)
    pygi:  thiago, yup
    pygi:  trapni, that's what I meant, yes
    thiago:  the idea of the wiki homepage is to describe what the repository is for, who's working on it, etc.
    pygi:  yea, basic description of it and stuff
    thiago:  exactly
    thiago:  for separate projects (like trapni's idea), it might need some actual webspace and a domain name
    thiago:  under the .kde.org hierarchy, so the project can have its own D-Bus namespace
    pygi:  a lot of magic under :)
    pygi:  so the "application" process would involve selection of services one needs, and a link to wikipage describing what the repository will be for, descriptions, and such
    thiago:  yep
    thiago:  the admin would also be able to select from the account list who's allowed to push there
    pygi:  thiago, why not the repository owner?
    thiago:  the admin = the repository owner
    pygi:  ah, ok! I was thinking of complete system admin
    thiago:  or anyone with push permissions. I don't see the need for making the distinction.
    pygi:  aha, got it
    trapni:  still admin != owner, their _maybe_ shall be a distinction, shouldn't it?
    thiago:  I don't see the need
    pygi:  ok, so no need for manual review process I take it
    thiago:  whoever requests the repository is given push/admin rights over it
    thiago:  and he can give those rights to anyone else
    pygi:  thiago, nod, but who has the ability to approve the request?
    pygi:  or is it approved magically?
    thiago:  should be automatic for clones
    pygi:  yea, but what for new repos? :p
    thiago:  for brand-new projects, sysadmins
    pygi:  yea, well .. that's what we're talking here =)
    thiago:  you have to fit it into one of the supermodules anyways
    trapni:  in my case, it's not a brand new project, I'm just looking for a sweet home :)
    thiago:  you'd have to justify the relation to kde
    thiago:  if it's in kdesupport, it's something kde uses
    thiago:  if it's in extragear, it's kde-based
    pygi:  ah, so one should be able to create project (say kdesupport) and then module under it!
    thiago:  yeah, things have to be organised
    pygi:  a lot of things to think about =)
    trapni:  kdesupport is the relation
    thiago:  my idea was to have a top-level hierarchy "stable" with kdesupport, kde, koffice and extragear supermodules
    thiago:  then a top-level hierarchy "projects" with cloned repositories as well as new stuff (i.e., current playground)
    thiago:  then a top-level hierarchy "users" for someone's own branches (no wiki page required)
    thiago:  people don't get today in SVN the difference between /branches/work and /trunk/playground, so we might as well merge the concepts
    pygi:  supermodules with projects under them
    pygi:  joy :)


    * Annex: create plasmoids that will make use of the Gitorious web service api.
    * Annex: create plasmoids that will make use of the Gitorious web service api.

    Revision as of 13:20, 14 July 2008

    Goal

    Improve and adapt the Gitorious Web Application for its use in the development of KDE

    Process and development of the project

    Evaluate the current peculiarities of Gitorious and the process or politics of the KDE development to define the new modifications needed to obtain the best integration with Gitorious.

    KDE repository structure

    The goal is to have a git repository for each KDE module, (Thiago: "given the current level of cross-repository copy/move support in Git, I'd be tempted to lump kdelibs and kdebase together"). Only playground would make use of submodules.

    This is what it looks like:

    Supermodules

    kdelibs.git

     README/TODO/CMake/...
     includes
     doc
     kdecore/
     kdeui/
     kio/
     knewstuff/
    

    kdebase.git

     README/TODO/CMake/...
     includes
     doc
     ...
     plasma/
     kdm/
     ...
    
    • all repositories are readable (clone, fetch)
    • "private" repositories are only push-accessible by its owner
    • "project" repositories have ACLs that can be controlled by whoever created that project
    • "mainline" repositories are push-accessible by anyone with an account
    • "release" repositories are locked and are writable by the "release dude" (Admins do manual control on these).

    KDE commit politic

    KDE has a history of being developer friendly and easily giving svn accounts to most people that apply for them. Git makes this even easier as anybody can commit to his/her own branch.

    That being said, KDE is more or less a meritocracy and some developers have more power than others when it comes to administration. This is necessary but SVN can make the administration painful for admins and frustrating for wanna-be developers as they have to wait before they can commit.

    Given that, we have thought of a simple user/role organization (no complicated rwx type of permissions system that nobody gets and uses).

    Type of users and their abilities

    Developer rights

    • create "project" or "private" repositories, either as clones or new repos
    • submit to any mainline repository

    Admin rights

    • anything on the server


    We have a few questions too

    1) The question is to know if each submodule of a supermodule will have one or several admin or if every developers of a submodule will have the ability to accept or reject new developers?

    2) If there are submodules admins, who does grant them there role? the supermodule admin?

    3) Gitorious would be a websvn replacement only or will it be a bugzilla replacement too? Gitorious devs are implementing ticket tracking as of this document writing.

    4) is it necesary to implement "applying for developer status" or will it continue to happen by mail as it does now?

    Answers to these questions we have so far

    According to Mario Ðanić: 1) first question: any commiter to mainline can grant access. It is assumed that developers will discuss that between themselves anyway.

    2) If there are submodules admins, who does grant them there role? the supermodule admin? --> Submodules admins are just commiters to mainline of the submodule.

    3) Gitorious would be a websvn replacement only, and not a bugzilla replacement

    4) it is neccessary to implement applying for developer status

    ruphy thinks that... 1) Several admins, for sure... but... do we really need that? I'd say that anyone should be able to send the occasional bugfix upstream. (at least in a mob-branch-fashon way?). However, this could be avoided if we make forking extremely trivial (as in pusing a button and waiting <5sec), and similarly also pull requests.

    2) I really think so....

    3) I know bugzilla sucks, but we're going to stick with it for a while still... handling bugs for KDE is just extremely complicated. :\

    4) I... guess so? however, the simpler it is, the better.

    Suggestions

    1) suggestion: anybody will be able to create projects/repositories/forks without any permissions, those projects will be moved to kdeplayground/<submodule> with the submodule created by the superadmin. The project creator could choose the submodule by him/herself. This seems to be in the spirit of kde playground, what do you think?

    2) Playground module for each supermodule(kdelibs, kdebase):

    kdelibs.git

     ...
     playground/
       project1.git
       project2.git
       ...
    

    kdebase.git

     ...
     playground/
       project3.git
       project4.git
       ...
    

    playground/ (is a supermodule)

     project5.git
     project.6git
     ...
    

    3) knewstuff2 server (git repo => knewstuff2 => user )

    4) If KDE is going to keep bugzilla, we can still hook up gitorious with bugzilla in order to make them work together. As an example when a commit message has "closes #123" it will close the bugzilla bug #123, that's just an example of interaction. To have this kind of interaction, we'd need bugzilla xml-rpc web service API turned on, is that the case?

    5) Automatically generate Feature Plans by mapping branches to feature, when a branch is created its status is marked as "TODO", when it starts getting some commits, its status is marked as "IN PROGRESS", when the branch is merged back in mainline its status is marked as "DONE".


    Sum up of the evaluation

    What needs to be done

    • adapt Gitorious to include roles for project members. Those would be developers and admins. For now, gitorious has the concept of "committers" who can commit to mainline (or a clone), those would be KDE developers.
    • it is necessary to implement the concept and managment of supermodules in Gitorious (those supermodules would be KDE super projects such as kdelibs, kdebase, kdeedu...). This functionality is already on the TODO list of Gitorious.
    • the visibility of a project's clone will be modifiable by its creator, ability that only developers will have
    • migration scripts to migrate bugzilla db to gitorious (suggestion)
    • adapt gitorious design to KDE web style
    • CIA integration...

    initial IRC conversation

    trapni:  thiago: I was looking for a git hosting service as my local host can't be up as good as a server, so I had to push the repo (including history, ideally) to such a service (I decided to use gitorious - hopefully it's good though :)
    thiago:  ok
    trapni:  thiago: finally, is kde moving from svn to git anytime soon? do you have an idea? (just to be curious)
    thiago:  hasn't had the time to do that
    thiago:  which means it hasn't moved, since I am the only person trying to do it
    trapni:  i know tha lag of time, yep
    trapni:  what are the issues?
    trapni:  are there some?
    thiago:  submodule support isn't good enough, we haven't finished writing the import rules, we don't know the layout of the repositories yet, we need some server software (web service) for creating and cloning repositories, setting ACLs; getting rid of svn:externals
    trapni:  might be interesting to know wether there actually is an interest in moving from svn to git for the majority of kde devs or not
    thiago:  ah, and support for copy-with-history
    trapni:  ah, yeah
    trapni:  one big lag of all those git hosting services is, that they don't provide a homepage for your project - to sad :)
    pygi:  yet ;)
    thiago:  we decided that all our repositories will have a mandatory wiki homepage
    trapni:  a wiki is nice, yeah
    thiago:  if you request a repo on the server, you have to write a wiki page saying what your repo is for
    trapni:  on *what* server are you talking about?
    trapni:  you mean the kde repository?
    thiago:  git.kde.org
    trapni:  oh, you really have one?
    thiago:  no
    trapni:  :(
    pygi:  could volunteer during SoC to work with thiago to improve gitorious for kde needs ;)
    trapni:  i also thought about having one of my libraries to be hosted at kde, in case you'd have a homepage *and* git for me (it's the capseo/libcaptury video capturing framework used by kwin) ;)
    trapni:  pygi++
    thiago:  I don't see why not
    thiago:  we also need a central DNS registry for us because of D-Bus names
    trapni:  yep
    trapni:  thiago: what do you need me to do got get my git repositories (and wiki pages) hosted their?
    trapni:  i'd be glad in fullfilling the requirements though :)
    thiago:  1) get the server to exist
    thiago:  2) get the software running on the server that doesn't exist yet
    thiago:  3) get kde to convert
    trapni:  aha, you mentioned the 'server' word - which i can't provide currently :D
    thiago:  we have the machine already
    trapni:  oh
    pygi:  trapni, would be nice to have ability to request project addition, admins allow it, and then the space for website and git are provided for you
    pygi:  (eventually even mailman auto-create magic even ;))
    trapni:  pygi: exactly
    thiago:  in half the cases, the new repositories are project branches
    trapni:  while i noticed very very poor feedpack performance on savannah.[non]gnu.org where I first tried to host my capturing libraries on
    pygi:  thiago, branches can be done without approval
    thiago:  so the software should clone and set up the alternates link
    trapni:  they do require admin acceptance, gitorious didn't - which i quited liked
    pygi:  thiago, yup, one would be able to see a network of folks who cloned certain repos/branches
    pygi:  only addition of new modules (repositories) would require admin approval
    pygi:  cloned/forked branches dont need web-space, or mailing list, or anything, anyway
    thiago:  pygi: right
    thiago:  but we still want to enforce a wiki page describing the project
    trapni:  forked branches maybe do, but general purpose branches don't (they're most likely to be merged back)
    thiago:  so that we can have a proper list of repositories with their purpose
    pygi:  thiago, well, yes ... wiki page per project, not per branch :)
    thiago:  instead of a "braindump" like http://git.kernel.org/
    thiago:  no, per *repository*
    trapni:  thiago: i highly encourage the wiki-requirement, i guess nobody will have a problem with it, at least I were explicitely looking for a git hosting service with homepage feature (which I failed in finding yet anyways)
    pygi:  thiago, yup
    pygi:  trapni, that's what I meant, yes
    thiago:  the idea of the wiki homepage is to describe what the repository is for, who's working on it, etc.
    pygi:  yea, basic description of it and stuff
    thiago:  exactly
    thiago:  for separate projects (like trapni's idea), it might need some actual webspace and a domain name
    thiago:  under the .kde.org hierarchy, so the project can have its own D-Bus namespace
    pygi:  a lot of magic under :)
    pygi:  so the "application" process would involve selection of services one needs, and a link to wikipage describing what the repository will be for, descriptions, and such
    thiago:  yep
    thiago:  the admin would also be able to select from the account list who's allowed to push there
    pygi:  thiago, why not the repository owner?
    thiago:  the admin = the repository owner
    pygi:  ah, ok! I was thinking of complete system admin
    thiago:  or anyone with push permissions. I don't see the need for making the distinction.
    pygi:  aha, got it
    trapni:  still admin != owner, their _maybe_ shall be a distinction, shouldn't it?
    thiago:  I don't see the need
    pygi:  ok, so no need for manual review process I take it
    thiago:  whoever requests the repository is given push/admin rights over it
    thiago:  and he can give those rights to anyone else
    pygi:  thiago, nod, but who has the ability to approve the request?
    pygi:  or is it approved magically?
    thiago:  should be automatic for clones
    pygi:  yea, but what for new repos? :p
    thiago:  for brand-new projects, sysadmins
    pygi:  yea, well .. that's what we're talking here =)
    thiago:  you have to fit it into one of the supermodules anyways
    trapni:  in my case, it's not a brand new project, I'm just looking for a sweet home :)
    thiago:  you'd have to justify the relation to kde
    thiago:  if it's in kdesupport, it's something kde uses
    thiago:  if it's in extragear, it's kde-based
    pygi:  ah, so one should be able to create project (say kdesupport) and then module under it!
    thiago:  yeah, things have to be organised
    pygi:  a lot of things to think about =)
    trapni:  kdesupport is the relation
    thiago:  my idea was to have a top-level hierarchy "stable" with kdesupport, kde, koffice and extragear supermodules
    thiago:  then a top-level hierarchy "projects" with cloned repositories as well as new stuff (i.e., current playground)
    thiago:  then a top-level hierarchy "users" for someone's own branches (no wiki page required)
    thiago:  people don't get today in SVN the difference between /branches/work and /trunk/playground, so we might as well merge the concepts
    pygi:  supermodules with projects under them
    pygi:  joy :)
    


    • Annex: create plasmoids that will make use of the Gitorious web service api.