Module Submission for developers

(Difference between revisions)
Jump to: navigation, search
m (corrected test/testing typo)
(Cloning your own moonbase.git (multi-stage system with bare repository))
Line 11: Line 11:
 
====Cloning your own moonbase.git====
 
====Cloning your own moonbase.git====
  
I don't have a public server of my own on which I can make my repository available for sharing, testing and cherry-picking by others, so I use a two-stage repository system. You may be able to work with just one.
+
Just like other source code management systems, git uses the concepts of a repository and working files. In the past git has allowed the repository and the working files to exist within the same top level directory structure. It seems that this has lead to some problems, so work is under way to separate out the repository and the working files. At the moment certain operations generate warning messages if you try to save to a combined repository and working directory. In the future you will need to have a "bare" repository and a separate working version. This article describes how to set up and use such a future-proof system.
  
I have a remote repository on the main [[Lunar Linux]] machine, '''doppio''', which was created using:
+
I don't have a public server of my own on which I can make a repository available for sharing, testing and cherry-picking by others. Therefore I use a multi-stage repository and working directory system. If you have a public server, you may be able to work with just one.
 +
 
 +
I have three repositories: a remote copy of the main [[Lunar Linux]] [[Moonbase]] repository; a remote working copy; and a local working copy. The first one is optional, but it allows me to share files between the working copies without having to commit them to the main [[Moonbase]] repository first.
 +
 
 +
First of all I login to the main [[Lunar Linux]] machine, '''doppio''', using ssh, and set two important global git defaults:
  
  doppio$  git clone git://lunar-linux.org/lunar/moonbase.git moonbase.git
 
  doppio$  cd moonbase.git
 
 
   doppio$  git config --global user.name "Full Name"
 
   doppio$  git config --global user.name "Full Name"
 
   doppio$  git config --global user.email "username@lunar-linux.org"
 
   doppio$  git config --global user.email "username@lunar-linux.org"
  
I have a local repository on my private machine that shadows the remote one. I shall call my local machine '''satellite''' so that it matches the instructions given in [http://www.kernel.org/pub/software/scm/git/docs/everyday.html Everyday GIT With 20 Commands Or So]
+
Then I create the remote "bare" copy of the main [[Moonbase]] repository and configure two useful properties:
  
   satellite$  git clone ssh://username@foo-projects.org/~username/moonbase.git moonbase.git
+
  doppio$  git clone '''--bare''' git://lunar-linux.org/lunar/moonbase.git moonbase.git
   satellite$  cd moonbase.git
+
  doppio$  git config --get-regexp '^(remote|branch)\.'
 +
  doppio$  git config remote.origin.push master:master
 +
 
 +
Next I create a remote "working" repository that shadows the remote "bare" one, again on '''doppio''':
 +
 
 +
  doppio$  git clone moonbase.git workbase.git
 +
  doppio$  cd workbase.git
 +
  doppio$  git config --get-regexp '^(remote|branch)\.'
 +
  doppio$  git config remote.origin.push master:master
 +
 
 +
The second git config command tells git that the default action for the "push" command is to push changes from the master branch of the current repository to the master branch of the parent repository. Otherwise you need to provide that information on the command line every time.
 +
 
 +
I have a local "working" repository on my private machine that also shadows the remote "bare" one. I shall call my local machine '''satellite''' so that it matches the instructions given in [http://www.kernel.org/pub/software/scm/git/docs/everyday.html Everyday GIT With 20 Commands Or So] but see the Note at the end of the section.
 +
 
 +
  satellite$  git config --global user.name "Full Name"
 +
  satellite$  git config --global user.email "username@lunar-linux.org"
 +
 
 +
   satellite$  git clone ssh://username@lunar-linux.org/~username/moonbase.git workbase.git
 +
   satellite$  cd workbase.git
 
   satellite$  git config --get-regexp '^(remote|branch)\.'
 
   satellite$  git config --get-regexp '^(remote|branch)\.'
   satellite$  git config remote.origin.push master:refs/remotes/satellite/master
+
   satellite$  git config remote.origin.push master:master
  
Notes:
+
This repository setup allows the following workflow:
  
* [[Lunar Linux]] is one of the projects on '''foo-projects.org''' and I could have cloned from '''lunar-linux.org''' instead. The admins take security very seriously, so take care to be consistent and set up your '''.ssh/config''' properly or you will find your access blocked.
+
* first ensure that the my central repo is up to date:
 +
 
 +
  doppio$  cd moonbase.git
 +
  doppio$  git fetch              # note: fetch not pull
 +
 
 +
* then, either work in the remote working directory:
 +
 
 +
  doppio$  cd workbase.git
 +
  doppio$  git checkout master
 +
  doppio$  git pull
 +
  doppio$  git checkout -b testing master
 +
  doppio$  ... edit files and commit ...
 +
  doppio$  git checkout master
 +
  doppio$  git merge testing
 +
  doppio$  git branch -D testing
 +
  doppio$  git push
 +
 
 +
* or work on the local working directory:
 +
 
 +
  satellite$  cd workbase.git
 +
  satellite$  git checkout master
 +
  satellite$  git pull
 +
  satellite$  git checkout -b testing master
 +
  satellite$  ... edit files and commit ...
 +
  satellite$  git checkout master
 +
  satellite$  git merge testing
 +
  satellite$  git branch -D testing
 +
  satellite$  git push
 +
 
 +
* Finally I can update the central [[Moonbase]] repository, although I'm sure there must be a better way:
 +
 
 +
  doppio$  cd moonbase.git
 +
  doppio$  git push /var/git/lunar/moonbase.git master
 +
 
 +
* I can only share files back and forth between the local and remote working repositories using this method. I can't pull from the working directory on satellite to the working directory on doppio because satellite is private and can only call out. I can't push from the working directory on satellite to the working directory on doppio either because it is not a bare repository.
 +
 
 +
''Note that [http://www.kernel.org/pub/software/scm/git/docs/everyday.html Everyday GIT...] recommends having the following line instead:
 +
 
 +
  satellite$  git config remote.origin.push master:refs/remotes/satellite/master
  
* I've discovered that a '''git push''' directly from '''satellite:master''' to '''doppio:master''' works but it is easy to get out of sync. Or rather, I kept screwing up and having to delete and re-clone the repository. The last '''git config''' line makes it easier to push from '''satellite:master''' to '''doppio:satellite/master''' and then '''git merge satellite:master''' as described in [http://www.kernel.org/pub/software/scm/git/docs/everyday.html Everyday GIT...]
+
but this doesn't work with the bare repository system. Or rather, it does something, but I haven't worked out how to use the information in the remote repository.''
  
 
====Reviewing module submissions====
 
====Reviewing module submissions====

Revision as of 17:38, 24 May 2009

Contents

Module Submission for developers

Why and how users should submit modules is described on the Module_Submission page. This page describes what a developer needs to do to handle these submissions. Or rather, it describes what works for me.

I'm no expert by any means and this is not intended to be a comprehensive guide to git. There is just enough information to get started, and to describe my workflow.

Developer access

Before you will be able to save anything back to the central git repository holding the Moonbase, you will obviously need to apply for a username and password on the central machine. You will need to have been a regular, constructive contributor to the #lunar IRC channel on Freenode.net, or the Lunar Mailing-List or have successfully submitted modules in the past.

Cloning your own moonbase.git

Just like other source code management systems, git uses the concepts of a repository and working files. In the past git has allowed the repository and the working files to exist within the same top level directory structure. It seems that this has lead to some problems, so work is under way to separate out the repository and the working files. At the moment certain operations generate warning messages if you try to save to a combined repository and working directory. In the future you will need to have a "bare" repository and a separate working version. This article describes how to set up and use such a future-proof system.

I don't have a public server of my own on which I can make a repository available for sharing, testing and cherry-picking by others. Therefore I use a multi-stage repository and working directory system. If you have a public server, you may be able to work with just one.

I have three repositories: a remote copy of the main Lunar Linux Moonbase repository; a remote working copy; and a local working copy. The first one is optional, but it allows me to share files between the working copies without having to commit them to the main Moonbase repository first.

First of all I login to the main Lunar Linux machine, doppio, using ssh, and set two important global git defaults:

 doppio$  git config --global user.name "Full Name"
 doppio$  git config --global user.email "username@lunar-linux.org"

Then I create the remote "bare" copy of the main Moonbase repository and configure two useful properties:

 doppio$  git clone --bare git://lunar-linux.org/lunar/moonbase.git moonbase.git
 doppio$  git config --get-regexp '^(remote|branch)\.'
 doppio$  git config remote.origin.push master:master

Next I create a remote "working" repository that shadows the remote "bare" one, again on doppio:

 doppio$  git clone moonbase.git workbase.git
 doppio$  cd workbase.git
 doppio$  git config --get-regexp '^(remote|branch)\.'
 doppio$  git config remote.origin.push master:master

The second git config command tells git that the default action for the "push" command is to push changes from the master branch of the current repository to the master branch of the parent repository. Otherwise you need to provide that information on the command line every time.

I have a local "working" repository on my private machine that also shadows the remote "bare" one. I shall call my local machine satellite so that it matches the instructions given in Everyday GIT With 20 Commands Or So but see the Note at the end of the section.

 satellite$  git config --global user.name "Full Name"
 satellite$  git config --global user.email "username@lunar-linux.org"
 satellite$  git clone ssh://username@lunar-linux.org/~username/moonbase.git workbase.git
 satellite$  cd workbase.git
 satellite$  git config --get-regexp '^(remote|branch)\.'
 satellite$  git config remote.origin.push master:master

This repository setup allows the following workflow:

  • first ensure that the my central repo is up to date:
 doppio$  cd moonbase.git
 doppio$  git fetch               # note: fetch not pull
  • then, either work in the remote working directory:
 doppio$  cd workbase.git
 doppio$  git checkout master
 doppio$  git pull
 doppio$  git checkout -b testing master
 doppio$  ... edit files and commit ...
 doppio$  git checkout master
 doppio$  git merge testing
 doppio$  git branch -D testing
 doppio$  git push
  • or work on the local working directory:
 satellite$  cd workbase.git
 satellite$  git checkout master
 satellite$  git pull
 satellite$  git checkout -b testing master
 satellite$  ... edit files and commit ...
 satellite$  git checkout master
 satellite$  git merge testing
 satellite$  git branch -D testing
 satellite$  git push
  • Finally I can update the central Moonbase repository, although I'm sure there must be a better way:
 doppio$  cd moonbase.git
 doppio$  git push /var/git/lunar/moonbase.git master
  • I can only share files back and forth between the local and remote working repositories using this method. I can't pull from the working directory on satellite to the working directory on doppio because satellite is private and can only call out. I can't push from the working directory on satellite to the working directory on doppio either because it is not a bare repository.

Note that Everyday GIT... recommends having the following line instead:

 satellite$  git config remote.origin.push master:refs/remotes/satellite/master

but this doesn't work with the bare repository system. Or rather, it does something, but I haven't worked out how to use the information in the remote repository.

Reviewing module submissions

There is a neat script on doppio called review that you can use to view and apply the module submission patches directly to the current moonbase.git. The review script assumes that the submitter has already tested the patch and that if it applies cleanly it will be accepted, otherwise it will be rejected. One really cool feature of review is that the submitter gets the credit in the commit message. Unfortunately, if there are any problems in your moonbase.git - and there often were in mine - the script can delete the patch.

By default review will work through all of the modules in the submission queue so it's best to specify the module that you are interested in. review will open an editor session containing the module patch file so that you can view the changes, and potentially tidy the subject line and annotate what will become the text of the commit message. After you leave the editor, review checks whether the patch will apply to the current moonbase.git without problems, suggests what should be done with the patch, and waits for your input. If the patch applies cleanly, the default action is to accept it. You could still reject it for some other reason; because it does not conform to the Module_Guidelines for example. If the patch can't be applied - and that might mean that the submission was valid but that a developer has already updated that module in the moonbase - the action is to reject and another editor session will open so that you can explain why. The two other possible actions are defer and quit.

 doppio$ cd moonbase.git
 doppio$ git pull                       # make sure we are up to date
 doppio$ git checkout -b testing master # create a test branch in case of problems
 doppio$ review submitted_module        # view patch in editor, then accept,reject,defer,quit
 # if accepted:
 doppio$ git checkout master
 doppio$ git merge testing
 doppio$ git branch -D testing
 doppio$ git push                       # may give warnings about pushing master to master
 # else:
 doppio$ git checkout master
 doppio$ git branch -D testing

If all has gone smoothly, git status won't show any added, deleted or modified files. Take care to clean up if there are. You might also need to git pull again here, but I don't understand why.

Testing the submission locally before committing

If you want to be really thorough - or paranoid - you should really check that the patch applies and builds on your own machine first. That takes a little more effort, because you will first need to grab the patch file, apply and test locally:

 satellite$  scp username@doppio.lunar-linux.org:/var/anonymous/submissions/submitted_module.patch /tmp
 satellite$  cd moonbase.git
 satellite$  git pull
 satellite$  git checkout -b testing master
 satellite$  git am /tmp/submitted_module.patch
 satellite$  # check lin submitted_module works as expected

Notes:

  • OK, I admit it. I did mix lunar-linux.org and foo-projects.org and my access was blocked until I set up my .ssh/config properly.

At this point, if you want the submitter to get a feel-good "Your patch was accepted" message:

 satellite$  git checkout master
 satellite$  git branch -D testing
 doppio$     # use git and review as described earlier

Otherwise, just continue with the usual git workflow:

 satellite$  git checkout master
 satellite$  git merge testing
 satellite$  git branch -D testing
 satellite$  git push
 doppio$  cd moonbase.git
 doppio$  git merge satellite/testing    # see git config tip earlier
 doppio$  git push
 doppio$  review                         # patch already applied, so review will reject it

As mentioned earlier, run git status, tidy up if required, and run git pull again.

Is there a better way?

I'm an apprentice rather than a wizard level developer, and I've just about understood git-101, but diagnosing git problems makes my head hurt.

So if you can see a better way of doing things, please add a section at the bottom and describe your workflow.

Personal tools
Namespaces
Variants
Actions
Wiki Navigation
Project Sites
Toolbox