Subject: Re: ch-ch-ch-changes...

From: "Mike Crowe" <[email-masked]>
Date: Mon, 1 May 2000 11:39:44 +0100

You wrote:
> > I'm looking for a bit of general advice from experienced kernel
> > developers.  Are there any clever techniques for integrating any changes
I
> > make to a kernel/patch version with new patch versions as they come in?
I
> > understand that in some cases, doing it by hand is the only way to go.
> > But with so many different groups working on the same project I figured
> > there might be some diff/patch tricks to make it easier.
>
> Since you work on SA1100, here what you could do:

[snip Nico's procedure]

Nico's manual method seems to be very close to that used by CVS with vendor
branches. We use them here and incorporating Nico's patches into our sources
and then producing a new patch for him containing our changes is relatively
straightforward. Actually patching up to a newer version is still not a
no-brain operation (particular if, like us, you don't actively track every
version) but it does make sorting out conflicts much easier.

See http://www.loria.fr/~molli/cvs/doc/cvs_13.html#SEC98

--
Mike Crowe
empeg ltd

unsubscribe: body of `unsubscribe linux-arm-kernel' to [email-masked]

---

From: Andreas Bombe <[email-masked]>
Date: Mon, 1 May 2000 16:11:56 +0200

On Fri, Apr 28, 2000 at 10:22:36AM -0400, Nicolas Pitre wrote:
> 
> 
> On Fri, 28 Apr 2000, Chris Blazie wrote:
> 
> > 
> > developers.  Are there any clever techniques for integrating any changes I
> > make to a kernel/patch version with new patch versions as they come in?  I
> 	mkdir 2.3.99-pre3-rmk3-np2
> 	cd 2.3.99-pre3-rmk3-np2
> 	tar -xzf linux-2.3.99-pre3.tar.gz
> 	zcat patch-2.3.99-pre3-rmk3.gz | patch -p0
> 	zcat diff-2.3.99-pre3-rmk3-np2.gz | patch -p0
> 	cp -a linux linux.orig

Using the -l option to create hardlinks instead of real copies is a
big win here...
 
> Now you have two copies of the source tree.  Make your changes in linux/
> but keep linux.orig/ untouched.  When you are done:
> 
> 	diff -urN linux.orig linux > my_changes.diff

... since diff finds out that e.g. linux.orig/Makefile and
linux/Makefile are the same file and then knows that they have
identical content and that there is no need to read and compare both
files.  For something as big as the Linux kernel sources this is a big
win (if the inodes are already in the cache it usually takes only a
few seconds).  And it saves you a lot of disk space for identical
files.

However, you need an editor that doesn't overwrite files (this would
change both links) but creates a new file everytime it saves.  Emacs
works like that (when saving x.c:  unlink x.c~; mv x.c x.c~; write
into x.c) unless configured otherwise.  Luckily, patch does also, so
you can apply your personal diff to the linked copy and have different
trees afterwards.

-- 
 Andreas E. Bombe <[email-masked]>    DSA key 0x04880A44
http://home.pages.de/~andreas.bombe/    http://linux1394.sourceforge.net/

---

From: Nicolas Pitre <[email-masked]>
Date: Mon, 1 May 2000 11:53:35 -0400 (EDT)

> Using the -l option to create hardlinks instead of real copies is a

Absolutely not.  The idea of copying the whole tree is really to preserve
a non-modified copy.  Otherwise the idea of having two separate trees
doesn't mean anything.

If you want to do such development you should manage
for disk space not to be an issue. 300-500 megs of disk space is pretty
afordable today.

Of course you could use CVS which is lot more efficient on disk usage when
you know enough about it to have a proper setup. But I wanted to
illustrate a straight forward procedure which works for sure all the time.

Nicolas

---

From: "Mike Crowe" <[email-masked]>
Date: Mon, 1 May 2000 18:48:20 +0100

> a non-modified copy.  Otherwise the idea of having two separate trees

But provided your editor isn't too intelligent the hard link case works too.
The first time you edit the file you lose the hard link to the original so
have forked versions.

> for disk space not to be an issue. 300-500 megs of disk space is pretty

Although I've never tried it I can believe that diff checks the inode number
and won't bother doing any work if the files are one and the same. I think
this was Andreas' point.

--
Mike Crowe
empeg ltd

---

From: Nicolas Pitre <[email-masked]>
Date: Mon, 1 May 2000 14:38:29 -0400 (EDT)

> The first time you edit the file you lose the hard link to the original so

Really?  Didn't know hard links are copy-on-write.  If it is the case then
it would be nice indeed.

Nicolas

---

From: Russell King - ARM Linux Admin <[email-masked]>
Date: Mon, 1 May 2000 23:28:39 +0100 (BST)

> Really?  Didn't know hard links are copy-on-write.  If it is the case then

What Mike is refering to is editors like MicroEMACS, which, when saving a file
out write out a temporary file, and then rename it to the desired filename.
This kills any symlink/hardlink you have which is nice because it gives the
impression of copy-on-write.
   _____
  |_____| ------------------------------------------------- ---+---+-
  |   |        Russell King       [email-masked]      --- ---
  | | | |            http://www.arm.linux.org.uk/            /  /  |
  | +-+-+                                                     --- -+-
  /   |               THE developer of ARM Linux              |+| /|\
 /  | | |                                                     ---  |
    +-+-+ -------------------------------------------------  /\\\  |

---

From: Andreas Bombe <[email-masked]>
Date: Tue, 2 May 2000 01:13:57 +0200

> a non-modified copy.  Otherwise the idea of having two separate trees

Exactly.  Using patch with standard backup option and editors with
move-to-backup-and-create-new instead of overwrite allow that.
Originals (=linked) files which are edited and patched are moved out
of the way, the original tree is therefore preserved.

Just as I described in the paragraphs you deleted.  I do it like that,
successfully, no problems.

> for disk space not to be an issue. 300-500 megs of disk space is pretty

It is still wasted when more than 95% of the files are unmodified
(unless you're doing heavy bug fixing all over the place you modify
only few files).  Plus you waste the time with reading and comparing
almost (more than?) 200MB of data everytime you do a diff.  Hardlinked
trees diff in 2 to 4 seconds (with inodes in cache, lowly K6 233MHz).

> illustrate a straight forward procedure which works for sure all the time.

Tried CVS.  It's slow as hell only checking in kernel patches, so I
didn't proceed along that line...  I guess it suffers from having to
read and diff every single file for a checkin or to create diffs.

---

From: Nicolas Pitre <[email-masked]>
Date: Mon, 1 May 2000 19:51:53 -0400 (EDT)

> of the way, the original tree is therefore preserved.

OK.  I understand now.  

> successfully, no problems.

Sorry for having missed your point.  The gain in speed is certainly worth
it.  However I would rather suggest it as an upgrade from my procedure
i.e. after the unfamiliar person got more familiar and is less likely to
forget about an extra parameter or configuration somewhere.  Also there is
still those dumb editors some people still use...

Nicolas

---

From: Mike Crowe <[email-masked]>
Date: Tue, 2 May 2000 09:14:29 +0100

> read and diff every single file for a checkin or to create diffs.

Odd, which version? I'll admit that it isn't quick on a complete kernel but 
it certainly doesn't do a diff on every file (I've run it over a 28.8 modem 
in a few minutes with an unchanged tree which certainly isn't possible if
it has to transfer the contents of every file).

Besides, once you've checked in your changes you no longer pay the diff
penalty anyway.

Oh, and back to the editors thing. My copy of xemacs is intelligent about
hard links and keeps the link which is obviously rather bad in this
case. It can probably be persuaded not to though.

-- 
Mike Crowe
empeg ltd

---

From: Mike Touloumtzis <[email-masked]>
Date: Wed, 3 May 2000 15:10:36 -0700

> case. It can probably be persuaded not to though.

GNU emacs has a 'backup-when-copying-when-linked' variable that (I
believe) controls its behavior in this case; maybe you could look for
that in xemacs.

miket