Monday, April 23. 2007
Improving svnmerge
There's been a lot of writing about svnmerge: Ken Kinder wrote a nice introductory article on the topic, and now there's a wiki and even a mailing list. Maybe someday soon it will depart the contrib/ purgatory!
One unusual use of svnmerge is to "branch" a public subversion repository into your local repository, to allow local development while still tracking the public trunk. This is related to vendor branches, but is more suited to the case where you'll be submitting changes back to the project, and is particularly useful if you have commit permission on the public repository. For me, I was merging from the Python repository (http://svn.python.org/projects/python/trunk/) to my own private repository (let's call it http://svn.v.igoro.us/python/trunk).
Svnmerge has a few weaknesses, but one that surprised me was this: while svnmerge can manage changes between different repositories, it can't do so when the repository-relative path is the same in each branch. In this case, the repository-relative path for both is /python/trunk, so svnmerge complains:
svnmerge: cannot init integration source '/python/trunk' It must differ from the repository-relative path of the current directory.
Getting Under The Hood
To understand why this limitation exists, you need to look at how svnmerge works its magic. For each managed branch, svnmerge keeps a list of the revisions in the source branch that have been merged. By default, this list is stored in the property svnmerge-integrated, looking like /python/trunk:1-54918,54920-54926. When merging new changes (svnmerge merge), this property gets updated to reflect the newly merged revisions. The problem, in this case, is that the identifier for the branch does not include any information for the repository: does this property list revisions in my repository, or in the Python repository?
The Fix
The solution I found to this problem was to qualify the properties with an identifier for the repository. For most repositories, the obvious choice is to use a full URL, e.g., http://svn.python.org/projects/python/trunk:1-54918,54920-54926. For repositories which might be accesed via different URLs by different people, the UUID might be a better idea, e.g., uuid://6015fed2-1504-0410-9fe1-9d1591cc4771/python/trunk:1-54918,54920-54926. To be general, I introduced the notion of a "location identifier" to specify the location of a branch. Currently, there are three location identifier formats:
- path: the "old way" with a simple repository-relative path
- url: a fully qualified URL for the branch
- uuid: a UUID-based identifier
When initializing a new branch, you can specify one of these formats with the --location-type flag:
$ svnmerge init --location-type uuid http://svn.python.org/projects/python/trunk property 'svnmerge-integrated' set on '.' $ svn pg svnmerge-integrated . uuid://6015fed2-1504-0410-9fe1-9d1591cc4771/python/trunk:1-54928
The Future
Subversion 1.5 promises to support merge tracking natively. From what little I've seen, it does this using a similar technique -- keeping lists of revisions in properties. However, the developers are not recommending that folks all convert to 1.5 immediately -- it looks like it will be a significant change that needs some serious testing first. Even if it were stable, most Linux distros are so slow to upgrade that it's reasonable to assume we'll all be using 1.3 for a good while.
This patch is in the submission process on the mailing list, but an updated version of svnmerge.py is available on this site, if you'd like to take it for a spin. Any feedback would be appreciated!
Hello Dustin! Your svnmerge.py improvement fixes exactly the limitation I'm currently facing, merging between two repos with identical paths. Thanks for this enhancement! I am however running into an error using your revision of the script, and I cannot debug it 'cause I'm a total python neophyte! Can you please help me? The error I'm experiencing is detailed here: http://apollo.homeunix.net/~juan/svnmergeerror (terminal output looks awful pasted here!). svnmerger23865.py is the name I gave to your revision of the svnmerge.py script (branched at r23865 off the official repo), to differenciate it from the HEAD revision. Would really appreciate your help, let me know if you need any system information to debug the problem [1]. Thanks in advance! [1] Intel Mac running Mac OS X Tiger 10.4.10 and stock Python 2.4.4 as shipped by Apple.
Juan -- the error you see (KeyError: 'Repository UUID') happens when svnmerge tries to examine a repository that doesn't have a UUID. The macports.org repository does have a UUID, so it must have been your local repository that didn't.
This is definitely a bug, so I'll take a look.
I would love to hear from you on the svnmerge mailing list! http://www.orcaware.com/mailman/listinfo/svnmerge
Hi Dustin! Thanks for inviting me to the svnmerge.py list. I joined and posted some feedback on a different failure I'm now experiencing after fixing the repository UUID errors in my local repository (the repository was OK, the checkout was somehow broken). Thanks for your help and time! -jmpp.
Thanks for the fix Dustin. Flag information you provided
$ svnmerge init --location-type uuid http://svn.python.org/projects/python/trunk property 'svnmerge-integrated' set on '.' $ svn pg svnmerge-integrated . uuid://6015fed2-1504-0410-9fe1-9d1591cc4771/python/trunk:1-54928
has helped in my case.
Thanks a bunch :)
