- Making your servers a bit more "green" with smart system adminstration, part 3: the right tool for the right job
- Integrating Subversion, Trac, and Mailman into a development environment: part 1, design
- Fixing permissions in Subversion post-commit hooks
- OpenSSL defaults for certificate generation
- "Maybe we should plug the guitars in"
Copying subversion repositories with history using svnsync
October 22, 2010,
Occasionally it is necessary to make a copy of a subversion repository, while preserving the revision history. Sometimes you need to do this without shell access to the server. In those cases, svnsync may solve your problem.
Recently we were confronted with the problem in a software forge we host, where developers sometimes need to copy the contents of a subversion repository to a new project. The downside of exporting the source repository and importing the source into the new repository is that the entire revision history is lost, which was not an option. The solution we would normally use in such cases is svnadmin dump to create a copy of the source repository, and svnadmin load to load the dump into the new repository, but the problem with that is that you need shell access to the servers hosting the subversion repositories.
We really don't like giving server access to anyone outside the company.
At the same time we also don't want to be bothered with requests for making subversion dumps, probably filtering out parts of the dump using svndumpfilter, and rolling the results into a new repository. At least, not if we can provide a perfectly useful solution that allows the developers to do it themselves.
We realized that this procedure can be generally useful. It will also allow you to reorganize a repository into multiple new repositories, or as in the case we were confronted with, you can fork a base project into spinoffs that require some customization, as is quite common in commercial software development. So we decided to invest some time in finding a reasonable solution. After a little research we found that svnsync can be (ab)used to create clones of a repository, as long as you fulfill a couple of requirements:
- you need read access to the source repository
- you need write access to the target repository
- the target repository should be empty
- the target repository must have a pre-revprop-change hook that exits with 0
The pre-revprop-hook needs a little explanation. It controls whether it is allowed to change revision properties, which is required for things like automatically expanding keywords in source files, but also if we want to add the revision history to a copy of a repository. If you're only going to copy the source repository once, you can get away with a little script like
#!/bin/sh exit 0
which allows all operations on revision properties by all who have access to the repository. Just remove it when you're done syncing. If you intend to allow more operations on revision properties, you probably need a more sophisticated hook script. See the pre-revprop-change.tmpl that subversion places in the hooks directory of all repositories for more information.
Once you have met the requirements, you need to prepare the target repository:
svnsync init --source-username <user> --source-password <pw>\ --sync-username <user> --sync-password <pw> <destination> <source>
(line break added manually for leggibility)
Obviously you should provide authentication credentials if you use these. The <destination> is the URL of the root of the target repository. The <source> is the URL of the source repository, or, as of subversion 1.5, the URL of a directory within the source repository. If all goes well, svnsync responds with:
Copied properties for revision 0
If it doesn't, fix whatever is wrong, rinse, and repeat. After that, you can sync the repositories with
svnsync sync --source-username <user> --source-password <pw>\ --sync-username <user> --sync-password <pw> <destination>
(line break added manually for leggibility)
The revisions will be synced one by one. If the process is interupted for some reason, you can try restarting this last command. With a little luck svnsync will restart where it left off.
Things to keep in mind:
- you can repeat the sync step to transport changes from the source repository, but once you check in modifications to the target repository, you can no longer sync it with the source repository.
- if you make a partial copy of the source repository, svnsync will still create the whole directory tree to the part you copied, so you might need to do some svn mv'ing afterwards.
- if your source repository contains many revisions, be prepared to make yourself comfortable. The sync process is not fast.