The aging source-code management (SCM) system we’ve been using for the JDK all these years is unsuitable for open development. To which newer system should we migrate as we open-source the code?
A bit of history For over twelve years the JDK source code has been faithfully maintained by a trusty old Sun-developed SCM system called TeamWare, and its predecessors. These systems built upon the seminal SCCS single-file revision-control system—which itself dates almost to the beginning of time—to support large-scale, distributed software development.
The key difference between TeamWare and more-traditional SCM systems such as CVS and Subversion is that there is no central source-code repository. Each working copy of the source tree is, in effect, its own repository. This approach does tend to take up a bit more disk space, but it has several advantages:
A branch is just another repository, not some state in a central database. You can create branches at will to explore new ideas and, equally easily, simply delete them if things don’t work out.
You don’t need to be connected to a server—or even be on the network—in order to get work done.
You don’t need to set up and manage a central SCM host with sufficient disk space, compute power, bandwidth, and backup to support the concurrent SCM operations of your entire development community.
The main trouble with TeamWare is that it depends upon NFS for distributed operation. This works reasonably well so long as you’re not too far from the repository you’re trying to talk to, but doing an update across the U.S. is pretty painful and doing one halfway around the world—to, say, St. Petersburg—is basically impossible. Aside from performance there’s also the problem of how to set up NFS authentication across administrative domains—theoretically feasible, but practically fairly difficult.
TeamWare has other flaws, to be sure—some of them quite fundamental. We could easily open-source it, but from a technical perspective it’s just not a very good solution. Rather than invest a bunch of effort to improve it, then, perhaps it’d be wiser to look further down the evolutionary path.
Darwinian forces The distributed-development facilities pioneered in TeamWare were taken further in the closed-source BitKeeper product, which was used to maintain the Linux kernel tree for a few years, and also in various open-source projects such as Bazaar, Darcs, Git, Mercurial, and Monotone.
These philosophical descendants of TeamWare abandoned the antique SCCS file format and replaced NFS with protocols built on top of HTTP or SSH, or in some cases both. They also introduced many other improvements, perhaps the most important of which was to redefine the notion of a commit from a group of updates to individual files in the repository to genuine a atomic transaction upon the entire repository. This is not only a (much!) more robust approach but it also allows file creates, renames, and deletes to be tracked right along with file-content changes.
Natural selection Now if we want to choose the best of the current crop of distributed SCM systems to use for the open-source JDK we should formulate a list of requirements, conduct an in-depth study of the various candidates, and then pick a winner.
It turns out—handily for us—that the OpenSolaris Tools Community has already done just that. They started out pretty much where we are now, trying to figure out how to move forward from TeamWare in an era of open-source development. They wrote up a list of requirements, conducted a thorough evaluation of the best alternatives, and eventually settled upon… Mercurial.
Definite leanings Our goal is to make the right SCM choice for the open-source JDK; we’re not going to blindly follow the OpenSolaris team. Given our very similar starting points we’d be foolish to ignore their conclusions, however, and after doing some research and experiments of our own we are strongly leaning toward Mercurial. If you think there’s a better choice here then by all means please do let us know!
Migrating a project of this size from one SCM system to another is, of course, a major effort. There are both technical problems (e.g., how do we transfer history information from TeamWare into the new system?) as well as process and planning challenges (e.g., how do we do this in a way that minimizes the disruption to the team?). My colleagues Kelly O’Hair and Martin Englund are busy working out these issues and have more to say in their own blogs.