A quick review of both the #drupal IRC channel and the activity on the dev list is that, despite everything, Drupal’s new release system is a little confusing. And, truth to be told, it is a little confusing. In my [not so] humble opinion, this is not the fault of the implementation or the design of the system, but more the fault that Drupal users and developers have gotten accustomed to a system that sort of worked, but left a lot to be desired.

To start with, the original system worked well for Drupal core itself. This meant that a fair number of people, who primarily deal with core, never really got to get face to face with the problems that manifested on the contrib repository. But that’s a relatively small group of people, and all of them are very smart people. While it took some good arguing, the bulk of them were convinced that the new way is a better way, and thus the improvements actually came to pass.

The second matter is that a lot of Drupal contributors have no experience with formal software release cycles. And, to be fair, I think this is true in a lot of open source, though it seems to be moreso in the Drupal community than elsewhere. This lack of experience, coupled with the former difficulty in the tools, led to a very lazy, laissez-faire, method of Drupal contrib. For one, it means that a lot of people found it easiest to simply update their sites using cvs. After all, you simply need to point your system at the branch, type ‘cvs update’ and you’re done. Except of course that you don’t really have a good idea what code you have, which led to some developers getting flak for having ‘unstable’ code in a ‘stable’ branch. And so on.

None of this really matters, at this point. The new system is in, and it is now what we have to work with. Any changes to it now are going to be very minor and geared toward enhancements and refinements as opposed to any serious changes. To me, this is for the better. So here are my brief instructions on how Drupal contrib maintainers need to handle their modules.

First, you have to even be in CVS. I’m going to assume that my target audience has already figured out how to check something into CVS and has a project and Drupal has created tarballs. But I do want to explain one thing, because even people who use CVS sometimes don’t quite get this.

  • A branch is a new direction of code; a fork, if you will. Your code automatically starts with one branch, and that is HEAD. When you branch, it’s a little like each repository is different. Whenever you check into a branch, the ‘branch tag’ moves up (which is how CVS describes it)…but the easiest way to think about it is that once you make a branch, it is now a different repository. Until you check into the new branch or the old branch, they happen to have the same code. But if you want them to have the same code, you will then have to perform two checkins. The most current code in a branch is often referred to as the ‘tip’ of the branch.
  • A tag is a ‘stable’ snapshot of a branch. Tags shouldn’t move. They represent an instance of the code that you want to preserve. Ordinarily tags are used for releases, be they development, beta, release candidates or full-fledged releases. Using a tag lets you quickly and easily identify what code was actually running.

The Old Way

You worked in 2, possibly 3 branches, based upon the version of Drupal that your software was meant to work with.

HEAD, referred to formerly as CVS, was the newest, latest, grandest code. It was the one place you had for ‘unstable’ development code. Most often people would stay away from this, but sometimes using this code was necessary. The worst part about this, though, is that during the transition phase from one version of Drupal to another, it was (and in fact will continue to) remain impossible to tell at a glance what version of Drupal this is for. 3 times a day, the tip of the HEAD branch would be packaged into ‘module-cvs.tar.gz’ and made available for download on your project.

DRUPAL-4–6 was and will likely remain the ‘stable’ branch of code meant for Drupal 4.6. Whenever you checked into the DRUPAL-4–6 branch this code was meant to be stable. Every night the tip of this branch was packaged into ‘module-4.6-0.tar.gz’. About the only way to tell one module-4.6.tar.gz from another is by the datestamp on the download, which was often not preserved.

DRUPAL-4–7 had the exact same treatment. DRUPAL-5 was opened up but not particularly used, and was never packaged, but has the same idea.

The New Way

As a module maintainer, I have to do a little more work. Well, that’s not true. I don’t have to. In fact, if I do exactly what I did before, I’ll get relatively similar results. DRUPAL-4–7 is a branch. DRUPAL-5 is a branch. And those are the branches I check code into.

And while there’s an extra step here, I still have the snapshots that get run twice a day. But those have been renamed. They are now package into module-4.7.x-1.x-dev.tar.gz — as you can see, a little bit has been added to the filename (and the release name) to distinguish what it is. The ‘.x’ indicate sthat it’s a development version, it’s not tied to a particular version, and that the snapshot is theoretically unstable. What you get is going to depend completely on the module developer’s policy.

Now, when the system was updated, you got all of the existing DRUPAL-4–6 and DRUPAL-4–7 release for free. But you’re going to find that when you go and make your DRUPAL-5 branch that you don’t get this for free! Why not?

Well, you got them for free because the release system wanted to basically preserve what was there. When you create a new one, you have to go and create a new development release. More about that a little later.

So basically, we still use DRUPAL-4–7. But we can now also make real, actual releases. The benefits of this are many, one of the most obvious being that when people file bug reports, I can know what release they filed it against; but also, it’ll become much easier to tell when modules on your site need to be upgraded. It’s not safe to upgrade to development snapshots all the time, but if a developer actually rolls a real release, it is probably a lot safer to upgrade to it.

Actual releases are called ‘module-4.7.x-1.0–beta’, for example. 1 is the major version number, 0 is the minor version number, and as a module developer I’m allowed to put in an extra if I like. Maybe for a maintenance release or to indicate a beta, or the like. It's completely optional. I could've simply had module-4.7.x-1.0.

Super Fast How To

So what you want to know is…how do I, the module developer/maintainer, actually use the system?

  • First, I commit code. cvs commit -m’this is my commit comment’.
  • Now, I’m ready to create a branch because I’m about to update my module to Drupal 5: cvs tag -b DRUPAL-5 which I must immediately follow with cvs update -r DRUPAL-5 or I’ll have created a branch that I’m not using.
  • I surf to my project page, and I select ‘Add new release’. The branch I just created will be visible under the Branches section in the version drop down. If it’s all that’s available it’ll be defaulted, but if I have other branches/tags hanging around that don’t have releases attached to them, they will be there too. Select the branch I just created and, and in the description I put some witty comment about how this is an unstable development branch, use at your own risk. Then drupal.org will tell me that my development branch will be available in 12 hours. I pout, because I like instant gratification. But I’ll get over it.
  • Now, I’ve checked in some more code. I’m ready for people to try this. I’m going to create a release. Two steps. First cvs tag DRUPAL-5--1-0-BETA (I pick BETA because I’m not sure that it’s been tested enough and I want people to test it). Second, I go to my project, click add new release, and I select that tag. I make some notes about what I’ve changed that might matter to the consumer, and I’m done. Then drupal.org tells me that my release will be created within 5 minutes! I jump for joy, because in this case, I get my instant gratification. I can now blog post about this new release and provide links.

In the above, you’ll notice that the tag is basically some encoded information.

  • DRUPAL-5 — that’s the version of Drupal. I bet you have this.
  • -- That’s the indicator that’s separating the version of Drupal that my module uses from the actual version of my module.
  • 1 — That’s the ‘major’ version number of my module.
  • 0 — that’s the ‘minor’ version number of my module.
  • BETA — that’s something extra that I can put in. I could call it the ‘underpants’ release if I were so inclined. I’d probably be scowled at heavily by the more serious developers and get a good laugh from the rest.

There’s just one piece left to talk about, and that’s the advanced usage.

So let’s use Views as an example. And let’s say after awhile I’m up to Views 1.7–2 [side note: I picked this partly out of ridiculousness, that I had Views 1.7 and I am abusing the system to have had a 1.7-2 rather than having moved to 1.8] and it’s trucking along nicely and I’m about to completely change the API. I’m going to rip it out and replace it with something 10 times better. But the downside is that every module that now uses the Views API will break. Don’t laugh, this is exactly the example that we used when we convinced people of why we need this level of complexity in the system. Because I do want to do something like that at some point.

And when it happens, I don’t want to abandon everyone using Views 1.7–2. And I don’t have to.

cvs tag -b DRUPAL-5--2

What did I just do? I created another branch. Yes, this is a somewhat confusing, complicated branch. The good news is? You’ll probably only ever see a dozen of them on drupal.org. What does this mean? Because I branched code, when I have to fix bugs or add new minor features to Views 1, I can still make a Views 1.8 release, and it’ll be independent from Views 2. What’s the downside of this?

Some bug fixes will have to go in all of these branches:

  • HEAD
  • DRUPAL-5--2
  • DRUPAL-5
  • DRUPAL-4–7

And by that time, possibly

  • DRUPAL-6

And of course, the downside for YOU is that you now have to pick which one you’re going to use: 1.x or 2.x. But that’s the price that we’re going to have to pay so I can write new API without breaking the all the installations out there that are relying on Views and want to keep their modules up to date.

 

 

Comments

Good writeup! When you say, "Views 1.7–2", it is not be clear where the "-2" comes from.

And the problem in that case is because the '-2' is the 'extra' field. It's the same place that -beta or -rc goes, and it's something I made up on the fly as though I had decided I had released views 1.7, and found a bug that I had to fix but didn't feel worthy of being 1.8, so I made 1.7-2.

thanks for writing this up and sharing your thoughts. overall, this is mostly acurate. however, there are a few rather important errors that will spread confusion. please edit the doc (and feel free to delete this comment when you do). thanks!

  1. in your definition of tag, the last sentance should read: "Using a tag lets you quickly and easily identify what code was actually running."
  2. old way: the old tarballs were called "module-4.6.0.tar.gz" (for added ridiculousness, since you couldn't change the .0 *sigh*)
  3. this is wrong: "They are now package into module-4.7.x-1.x.tar.gz —" should be: "They are now packaged into module-4.7.x-1.x-dev.tar.gz —"
  4. this is also wrong: "Actual development snapshots are called ‘module-4.7.x-1.0–beta’, for example." EEEK! the docs call the nightly tarballs "development snapshots", and real releases "official releases". should be "Actual releases are called ..."
  5. technically, we're going with "major" and "patch level" as the 2 digits for core, and as the 2 digits for the second half of the version numbers for contrib. "minor" has all but disappeared from the drupal terminology...
  6. what is "Views 1.7–2" supposed to mean? ;) are you using "major.minor-patch" for views version numbers? eek. i mean, more power to you, but i can't be held responsible for any resulting confusion. ;)

Doc edited; comment retained so people who read it before the edits might someday realize I left some errors in. Thanks for the comments!

Let's say I have a site where I've checked out the 4-7 versions of modules. If I do a cvs update -dP what will I get? The latest stable releases, or the tip of the 4-7 branch (and hence a development version)?

You will get the tip of the DRUPAL-4-7 branch, which is a development version.

While this sounds scary, keep in mind that that is what you *always* got before; since we had no releases, there was no way to clarify what version you were actually getting, beyond the 'newest'. Any sense that you got a 'stable' version at that point was a mirage whose reality was based purely upon the whims and abilities of the individual developers to ensure their checkins.

Hopefully soon we'll have features that'll help power users quickly scan their module list for current versions and see what versions have been updated, help download and install them, and do the right things, and that will return us to the glory days of keeping our sites up to date with CVS.

DRUPAL-4-7 branch is (supposed to be) ONLY the stable branch for the 4.7.x-1.* series of a given module.

of course, since "cvs update" just gives you the end of the branch, it's less stable than an official release, e.g.:

cvs update -r DRUPAL-4-7--1-3

but it shouldn't be less stable in the sense of new features, major redesigns, etc. those should be happening on other branches (e.g. DRUPAL-4-7--2 or HEAD, etc).

please, please read New release system: debate on branches and versions (http://drupal.org/node/92452)

DRUPAL-4-7 is only for the 4.7.x-1.* release, and there is no such thing as DRUPAL-4-7--1. yes, i know it's slightly confusing, but my hands were tied by a) past practice and b) people who insisted on being able to deploy sites via "cvs update". ;)

thanks,
-derek

Cheers, that was an easy-on-the-brain tutorial.

thanks for the writup!

Sorry for asking this, but I'm really curious. Why CVS when it has severe architectural problems that Subversion was created specifically to fix. Subversion has been out for quite a while now and is extremely stable and you can load a CVS repository in it, including all the history. So, why CVS?

(And, as you might have guessed, I'm extremely partial to subversion over cvs. And, I realize that you probably didn't pick the system, but since you posted about it, I thought I'd ask.)

Because it was set up, has been up and running, and a switchover is a very serious undertaking with a fairly minimal real gain. Sure, CVS has some issues but it works for our purposes, and a major change in architecture is hard. There are much better places to spend our resources.

IMHO, part of innovation (or being innovative) is working with the resources that you have. CVS isn't the latest and greatest... but I works well and is still used by a majority of devs in our own projects. I think Merlin hit the nail right on the (pun intented) by writing this post. The major breakdown isn't in the system... although enhancements are part of any evolutionary process ... its properly educating users and contributers. Creating standards and from time to time auditing these standards will help Drupal go the next step, and teach developers about development and release cycles... which they can apply inside of Drupal and in their own projects. These practices are good to learn and will make you more successful and effecient.

Thanks Merlin!

before you create a branch, you have to add your module as a project to the drupal site. i did not do so in the first place, and got the error message:
"error no published project was found under this directory drupal cvs"
-alex

Nice tut merlin. e-z to understand :)

Thanks for your ideas Merlin.

Dang, I dont' know what is going on here. Every time I try do anything like create branch or try to add a module, I just get errors. I found it, that it is probably from not updating my database, but it is like impossible to do with stupid godaddy hosting and using linux. Anyone now how to update to the new 5.0 or better.

thanks for the writup!

Add new comment