Archive for June, 2007

The promotional model

Friday, June 29th, 2007

Yesterday I modified a Bugzilla entry to indicate that the change was ready for QA testing. QA responded that not only was the bug fixed, but it was also working in production. That's when I realized that the build that went to production the night before actually included my fix, before QA had had a chance to validate it. Our promotional tracking failed.

If you manage a live production system, you need a good set of procedures to define how changes are rolled out. It's a common problem. How do you control what changes to into production? How do you notify QA of all the bug fixes that they should be testing? How do you know what version of the software currently in QA? How do you pinpoint an issue to a particular build? Although the problem is a common one, it seems that every place I've worked has had a different solution.

The ideal solution is known as the promotional model. This is the world view whereby builds are promoted from one platform to the next. At a minimum, you need five:

  • development
  • integration
  • QA
  • staging
  • production

The purpose of these five platforms is to test different parts of the system with reasonable isolation. Programmers can test their individual features in development, and then make sure they don't break each other in integration. Meanwhile, QA is running regression tests on a stable build. The staging area is used to test the deployment process itself before it finally sees the users in production.

The promotional model forms a pipeline. A build of the software moves through this pipeline, stopping at each of the five platforms along the way. When it fails to pass the tests at a particular platform, it falls out of the pipeline and another build comes up behind it. No build proceeds to the next platform until it passes all tests at the previous platform.

The promotional model has no forks. So while it is likely that you will have different builds installed on different platforms at any given time, a build cannot pass another one in the pipeline. Unfortunately, this makes emergency releases extremely disruptive. If QA is testing the new features when you find a problem in production, you have to take the feature build out of the pipeline to allow the fix build to flow through. That means that QA has to stop testing new features in order to perform a regression test on the fix. I've been on teams that have tried to short-circuit this process to their own detriment. Just take it as a given that you will disrupt QA testing to push an emergency fix.

The promotional model is not the same as source control, but they do touch each other. The source control system is allowed to branch. This is necessary when a change needs to be applied to a build that occurred in the past. So while it is not possible for a build to pass another one within the pipeline, it is possible to push through a build from an older branch.

The promotional model is not the same as bug tracking, but they do touch each other. A bug is opened against the particular build currently installed on a particular platform of the pipeline. It should be possible to identify the exact branch and revision that were used to create the build in question. So it should be standard practice for a developer to pull the code in question before looking for the bug. It should be, but in my experience it rarely is.

The industry has known about the promotional model for many years. In fact, some source control systems claim to support the promotional model. They usually support this claim with branching and tagging features. It's not the same thing. To do promotion right, you should implement both source control and bug tracking, and then integrate them with a database that tracks builds through the pipeline.

Phones are weird

Thursday, June 28th, 2007

I work in the industry, so I can say it. Phones are weird.

Have you noticed that the iPhone plans have unlimited data, but a limited number of SMS messages? Don't they realize that we can just use web-based IM?

Have you noticed that a song in iTunes costs $0.99, while a piece of a song in a ringtones WAP deck costs $2-$5? Don't they realize that we can just drop an mp3 file in the right folder?

Have you ever called someone intending to leave a voice mail, only to have them pick up? Shouldn't we be able to choose to record a message rather than wait for the phone to ring four times and to listen to the greeting?

Have you noticed that you need a special app to get YouTube content on the iPhone? Isn't that a full version of Safari in there?

Have you noticed that the iPhone doesn't work with Exchange Server? How is this going to attract the executive Treo users who can afford to carry this status symbol?

Have you noticed that you usually buy your phone from your network provider? When's the last time you bought a computer from your ISP?

Have you noticed the missing feature on iPhone's maps? You have to enter a city name to do a search. Can't it tell where you are based on the towers it's connecting to?

With all of the entrenched economics and politics, phones just don't work the way they should. They are just weird.

AiS 31: The Switch

Tuesday, June 26th, 2007

Listen Now

If you're new to the show, then welcome. If not, then you probably know that I am a Java programmer by day, and .NET by night. So it may come as a surprise to you that I have purchased a Mac. I got a 15 inch MacBook Pro. The "entry-level" version with a 2.2 GHz processor, not the 2.4. And certainly not the 17 inch boat that frankly should come with a set of trucks so you can ollie and rail slide your way to work.

If you are not new to the show, then you probably know that I am passionate about quality. I'm on a constant quest for quality in my own software and in the software that I use. I've heard it said that people who love software want to build their own hardware, but I didn't really understand that until I used my new Mac. This thing was built to run quality software.

For one thing, you have the specs. This, like I said, is a 2.2 GHz laptop. It comes with 2 Gig of memory and a 5400 RPM hard drive standard. You can't choose anything less and still have a MacBook Pro. Apple just won't let you. Other manufacturers sell similar specs for the money, but their stuff appears less expensive because you can buy less machine.

For another thing, you have the screen. This 15.4 inch monitor has a 1440x900 resolution. That's the same as the 22 inch panel on my desktop. Do the math ... the not new among you know that I love to do the math ... and you see that this screen is twice as dense! Apple won't let you see a pixely picture. There's a reason that Safari looks bad on Windows ... it was designed for a Mac!

So this is hardware worthy of quality software. It begs for quality software. In fact, if you write crappy software for the Mac, nobody will use it. The Apple community has come to expect a certain level of polish on their apps. Sure, some of them only go skin deep, but at least the effort is visible. I use an open-source instant messenger called Adium, and even this little app has the clean simple look of OS X. It even has a gorgeous little green duck icon that looks great sitting in the dock next to the Garage Band guitar. I'm a Java developer, so I know. You're lucky if we replace the coffee cup. And when I'm writing a Windows app, 16 colors and a copy of Paint is good enough.

But you know that I can't live without my .NET. Having written in C#, I never want to go back to C++. The Mono Project isn't keeping up, so I, of course, installed Boot Camp and Vista on this baby. And let me tell you, it runs Vista like a dream. I can't imagine the heat that the Boot Camp team gets in Cupertino for writing Windows drivers, but they did an excellent job.

Anyway, it's not my intent to become a Mac bigot or to convince you to switch. I just know quality when I see it, and the not new know that I'm all about the user experience.

My name is Michael L Perry, and I run Windows on the Mac.

Subclipse: Problem Occurred. Cannot Checkout.

Wednesday, June 20th, 2007

Here's one for the record books.

Problem Occurred: cannot checkout

I have just tried to import a Team Project Set into Eclipse from an SVN repository using the Subclipse plugin. Checking the Console window in Eclipse, I see that the problem was an obstructed update. I already had a project directory by that name. I need to delete it first.

So the plugin already knew what the problem was. After all, it put the error message in a different window. It could have at least displayed it in the popup.

In this situation, there are two correct actions for the software:

  1. Fix the problem itself (after prompting, since this would be a destructive fix), or
  2. Tell me how to fix the problem.

Those are the only correct options, but at least displaying the error message would be a start.

Eclipse's plugin architecture may be to blame for this one. It is difficult to get different components to work together, especially if they were written by entirely different teams and integrated only on the user's desktop. Perhaps there was no easy way for the plugin to give its host enough information. Composite applications often turn into Frankenstein's Monsters. But Eclipse is usually better integrated than this.

AiS 30: Web Development Techniques

Wednesday, June 20th, 2007

Listen Now

Charles Martin of Before You Are Gone shares some of his time-tested web development techniques. And we even make some up on the spot.

First, consider creating separate servers. You need at least four. Start with a development server where the team can first integrate their code and see if it works. Then deploy to a test server where your QA team can exercise it. Practice your deployment on a staging area separate from the others so that you can be certain that you can upgrade the current production release to the next one. Finally, of course, you have the production environment, far removed from the others.

For the love, don't develop in production!

Second, the separation of style from content is essential to modern web development. Always create a CSS file rather than embedding style within the page. Understand how specificity determines how styles are applied to elements of a document. And to test your CSS file, consider creating a single template page that includes examples of all elements that you use in the entire site. This gives you one place to look to see if your CSS change affects anyone else.

And third, configure your environment to report all Javascript errors. Web browsers are very forgiving by default. They have to be because of all the legacy code that is out there on useful sites today. Since you don't have a compiler to check for errors, you have to rely on the browser to perform that function. Let it tell you about all the errors on your pages, and test each page on all major platforms. Now that we have Safari for Windows, the Mac - and the iPhone - cannot be ignored.

Three degrees of freedom in ant

Friday, June 15th, 2007

I've just finished updating the build process at Handmark (again). This time, the goal was auditability and repeatability. We wanted to have a permanent record of exactly which revision was built when, and the ability to quickly get back to that exact build.

The prior build process had two degrees of freedom. On one axis, we have the set of projects. On the other, we have the set of build steps. We represented these two degrees of freedom with two separate ant scripts. build.xml lists the many different jar and war files and the relationships among them. It calls actions.xml via the <ant> task, which lists the build steps like clean, get, compile, unit test, jar, and war. For details, please see Separation of Concerns in Ant.

The command line to run the build took the branch and revision, like this:

ant -Dbranch=trunk -Drevision=24516 clean,war

It would pull the source code and build the war files. When it was finished, the branch and revision number were embedded within the manifest of the war files, but there was no permanent record that this build occurred. In addition, if we wanted to go back to a previous build, we would have to find the branch and revision number and reissue the command. And to be absolutely safe, we would have to pull the previous versions of the build scripts and properties files to ensure that we got exactly the same results.

Here's my solution
I wrapped this build process within another layer. We now have a build script that loads a file called This file contains two lines:


It then creates a temporary working directory, pulls the build scripts by that branch and revision into that directory, and launches them.

<!-- Load the tag file. --><property prefix="tag" file="${basedir}/" />
<!-- Get the project files. --><svn username="${username}" password="${password}">    <export srcUrl="${svnbase}/${tag.branch}/Projects/MyProject/" revision="${tag.revision}" destPath="${work}" /></svn>

<!-- Build the war files. --><ant dir="${work}" target="war" inheritall="false">    <property name="branch" value="${tag.branch}" />    <property name="revision" value="${tag.revision}" />    <property name="username" value="${username}" />    <property name="password" value="${password}" />    <property name="thirdparty" location="${thirdparty}" /></ant>

Now we can change and check it in. We can go back to any prior build by checking out the file that was current at the time. We can be sure that the build process is repeatable because even the build scripts themselves are pulled by revision. And best of all is the new command line:


AiS 29: The New Programming Model

Tuesday, June 12th, 2007

Listen Now

Several recent events are converging on a common theme.

Windows Presentation Foundation brings new user interface possibilities to Vista. Silverlight delivers WPF to the browser. Microsoft’s surface computer let’s us interact with the computer on a near-tactile level. Steve Jobs announced that the programming model for the iPhone is Ajax in Safari. Apple ported Safari to Windows. And Google Gears allows you to run a web application while not on the web.

All of these events demonstrate that the new programming model is markup and script. This has been the programming model for Web 2.0, but it is now it is breaking out of the browser.

This programming model has some promise, but there is a dark side. This moves us away from computer science based in proof. The tools that we have chosen to use for the new programming model are usually dynamically typed. With the exception of C# in WPF, the scripting languages that we bolt on to the markup are neither compiled nor statically typed.

Static typing is not just a way to give us intellisense, but it is a way to express intent in the form of a contact. Type checking is one simple way for the compiler to check a contract. But wouldn’t it be great if the compiler could do even more for us? Current research languages can prove that null pointers are not derefferenced. And with existing theorem provers, it is feasible to verify preconditions and post conditions at compile time. Despite the apparent evidence to the contrary, the web model is leading away from a path of increased productivity.

In addition, the web model does not support real-time collaboration. The best we can achieve with a request/response protocol like HTTP is rapid refresh. That solution is not scalable, and not feasible for many kinds of highly collaborative applications. And even when it works, the programming model forces developers to be aware of the fact that they are checking for updates. They can’t just express the relationships among objects and leave it at that.

I see a new programming model on the horizon after the markup/script wave has passed. In this programming model, the code represents the relationships between objects, not the triggers that cause the next domino to fall in a Rube Goldberg machine made of script. The compiler and the runtime can work together to make inter-related objects behave in a provably consistent way.

Update Controls is the first step along this path. With this library, you express the relationships between your user interface components and your data. You say what is to be displayed; the runtime takes care of when. If the data is changed in one window, the library pushes that change to the others automatically.

The next step is a collaborative object model that pushes those changes across the wire. When that step is ready, data can be changed on one computer, and the effect will automatically appear on another. Best of all, application developers won’t need to code that behavior. They simply express the relationships among objects and the runtime makes it happen.

I hope you will be with me for the next few steps in our evolution. The future is rapidly approaching in which we bring great value to our clients. It’s closer than you think.

Microsoft Surface
iPhone programming model
Safari for Windows
Google Gears
Update Controls

AiS 28: Closure

Wednesday, June 6th, 2007

Listen Now

A recent article in MSDN magazine introduces several new features of C# 3.0. The goal of these changes was to add query capabilities to the language. Lambda expressions were added so that the where clause could be written. Lambda expressions required type inference. Type inference allows for anonymous types. But anonymous types could not be stored without implicitly typed variables.

These features could not be added individually. Each feature brings capabilities to the language that are incomplete without the others. This is an excellent example of closure.

The idea of closure appears often in mathematics. The set of integers, for example, is closed under the operations of addition, subtraction, and multiplication. If you add, subtract, or multiply two integers, the result will be an integer. Introduce the concept of division, however, and the system is no longer closed. Now you need a larger set to express the possible outcomes. You need rational numbers. Introduce exponentiation, and now you need the reals.

In software, closure is often a consideration in language design. But it can also be applied to applications. The set of states that can be represented should be closed under the set of operations that the user can perform. If the user can perform an operation on the system that would put it in an invalid state, then that operation is a design flaw.

Join the panel as we discuss several issues from past experience that were solved or could have been avoided by applying the concept of closure.

Bill Gates and Steve Jobs on the magic of software

Friday, June 1st, 2007

Bill Gates and Steve Jobs participated in an interview for D5. The interviewers and audience were generally looking for points of tension, but Bill and Steve gave them so much more. The message was one of passion. Bill summed it up here:

If you look inside my brain, it's filled with software, and the magic of software, and the belief in software, and that's not going to change.

-- Bill Gates

Video on D5