The promotional model

Yesterday I modified a Bugzilla entry to indicate that the change was ready for QA testing. QA responded that not only was the bug fixed, but it was also working in production. That's when I realized that the build that went to production the night before actually included my fix, before QA had had a chance to validate it. Our promotional tracking failed.

If you manage a live production system, you need a good set of procedures to define how changes are rolled out. It's a common problem. How do you control what changes to into production? How do you notify QA of all the bug fixes that they should be testing? How do you know what version of the software currently in QA? How do you pinpoint an issue to a particular build? Although the problem is a common one, it seems that every place I've worked has had a different solution.

The ideal solution is known as the promotional model. This is the world view whereby builds are promoted from one platform to the next. At a minimum, you need five:

  • development
  • integration
  • QA
  • staging
  • production

The purpose of these five platforms is to test different parts of the system with reasonable isolation. Programmers can test their individual features in development, and then make sure they don't break each other in integration. Meanwhile, QA is running regression tests on a stable build. The staging area is used to test the deployment process itself before it finally sees the users in production.

The promotional model forms a pipeline. A build of the software moves through this pipeline, stopping at each of the five platforms along the way. When it fails to pass the tests at a particular platform, it falls out of the pipeline and another build comes up behind it. No build proceeds to the next platform until it passes all tests at the previous platform.

The promotional model has no forks. So while it is likely that you will have different builds installed on different platforms at any given time, a build cannot pass another one in the pipeline. Unfortunately, this makes emergency releases extremely disruptive. If QA is testing the new features when you find a problem in production, you have to take the feature build out of the pipeline to allow the fix build to flow through. That means that QA has to stop testing new features in order to perform a regression test on the fix. I've been on teams that have tried to short-circuit this process to their own detriment. Just take it as a given that you will disrupt QA testing to push an emergency fix.

The promotional model is not the same as source control, but they do touch each other. The source control system is allowed to branch. This is necessary when a change needs to be applied to a build that occurred in the past. So while it is not possible for a build to pass another one within the pipeline, it is possible to push through a build from an older branch.

The promotional model is not the same as bug tracking, but they do touch each other. A bug is opened against the particular build currently installed on a particular platform of the pipeline. It should be possible to identify the exact branch and revision that were used to create the build in question. So it should be standard practice for a developer to pull the code in question before looking for the bug. It should be, but in my experience it rarely is.

The industry has known about the promotional model for many years. In fact, some source control systems claim to support the promotional model. They usually support this claim with branching and tagging features. It's not the same thing. To do promotion right, you should implement both source control and bug tracking, and then integrate them with a database that tracks builds through the pipeline.

15 Responses to “The promotional model”

  1. Valerie Says:

    My previous company has six steps for their own production model. That includes "software testing" in between staging and production, wherein users (usually, our clients) test out the applications for themselves. If they find it satisfactory, then it's on to production. If not, then it goes back to the QA to iron out the bugs. It works fine with us.

  2. Robert Says:

    Wow. I just realized that the question I made was really stupid and somewhat insensitive. I apologize for that mistake.

    Sorry. I can't erase nor edit that comment.

  3. Michael L Perry Says:

    I didn't see the problem, but I deleted the comment for you. Go ahead and post again if you like, and I'll delete these intermediate posts.

  4. Adriel Karn Says:

    I've undergone these promotional models but one thing I gonna ask you, what's the difference with these promotional models to SDLC Software development Life Cycle? @Valerie: we six also!

  5. Jeannie Says:

    "The promotional model is not the same as bug tracking, but they do touch each other. A bug is opened against the particular build currently installed on a particular platform of the pipeline. It should be possible to identify the exact branch and revision that were used to create the build in question. So it should be standard practice for a developer to pull the code in question before looking for the bug. It should be, but in my experience it rarely is."

    I thought this was interesting about what should be standard practice. It makes perfect sense. Too bad it is not followed, as I have gotten some pretty buggy software at time.

  6. Valerie Framo Says:

    @Adriel: Oh yeah? Well, almost all other companies has six.

  7. Willilam Says:

    Yes I must agree that there are really stages on how to do things in production business. My working company right now almost have the same thing like this.

  8. Michael L Perry Says:

    Adrian,

    The promotional pipeline is just one component of the SDLC. Its only focus is on getting changes from development to production. SDLC covers the entire process, from discovery through maintenance and support. And while promotion has some very strict, nearly universal best practices, SDLC is more flexible and can be tuned to the organization.

  9. David Brown Says:

    I think it is remarkable that you could decide to put a great deal of effort into the design stage. I am frequently let down in just how minimal an effort may be made by designers and experiencing your commitment to the enhancement of high quality in the software industry makes me extremely pleased. I always look for brand new ideas whenever searching for methods to motivate our designers to improve the travel insurance sites I am involved in taking care of, for example in http://www.mycover.com.au please visit the website and provide us with your remarks. It is through this valuable interchange of information, particularly of such a specialized nature, that quality will be improved for the benefit of all. We sincerely do value your contribution and hope this is also valued by our good friends and colleagues also. Best regards, David Brown

  10. Samuel Says:

    But there are some common problem occurred after the development process, like miscommunication unrealistic schedule and poor requirement. Managing all these problems is the hard part.

    @valerie: Yes we also have six. After we did the QA we do the software testing.

  11. Michael Says:

    The problem is, many small companies skip these stages. Many a times there is even no QA person - the developer does some unit testing, followed by some UAT and that's it!

  12. Mark Fades Says:

    Anther problem I see is that most companies neglect their duties just right after production... Come to think of it, there should be 2 stages of QA's one on pre-production and one for post production... things may happen right after production and this is very important.

  13. Chaser Cruz Says:

    Promotional model for every production company is really the key to its success. Production consist of various pipeline where it should work in process to attain the goal the want to be. Software testing is really important in line to detect the effect of it in each pipeline.

  14. Nicotinamide gel Says:

    QA is very important, imagine, even in call centers they do have QA specialists to ensure that every agent provides the best services. This is not even about the product released it's even monitoring and preventive measures after release.

  15. Monica Says:

    I think it is remarkable that you could decide to put a great deal of effort into the design stage. And by the way, I've undergone these promotional models but one thing I want asks you, what's the difference with these promotional models to SDLC Software development Life Cycle? Can you help me, please?

Leave a Reply

You must be logged in to post a comment.