Archive for January, 2011

Turn software architecture on its ear

Monday, January 31st, 2011

Software architectures have evolved since the first computers were put to use. Early computer operators would hardly recognize the systems we build today. Architecture continues to evolve at a pace where we will not recognize the systems of tomorrow.

image A computer used to be a large, noisy machine enshrined within a windowless room. The only access to it was through punched cards, teletype printouts, and eventually dumb terminals. Computer programs were input/output routines that ran business logic over data.

Then the personal computer revolution hit and a computer became a briefcase-sized box sitting on your desk with a television on top. People could run their own logic on their own data without having to lease space on the mainframe. Rather than building custom programs for every need, people could buy applications. Thus the software industry was born.

image Unfortunately, applications didn’t make it easy to collaborate with other people. The data and the logic were both on your PC. And so people adopted the World Wide Web as a way of sharing data. With the advent of the web, applications again became centralized. The personal computer started to resemble the dumb terminal again, becoming only a host for a web browser.

Then rich web applications brought interactivity and power back to the personal computer. The browser gained the ability to run JavaScript, making the dumb terminal a little bit smarter. This trend continued with Flash, JQuery, and Silverlight.

Smart phones broke the user experience out of the browser and moved it into apps. Apps had some of the benefits of browser-based clients in that they were quicker to develop, and easier to deploy. But they also were like an application in that they were useful offline.

image This is our current state of software architecture. The data still resides in a large noisy machine in a windowless room, but we have installed layers of access in front of it. We can call an API – whether it be REST or Web Services – to invoke the business logic and access that data. We have a service agent on the client side – perhaps AJAX or RIA Services – to invoke that API. It brings back a model – JSON or XML converted into objects – that we can translate through a view model and display on a view.

Writing software today, it seems like we’ve always done it this way. But it was only 50 years ago that we started on this journey. Some of the people who made the earliest advances are still with us now.

Where we are going

Take a look at the work of people like Udi Dahan, Greg Young, and Ayende Rahein to see the next step in this architectural evolution. Look at where NoSQL projects like MongoDB and CouchDB are leading us. This architectural stack is about to be turned on its ear.

One of the big problems that we are about to solve is the synchronous nature of our current APIs. REST and Web Services are based on the idea of request/response. Synchronous protocols have scaling and reliability issues. Udi and Greg are proposing new ways of creating asynchronous services.

Another problem that we are working on is impedance mismatch. The database is relational. The logic is object-oriented. The API is document-oriented. And the view is a scene graph (HTML or XAML). At every layer, we have to map one format onto the next. NoSQL databases skip the relational and object-oriented layers and go straight to documents. Microsoft’s data binding, John Resig’s JQuery, and Steve Sanderson’s Knockout bridge the object/scene graph gap.

What will the architectural diagram look like in 5 years? I have a guess, but I can’t say for sure. The only thing certain is that it will look nothing like it does today.

Itzben

Monday, January 24th, 2011

You know those little useful snippets of code that you keep copying from one project to another? Where do you put those? I’ve always had that problem, but never so much as when I started working in XAML. There’s something about WPF, Silverlight, and Windows Phone 7 that makes me write the same little utilities over and over again.

I’ve finally decided to create a place for these universally useful XAML helpers. It’s called Itzben, and it’s on Codeplex. Here’s what I started with:

BindVisualStateBehavior

Use this behavior to databind an enumeration or string property to a visual state. When the property changes, your state transition animation fires.

EnumEqualsConverter

Use this value converter on the IsChecked property of a radio button. Set the converter parameter to the enumeration value that the radio button assigns.

CollapsedWhenNull

Apply this style to a Grid acting as the details pane for a selected item. When no item is selected, the grid will be collapsed.

There are more behaviors, converters, and styles that I write over and over again. I’m sure that you have some yourself. If so, please post an issue on the project site and include your code. I or one of the other committers (once I get them to agree) will add it to the project.

Enjoy!

Dallas Give Camp 2011

Monday, January 17th, 2011

team pic2011 was my second year at Dallas Give Camp. It was just as much fun this year as it was last year.

Give Camp is a chance for developers to donate their time to local charities. While it started in Dallas four years ago, it has quickly grown to include over 25 cities.

Second Wind revisited

Last year, I had the opportunity to help out Second Wind, a local organization that helps families in need to have Christmas gifts for their children. We built them a web application in which the volunteers could organize schools, families, and sponsors. It was an ASP .NET Web Forms application built using Entity Framework 1. I was pleased with the progress that we were able to make, and the charity was happy to move away from a paper-based system.

This year, Second Wind returned for some enhancements to the application. I was not on that team, but I was proud nonetheless to see their results. They ported the Web Forms application to MVC 3 and greatly improved the look and feel. They also added reports that the volunteers found that they needed. Over this past year, they integrated the app into their workflow and discovered some manual processes that they could automate. This is iterative development at its best: put the app into production, see how you actually use it, and then come back to add features.

Legal Hospice of Texas

While my old charity was getting the help they needed, I was assigned to a new charity. New for me, at least. Legal Hospice of Texas ensures that people with incurable diseases get the legal help that they need to put their affairs in order. Two years ago, they participated in Dallas Give Camp to get a new web site built. They were very happy with the look and functionality of the site, but it was difficult for them to make changes on their own. They needed a Content Management System (CMS).

We evaluated several alternatives, including SiteFinity, MojoPortal, and DotNetNuke. We chose SiteFinity for its ease of use. We liked how a non-technical person could change not only the content, but also the layout, without editing code. Styling required some CSS knowledge, but it was not necessary to create a custom master page. It had all of the features that our charity needed out-of-the-box, and Telerik was kind enough to donate licenses. It seemed to be a clear choice.

SiteFinity 4

When we started using SiteFinity, however, we ran into some serious issues. The first couple of issues were related to IIS setup, which Matt Legrotte from Verio was quickly able to resolve. But then additional problems arose. They didn’t just affect us; all charities using SiteFinity this weekend were having trouble.

I am not just talking about Dallas. It turns out that half of the Give Camps in the United States were holding their events on the same weekend. Coincidentally (or perhaps not so coincidentally), Telerik was released their new version of SiteFinity on the same weekend. Gabe Sumner, the Telerik Developer Evangelist who supported SiteFinity this weekend, was stationed in Houston. He provided technical training and support via Net Meeting, Twitter, and email. He was in high demand, as all charities using SiteFinity 4 in all cities were having difficulty.

The problems were numerous. When more than one person publish page content at the same time, SiteFinity throws an exception that puts the application into a continuous restart mode. The only resolution is to recycle the app pool. Furthermore, there were some modules that were missing resources, like background images. When someone tried to add one of these modules to a page, it would bring down the application.

Most ominous was the fact that SiteFinity 4 would sometimes loose data. When building a page, you had to be careful to allow SiteFinity to complete all of its AJAX calls. If you navigated to another page too soon, the content that it was saving would not be stored. In most cases, you could correct this by reverting to a previous revision in history, then reapplying your intended changes. One team, however, changed their template after creating several pages. After doing so, not only did the pages no longer contain content, but the revision history brought up “yellow screens of death”, error dumps all too familiar to ASP developers.

In addition, SiteFinity 4 either has a memory leak, or simply consumes more memory than your typical web application. It was exceeding the allocated memory limits at many hosts. Fortunately for us, Verio set their limits much higher than others, so this wasn’t one of our issues.

With Matt’s help, we diagnosed many of SiteFinity’s bugs and learned how to avoid them. We announced “Publishing” before hitting the button, so others would know to wait. We modified the site with FireBug open so we could monitor the AJAX traffic. In the end, SiteFinity was the correct choice. We produced a site that the charity could maintain on their own, and we fully expect that Telerik will fix their bugs. But it would have been better for them to be more honest about the status of their product. They should have told us that version 4 was not production ready, and encouraged us to remain on 3.7, providing an upgrade path. Instead, we were free labor for Telerik, performing the quality assurance that they obviously either did not complete or ignored.

We’re growing

As compared to last year, the venu and the food were both improved. Last year, BravoTech hosted the event. While they continued to be a sponsor this year, their offices were just too small for the sheer number of participants we had. This year, Bearcom provided space. And there was plenty of space to be had. This being a developer event, there was of course pizza. Dominoes was a gracious sponsor of all cities having events this weekend. But in addition, we had barbeque from Spring Creek, and enchiladas from Mexican Inn. And I got a chance to make my 5-4-3-2-1 Waffles.

Despite the technical issues, the charities received some much needed IT work. It was estimated that we provided one man year of effort in 48 hours. Without this event, these charities simply could not afford an effective web site. I look forward to next year.

TDD Dojo

Thursday, January 6th, 2011

I’ve been conducting lunch classes at work to help us improve the confidence in our software. I call these sessions “Q.E.D.”, based on my foundation in mathematics and my practice of applying mathematical proof to software (see my other blog qedcode.com).

In the past, these have been lectures, where I present mathematical concepts like Degrees of freedom, Closure, or how to write Provable APIs. These sessions have earned me the nickname “Professor”, which I answer to proudly. But lately, I’ve been thinking it would be better for the team if I were a “Sensei” instead.

The end of lectures

Yoda As the Professor, I would get up in front of the class and give a lecture. This would include not just PowerPoint, but also a good deal of coding in Visual Studio. I would get some good questions from the group, and I would also throw out questions of my own.

Then I would assign homework. Make this unit test pass, or refactor that API to make it more provable. People would do the homework individually and email me the results. At the beginning of the next class, I would choose the person who did the best homework and recognize them with the Yoda Award. They get to keep Yoda at their desk until next week’s class.

The main problem I saw with this kind of session was that not many people were putting the concepts into practice. Only two or three people would email me homework each week. It was a different set of people each time, so I wasn’t concerned about particular individuals. But work and other obligations got in the way, people didn’t seek help from me or each other, and the week just slipped by.

To solve this problem, I decided that we should do homework in class. When thinking about how practically to do that, I remembered the evenings I spent at a local meetup group called Geek Night, hosted at Thought Works and led by local Though Worker Paul Hammant. At Geek Night, we pair program on open source projects.

To get people practicing together, we need the pair programming bit more than the open source bit. So I decided to have them pair on very simple exercises at first. We are starting off with the Bowling Game Kata. But instead of solving the whole thing, I’ve broken it down. The original kata has you write test like “When I roll 20 gutters, my score is zero” and “When I roll 12 strikes, my score is 300.” That’s too big, in my opinion. The problem should be broken down into testable pieces that you can then assemble into a working solution.

The first piece that we are working on is a class called GameData. This class will record all of the rolls. It will eventually be used by the Frame class, which will observe rolls by index. This leads to a number of simpler test, like “there are initially no rolls” and “after one roll, you can retrieve the first roll by index.”

With that inspiration, I closed the classroom and opened a dojo. In a martial arts dojo, the sensei doesn’t stand at the front. He walks around and interacts with the students. The students work individually or in pairs on routine forms. The sensei will call out commands, and correct individual students on their form.

The rules of the dojo

I brought in a deck of cards. I leave two cards of the same value on each desk (for example the 3 of hearts and the 3 of spades). The other two (3 of clubs and 3 of diamonds), I put in a deck near the door. When you enter the room, you take a card. You sit at the table with that value. This creates up to 13 pairs. I have on average 12 people attend, so I only used Ace through 6. If we have extra people, then we create additional pairs. If we have empty chairs, we consolidate. And if we have an odd number of people, one pair becomes a triple.

Each person brings a laptop with Visual Studio 2010. They have all pulled the same solution from TFS and mapped it to their drive. When we are in the dojo, we cannot connect to the network. So we all open the solution and select “Go Offline”.

I give the group a requirement to test. For example, “Given a GameData class, the Rolls collection should initially be empty.” One person writes that test, which should initially not compile (the test uses a property that the code doesn’t have yet). The other makes it compile (by adding the GameData.Rolls property), sees it fail, and then makes it pass.

I walk around the room observing how each pair is doing. If they write working code without seeing the test fail, I ask them to delete the code and see the failure. If they are leaving sloppy tests behind, I ask them to refactor. If they are using the debugger, I ask them to make the test fail in such a way as to tell you why it failed. And if their test names don’t reveal their intention, I ask them to rename them.

When every pair has the test working, I call out the next test. “When I add a roll, I can get that roll by index 0.” The person who wrote the code last time writes the test this time. Then they pass it off to their partner to write the code.

In the first hour, we were able to get through 4 tests. This included learning the rules of the dojo, setting up the project (for those who didn’t come prepared), and leaving 10 minutes at the end.

During the last 10 minutes, I ask each pair to bring their laptop to the projector and show us their tests. We all ask questions about the tests and the code. The pair defends their naming, their refactoring, and their implementation. “The test”, I explain, “is the example that you give to the consumers of your API. It says, ‘I want you to use this class in this way.’ Is this the way you want people to see your API?”

Finally, each pair selects one other pair that has the best code. They cannot select themselves. The pair must vote as one. They write down that pair’s number (Ace through 6 in my case). Then we reveal the votes and decide the winner. The winners get to share Yoda for the week.

When they get back to their desks, they hit the “Go Online” button to reconnect the solution to TFS. The winning pair gets to check in their code. Everyone else must roll back. Then next week, everyone gets latest and starts from there. This encourages people to vote for the code that they want to work on.

The reaction so far

People really seemed to enjoy the first session. More importantly, it was the most engaged that I saw them since we started these lunches.

All of the pairs made some pretty basic mistakes. Not all of them realized that they were mistakes. For example, one pair did not refactor between tests. They took each test as a separate requirement that, once completed, could not be altered. So when they saw that another pair had refactored the first test, they complained that they had rewritten requirements. I explained that the requirement was still met, but now the design was improved.

If I were a real sensei, I would not feel the need to explain. I would simply correct, expecting that understanding will come in time. I don’t feel that I can take this approach with my team at this point. Perhaps eventually we will be able to leave our egos at the dojo door.

I had to come up with the rule that you vote for another pair because initially everyone voted for themselves. Rolling back your own code is painful at first. Over time, they will recognize that deleting your project at the end of the kata is actually quite a cleansing sensation.