Archive for October, 2009

The Proportional Change Principle

Thursday, October 29th, 2009

The change in the solution should be proportional to the change in the problem.

Demoware
Have you ever attended one of these demos? "No code required!" "Drag and drop!" "It's automagic!"

The presenter points and clicks his way through a canned problem and creates a solution extremely rapidly. He data binds to an Access database. He lays out a report. He generates a web site directly from tables. Then he walks off stage.

If you've tried to use VB 6 data binding or Dynamic Data in a production system, you know the problem with demoware. It only works for a simple problem. Once the problem becomes complex, the demoware does not work. Your only recourse is to rewrite.

If you were to plot the total amount of work against features, you might see something like this:

Rewrite

The slope of the line before the rewrite, when you were relying upon automagic demoware,  is fairly constant. Each new feature you added required you to drag a new column onto the designer. Not only is it easy to perform each change, each change is roughly the same size.

The slope of the line after the rewrite is also fairly constant. Here you are no longer using the automagic demoware framework. You have switched to a production-ready framework. Even though have to actually write code for each feature, the amount of code per feature is about the same. Moreover, since you've learned the new framework and the pattern that it demands, it's really not that hard to write that code. The slope is steeper than it was with the automagic demoware, but at least it's constant.

Up front design
Let's imagine that we skipped that demo and instead designed the infrastructure that our problem demands. Our total effort vs. features might look like this:

Design

We spend significantly more hours before we can deliver the first feature. But once we do, the effort per new feature is constant. We avoid the big rewrite cliff because we create a framework that can handle all of features that we'll need to write. We don't know ahead of time what all those features will be, but we do know that our chosen framework will handle whatever we throw at it.

I evaluate everything from the principle of Proportional Change. When the problem changes a little, I should only need a small change in the solution. Big changes in the solution should only occur when there are big changes in the problem. I've found that many things violate this principle, not just demoware. Here's a few more examples:

  • YAGNI - Yes, you really are gonna need it.
  • Refactoring to a design - Small steps cannot always get you to your goal.
  • Code generation
  • Event-driven code

Don’t abdicate the ubiquitous language

Friday, October 16th, 2009

The Ubiquitous Language of Domain Driven Design is a call for developers and domain experts to communicate openly, instead of erecting false barriers of obfuscation. It admonishes developers not to put pattern names or technical terms into the domain model.

Yet it also does not suggest that developers should abdicate the language to the domain experts. Imprecise language cannot be tolerated. When a domain expert uses it, it is the developers' duty to correct it.

The following four-letter words are indicators of imprecise language:

  • Type
  • State/Status
  • Code
  • Text

Type
When a domain expert uses the term "type", they are trying to classify objects in a meaningful way. Work items, for example, can either be bugs or enhancements. Orders can be purchase orders, quotes, or returns.

Domain experts will often model type as an attribute of a single entity. The behavior of that entity is sometimes dependent upon this "type" attribute. Entities of one type are treated differently than entities of another type, but only in certain situations. Pricing is run on purchase orders and quotes, for example, but not on returns.

When a developer sees the word "type", they immediately attach some assumptions to it. An entity can only be an instance of one concrete type. Its type is determined at the time of creation and cannot change during its lifetime. Type determines the attributes and behaviors of the object. These are not the assumptions that the domain expert means to imply. A work item may be entered as a bug, but later determined to be an enhancement. The work item type can change. If we modeled this domain using a static type system, such a change would be impossible.

It would be more useful to identify the reason that objects have different "types". A work item is either a bug or an enhancement because of the feature set it affects: existing features or new features. Orders are purchase orders, quotes, or returns because of their intent. Let's use the more precise terms "feature set" and "intent", and not the overloaded term "type" to represent these attributes of an entity.

State and Status
When a domain expert uses the term "state" or "status", they are trying to express a workflow. A work item is in the "new", "fixed", or "closed" state. An order status can be "received", "processed", or "shipped".

State and status are modeled as attributes of an entity. But they are in truth events in the workflow. As an entity moves through a workflow, the intent is not so much to modify that entity. Rather, it is to gather information about the entity. This is a concept discussed by Greg Young in Unshackle Your Domain. The events should themselves be part of the domain model.

States are only meaningful within a context. But entities are often found at the intersection of several contexts. An entity can be in several states simultaneously, depending upon what question is being asked. But if "state" is an attribute of an entity, it can have only one value. This can lead to a combinatorial explosion of states that attempt to simultaneously represent two or more contexts.

It would be more useful to identify the events that can occur within a workflow, and ask questions in terms of that history.

Code
When a domain expert uses the term "code", they are usually calling out an implementation detail about how a problem is currently solved. Our ordering system currently has a "signal code" attached to a product. This three-letter code is used to signal the system that special processing needs to take place.

For example, one signal code indicates that the product is a controlled substance, and therefore requires DEA license validation. Another signal code indicates that the product is drop shipped, so different inventory behavior is applied.

The problem with codes is that they are already entrenched. It is hard to convince a domain expert that there is a problem. Telling them that codes represent different dimensions is too abstract. But if you give them a concrete example, they already have an answer for it.

"What if a product is both a controlled substance and drop shipped?"

"Are you crazy! We don't drop ship controlled substances!"

It would be more useful to identify the dimensions along which entities vary and make those independent attributes. A product has licensing requirements, and also has an inventory strategy. Interactions among those various dimensions can be captured as business rules.

Text
Unless the domain is specifically about books, the term "text" is often used to represent a note to be displayed in a specific place. A customer service representative can enter "footer text" when placing an order, which appears on the footer of the invoice.

These notes are important to capture, as the primary purpose of any system is to facilitate communication among its users. But the notes should be named according to their purpose, not according to where they appear. Formats change over time, and are not an appropriate concept for the domain model. Our "footer text" for example no longer appears on the footer of the invoice. It would be more useful to call it "invoice instructions".

The four-letter words listed here are just the most egregious examples of imprecise language. There are many other examples that could be cited. As a developer, it is your job to ask questions when you see them. You need more information about what these terms mean in context. If you have to ask a domain expert to elaborate, then they have not chosen a precise term. Once they do elaborate, suggest that they use that term instead.

Merit vs. Credit

Thursday, October 15th, 2009

Domain Driven Design (DDD) advocates for a ubiquitous language. This is more than just a fancy way of saying that developers and domain experts should use the same words. It's more than a glossary or a dictionary.

I've fallen victim to the hubris of making up my own language that is "better" than the business'. It's a personality flaw that I fight constantly. The worst example occurred when I designed a frequent diner program for a prior employer. The business called points earned "credit". I insisted that the correct term was "merit". "Credit" is for accounting systems (like our gift card program). "Merit" is something you accrue for which you are eventually rewarded.

I debated the issue at length with the business analyst. She understood my position, but informed me that our users use the term "credit". She performed an experiment in which some potential customers were shown specs with "credit" and others with "merit". The "credit" group had fewer misunderstandings than the "merit" group. Even in the face of scientific evidence, I stood my ground. "Merit" was right, and they just needed to be educated.

The debate continued. People lined up behind The Architect claiming that "merit" was technically correct and semantically concise. Other people lined up behind The Business Analyst claiming that "credit" made more sense and did not conflict with other definitions in other contexts.

In the end, we resolved that all of the functional specifications and end user documentation would use the term "credit", while all of the technical documentation and code would use "merit". The user interface code was a mess, since it had to display "credit" but read the value from the "merit" property. For years afterward, I received questions from new developers about what was "merit" and what was "credit" and what they had to do with each other.

In this, I was wrong. My continued insistence on imposing my own language added no value. I created a situation which only bred confusion. I was correct only in proposing a new term and in soliciting additional feedback. But once the term "credit" was corroborated by our customers, I should have ended my crusade for the sake of ubiquitous language.

Topic specific sites

Saturday, October 10th, 2009

I now have three sites for topic-specific content.

I have seeded these sites with articles from Adventures in Software. But from now on, articles specific to those topics will go on those sites. Please subscribe to those RSS feeds as well as this one.

A provable Socket API

Thursday, October 8th, 2009

Two weeks ago, I assigned homework. Rewrite the .NET Socket API so that you can prove the following assertions:

  • You cannot use a Socket returned from Accept to accept any additional connections from the connection queue.
  • You cannot call Accept, Bind, Connect, Listen, Receive, or Send if the socket has been closed.
  • You must call Bind before you can call the Listen method.
  • You must call Listen before calling Accept.
  • Connect(string host, int port) can only be called if addressFamily is either InterNetwork or InterNetworkV6.
  • Connect cannot be called if Listen has been called.
  • You have to either call Connect or Accept before Sending and Receiving data.
  • If the socket has been previously disconnected, then you cannot use Connect to restore the connection.

These assertions are prominently listed in the MSDN documentation. Why not roll them into the API itself? Let the compiler prove that we are using sockets correctly.

Last week, I gave you a hint. I moved the Send and Receive methods out to a different class so that we can prove that you've called Accept or Connect prior to Send or Receive. Both Accept and Connect became factory methods. You must call the factory before you get the object, and you must get the object before you call its methods. Q.E.D.

Let's start from there and prove the rest of the assertions.

Starting point

  • You have to either call Connect or Accept before Sending and Receiving data.
public class ConnectedSocket
{
    public int Receive(byte[] buffer);
    public int Send(byte[] buffer);
}

public class Socket : IDisposable
{
    public Socket(SocketInformation socketInformation);
    public Socket(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType);

    public ConnectedSocket Accept();
    public void Bind(EndPoint localEP);
    public ConnectedSocket Connect(EndPoint remoteEP);
    public ConnectedSocket Connect(string host, int port);
    public void Dispose();
    public void Listen(int backlog);
}

Looking through the list of assertions, we find that this one is already taken care of by the first change:

  • You cannot use a Socket returned from Accept to accept any additional connections from the connection queue.

The ConnectedSocket class doesn't have an Accept method, so you can't call it. Q.E.D.

Other prerequisites
There are three other prerequisite assertions that we can prove in exactly the same way:

  • You must call Bind before you can call the Listen method.
  • You must call Listen before calling Accept.
  • Connect cannot be called if Listen has been called.
public class Socket : IDisposable
{
    public Socket(SocketInformation socketInformation);
    public Socket(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType);

    public BoundSocket Bind(EndPoint localEP);
    public ConnectedSocket Connect(EndPoint remoteEP);
    public ConnectedSocket Connect(string host, int port);
    public void Dispose();
}

public class BoundSocket
{
    public ListeningSocket Listen(int backlog);
    public ConnectedSocket Connect(EndPoint remoteEP);
    public ConnectedSocket Connect(string host, int port);
}

public class ListeningSocket
{
    public ConnectedSocket Accept();
}

Bind becomes a factory for the class that gives you Listen. Bind does not take away your ability to call Connect. Listen does.

Factory and product
Factories are great prerequisites. Now that it is becoming more obvious that we are using that pattern, let's rename Socket and move IDisposable to the product.

public class SocketFactory
{
    public SocketFactory(SocketInformation socketInformation);
    public SocketFactory(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType);

    public BoundSocket Bind(EndPoint localEP);
    public ConnectedSocket Connect(EndPoint remoteEP);
    public ConnectedSocket Connect(string host, int port);
}

public class ConnectedSocket : IDisposable
{
    public int Receive(byte[] buffer);
    public int Send(byte[] buffer);
    public void Dispose();
}

The ConnectedSocket is the ultimate product. That's the socket that needs to be closed. And what do you know, this change proves another of our assertions:

  • If the socket has been previously disconnected, then you cannot use Connect to restore the connection.

It is impossible to call Connect on a socket that has been disposed, since they aren't the same object anymore.

Dispose once
Now that the Dispose method is where it belongs, let's focus on the assertion most concerned with its use:

  • You cannot call Accept, Bind, Connect, Listen, Receive, or Send if the socket has been closed.

Accept, Bind, Connect, and Listen have already been taken care of, since they have become factory methods. To handle Send and Receive, we turn every ConnectedSocket factory into a callback:

public class ConnectedSocket
{
    public int Receive(byte[] buffer);
    public int Send(byte[] buffer);
}

public void Connect(string host, int port, Action<ConnectedSocket> action);

We have taken away the capability of disposing a socket. The factory does it on the consumer's behalf. Since there is no way for the consumer to call Dispose, and since the callback returns before Dispose is called, there is no way to call Send or Receive after Dispose. Q.E.D.

Specialized factory
Which leaves us with one final assertion:

  • Connect(string host, int port) can only be called if addressFamily is either InterNetwork or InterNetworkV6.

Again, we want to move the restricted method into its own special class. But now, we want to enforce that certain values are provided, not that certain methods are called. We'll limit this by creating our own smaller enumeration.

public class IPSocketFactory : SocketFactory
{
    public enum Version { V4, V6 }
    public IPSocketFactory(Version version, SocketType socketType, ProtocolType protocolType)
        : base(
            version == Version.V4 ? AddressFamily.InterNetwork : AddressFamily.InterNetworkV6,
            socketType,
            protocolType) { }

    public void Connect(string host, int port, Action<ConnectedSocket> action);
}

If you want to call the Connect overload with two parameters, you have to use the limited enumeration to get the right factory. I didn't mess with the overloaded constructor, because it complicates things.

One slight wrinkle. In a previous change we created a copy of the Connect method, since it can be called after Bind(). So we make the same change there:

public class IPBoundSocket : BoundSocket
{
    public void Connect(string host, int port, Action<ConnectedSocket> action);
}

public class IPSocketFactory : SocketFactory
{
    public enum Version { V4, V6 }
    public IPSocketFactory(Version version, SocketType socketType, ProtocolType protocolType)
        : base(
            version == Version.V4 ? AddressFamily.InterNetwork : AddressFamily.InterNetworkV6,
            socketType,
            protocolType) { }

    public IPBoundSocket Bind(EndPoint localEP);
    public void Connect(string host, int port, Action<ConnectedSocket> action);
}

The only way to get an IPBoundSocket is to use the IPSocketFactory. Therefore, you must have selected one of the Internet protocols. By the way, this is taking advantage of a little-used feature of C# called covarience. This is the one part of the proof that might not work in other languages.

So there you have it. A provably correct Socket API. There is no way to use it incorrectly. The compiler enforces all of the assertions that we would otherwise have to read the documentation to discover.

A new example of historic modeling

Wednesday, October 7th, 2009

I wrote up an example historic model that illustrates very succinctly some of the major points of the theory. It is a simple work item tracker.

So far, the example model can assign developers to projects. The next step is to allow developers to move work items between folders to track progress.

Currently, the model is only in text form. Soon, I will render it in source code using Correspondence. Eventually, it will integrate with TFS and HP Quality Center to act as a front end and perhaps a conduit between those systems.

If you would like to see a simple and reliable way of writing distributed smart clients, please follow along.

Clean URLs in Drupal with Apache and Tomcat

Tuesday, October 6th, 2009

I've just returned from the land of jar -xzf. I've installed a couple of new Drupal sites (details to follow). One of those sites takes over the whole domain. The other is just a subset. I ran into trouble with the infamous "Clean URLs" feature, and only found the solution after hours of fruitless Googling.

Drupal is a PHP content management system. In PHP, the URL addresses the program, and any parameters must be supplied in the query string. Drupal URLs therefore look something like this: http://drupal.org/index.php?q=node/1.

There are a few things we can do to make the URL prettier. First, we can configure the web server to serve index.php by default. Most web servers are initially configured to do just that, so the URL could be written like this: http://drupal.org/?q=node/1.

That's better, but SEO is harmed by the question mark. So we turn on mod_rewrite to change that URL to this: http://drupal.org/node/1.

A tale of two sites
This was all well and good. I followed the directions found on blogs too numerous to mention for configuring mod_rewrite. These instructions worked fine for one of my sites, but not the other.

What's the difference? On the site that worked, Drupal was installed in a subdirectory. On the site that failed, Drupal was installed in the root. The problem has something to do with the lack of a subdirectory in the URL.

In the course of troubleshooting, I tried removing the "?=q" from various URLs. It worked for content pages, but failed for admin pages. So, I reasoned, the problem is with "/admin" as the top folder.

Tomcat
What was most telling is what happened when I entered a URL with just "/admin". The Tomcat administration page appeared. I have Tomcat installed on my server, but I'm not using it on the full Drupal site. Drupal is PHP, not Java, and this site has no other content. So why is Tomcat showing up?

My host (eApps) defines a default configuration for any new site I create. This configuration includes Tomcat, because I typically host Java applications rather than PHP applications. The configuration includes this line in httpd.conf (located in /etc/httpd/conf):

JkMount /admin* ajp13

This directs the "/admin" URL to Tomcat through mod_jk. I just removed this line from this site's configuration, and I was able to turn on clean URLs.

Every site is different. The solutions that you find on blogs (including this one) won't necessarily work for you. It is best to hone your troubleshooting skills and attack the problem systematically than to Google for the error message. I mean, what are the odds that your environment is exactly like mine?

Non-recursive WPF TreeView controls

Friday, October 2nd, 2009

A WPF TreeView control can display data in any tree structure. There are essentially two kinds of tree structures: recursive and non-recursive. Non-recursive trees are simpler.

SurveyDataModelItems in a recursive tree can have children of the same type. Items in a non-recursive tree cannot. A non-recursive tree has a fixed depth, and every item on the same depth is the same type as its siblings. This makes it particularly easy to model in WPF.

Pictured here is a non-recursive tree structure for a survey game. A Game has Surveys, which in turn have Answers. It's a very simple tree structure with no recursion. The depth of this tree from the Game to the Answers will always be 3.

Binding the TreeView
Let's start with a TreeView control that displays the surveys. We'll bind the collection to the ItemsSource of the TreeView.

<TreeView ItemsSource="{Binding Surveys}">
</TreeView>

The TreeView control creates a TreeViewItem for each element in the collection. We need to tell the container how to populate those TreeViewItems. We do this with a DataTemplate.

A DataTemplate defines the contents of an item in a container control. DataTemplates are used for things like list boxes, menus, and tree controls. A DataTemplate does not define the item itself -- the container determines that. ListBox controls have ListBoxItems. TreeView controls have TreeViewItems. The DataTemplate just defines what goes into that item.

We want each TreeViewItem to contain a TextBlock bound to the survey question. So we set the ItemTemplate property to an instance of the DataTemplate class. This DataTemplate contains a TextBox with the right binding.

<TreeView ItemsSource="{Binding Surveys}">
    <TreeView.ItemTemplate>
        <DataTemplate>
            <TextBlock Text="{Binding Question}"/>
        </DataTemplate>
    </TreeView.ItemTemplate>
</TreeView>

We want each TreeViewItem to have child items. The child items are not contained within the parent item. They are just a property of the parent item. We can't use a DataTemplate to set properties. So we switch to a special DataTemplate: HierarchicalDataTemplate.

HierarchicalDataTemplate
A HierarchicalDataTemplate is a DataTemplate with an additional property: ItemsSource. This property is bound to a child collection. The template sets this property on the TreeViewItem itself.

<TreeView ItemsSource="{Binding Surveys}">
    <TreeView.ItemTemplate>
        <HierarchicalDataTemplate ItemsSource="{Binding Answers}">
            <TextBlock Text="{Binding Question}"/>
        </HierarchicalDataTemplate>
    </TreeView.ItemTemplate>
</TreeView>

This template will set the ItemsSource of each Survey TreeViewItem to the bound collection of Answers. But, as before, we need to tell the control what to put into those TreeViewItems. We do this by setting the ItemTemplate property of the HierarchicalDataTemplate.

<TreeView ItemsSource="{Binding Surveys}">
    <TreeView.ItemTemplate>
        <HierarchicalDataTemplate ItemsSource="{Binding Answers}">
            <TextBlock Text="{Binding Question}"/>
            <HierarchicalDataTemplate.ItemTemplate>
                <DataTemplate>
                    <TextBlock Text="{Binding Response}"/>
                </DataTemplate>
            </HierarchicalDataTemplate.ItemTemplate>
        </HierarchicalDataTemplate>
    </TreeView.ItemTemplate>
</TreeView>

Because the Answer TreeViewItem does not need to have children, there is no need to use a HierarchicalDataTemplate at this level.

A non-recursive tree structure is easy to model. Since an item does not have children of its own type, there are no self-referential templates. When the tree structure can be recursive, however, things are a bit tricker. That's coming up.

Hint on socket homework

Thursday, October 1st, 2009

Your homework is extended for another week, but let me give you a hint what I'm looking for. The problem was to Fix the Socket API. Rewrite the .NET Socket interface so that the assertions made in the MSDN documentation can be proven.

To give you an example of what I'm looking for, take the following assertion:

  • You have to either call Connect or Accept before Sending and Receiving data.

This is a prerequisite assertion. We must first call Connect or Accept, then we can call Send or Receive. To prove this assertion, I'll use factory methods. Here's the original Socket class:

public class Socket : IDisposable
{
    public Socket(SocketInformation socketInformation);
    public Socket(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType);

    public Socket Accept();
    public void Bind(EndPoint localEP);
    public void Connect(EndPoint remoteEP);
    public void Connect(string host, int port);
    public void Dispose();
    public void Listen(int backlog);
    public int Receive(byte[] buffer);
    public int Send(byte[] buffer);
}

Move the Send and Receive methods out of this class. We cannot call them yet. Move them into a new class called ConnectedSocket.

public class ConnectedSocket
{
    public int Receive(byte[] buffer);
    public int Send(byte[] buffer);
}

To prove that we have to call Connect or Accept before any of the ConnectedSocket methods, we make Connect and Accept factory methods:

public class Socket : IDisposable
{
    public Socket(SocketInformation socketInformation);
    public Socket(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType);

    public ConnectedSocket Accept();
    public void Bind(EndPoint localEP);
    public ConnectedSocket Connect(EndPoint remoteEP);
    public ConnectedSocket Connect(string host, int port);
    public void Dispose();
    public void Listen(int backlog);
}

This API proves one of the assertions in the MSDN documentation. Try the others on your own. Answers will be posted next week.