Archive for September, 2007

Well Enabled Services

Thursday, September 27th, 2007

As part of my current software-as-a-service project, I've been investigating Microsoft's Connected Services Framework (CSF). This is Microsoft's answer to the service management problem. Service management is an issue in any IT department, but it is especially important when those IT services are offered to your customers.

At the core of CSF is the well-enabled service (WES). This is a web service with additional interfaces. These interfaces provide support for:

  • Discovery
  • Provisioning
  • Health
  • Utilization

Web services in general define a mechanism for discovering (WSDL) and consuming (SOAP) the service itself, but offer no management assistance. A WES adds these missing management interfaces.

Discovery
WSDL provides for the consumers of a web service to discover its capabilities. But a WES adds discovery of resources managed by that service. This set of resources differs from one service to the next. For example, an email service would manage mailboxes. The WSDL would tell the consumers of the service that they can check their email, but the WES discovery interface tells the IT technician that he can create and delete mailboxes.

Provisioning
In a single-tenant situation, the IT technician provisions services on behalf of his users. In our email example, this means that he would create a mailbox when someone is hired, and delete it when they are fired. In a multi-tenant situation, however, manual provisioning doesn't scale. Instead of the service provider handling every provisioning request manually, he would prefer to automate the task and put the power in his customer's hands.

Most services are not designed for multi-tenancy. They were built to solve a specific business problem. The burdens of scalable IT were probably not even considered during their design and construction. And even those that were designed for multi-tenancy have their own application-specific provisioning interface.

The WES provisioning interface is a small but important step toward a scalable IT solution: it gets us thinking about multi-tenancy during the design phase, and it gives us a common interface to implement. Granted, that interface is extremely weak (the ProvisioningRequest message has one string field: Data), but it's a start.

Health
A single-tenant IT technician can answer his user's support questions by using an application-specific interface to check the health of their resources. But in a multi-tenant environment, that doesn't scale. The customers should be able to check on the health of their own resources.

The WES health interface provides an automated mechanism to query the status of a resource. With this, a user could see, for example, whether their mailbox is online or offline, whether they have had any failed login attempts, and whether they are approaching their quota. These particular health metrics are specific to email, of course. Each WES advertises its own available metrics through the discovery interface.

Utilization
The single-tenant service has an on-going operational cost, regardless of utilization. The consumer of the service has already paid for it. The multi-tenant service, however, is paid for by the service provider. The service provider must ultimately bill the consumer for its use. One way to do this is to bill based on utilization.

The WES utilization interface allows a billing system to subscribe to usage events. When a billable event occurs -- exceeding a mailbox quota, for example -- the billing system is notified, and the invoice is sent.

Unfortunately, this subscription model creates several race conditions. The WES must be started before the billing system so that it can accept subscriptions. If the WES is ever restarted, the billing system must know to re-subscribe. And if the billing system hasn't subscribed when a billable event occurs, that information is lost.

Here's my opinion
Well-enabled services are a good start. Unfortunately, they are just a start. They give us a defined entry-point for provisioning, but do not prescribe a format for those messages. This means that tool support will be limited to routing messages, since generic tools cannot be built to generate or understand service-specific messages.

Furthermore, well-enabled services are built on unreliable technology. HTTP web services do not guarantee delivery. This may be acceptable for certain services, such as checking email, but it is not acceptable for provisioning requests and billable events. Lost bills means lost money.

More information
Microsoft and British Telecom have created the Connected Services Sandbox: a listing of well enabled services. They have posted an excellent article on Building a Well Enabled Service. This goes into much better detail, describing what a WES is, who should build them, and how to do it.

Microsoft's landing page on the Connected Services Framework is mostly executive-level propaganda. I'd recommend starting at their Communications Sector Developer Center for more relevant information.

What is Active Directory?

Wednesday, September 26th, 2007

Active Directory is Microsoft's identity solution. It is part of Windows Server, and controls access to all the resources (computers, shared folders, printers, etc.) in a network. Active Directory is likely an important part of any software-as-a-service offering. Here are the things it contains.

Domains
A domain is the container and namespace for all objects in the directory. Active Directory cooperates with the DNS server built into Windows Server, so the AD domain is synonymous with the DNS domain.

Organizational Units and Groups
Both organizational units (OUs) and groups are ways to collectively manage rights. They each describe a collection of users, computers, or shared objects (like folders and printers). You can define rights and policies at either the OU or group level, and those rights and policies apply to all members.

The difference between an OU and a group is one of ownership. An OU owns the objects it contains. It is a strict hierarchy. A group, however, is a loose collection. An object can be a member of more than one group, but it can belong to at most one OU.

OUs typically break a domain down by region or department. All objects in the OU are typically geographically located, served by the same IT team, and under the same management. Groups, on the other hand, typically break a domain down by role. All objects in the group serve the same function, regardless of their OU.

Trees and forests
In AD vernacular, you'll hear people talk about trees and forests. Besides inspiring obvious puns about not being able to see things, these terms can cause confusion. There are no explicit objects in AD called "tree" or "forest". These are just concepts for how domains relate.

A domain has a dotted distinguished name -- a.k.a. a DNS name. For example, I have a domain called "mallardsoft.com". In my virtual lab, I defined another domain called "mlp.mallardsoft.com". Since they share a common root, these two domains together form a tree. Any domain that ends in "mallardsoft.com" is part of this tree.

Domains deeper in the tree implicitly trust those higher up. So "mlp.mallardsoft.com" trusts "mallardsoft.com". But you can establish a trust relationship across trees. When you do, the conglomeration of trees defined by the web of trust is a forest. The domain at the center of this web of trust is sometimes synonymous with the forest itself.

Best Practices
The best advice I've found related to Active Directory is to keep it simple. Always start with just one domain, until you are compelled to create more. Don't define a domain tree to represent regional or organizational divisions; that's what OUs are for. And don't join domains into a forest unless you really need to merge two disparate trees.

Don't go crazy creating many nested OUs or groups. You don't need to represent your entire org chart in AD. Just define a few high-level departments in which members tend to share resources, and roles in which member tend to share responsibilities.

For software-as-a-service, create an OU specifically for your users. Don't let them get mixed up in your internal OUs. Users and employees can peacefully coexist within one domain, but they should never occupy the same OU.

activedirectory.png

Engineering notebook

Thursday, September 20th, 2007

I started my software career working for ESystems, an aviation contractor now part of Raytheon. There, I had a mentor who taught me how the software business worked. Sure I could sling some code. That was my hobby through high school and my major in college. But he taught me how software was done in the real world.

One of the skills he taught me was keeping an engineering notebook. I've adapted his system over the years, but the spirit of his system is still present.

Composition style
I start with a composition notebook. Some people choose to use graph paper, while others like the elegance of Moleskine, but a simple composition notebook works for me. Do not use loose leaf paper, a legal pad, or a spiral notebook, as you should never be tempted to remove pages.

Keep a separate notebook for each project. You can have just one for your employer, if you like, but definitely separate that from your personal projects. I even go so far as to keep my personal projects separate from each other, even though that means I have a few books on my shelf with only the first three pages filled in.Notebook

Record the date
I start each morning by entering the date. By its very nature the engineering notebook is chronological, so embrace it.

I also record the first date on the cover of the notebook. When I run out of pages, I'll record the end date as well. Unfortunately, chronological organization means that I always have to look things up by date, but this information is usually available.

Task hierarchy
Under each date, the notebook is a list of trees. At the root of each tree is a task. A task might be a feature, a bug, a research item, or anything else that is small enough for me to finish within an hour or two. Below that, I'll use indentation and lines to break down the task into individual units of work. Lines are especially useful for carrying the hierarchy from one page to the next.

Next to each entry, I'll use symbols to indicate progress. A check mark means done. An open box means that the unit of work is done but needs to be tested. It gets a check mark after the test passes. I put a star by information that I'll probably need in the future. An arrow means that it'll get done later. An arrow pointing to someone's initials means that I've delegated the work to them. An arrow pointing to a vertical line means it'll probably never get done. My system of symbols has evolved over time, but it pretty much covers all of the situations I need.

If I get interrupted, I'll start a new task tree for the new task, leaving the old one hanging. When I get back to the original task, I start a new tree below the interruption. This is the only practical solution in a paper-based system. And it is also a good way to measure the frequency and cost of interruptions.

Reference numbers
I use reference numbers to cross-reference items outside of the notebook. Whenever I check in some code, I write down the revision number in parentheses. Whenever I work on a bug, I write down its ID with a pound sign. And when I prepare a change for deployment, I write the ticket number underlined. I can't count the number of times that I've been able to glance at my notebook during a meeting and find exactly the information I need because of this system.

Pull it forward
Once a week I go back through the notebook to find tasks that have not been checked. If they have been finished, I'll check them off. If they can be delegated or ignored, I'll do so and make the appropriate notation. If they still need to be finished, I'll write them under the current date. I'll then draw a box around all of the tasks that I've carried forward. This gives me a backlog that I can refer to quickly to ensure that tasks don't slip through the cracks.

We deal with far too much on a daily bases to keep everything in our heads. I've found that this system helps me to act more like a professional by maintaining a history of important facts and tasks in a portable and accessible package.

What is your system?

Virtual Network in Microsoft Virtual PC

Wednesday, September 12th, 2007

For various reasons, I find myself needing to switch from VirtualBox to Microsoft Virtual PC. Virtual PC lacks the easy snapshot feature, but it has one significant advantage. Everyone else on the project is using it. This makes it possible to share virtual hard drives with each other, something that is surprisingly difficult with VirtualBox. And these virtual hard drives can be hosted with Microsoft Virtual Server for staging in the integration environment.

The virtual network that I built in VirualBox did not port to Virtual PC or Virtual Server. Virtual Server has no NAT capability, and Virtual PC's NAT does not let the VMs talk directly to each other. I had to find a new way to accomplish the same result. Fortunately, with a little help (OK, a lot of help) from Virtual PC Guy (on loopback, NAT and ICMP), I got it working.

Here's his solution
Go to Ben's blog to follow his detailed instructions on installing the Microsoft Loopback Adapter. Please note that in Vista you have to switch control panel to the classic view in order to find the Add Hardware icon. And the wizard is slightly different: manually select, network adapters, Microsoft, Microsoft Loopback Adapter. When it gets installed, it creates a new adapter called "Local Area Connection 2". I renamed this to the more meaningful "Virtual Network".

In Vista you can change the IP address of the loopback adapter, which according to Ben is impossible in XP. Just go to the Network Connections page. (The simplest way I've found to get there is to open the menu formerly known as Start, right-click Network, properties, and then "Manage Network Connections".) Once there, you can rename the connection or change its properties. I set my virtual network's IP to "10.123.1.1" and the subnet mask to "255.255.255.0" to avoid any routing collisions.

In Virtual PC, open the network settings and set Adapter 1 to "Microsoft Loopback Adapter". Then start up your VMs and configure their IP settings. My VMs are configured thus:

  • MLP-AD (active directory and DNS)
    • IP: 10.123.1.100/255.255.255.0
    • GW: 10.123.1.1
    • DNS: 127.0.0.1
  • MLP-WSS (SharePoint)
    • IP: 10.123.1.101/255.255.255.0
    • GW: 10.123.1.1
    • DNS: 10.123.1.100

The loopback adapter does not respond to ICMP pings from within the VMs, so you can't ping your default gateway to see if your configuration is working. But you can ping the VMs from each other, and you can ping machines outside of the virtual network.

iPod jukebox, please?

Wednesday, September 5th, 2007

I love my iPod. And I'm certainly looking forward to the new iPod touch. But it appears that it is missing a feature that I use often on my fifth generation. Any iPhone owners reading this want to confirm?

I use the on-the-go playlist to queue up songs based on my mood. You can hold the center button to add a song, album, or even another playlist to the on-the-go playlist. And you can do this while the on-the-go playlist is playing. So it works like a queue.

But think about the UI for this. I browse my library for the first song, hold the center button until it flashes, the go out, out, out, into the playlists, into the on-the-go playlist, and pick the first song (the one I just picked from the library). Only then does it start playing. Now I go out, out, and back to my library to add more songs. From this point on, it works as I would like it to, but it took quite a bit of excise to get started.

So why doesn't the on-the-go playlist just start playing after the first song is added? In fact, why not just add the current song to the playlist if I select the next one. That would make my iPod work like a jukebox: the current song finishes before it starts my next selection. That would keep me in my music-listening mood instead of forcing me to think about the UI.

Install Windows SharePoint Services in account creation mode

Wednesday, September 5th, 2007

The installation of Windows SharePoint Services (WSS) is by no means straight-forward. WSS reaches its appendages into Active Directory, SQL Server, and IIS. You have to dance from one to the next in order to make it all work together. And any error messages you run into won't tell you how to fix the problem. It will just leave you scratching your head and running to Google. Hopefully such a search brought you here, and together we can find a solution.

I highly recommend that you get a book to help you with the process. I'm using Jim Buyens' Microsoft Windows SharePoint Services Inside Out from Microsoft Press.

The following steps assume that you have created a virtual network having two machines. Mine are called MLP-AD and MLP-FULLSTACK. My domain has the short name MLP, and the long name mlp.mallardsoft.com. Please make the appropriate changes for your configuration.

Prepare Active Directory
In account creation mode, WSS creates user accounts in Active Directory. We need to set up a place for those accounts to live, a domain user with permission to create them, and an owner to start us off.

On MLP-AD, create a user account named SharePoint_admin with a password that cannot be changed and never expires. Also create a domain user to be the owner of your sharepoint site (ex. MySiteOwner) with the same password lifetime. Finally, create an organizational unit called “sharepoint_ou” for the new user accounts that WSS will be creating. Right-click this new organizational unit and delegate control to the SharePoint_admin user.

Then switch over to MLP-FULLSTACK and grant SharePoint_admin permission to log on as a service. This is located conveniently in "Administrative Tools", "Local Securty Policy", "Local Policies", "User Rights Assignment". Double-click "Log on as a service" and add the SharePoint_admin account.

Prepare SQL Server
Account creation mode requires SQL Server 2003. It cannot run in MSDE. So go to the SQL Server Management Studio and add a login for SharePoint_admin ("Security", "Logins", right click, "New Login"). Be sure to use the full name including the domain (such as "MLP\SharePoint_admin"). Check the "dbcreator" and "secirityadmin" roles.

Prepare IIS
To keep the SharePoint site separate from the default web site on MLP-FULLSTACK, we want to create a new virtual host. Start by creating an A record entry in the DNS server. On MLP-AD, open the DNS management console from Administrative Tools. Open the “Forward Lookup Zones” folder, right-click “mlp.mallardsoft.com”, and select New Host. Set the host name to mysite, and the IP address to 192.168.0.3.

Now back on MLP-FULLSTACK, open the IIS Manager from Administrative Tools, and drill down to the Web Sites folder. Right-click the folder and select New Web Site. Using the wizard, enter the description “MySite”. Set the host header to “mysite.mlp.mallardsoft.com”. Set the path to “c:\SharePoint Sites\MySite”. Allow permissions to read, run scripts, and execute on this site.

Install WSS
At this point, you are ready to install Windows SharePoint Services on MLP-FULLSTACK. Do not use the “Add Role” feature in “Manage Your Server”, as this will only perform an MSDE based installation. Instead, unpack the installation files from your MSDN disk and run "C:\Program Files\STS2Setup_1033\setupsts.exe". Select the “Server Farm” option. This installer takes a while and doesn’t really let you know that it’s working, so be patient. When the installation is finished, it launches a configuration page.

Create the new application pool
Using the configuration page that appears after the installation, create a new application pool called SharePointAdministration. Use the “MLP\SharePoint_admin” account that you created earlier, and select the NTLM security configuration. If you receive the error System Error 1057 while trying to query service “SPTimer”, then you forgot the domain name (MLP\…).

When prompted to do so, type iisreset at the command line. Then click OK on the config page. At this point you will be prompted to create the configuration database. Close the browser now, as you need to do this from the command line.

Create the configuration database
From a command line, change to the directory “C:\Program Files\Common Files\Microsoft Shared\web server extensions\60\BIN”. Run the following command:

stsadm -o setconfigdb -ds MLP-FULLSTACK -dn sharepointconfig -adcreation -addomain MLP -adou sharepoint_ou

This creates a database called "sharepointconfig" on the server MLP-FULLSTACK. It uses Active Directory account creation within the domain MLP and the organizational unit "sharepoint_ou".

Now specify an email server for sending invitations to new users. This should be an internal SMTP server, since you cannot configure authorization. It will have to be an open relay.

stsadm -o email -outsmtpserver smtp.mycompany.com -fromaddress admin@mycompany.com -replytoaddress admin@mycompany.com -codepage 1252

Create a SharePoint site
And finally extend the mysite virtual server to add SharePoint services. This also creates a new application pool for the site called “MySiteApplicationPool”, and a content database called “mysite”. This step may hang. If so, hit Ctrl+C and continue.

stsadm -o extendvs -url http://mysite.mlp.mallardsoft.com -ds MLP-FULLSTACK -dn mysite -apcreatenew -apidname MySiteApplicationPool -apidtype configurableid -apidlogin MLP\SharePoint_admin -apidpwd "SharePoint_admin password" -ownerlogin MLP\MySiteOwner -owneremail mysiteowner@mycompany.com -exclusivelyusentlm

If you receive the error message “The virtual server was extended with Windows SharePoint Services, but the following error occurs in creating the default site…”, then you will have to fix the problem and manually create the default site:

stsadm -o createsite -url http://mysite.mlp.mallardsoft.com -owneremail mysiteowner@mycompany.com

Set an administrator group
Access your new SharePoint site by browsing to http://mysite.mlp.mallardsoft.com. If you get the error message “Access denied. You do not have permission to perform this action or access this resource”, then you will have to set an administrator group. This can happen if you install WSS while logged in as a local administrator rather than with a domain account. Go to the SharePoint Central Administration page and click “Set SharePoint Administration Group”. Enter “MLP/MySiteOwner” and press OK. Now that can access the site.