I joined Microsoft mid-way through Whidbey's lifecycle. Mid-way means post-feature development, for the most part. There were plenty of unplanned features that I got to design and work on, but those were handled quite differently than the initial development process. The impression I formed during this period was that of a very regimented, structured, and process-heavy software engineering practice. Clearly this is good to ensure people don't screw up too badly, but it also places unnecessary constraints on your best talent.

Or, as Paul Graham said in his essay Hackers & Painters:

Big companies want to decrease the standard deviation of design outcomes because they want to avoid disasters. But when you damp oscillations, you lose the high points as well as the low.

At first, I thought this was necessary for the type of project I was working on, when compared to the projects I'd worked on in the past. But now that 2.0 is out the door, I'm enjoying myself quite a bit more.

Scrounging for change

Planning the direction for future releases is clearly a complex game, consisting of a mixture of top-down (where must the business go?) and bottom-up (what features do we want to do?) analysis. Customer needs come from all directions. At some point, somebody presumably must pull the trigger and unleash the team in a concrete and coordinated direction. Some don't have the stomache for this, but without it paralysis analysis sets in.

Of course, direction is a funny thing. It typically emerges over time rather than being planned explicitly, whether those doing the planning realize it or not. This particular case is no different. Before we even shipped Whidbey Beta2, we knew the primary focuses for Orcas, even down to the feature level in many cases. I suspect most people are already half-way down the path we'll eventually go, e.g. because they've been dreaming of and prototyping the new features they would like to implement for well over a year already.

But (in theory) somebody has to capture that, refine it, and communicate it to form a shared understanding. Presumably for purposes of making sure management is OK with what everybody wants to implement. But of course, what everybody wants to implement must first be turned into market segmentation and value propositions. OK, that statement is a tad cynical. Although I'm sure the process in this case will ferret out some thought bugs before they get put into code--which is clearly a good thing--I can't help but wonder whether the cost is worth the benefit. This deserves separate analysis, of course.

Flying under the radar

Planning aside, the projects I am most interested in as we move into the next few releases are the tiny incubation efforts. These are ordinarily small groups of individuals from across the company (including product teams and research). Such groups are a diverse mix of people with different backgrouns and goals, yet are drawn together because of a shared interest. If people are responsible for allocating their own time "on the side" to work on something, you can safely bet they are passionate about it and (more than) qualified to work on it. Being united by a shared interest can lead to fun collaboration and great end results. Generics is an example of a (big) project that evolved this way.

In many cases, there is no clear support from management in terms of funding for these projects. A head nod about its importance is about all you'll usually get at first. And in fact, often such wild-west efforts can go against the spirit of the planning. In terms of funding, research is funded to research it and obviously product teams can communicate with research. But for product teams to get something worthy of the productization stamp of approval (that's a vigorous management head-nod), clearly there's some level of prototyping that is needed. But simultaneously, most capable developers are focusing on staying inside the bounds of the aforementioned processes and fixing bugs. This is a slight catch-22.

Regardless of tedious funding problems, I enjoy these efforts the most. The path is not pre-defined--we must find it--and the ability for one individual to contribute substantially is very high. They feel like start-ups. But you don't have to worry about paying the bills and recruiting the best talent (the perks of working at a place like Microsoft). Many such projects I've been involved in have been primarily thought exercises. But a few have recently been given some level of funding. The groups of people that are working on them are, as noted above, often genuinely interested in what's being built. This is great. Very little process is needed...only enough to ensure we hit the deadlines for integration with the main product, and to report back to management to make sure they feel comfortable.

Only time will tell whether this approach will lead to better results. But something seems intuitively correct about the attitude that enabling your best people to do what they were hired for will lead to huge successes. Obvious corollaries can be drawn from this statement.