Microsoft Rebooted
Posted by Jacques Chester on Monday, October 20, 2008
Despite having a reputation for producing poor software, Microsoft have for decades published some of the leading authors on software engineering practice and know-how. They have vast resources and thorough processes for developing software along the lines of 1980s best-practice. By and large, with caveats and shortcomings, they made it work. For the period up to the late 90s, “How Does Microsoft Do It?” was a reasonable question for project managers to ask.
Windows Vista spelled the death of the old way of working at Microsoft. It had to: here was a project that ran several years late and billions of dollars over budget, which upon its release was greeted with the deafening sound of yawning. Windows XP still remains the Windows version of choice for many consumers — that is, when they are given a choice.
For a smaller firm this would have been a mortal setback. Microsoft however retains generous income from its Office business and has tens of billions of dollars on its cash balance (though most of it will be spent on share buybacks to placate grumpy shareholders). But even they have realised that while they could afford one Windows Vista, it would be uncertain if they could survive a second.
The old way of working didn’t scale, and consequently Microsoft’s approach to developing Windows 7 has undergone drastic changes.
In this blog entry by Microsoft employee Larry Osterman, the differences between the old and new methods of development are laid out.
Here’s my diagram of the old Microsoft way of project management:
The Old Microsoft process
There are some important things to notice about this process:
- It’s full of ‘blocking’ elements. The tester can’t start writing test plans until the developer has written the design document. The developer can’t work on design until the specification is done. Then the tester is stuck waiting on the developer implementing the actual code before they can write the tests. To prevent wasted staff time, Microsoft assigns multiple features to multiple people. A given staff member has a bunch of things on his or her plate, in various states of completion. The tester isn’t twiddling their thumbs on a single issue. But overall each given issue can become blocked in a number of places.
- The “Ship Room”, which decides whether a given implementation of the feature will actually be included in the product (whether it ships), is right at the end of the process. Twice. A feature could go a long way and consume a lot of resources before the change is rejected. In the article Osterman says that as a project progresses the Ship Room becomes more and more reluctant to accept any changes to the mainline product, leading to some acrimonious arguments in a very high stakes game.
- Code can apparently get past the Ship Room before it has been fully tested. Osterman says that for Vista, once code was complete it would be checked in, with testers left to deal with any fallout. This means code can be shipped to customers that isn’t fully tested.
- Bugs found by testers might require fixes that don’t get past the Ship Room.
- Throughout the process, senior management are constantly giving “input”.
Throughout the project there are regular Milestones. At each Milestone a certain number of features is meant to be complete. One problem is that the old process would mean that when a Milestone arrived, code and other project products could be wildly inconsistent states of completion and still be counted.
Three major things changed for the Windows 7 effort:
- Organisation is now based on “triads”: groups of three individuals — one project manager, one developer, one tester. Features are assigned to triads, project areas are managed by triads, and there are specialist cross-project triads (such as the Security triad) reviewing the work of other triads in their areas of expertise. This means that the testers are present from the very beginning of a feature’s inception, giving testability and testing generally much greater emphasis.
- A feature is complete when it is fully coded and fully tested. No code or products are allowed into the mainline which are not complete and tested. No half-done features are allowed. The mainline is essentially shippable at all times, rather than being incomplete at all times including at Milestones.
- Triads and project teams generally have been given the authority to push back features. At the start of every development cycle (every 3 months), teams can decide which features they will be able to implement in time for the next Milestone. If they determine it cannot be fully implemented and tested in time, they will either break into smaller features that can be done separately, or drop the feature altogether. This is a very big change from the old way of working and allows a much greater degree of decentralisation between teams.
Here’s my diagram of how the new process looks:
The New Microsoft Process
It remains to be seen if this can turn Microsoft’s fortunes around. They’ve had a bad couple of years — Google have been hiring away of lot of top staff and Linux is busily chewing them up from the bottom. However they still retain massive strategic advantages and apparently a willingness to learn from their mistakes. Don’t count this wily giant out yet.
This entry was posted on Monday, October 20th, 2008 at 8:05 PM and filed under Business, Geeky Musings, IT and Internet.
Follow comments here with the RSS 2.0 feed.
Post a comment or leave a trackback.
I think you hit the nail on the head in that last bit – Microsoft have shown an exceptional willingness to learn from their mistakes, of which they’ve made many. And of course a necessary, though not sufficient, condition for learning from your mistakes is surviving them – ie be big, cashed up and with lots of market power.
Doesn’t the new structure risk the final product ending up with bullet-proof features which the customer doesn’t want? Having the ship room entirely at the end seems to me to eliminate an important customer feedback loop.
Posted on 21-Oct-08 at 12:39 pm | PermalinkI’m not sure, derrida. But I think one thing that has changed in the past few years is that Microsoft have embraced blogging company-wide. It’s pretty trivial for employees to set up and blog about their work, which means they’re likely to get feedback pretty soon on stuff they’re thinking about.
Contrast this to the 1980s and 90s, when everything the world knew about Microsoft development practices was in a few books and a handful of articles.
So in a sense they’ve adopted two big things from the opensource world: a transparent developmental process where outsiders can easily see into proceedings, and a continuous integration approach per-feature (though they’ve had the first half of continuous integration for some time — they call it the daily build-and-smoke).
Posted on 21-Oct-08 at 1:01 pm | Permalink(1) So who made the decision to include FlightSimulator easter eggs in an Excel release a few years back? Who let it through?
(2) Actually, the “nagging” of Vista when trying to execute stuff, even as Administrators group member, is one reason why I’ll be dual booting my new laptop rather than completely SuSE-ifying it. It’s the first MS OS I’m even moderately happy with.
(3) The new plan, if carried through, is a GOOD thing. Mind you, it can stall inventiveness (just imagine how stuffed things across many operating systems would be if McIlroy hadn’t kept going on about pipes and a simple notation… even ken and dennis were underwhelmed when the idea for inclusion of simple pipes in unix was proposed).
(4) “Dont count this wily giant out yet.” Naaaa… never discount the effectiveness of a monster advertising budget!
Posted on 21-Oct-08 at 4:10 pm | PermalinkI’m sure the new way is very much the way Microsoft was a decade or more ago. I recall reading an article about their development process that described how a developer and a tester would work as a team, and a new build was made every night.
In response to derrida, being driven by user requirements in a high innovation field is often a mistake because you miss the things that people didn’t know that they wanted because they had never thought of them. Whoever stated a user requirement to develop Google? After all there were reasonable web search engines already. Altavista for example, started by a few DEC engineers purely as a showcase for the new DEC super-fast Alpha processor chip, aspired to index the entire world wide web – come to think of it, there was no user requirement stated for that either.
In fact no product manager or marketer would have had a bar of either one as they were both designed to be given away
An even simpler example: mobile phone text messaging capability. That was never stated by product managers as a fundamental user requirement, but today there are millions of people round the world who use their phone for little else.
Posted on 21-Oct-08 at 4:22 pm | PermalinkHave you considered the X-Box factor?
Microsoft dumped heaps of development resource into what was supposed to be their household gaming and set-top box. To some extent it has paid back, in as much as they have opened some decent competition in the console market. On the other hand, so many companies have dreamed of conquest in the set-top box market and it’s a tough one to make profitable with big costs, narrow margins and fickle consumers. Microsoft will never get real dominance in the gaming and console market like they have done with PC desktops, so they will never be able to milk this one hard for revenue.
Meanwhile, fighting on the console front left them short of troops for the business desktop front and largely sacrificing their already weak server position. Plus, DRM and “Trusted Computing” got a harsh reception from the customers (remember customers? the guys who pay the wages) so they had to water down the transfer of “closed platform” technology from the X-Box to Vista (note that Vista was originally designed to be almost a totally locked platform).
Microsoft also expected their dominance in the browser wars to be much more comfortable than it turned out to be. No one expected Firefox to make such an impact (least of all Microsoft) so once more they had to swing resources back to browser development where they had planned for that to be all finished up already.
Just a few reasons why neither the engineers, nor their immediate managers were directly to blame for Vista’s lateness.
Posted on 21-Oct-08 at 7:39 pm | PermalinkTel_
As a business Microsoft has been struggling for years to grow explosively like it did in the old days. The massive profit margins they sustain mean they can afford to spend years and years hammering away at a market in an attempt to break into it. Consumer electronics is a big market and Microsoft wants the kind of platform-controlling profits it makes on Windows and Office.
It must really piss them off when Apple effortlessly segues from music players to mobile phones, though on the other hand the Xbox is far more successful than the Apple TV.
As a side note, Google has become like Microsoft in this respect. Their central business is advertising; that’s where all their money comes from. Nothing else is really turning a profit worth a damn. So essentially they’re busy squirting money at anything that moves in the hope of breaking open a new growth vector. Hence forays into green-tech venture capitalism, phone platforms, browsers … it goes on and on.
As I noted in the article, engineers had no say in the featurelist of Vista, it was driven completely from the top with no regard for practicality or polish. And even then senior management couldn’t have everything they wanted.
Posted on 21-Oct-08 at 7:47 pm | PermalinkMikeM;
Good points about things being “retroactive” requirements — a market need nobody knew existed until the solution did. It’s given me some ideas for a ditty on requirements engineering.
Posted on 21-Oct-08 at 7:49 pm | PermalinkMikeM, I agree that starting things off with asking what the customer wants is over cautious where you’re hoping to create new markets, but not so when you’re just trying to maintain existing ones.
The market for desktop operating systems and office software is now very mature so it’s not sensible to go to the other extreme – just letting the engineers (or, far worse, the managers) have their heads and then not considering whether the product will sell until the end.
Tel_’s comment makes my point. If they’d talked to the customers they’d have soon learnt that what MS had in mind as “Trusted Computing” and what customers had in mind were quite different things. And again, talking to customers would have told MS that IE was a PoS that needed urgent fixing – if they didn’t fix it someone else soon would.
Posted on 22-Oct-08 at 11:24 am | PermalinkJacques,
None of this changes the fact that since Microsoft Office 2000 (and to a lesser extent, Windows XP), these core packages are essentially “done”.
Since then there have been no major improvements that are either compelling reasons to upgrade or demanded by all customers.
What are the major feature improvements we’ve seen in the past 8 years? XML support? SharePoint support? (Both of these are half-baked in any case!) Don’t even get me started on the ribbon — a classic case of a solution looking for a problem if I’ve ever seen one.
So really, Microsoft should just charge people $50/year for access to “Feature Packs” and build off the same core. If that’s not enough money, Microsoft should be investing in the next big thing rather than just moving the deck chairs around and trying to extract huge amounts of cash from selling the same product over and over.
No matter what development model Microsoft uses, I don’t believe they will ever be able to build a compelling upgrade for either Office or Windows from this point on — well, at least without largely dropping compatibility such as when Apple moved from OS 9 to OS X.
Posted on 23-Oct-08 at 7:33 pm | Permalink