But I’m not sure we will learn the right lessons.
In the early 1980’s I worked on applications that had to process 300 transactions per minute. At the time that was considered a heavy load for a Dec Vax to deal with. By the late 1980’s the same applications were dealing with 1,000 transactions per minute because the hardware had got a lot faster. In the mid-1990’s I worked on a small scale credit card processing application, running on a later incarnation of the Dec Vax that was able to handle nearly 20 transactions per second. In the late 1990’s I worked on a stock exchange system that had to deal with what we thought at the time was a stupidly large number of messages per second … little did I know. Fast forward to the early 2000’s and I worked on consumer facing web applications that had to withstand 10,000 requests per second. By 2010 I had the fun of being on a real web scale project, experiencing the joys of being linked to by Digg and CNN and hoping that the ensuing millions of requests per hour would not being the system down.
All this is to say that dealing with internet scale applications is a solved problem, but it seems that whoever was involved in the Healthcare.gov fiasco did not realize that. Dave Winer pointed out that the Government develops software differently, but there is no excuse for building a site that cannot handle the traffic.
I disagree with Bob Goodwin - there is no software engineering crisis - OK I have to say that because I wrote the Software Craftsmanship book. But that is not the real reason I have to disagree - I have to disagree because whoever built the site went about it the wrong way. Dave Winer parodied the approach that big consulting companies take
They’d fully specify the software, the user interface, its internal workings, file formats, even write the user documentation, before a single line of code was written. Then they’d hand the parts off to development teams who would independently of each other create the components. Another team would do the integration.
The sad fact is that the big corporations that are awarded these big government contracts do not have a clue how to build web scale applications that work. They over promise and massively underdeliver. All too often large companies are awarded contracts to build large systems and fail to deliver anything of value except to their own shareholders.