It was only 25 years or so ago that Dinosaurs ruled the earth.

No, really.  Computing Dinosaurs, that is.

Back in the early 1980s, most technical computing was done on “large iron” - Cray supercomputers and the like.  These systems took many years to design, cost many millions of dollars, required custom facilities, and employed a cadre of highly trained programmers who could extract every last drop of performance from those architectures.

I’ll never forget the first time I saw a Cray-2 supercomputer.  The Cray-2 was liquid cooled, and you could actually see the fluorocarbon cooling liquid bubbling over the electronic circuit boards.

In the late 1980s and early 1990s, parallel computing was the rage, and a number of approaches were explored — a “diversity bloom”, if you will.  Massively Parallel Processors based on custom bitwise processors like Thinking Machines’ Connection Machine and MasPar’s MP-1 and MP-2, those based on commodity microprocessors, such as the Intel iPSC, and the BBN Butterfly, and those based on parallel/vector architectures like the Alliant FX series, and Convex Computer systems all roamed the earth.

Towards the mid-1990s most of these architectures had died out, giving way to two predominant high performance computing architectures:

  • Symmetric Multiprocessors (SMPs) - the scale-up approach
  • Beowulf clusters - the scale-out approach

(For a more detailed treatise on scale-up and scale-out, see the case study I wrote for Microsoft a few years ago on the Cornell Theory Center’s use of scalable computing technology).

In the early part of this decade, many of the large SMP architectures  also fell, leaving a landscape in which Beowulf (scale-out or cluster computing architectures employing commodity servers and networking) rules the land.  Indeed, the Top500 web site, which lists the 500 largest computer systems in the world, indicates that “365 systems are labeled as clusters, making this the most common architecture in the TOP500″, with Intel, AMD, and PowerPC processors at the core of nearly all of the systems in the list.

Today, scale-out computing architectures are clearly the dominant architecture for technical computing applications.

I contend that we are presently in the midst of a similar transition in the storage industry.  The monolithic storage arrays of the past twenty years are giving way to midrange systems, and eventually to scale-out storage architectures employing commodity-component -based “storage bricks”.

I believe we’re seeing now the equivalent of the parallel computing diversity bloom in the storage market — proliferation of a wide variety of scale-out storage systems and clustered file systems, including BlueArc, IBM’s GPFS, Ibrix, Isilon, Lustre, Panasas, Polyserve, SGI’s CXFS, Rackable’s Terrascale, and, of course, NetApp’s ONTAP GX.

And, as with scale-out computing architectures, which are now at the core of enterprise computing operations, I believe we will see these approaches evolve and mature - and no doubt consolidate - into the enterprise storage systems of tomorrow.