“If it looks too good to be true, it probably is.”

I’ve counseled others with that same advice.  But now, I find myself frequently fighting that very conception – most recently, this past week at Oracle Open World.

On Monday, I had the opportunity to present NetApp’s Snapshot and FlexClone technologies and their role in supporting rapid application development (RAD) methodologies for database development, test, and QA:

S283383: “Reducing Time to Deployment: Improving Agility for Database Development, Test, QA, and Migration”

Rapidly creating copies of production and test databases for use in a variety of development, test, quality assurance, and migration scenarios is fundamental to today’s application development lifecycle. This session reviews traditional and emerging approaches to “database cloning” and demonstrates how space and time-efficient copies, such as Network Appliance’s FlexClone technology, can drastically reduce time to deployment and improve business agility for Oracle Applications and other database-centric application development.

I had written previously on this topic in NetApp’s Tech OnTap - a monthly newsletter aimed at IT professionals that implement and manage NetApp storage technology.  And I’ve spoken on many occasions with NetApp customers about the ways in which these technologies can transform database development.

So once again, after describing how NetApp’s underlying storage system (WAFL) works, and explaining how Snapshot and FlexClone technologies leverage that implementation for space-efficient, near-instantaneous copies of file systems and LUNs, I proceeded to explain how NetApp customers can make working copies of multi-hundred gigabyte databases nearly instantaneously (a few minutes) without consuming any additional storage.

You’d have thought I was selling snake oil!  As often happens, the discussion was met with a range of responses, from keen interest — because, if true (which it is!), it can revolutionize the manner in which database development is done — to Missouri-like healthy skepticism.

I guess I should have felt slighted, when, after my presentation a number of attendees came to the front of the room - not to discuss the topic with me, but to speak with a young woman who had volunteered the fact that she was a NetApp customer, and in fact had experienced some of these effects in her organization.

But I’ve experience this reaction before - and fully expect that I’ll continue to experience it in the future.

After all, it really does seem too good to be true …

References

[1] “A Thorough Introduction to FlexClone(tm) Volumes”, NetApp TR-3347.

[2] “Oracle 9i for UNIX: Cloning Database Using NetApp Filer Flexible Volumes in NAS and SAN Environments”, NetApp TR-3355.

[3] “DB2: Cloning a Database using NetApp FlexClone(tm) Technology”, NetApp TR-3460.

Reminds me of an old cartoon …

Pinky: “Gee, Brain, what do you want to do tonight?”

Brain: “The same thing we do every night, Pinky, try to take over the world!” [1]

The plan includes:

• a sweetheart deal to get RedHat subscribers to switch to an Oracle Unbreakable Linux subscription
• the promise of lower longterm maintenance costs, and
• indemnification from SCO, who has devolved a once innovative business into one of the most despicable business models I’ve ever seen (sue, sue, sue).

Larry claims that Oracle’s initiative is an effort to accelerate the robustness of Linux for enterprise data center applications.  Looks like investors have a slightly different take … RHAT is down more than 16% in after hours trading.

So the Oracle vs. Microsoft title bout looks to be shaping up nicely - with both parties sporting a full software stack - OS, storage, database, mail, apps, …  Wonder who they’re gonna get from the DOJ to officiate?

Oh well, maybe Larry will follow Bill and Warren’s lead and decide that his continued good fortune should be used to help address some of the world’s ills.

References

[1] Pinky and the Brain, YouTube.  Or try the politically slanted alternative version if you’re so inclined.

I recently sent Jon Toigo (DrunkenData) a formal response to questions he and some of his readers had about Data ONTAP GX, NetApp’s scale-out storage architecture.

In a followon comment, one of Jon Toigo’s readers, who failed to indicate his affiliation, but whose name links to LeftHand Networks :-), wrote:

“It’s nice to see NetApp being open about its GX architecture. This being said, the architectural deficiencies were clearly spun in a positive light with marketing hype.”

So, in the interest of educating that reader (and perhaps some other readers) about scale-out storage and the scale-out computing revolution, I thought I’d take the opportunity here to share some of my experience and personal observations.

And … sorry to disappoint, but no “spin” here.

In that post, Jon’s reader missed two key points, imho.

First, the *primary* purpose of almost all scale-out storage architectures is to provide scalable AGGREGATE performance for a large number of clients - not to speed single stream performance to a single host.  This is true both in technical computing apps and in largescale enterprise deployments.

In fact, most hosts in cluster computing configurations (or enterprise server environments) have only a single GbE (or 2Gb FC) into the client fabric - so you’ll never get more than a wire’s worth of bandwidth (90 or 100 MB/s for GbE) to a single host anyway.

For higher performance interconnects - including 10GbE and IB, it *is* possible for single hosts to obtain higher throughputs, but you still won’t exceed the single wire (or single “logical wire” if you’re using and can leverage link agg) performance from host to storage subsystem.  So the fact that a host goes through an N-blade in the GX architecture to get the data is irrelevant as long as the N-blade can deliver the single session bandwidth expected by the host - which, of course, it can.

That having been said, technical computing apps requiring high aggregate I/O for “single problem” scenarios do exist.  But those applications are structured in one of two ways to leverage parallel computing - either as a set of “embarrassingly parallel” processes, or as part of a large parallel job (using, e.g. MPI).

In the former case, this is simply an aggregate I/O problem with lots of hosts accessing lots of files on the shared storage system (perhaps in the same directory, perhaps in multiple directories).  In the latter case, the individual “workers” in an MPI communication group (node/process set) will EACH perform their own I/O - either accessing individual files or accessing (typically) disjoint portions of large files (perhaps through a parallel I/O layer such as MPI-IO).  For either case, GX does just as it is supposed to - it spreads the accesses across *BOTH* the N-blades and the D-blades to provide scalable I/O.

This is basically the manner in which *all* parallel / clustered file systems are employed for scale-out computing applications.  There are URLs to many of the companies developing such systems in my aforementioned post on The Scale-out Evolution.

As I indicated in my response to Jon, pNFS is an NFSv4 proposed extension that will effectively allow you to take the VLDB functionality “out-of-band”, so that a pNFS-savvy client can get a “map” of the locations of all of the file segments through the metadata server (at time of open or in response to file extend callbacks), then go directly to those segments for its “data access” commands (read, write) - in essence, eliminate the “hop” that Jon initially asked about.

There are other performance/efficiency options that can be effected in such architectures as well - both in an out-of-band metadata scenario AND in the current N-Blade implementation.  These have to do with how those components may choose to do read-ahead across distributed segments of a large shared file - effectively issuing requests for multiple portions of the file at the same time to “fill” a larger pipe back to the requestor.  With a pNFS client, the multiple segment read-ahead could be effected by the host.  With the current GX implementation, by the N-Blade.

Bottom line … yes, it’s true that a single host’s I/O will be limited by the *smaller* of: (a) it’s network ingress capability (typically 1 GbE); (b) the network egress capability of the N-Blade to which the host session (mount) is connected.  Not a problem for most apps I’ve ever seen given the hardware on which GX currently runs.

The reader’s second point has to do with the functionality that is part of the current ONTAP GX release.  The initial release of ONTAP GX, as I had indicated, is targeted at technical computing applications, for which the set of features currently provided are critical.

Technical computing applications work almost exclusively against files and filesystems.  What I think Jon’s reader is missing is the difference between a block storage platform (in which you can add bricks and controllers under a single management domain and provide striped LUNs - stuff many of the vendors’ volume managers have been doing for years) and a scale-out FILESYSTEM, which ONTAP GX presents.

As many users of parallel and clustered filesystems know, this is a very important distinction.  Large storage systems can effectively be crippled by a filesystem that hasn’t been adequately designed for scalability.  The old “metadata bottleneck” in SAN filesystems.

ONTAP GX provides both a scale-out storage platform (distributed storage “bricks”, distributed controllers, single management domain) as well as a parallel filesystem with DISTRIBUTED metadata - alleviating file system bottlenecks that would otherwise prevent full utilization of the underlying architecture.

What you’re seeing at NetApp, and I believe elsewhere in the storage industry, is the gradual maturation of scale-out storage technology (and particularly parallel/clustered filesystems) - across the industry, as companies look to leverage these innovations first in production technical computing applications (digital animation, EDA, bioinformatics, automotive/aerospace design, …), and subsequently as highly scalable enterprise storage systems (e.g. for enterprise grid deployments).

The truth of the matter, despite what some marketing departments will tell you, is that (a) building parallel/clustered filesystems is hard (especially in delivering scalable, full featured, robust, enterprise-class services); and (b) we (as an industry) are very much in the early days of the scale-out storage revolution.  Just ask someone who’s tried to deploy any of these systems, or do an in-depth survey of the scale-out storage solutions and ask them (and their customers) about feature sets and robustness.

From my (admittedly biased) perspective, the HUGE advantage NetApp has is a very successful enterprise business based on a proven scale-up approach to unified storage (SAN, iSAN, NAS) with Data ONTAP, and an emerging scale-out storage platform based on the Spinnaker legacy — all based on the same physical hardware components,  same core storage “container” architecture (WAFL), and same data management services (snapshots, mirrors, etc.).

Getting all of that right requires significant innovation, and takes time.  Hence the evolutionary, multi-release rollout of ONTAP GX.

Leonard Bernstein (or more accurately, Betty Comden and Adolph Green) had it right.

As I do from time to time, I had the chance to travel to New York on business this week.  And as usual, as I came through Queens and the Manhattan skyline approached, I fell into the magical grip of the city.

Circumstances this trip brought me to town a day early, and afforded me the chance to explore.  It was a beautiful day, so I set out from my midtown hotel late Sunday morning for a stroll through Central Park.

I’ve posted some of the pictures I took on my walk.

One of the highlights of my afternoon came when I stumbled across the Central Park Dance Skaters Association, a group of skaters on inline and traditional roller skates all skate-dancing to the disco and funk sounds of a couple of DJs.  The energy was infectious, and the ebullient skaters drew quite a crowd.

Only in New York … you gotta love this town!

No, really.  Computing Dinosaurs, that is.

Back in the early 1980s, most technical computing was done on “large iron” - Cray supercomputers and the like.  These systems took many years to design, cost many millions of dollars, required custom facilities, and employed a cadre of highly trained programmers who could extract every last drop of performance from those architectures.

I’ll never forget the first time I saw a Cray-2 supercomputer.  The Cray-2 was liquid cooled, and you could actually see the fluorocarbon cooling liquid bubbling over the electronic circuit boards.

In the late 1980s and early 1990s, parallel computing was the rage, and a number of approaches were explored — a “diversity bloom”, if you will.  Massively Parallel Processors based on custom bitwise processors like Thinking Machines’ Connection Machine and MasPar’s MP-1 and MP-2, those based on commodity microprocessors, such as the Intel iPSC, and the BBN Butterfly, and those based on parallel/vector architectures like the Alliant FX series, and Convex Computer systems all roamed the earth.

Towards the mid-1990s most of these architectures had died out, giving way to two predominant high performance computing architectures:

• Symmetric Multiprocessors (SMPs) - the scale-up approach
• Beowulf clusters - the scale-out approach

(For a more detailed treatise on scale-up and scale-out, see the case study I wrote for Microsoft a few years ago on the Cornell Theory Center’s use of scalable computing technology).

In the early part of this decade, many of the large SMP architectures  also fell, leaving a landscape in which Beowulf (scale-out or cluster computing architectures employing commodity servers and networking) rules the land.  Indeed, the Top500 web site, which lists the 500 largest computer systems in the world, indicates that “365 systems are labeled as clusters, making this the most common architecture in the TOP500″, with Intel, AMD, and PowerPC processors at the core of nearly all of the systems in the list.

Today, scale-out computing architectures are clearly the dominant architecture for technical computing applications.

I contend that we are presently in the midst of a similar transition in the storage industry.  The monolithic storage arrays of the past twenty years are giving way to midrange systems, and eventually to scale-out storage architectures employing commodity-component -based “storage bricks”.

I believe we’re seeing now the equivalent of the parallel computing diversity bloom in the storage market — proliferation of a wide variety of scale-out storage systems and clustered file systems, including BlueArc, IBM’s GPFS, Ibrix, Isilon, Lustre, Panasas, Polyserve, SGI’s CXFS, Rackable’s Terrascale, and, of course, NetApp’s ONTAP GX.

And, as with scale-out computing architectures, which are now at the core of enterprise computing operations, I believe we will see these approaches evolve and mature - and no doubt consolidate - into the enterprise storage systems of tomorrow.

What an incredibly beautiful city.  Of course it helped that the weather was simply spectacular during my visit, 70 F.

I took some photos in a morning walk around the city.

Thanks of course to my NetApp hosts - you know who you are

• Google signs a $900m deal with News Corp to provide search and advertising for MySpace.com and other Fox Interactive properties
• Apple announces an initiative to provide movie downloads from the iTunes store
• Yahoo announces an online video programming venture with Current TV
• Yahoo is reportedly in “serious talks” to acquire social networking site Facebook for $1B

So, is this deja vu all over again?  Are we experiencing the frantic upside of a second internet bubble?  Will it burst too?

While there are perhaps similarities to the Great Internet Bubble (note I don’t say “First Great …”) in the fervor with which web 2.0 applications seem to be catching the attention of the business community, I think there is one very significant difference.

And that is the fact that there is *real* money behind these deals - not the worthless-stock-for-worthless-stock transactions of the late ’90s.  But real, honest-to-God money, coming from very profitable media companies.

Will there be a shakeout once the fervor settles down?  Undoubtedly.  Will there be a wholesale bubble burst?  I don’t think so.

The internet is becoming the new entertainment medium - pulling viewers and advertising dollars from traditional broadcast and cable TV networks.  What we’re seeing is a transition in consumer behavior from passive viewer to active participant - in an effort to take back control of how we choose to entertain ourselves (not “be entertained”).

So, rather than asking when the internet bubble 2.0 will burst, maybe we should all be humming “Internet killed the ‘cast TV star” [1]

[1] shameful, unabashed reference to the Buggles’ “Video Killed the Radio Star”

 

I contend that a Storage Grid is indeed a useful architectural concept. 

The usual definition of “Grid” is more than the “network architecture” used to connect components together.  “Grid” really is the concept of an infrastructure (not just wires) into which a variety of devices (appliances) can plug to provide network-based services to applications. I often use the following definitions of Grid when talking with enterprise-oriented customers …

• Cross-application (enterprise-wide), holistic IT model based on complete virtualization of underlying resources (virtual everything).
• A heterogeneous environment that supports distributed application deployment, execution, management, and monitoring

In which case we can talk about the Storage Grid being a virtualization of underlying storage resources that provides a range of services to applications – data storage, data management, replication, backup/recovery, compliance, … at different performance levels and price points.  Throw in transparent data migration across backend tiers, and you have the Information Lifecycle Management (ILM) story.

 

The initial definition, as posed in Ian Foster’s “The Anatomy of the Grid” [1], encompassed “coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”.   As indicated in a subsequent paper, “the key concept is the ability to negotiate resource-sharing arrangements among a set of participating parties (providers and consumers) and then to use the resulting resource pool for some purpose” [2].

Since then, the term has broadened to refer generally to the use of shared (commodity) computer components — processing and storage — in a distributed networked architecture.  In essence, an architectural alternative to the development of monolithic, centralized computation and storage architectures.

That having been said, there are at least three common uses of the term “Grid” in the present IT lexicon:

• Technical Computing Grids employ rack-mount computer systems in scale-out configurations to bring the aggregate processing power of many CPUs to bare on problems of interest.
• (Enterprise) Utility Computing Grids provide an agile, on-demand model for application provisioning and migration based on sharing of common infrastructure resources implemented through commodity computer components (CPUs, networking, storage).
• Data Grids provide for the distributed capture, management, and sharing of information (and sometimes instrumentation) — typically across mutliple authority domains.

Technical Computing Grids

This artchitectural approach is common in High Performance Computing (HPC) applications — ones employing large amounts of computing power to solve computationally intense problems.

Traditional HPC apps in the scientific computing space include:

• Energy research and simulation, and high energy physics research

• Earth, ocean, and atmospheric sciences, global change, and weather prediction
• Complex multi-physics simulations for aerospace design
• Seismic data analysis
• Large scale signal and image processing applications

These are the types of applications that used to run on large supercomputers, such as those developed in the early 80s by Cray and others.  As commodity-based cluster computing (Beowulf clusters) emerged, many of these application were re-hosed on large (1000+ node) compute clusters (often called “grids”).

Additionally, many new applications have arisen, in part due to the increased availability of these new commodity supercomputers.  These applications are increasingly at the core of business critical operations, and include:

• Drug discovery (computational chemistry, genomics research)
• Circuit design simulations
• Automobile design simulations (aerodynamics and crash analysis)
• Risk analysis of financial portfolios
• Digital media applications (animation and rendering)

Utility Computing Grids

Utility computing is all about leveraging modular compute, network, and storage components to improve resource utilization, increase enterprise agility through rapid application provisioning (and re-provisioning), and simplify IT operations.  In short, creating a more nimble, more cost effective IT organization.

The terms Grid, Utility Computing, and On-demand computing are often used almost interchangeably to describe a wide variety of approaches that are generally aimed at these objectives.  These approaches are typically based on two key principles - the foundational pillars - of utility computing: Consolidation and Virtualization.

These two principles go hand-in-hand to support hosting of multiple applications - either concurrently, or in a time-share model - on the same physical resources.  E.g. server virtualization, as exemplified by VMWare, Xen, and Microsoft Virtual Server supports the hosting of multiple (virtual) servers on a single physical host.  The physical resources of the host - cpu, memory, I/O, network connectivity - are shared amongst a number of virtual servers.  Each looks like a standalone server - with its own IP address(es), its own network and security settings, and its own OS and applications - but shares the underlying physical resources.  Server virtualization improves utilization by consolidating multiple applications onto a common physical hardware platform - eliminating capex and opex costs associated with deploying multiple physical servers.  This approach is particularly effective in containing the “server sprawl” that has occurred in many IT organizations where every application instance required its own server (and local storage).

Similarly, network virtualization strategies (including vlans) and storage virtualization strategies serve to allow applications shared use of those infrastructure resources, often employing Quality-of-service (QoS) provisions.

Storage virtualization strategies, in particular, aim at delivering logical storage containers (filesystems and LUNs or volumes) that transcend the physical nature of storage systems – disks and controllers.  Together with tiered storage strategies (using different classes of storage for different types of data) and transparent data migration, they deliver a “storage as services” model where those services are provided in the storage network and not by individual physical devices or servers.  A realization of what many have been calling information lifecycle management (ILM).

Data Grids

The notion of a data grid may be the closest concept to the original Grid concept developed by Foster et al. It represents a physically distributed set of information resources (services) contributed by multiple authorities under a common set of protocols. In some ways, the World Wide Web represents the first generation data grid, in which information is “published” by individual sites, indexed by crawlers, and accessed via search engines and explicitly represented links.

The San Diego Supercomputer Center’s Storage Resource Broker is another data grid model that presents cataloged data “collections” presentable to a community of interest [3]. SRB “presents the user with a single file hierarchy for data distributed across multiple storage systems. It has features to support the management, collaboration, controlled sharing, publication, replication, transfer, and preservation of distributed data.”

In many ways, these efforts represent first generation data grids – with static or quasi-static data “published” into web-based documents or files, and consumed by browsers and file-savvy applications. The next generation of data – or information – grid is one based on Web 2.0 technologies that affords a richer model for compositing individual information sources into a rich set of distributed web services.

References

[1] Ian Foster, Carl Kesselman, Steven Tuecke, ”The Anatomy of the Grid”, http://www.globus.org/alliance/publications/papers/anatomy.pdf

[2] Ian Foster, “What is the Grid?  A Three Point Checklist.”, http://www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf

[3] SDSC Storage Resource Broker, http://www.sdsc.edu/srb/index.php/Main_Page