Isis2 A library to help you build distributed systems.
project at Cornell is a long-term effort to create a new option for cloud computing that can enable reliable, secure replication of data even in the highly elastic first-tier of the cloud. Today’s cloud platforms tend to adhere to an approach
called CAP in which one relaxes consistency in order to get better availability and fault-tolerance. With Isis2
we’ve focused on creating new options: strong forms of consistency (e.g. updates are applied to all replicas, and in the same order),
but in ways that work around the limitations CAP highlights.
is our most recent platform in a long history dating back more than 25 years. Cornell’s work in this area started with an earlier system, called the Isis Toolkit, with many of the same ideas that Isis2
embodies. The Isis Toolkit
was ultimately commercialized and became the basis for reliability in settings that included the New York and Swiss Stock Exchanges (Isis ran the NYSE for more than a decade without a single disruptive outage), the French Air Traffic Control system, the US
Navy AEGIS warship and many others. Spin-off technologies are still to be found today in products from a wide variety of vendors in the distributed systems and cloud management spaces.
The Isis Toolkit inspired all sorts of follow-on systems both here at Cornell and elsewhere. Cornell’s systems included Horus, Ensemble, JGroups (later commercialized by JBoss) and Quicksilver. Each of these specialized in different aspects of the broader reliability
problem: security, raw speed, elegance of structure, leveraging formal methods to achieve strong correctness proofs, embedding into Java. A collaboration with the Transis project at Hebrew University helped us understand how to adapt these techniques to maximize
availability during network partitioning failures. You’ll find many, many papers on these systems here:
draws on ideas from all of these ancestors. The system looks like a library and one uses it from programs written in languages like C# (our favorite), IronPython, C\\
and so forth. In fact we can work with any language available in the
.NET framework (about 40 in total) and over time we expect to offer a similarly wide range of options for Linux users. Today, our Linux support is via a cross-compiler called Mono, and we can support any Linux application that can link with a Mono library.
In theory this is already quite a long list of options.
Isis2 has built-in data replication, key-value storage and security solutions, among others.
To enable these kinds of powerful functionalities all you need to do as the user is to configure your group appropriately. For example, if an application replicates data over some set of nodes, a single call to g.SetSecure() will activate the group security
framework, encrypting all group-related communication so that nosy onlookers will learn… nothing.
(Do I) Need Paxos? Virtual Synchrony?
uses a novel new “dynamic reliability” model that integrates the two best known replication technologies into a single easily-used solution. So, if you’ve read about Leslie Lamport’s Paxos protocol, which implements State Machine Replication,
you’ve come to the right place: the Isis2
SafeSend primitive implements exactly that technology! If you want a faster but slightly less fault-tolerant option, consider the virtually synchronous Send primitive, or OrderedSend. Isis2
designed to let you justify your application design against even a tough critic who demands formal correctness models and proofs, even as you build solutions that can scale out while pegging the network at 100% loads and keeping even hundreds of thousands
of server processes busy. If you own a data center you’ll want it working hard for you. With Isis2
we can put those resources to work, and guarantee high assurance too.
Designed to be easy for object oriented programmers to master.
The programmer works in an object oriented style, but by associating application objects with Isis2
groups, is able to achieve data replication, security, automated checkpointing and recovery, fault-tolerant parallel computing and a wide range of
other behaviors. Our programming manual, at 100-pages and counting, illustrates the programming styles you’ll need to use. A collection of demo programs is planned for 2012 and will be downloadable examples that you can copy to your machine and customize to
create new applications.
Newcomers to distributed computing might consider ordering a
copy of Ken’s new textbook
. While the textbook is much broader in coverage than just focusing on Isis2
, it does discuss the technology underlying the system and includes a number of chapters on designing and implementing complex distributed application
using these kinds of tools.
Fast, scalable, elastic: a platform designed for the cloud
The entire goal of our project is to create a new option for consistent, secure cloud-scale computing. So this naturally raises basic questions: How fast is Isis2
? How big can a deployment be?
These are good questions, but in fact we just don’t know the answers yet. One reason is that Isis2
is a moving target. We’re evolving the system almost day by day. Another is that the system is new and still a bit unstable, as one might expect for
a new platform. Don’t use it if you need a professional product with professional Q/A and a 24x7 support telephone line. And don’t count on Isis2
scaling to a half-million nodes anytime in the immediate future. But don’t bet that we won’t get there
In smaller configurations with IP multicast enabled, we’re running at about 1000 application-initiated multicasts per second (small messages), but the data rates sustainable by the system vary widely as we change the size of an application or the loads to which
we subject it. We’re aiming for the fastest data rates possible, but of course we also want stability, flow control, seamless handling of failures and all of those can fight against super-high data rates. Performance will be less good if IP multicast isn’t
enabled, forcing us to tunnel over TCP, or if strong security is used, or if membership of an application changes rapidly while the application is running.
In terms of deployment scale, for the initial release we’re recommending that applications use smaller numbers of replicas. For example, things seem very solid with low levels of replication (e.g. applications with 3-5 members in a group).
Right now our heaviest testing has been in the 25-500 member range, and we think Isis2
is stable for most uses at that scale.
Early in 2013 we ran experiments with 1000+ members, and Isis2 was quite stable. Still, by this size you must work with master/worker startups and the batched MultiJoin API. We're starting experiments now, in early summer 2013, at a scale of 2000-4000 members
and are hoping to be at 10,000 by the end of 2013.
Why is it free? How can I get support when I find a way to crash it?
We believe that when public funding is used to develop a platform, the public should reap the benefits! So we’ve made a decision to release Isis2
under a free BSD license. Our group at Cornell is providing support, and we’re setting up a user discussion
forum where you can post questions, suggested improvements or even bug reports. Ken Birman is the main author of the code, and will be doing much of the support personally. We plan to do our best to make the experience a positive one for everyone. But do keep
one thing in mind: you didn’t pay anything for this system, and it isn’t a product with a multibillion dollar company behind it. So do cut us a little slack in terms of your support expectations.