Isis2 Cloud Computing Library
Isis2 is a new option for cloud computing that can enable reliable, secure replication of data even in the highly elastic first-tier of the cloud. Isis2
is a project of Ken Birman at Cornell University. Although the system started out as a data replication technology (groups of programs that can share updates), in 2013 Isis2 became much more big-data oriented.
The need for high assurance has never been greater: with the trends towards data centers of all sizes and shapes (ranging from small racks of just a dozen or two machines to massive cloud computing data centers with hundreds of thousands of them), developers of modern computing systems need to target the Web, employ Web Services APIs, and yet somehow ensure that the solutions they build can scale out without loss of assurance properties such as data security (who knows what the other users of the cloud might be doing… or what might be watching?), consistency and fault-tolerance.
library was built to help you solve this problem in an easy way, closely matched to the style of development used for standard object-oriented applications that use GUI builders.
A few quick comments aimed at those revisiting the site who might be wondering what's new.
We added a distributed sharding scheme (a DHT) that integrates with LINQ and has the remarkable feature that it can do consistent queries even on rapidly changing data, and yet scales as well as any sharding scheme of which we're aware. This takes Isis2 squarely into the world of big data, and by using LINQ in conjunction with this new technology, you can even do database-like programming in a new and rather elegant way, especially if you combine your queries with our scalable "aggregation" layer.
A second new mechanism, added in summer of 2013 and just being released as of August is a file copying and "shuffling" tool. The basic idea is that you have a cluster of machines and are running, say, Hadoop. As part of your computation you generate intermediate results and store them as memory-mapped files (this is popular lately). With the new tool, you can declare these to Isis2 and then ask the system to move them around: maybe you had a copy at A and want copies added on B..F. You want to move some other file outright: it was at D and you want it now at H and J. The number of copies can be large and the files can be huge. Isis2 will concurrently carry out this file movement actions at very high speed using a form of multicast that is reliable and (if desired) secure. The speed is extremely good; we seem to be able to peg even a large cluster network at near 100% of capacity, with very low overheads.
Right now these features can only be used by linking to the Isis2 library, but within a few weeks we plan to release a "daemon" that will run Isis2 and be accessible via simple RPC commands and even via command line. At that point you can launch the daemon on your cluster nodes and use these features of Isis2 without needing to touch the library itself.
So... check it out and let us know what you think!
Please see the Documentation
page for video summaries, release description, as well as complete documentation.
For compilation under Linux environment, please see the Compile
page. With proper setup, Isis2 can run on Windows, Linux, Amazon EC2 or Eucalyptus-style virtualization platforms, Azure... even the Android mobile phone platform! (But we're not convinced that it makes sense to use Isis2 on an Android phone, even if this does work)