If you want to use the Isis2 Cloud library in your application, but your code is not written in C#, one option is to work with one of the other languages that can talk to C#, such as the ones .NET supports (C++/CLI, IronPython, etc), to cross-compile Java into .NET using IKVM.NET, or to wait for Ken to finish the C++ port of the library later in 2015.

However, another and quite flexible option is to use the Isis2 Daemon Server (I2DS). The I2DS enables developers to incorporate the Isis2 Cloud library into applications written in languages other than C#.

For example, a C/C++ application consists of three distributed processes. Each process maintains a replica of some data. To keep consistency among replicas, the application is able to call Isis2 “ordered send” API through I2DS to make sure updates to the data is totally ordered. The application can also access I2DS wrapped Isis2 distributed lock to protect shared resources. Enforcing ordering on gigabytes of data is inefficient, the application can call I2DS wrapped Isis2 Out-Of-Band transfer to initiate an OOB transfer asynchronously, before sending an ordered notification. We provide a command line application demonstrating these capabilities along with I2DS source.

The I2DS source code package includes three components: a server side, a client side, and a sample application. The I2DS server side is a daemon server running against each application process. Each server side daemon maintains an Isis2 instance that talks to other Isis2 instances to handle hard distributed tasks like consistency, fault-tolerance, and self-management. The client side is a library encapsulating Isis2 APIs with the language for the application. We provide C/C++ client side library so far. Extending to other languages is easy for the client and server sides communicate in HTTP protocol. The sample application is a command line tool that enables manually call to Isis2 APIs like starting/stopping the daemon server, sending an ordered group message, and acquiring/releasing a distributed lock. The developer can refer to the its source code (<i2ds>/Isis2DSClient/SampleClient/SampleClient.cpp) as reference.

For support please post questions to the discussions or issues tab. Weijia Song will respond with help.

In this document, we assume you have basic knowledge of the Isis2 Cloud Library (

Compilation and Installation

Please download the Isis2 Daemon Server package: i2ds-0.9.src.tar.bz2 from Unpack it.

1. To build and run on Windows

System Prerequisite: You also need to download the latest version of Isis.cs source code ( and put it under Isis2DaemonServer/Isis2Library/. To build the server side, open Isis2DaemonServer/Isis2DaemonServer.sln with Visual Studio and build the solution. Then, you can run the server by open "Isis2DaemonServer.exe" with administrative privilege(right click and choose 'Run as administrator'). To build the client side, open Isis2DSClient/Isis2DSClient.sln and build the solution. You will get a dll library:Isis2DSClient.dll and a tester program: SampleClient.exe

2. To build and run on Linux

You must have the following packages installed:
To build the server side:
# cd i2ds/Isis2DaemonServer/linux
# make

When it is done, you can see the i2ds.exe is in the current directory. Run the server by
# mono i2ds.exe

when it shows "Isis WCF Service Host is running..." and waits there, the daemon server runs successfully. To verify, open another console and issue
# netstat -an | grep 8083"
tcp        0      0      *                   LISTEN

If the above line shows up, the daemon is verified to be running. Port number 8083 is the default configured in the <baseAddress> element i2ds.exe.config, you are free to choose any other unused port number. To stop the daemon server, simply press Ctrl+C on the waiting console to interrupt i2ds.exe. NOTE: if your mono is installed to a directory other than /usr, please modify 'MONOLIBS' variable in Makefile to point to correct directory.

To build the client side and the sample application:
# cd i2ds/Isis2DSClient
# ./
# ./configure --prefix=/usr
# make
# make install

Now, the header 'Isis2DSClient.h' and library 'libi2dsc.a' along with a test program 'sampleclient' is installed to the /usr/include folder and /usr/lib. The sample application "sampleclient" is also installed to /usr/bin.

3. Configuration

The WCF configuration file i2ds.exe.config (linux) or Isis2DaemonServer.exe.config (windows) are in the same directory of the executable. You can reconfigure them to change the server behavior. For example, by default, the daemon serves at http://localhost:8083/Isis2Service/. You can change the 'baseAddress' attribute in host element to serve on other address, port, or path.

4. A detailed example

It's great to have multiple hosts in a same LAN. For testing however, running multiple daemons on a same host but listening on different port works too. Here we show a simple test example. Please refer to 'sampleclient' help information and scripts in linux/test_scripts/ for extensive testing examples. The sampleclient source is at <i2ds>/Isis2DSClient/SampleClient/SampleClient.cpp.

4.1 start the daemon servers

For easy to test, we run three daemons on a single Linux host. The steps below shows how to setup the daemon servers:
1) Make three working directory for three daemon servers:
# mkdir ~/ds1 ~/ds2 ~/ds3
# for wd in ds1 ds2 ds3; do cd ~/$wd; tar -jxf <i2ds>/Isis2DaemonServer/linux/i2ds.tar.bz2; done

2) Change the configuration so that three daemon servers listen on different ports:
# sed -i "s/8083/8081/g" ~/ds1/i2ds.exe.config
# sed -i "s/8083/8082/g" ~/ds2/i2ds.exe.config

3) start the daemon servers
# for i in 1 2 3; do cd ~/ds$i/;nohup mono i2ds.exe & done

4) verify that the daemon servers is running:
# netstat -an | grep 8080}
tcp        0      0      *                   LISTEN
tcp        0      0      *                   LISTEN
tcp        0      0      *                   LISTEN

NOTE: if a daemon server does not start properly, the sampleclient talking to it will received an error like "curleasyperform returns error code:7".

4.2 start the Isis2 instances

# sampleclient http://localhost:8081/Isis2Service/ start

It takes a while for the Isis2 instance to start completely. After the command finished, you can check the server side log:
# cat ~/ds1/nohup.out
Isis WCF Service Host is running...
startIsis() is called with:
startIsis() ... succeeded with:
        Err = False
        Reason =

Continue to start the other daemon servers
# sampleclient http://localhost:8082/Isis2Service/ start
# sampleclient http://localhost:8083/Isis2Service/ start

4.3 register message handler

Then we register a message 100 as a SEND message in group "mg":
# sampleclient http://localhost:8081/Isis2Service/ regmsg mg 100 send
# sampleclient http://localhost:8082/Isis2Service/ regmsg mg 100 send
# sampleclient http://localhost:8083/Isis2Service/ regmsg mg 100 send

In the log, we can see:
registerMsgHandleID() is called with:
        MsgHandleID = 100
        IsQuery = False
        GroupName = mg
registerMsgHandleID() ... succeeded with:
        Err = False
        Reason =

4.4 join group

Then we join the group "mg"
# sampleclient http://localhost:8081/Isis2Service/ join mg

In the log, we can see:
joinGroup() is called with:
        GroupName = mg
joinGroup() ... succeeded with:
        Address = (20559:
        Rank = 0
        Err = False
        Reason =

All daemon servers need to join group "mg"
# sampleclient http://localhost:8082/Isis2Service/ join mg
# sampleclient http://localhost:8083/Isis2Service/ join mg

4.5 send message

Then we send two total ordered messages through different daemon servers:
# sampleclient http://localhost:8081/Isis2Service/ send ordered mg 100 "Hello,I2DS."
# sampleclient http://localhost:8082/Isis2Service/ send ordered mg 100 "Hello,Isis2."

When this command is done, the message is buffered in the daemon server. The receiver can pollmsg() to pick the message he received.

4.6 receive message

Then we try to receive the message "Hello,I2DS." and "Hello,Isis2."
# sampleclient http://localhost:8081/Isis2Service/ pollmsg

I received message: "100Hello, I2DS" from group: mg
  1. sampleclient http://localhost:8081/Isis2Service/ pollmsg
I received message: "100Hello, Isis2." from group: mg

Same thing happens when the application polls through other daemon server.
# sampleclient http://localhost:8082/Isis2Service/ pollmsg
I received message: "[100]Hello, I2DS" from group: mg
# sampleclient http://localhost:8082/Isis2Service/ pollmsg
I received message: "[100]Hello, Isis2." from group: mg
# sampleclient http://localhost:8083/Isis2Service/ pollmsg
I received message: "[100]Hello, I2DS" from group: mg
# sampleclient http://localhost:8083/Isis2Service/ pollmsg
I received message: "[100]Hello, Isis2." from group: mg

NOTE: to receive the group message at the first time, the application only needs to create a "doorbell" thread to call pollmsg(), which will block the thread until some messge is coming. Then the doorbell thread can notify other application modules as messages arrive.

4.7 shutdown

To shutdown a daemon server, we simply issue a shutdown command:
# sampleclient http://localhost:8083/Isis2Service/ shutdown
# sampleclient http://localhost:8082/Isis2Service/ shutdown
# sampleclient http://localhost:8081/Isis2Service/ shutdown

Importance Notice

It seems that Mono WCF HTTP binding (till version 3.4.0) couldn't handle WCF requests concurrently. So, if you send a 'query' to yourself and then perform 'pollmsg', deadlock would occur. The query message is waiting for reply and 'pollmsg', which is expected to send the reply, is waiting for the query to be finished. Please be careful about such a situation.

More Documentation

Please refer to the following documents for how to incorporate Isis2 Daemon Server into your application:
  • Isis2DSClient/linux/test_scripts/ : how to utilize isis2 functions
  • Isis2DSClient/SampleClient/SampleClient.cpp : how to use C/C++ client api
  • Isis2DaemonServer/Isis2WcfServiceLibrary/IIsis2Service.cs and Isis2DaemonServer/docs/Isis2DaemonServerProtocol.pdf : Isis2DaemonServer WCF protocol

Last edited Feb 2, 2015 at 5:13 PM by birman, version 5


No comments yet.