DMC in pure C++: Tabled, for now (but you can certainly use the system from C++)

Aug 13, 2015 at 2:27 PM
Edited Aug 13, 2015 at 3:04 PM
For more than six months I've been trying to get Isis2 translated into pure C++. In fact I really started more than a year ago, but I guess it has only been six months of really working on it. Unfortunately, I think the time has come to give up on this.

Generating a version where all the Isis symbols are changed to DMC is, of course, trivial. So we have DMC (like Isis2) that can be used from C#, C++/CLI, IronPython, IronRuby, F#, etc. It can also call into code written in native C++ or C. But this is no different than saying that you can use Isis2 with those languages.

My goal had been to translate DMC entirely into pure C++, and this aspect of the plan has stumbled. The core problem is that to do it, I needed to use a whole bag of C++ 11 features: std::shared_ptr (the C++ 11 version of Boost smart pointers), std::thread, recursive variadic templates, type traits (static compile-time type information), etc. As mentioned on a different thread, I found an automated translation tool, then planned to make a human pass over its output and fix any issues.

In fact this is feasible; by now I know every aspect of what has to happen. Step one, though, involved using that automated translation tool and this is where things stumbled. The best available tool is inconsistent about the way it translates to smart pointer code. For example, it will transform a DMC Msg reference into std::shared_ptr<DMC::Msg> in straight-line code, but if it sees something of type List<Msg>, it translates it as std::vector<Msg>, which means that every access to that data structure will now have to be edited by hand to fix the mistaken translation. I have a lot of situations where things are passed around in lists, and this ends up meaning that pretty much every line of code has to be reviewed and edited extensively. In effect, they don't generate code with correct pointer reference counting. Mistakes would result in leaks or memory exceptions, and of course this would be a disaster for me. But this level of review and editing is very close to a complete rewrite of the system by hand.

Then it actually gets worse. C++ 11 isn't widely supported yet, so it turns out that only a few compilers exist (Clang, GCC, Microsoft) and of those, GCC can't be used because the GCC version of std::thread has copyleft licensing provisions in it. Microsoft's version has some features tied to Microsoft's own platform. Clang really works only on Linux. Threads don't have priorities; DMC needs them. Type "traits" are the way you get to compile time type information; they don't have standards for this part of C++ 11, and only GCC really supports them seriously, so DMC polymorphic callbacks are hard to support on the other C++ variants. The list goes on and on -- I'm only sharing the more extreme issues.

And now I'm going to editorialize for a second: I just don't like C++ 11. I assumed it would turn out to be like any other language (I normally work in C, standard C++, C#, Java and Python, and now and then I use Haskell or O'Caml), but I was wrong. C++ 11 is quite a different critter. So... with genuine regret (I've spent months on this!) I'm going to table the idea. I liked the concept of a pure C++ version of DMC, but what I can see now is that just as I settled on this, C++ 11 came along and what I think of as native C++ has become deprecated. Whether they know it or not, the C++ community itself is suddenly in disarray.

In fact, C++ 11 is not going to be as popular as C++, in my personal opinion. What has happened here is quite fascinating in a sociological sense: the C++ people have tried to turn C++ into C# or Java (both of which have followed a similar evolution path), adding lambda expressions and reflection and threads and managed memory, but being firmly contrarian, also wanted all type information to be resolved at compile time (constexpr...) and all runtime behavior to be "pure code". They ended up with two languages that co-exist: a complete programming language that runs at compile time (all the constexpr and type-traits and recursive variadic template stuff), kind of like Haskell, and then a second language that runs at run time, with threads and these reference-counted shared pointers and so forth, and all sorts of new operators to express lvalue and rvalue distinctions so that the code will compile correctly (never heard of lvalue/rvalue distinctions as defined by the C++ 11 language designers? Then don't try to use C++ 11!)

Beyond the lvalue/rvalue impact on syntax (which is huge and not easy to get used to), there are deeper issues. For example to do reflection-based callbacks, what I turn out to have to do is to capture the signature information for your methods at compile time, encode it into a constexpr class of my own (call it Type, if you like....) and then I can pass a std::vector of these Type objects into DMC, and reflect using it. To actually do a callback I would build the call stack frame and than invoke your method. In C# this is tricky too -- I do it with something called "dynamic" -- but it isn't remotely as mind-numbingly complex. And by the time I do all this, DMC wouldn't be faster than Isis2 is now.

In fact C++ 11 won't be such a fast language at the end of the day. These new features make compilation harder, and code more complex to write and debug, and more code to run means slower code.

I teach object oriented programming to undergrads and I find that Java and C# are actually fun languages to teach, but late in the semester I generally feel that I should show my students some inline lambdas and some code that uses LINQ or JQUERY. But the students have trouble at that moment, and as an instructor I don't blame them. Those features can really feel strange at first: purely functional stuff dropped into an imperative language. So here with C++ 11, we see something similar, but rather than being runtime mechanisms, the functional side is compile time, while the runtime side only can access what was captured at compile time. I think C++ 11 would be VERY hard to teach, now that I've actually learned it myself (the main accomplishment of my six months of struggling with it).

Hopefully, nobody was holding their breath for this.

Keep in mind that in fact Isis2 (or maybe I'll rename it DMC anyhow...) can still be used with all sorts of applications, including from C++/CLI or C/CLI. It can also call into code (libraries) coded in standard C++ or standard C -- this is how we did the OOB layer, which uses RDMA verbs and needed to be in C. Further, with Microsoft now pushing to open source .NET and port it to small Internet of Things devices, and C++ 11 more or less "deprecating" the old pure C++, I think the world is simply evolving towards platforms like .NET and Java, and C++ may be a language of the past. Feel free to debate this point if you disagree...

Let me know if you think I have all this totally wrong!