Niall’s virtual diary archives – Saturday 27th August 2016

Saturday 27th August 2016: 12.49pm. You've not seen much from me these past six weeks here, this is because I've been spending every spare hour preparing the material for CppCon 2016 which is just three weeks away now. As a quick summary of my outside-of-work programming time since the ACCU 2016 conference last April:

* May-June: Writing a next generation code testing engine for C++ which is how code testing ought to always have be done. This thing's design is capable of formal proof of C++ code [1], though a very long gap exists between (Boost.) KernelTest's current facilities at present and that final goal. Even still, it's already amazing because it lets you functionally specify an exact behaviour for every possible call of an API in a very terse succinct form because the testing engine permutes calls of every tested API with many combinations of parameters (but unlike automated fuzzing, the programmer hand specifies the preconditions, permutations and postconditions to be met). You will almost certainly be seeing KernelTest at a 2017 C++ conference.

[1]: To clarify, by "formal proof of C++ code", I mean only to the extent that C has commercially available formal verification tooling (which is a subset of that possible for other programming languages). I believe that the design of KernelTest could, with a lot more extra development, make possible a similar level of formal proof tooling for C++ if and only if the code is written in a provable subset of the C++ language. Hopefully this will clarify any confusion people may have had about my claim.

* July-August: Writing next generation cmake tooling for a collection of C++ standards aspiring libraries, a collection not encumbered by the many legacy problems in the Boost collection of C++ libraries. This next gen cmake tooling is amazingly powerful and really shows off what cmake is capable of, and it already provides build, testing, dependency management, prebuilt binaries, static analysis, code sanitisation, CDash dashboarding, C++ Modularisation and precompilation and a long list of other cool stuff for AFIO v2, Outcome, KernelTest and of course Boost-lite, my minimal Boost C++ libraries emulation layer. This tooling also uses a functional programming style made possible by cmake 3.0's improvements, so instead of saying how your code should be built, you simply say things like "give me a header only library from the code in this directory" and the tooling automatically gives you a C++ Module or a precompiled header depending on what is consuming the library. This stuff is amazingly cool, but it's still very immature and it's an open question if I'll be allowed to present it at a 2017 C++ conference as the program committees may not take to it.

* Last two weeks: Taking advantage of Megan and Clara being in the States for their annual holiday there, I've invested about ninety hours outside of work (I took two days off work) in writing the prerequisites for the code to be presented at CppCon and the code itself.


CppCon 2016 will be my last "big push" conference where I work a never ending sequence of sixty hour weeks between January and September every year to get exciting new software libraries written - I've been doing that in one form or another since 2012, and as you can see from the above I've now got enough exciting new software to present at conferences until the end of 2017, so I'll be taking it easier from this September onwards.

In 2018 I expect to finally deliver the end goal of all my outside-of-work programming which is the embedded transactional ACID (Atomic Consistent Isolated Durable) key-value store that the ISO C and C++ standards - and thenceforth to all programming languages - so sorely need in my opinion. One of the ongoing major causes of software unreliability, user data loss and bad programming is how amazingly hard it is to write formally correct data persisting code even today in this modern world. And because it's so hard to do, programmers don't do it, leading to an never ending endemic quality problem across any computer programming which stores data outside a database.

Unless I have had no idea what I've been doing these past years, all the major foundations are now in place to implement a very low level reliable way of storing data in AFIO v2, indeed I could whip up a low quality toy ACID key-value store implementation in less than a week with what I've got already. Late last night after those two weeks of work the final major foundational piece came to life, the bit I am presenting at CppCon 2016 which is a file system algorithm for implementing mutual exclusion based on memory maps. This lets multiple programs concurrently use the same store of data without tripping over one another. Unlike the algorithms implementing the same which I presented at last spring's ACCU conference, this one is very considerably faster, indeed a full order of magnitude so and it would make any forthcoming embedded key store quite performance comparable with the best of the NoSQL databases out there (even though I am not developing a full database, it would be nice if someone could extend this stuff easily to implement a competitive NoSQL database).

It could even be possible I'm going to finish the new software's unit testing a bit earlier than expected, so I'm currently thinking of what R&R I could do before Megan and Clara return on Tuesday. In particular, I'm thinking why not play some computer games, something I haven't done since Clara was born. I doubt I'd finish them before they get back, but hell, why not? I think I've earned some R&R time!