I’ve been exploring C++ reflection a bit, using the Bloomberg fork of clang. I’ve yet to get my head fully around the syntax and the implications, but I have an obvious use case: more serialization functionality in libDwm.
In order to play around in ‘production’ code, I needed some feature test macros that are not in the Bloomberg fork in order to conditionally include code only when C++26 features I need are present. P2996 tentatively proposed __cpp_impl_reflection and __cpp_lib_reflection and hence I added those. I also need features from P3491, whose proposed test macro is __cpp_lib_define_static which I also added. Finally, I added __cpp_expansion_statements (feature test macro for P1306).
Technically, __cpp_lib_reflection and __cpp_lib_define_static should be in <meta>, but I added them to the compiler built-ins just because it’s convenient for now.
I’ve run into some minor gotchas when implementing some generic reflection for types not already covered by serialization facilities in libDwm.
As an example… what to do about deserializing instances of structs and classes with const data members. Obviously the const members can’t be cleanly written to without going through a constructor. I haven’t given much thought to it yet, but at first glance it’s a bit of a sore spot.
Another is what to do about members with types such as std::mutex, whose presence inside a class would generally imply mutual exclusion logic within the class. That logic can’t be ascertained by reflection. Do I need to lock the discovered mutex during serialization? During deserialization too? Do I serialize the mutex as a boolean so that deserialization can lock or unlock it, or skip it?
For now, since we have P3394 in the Bloomberg clang fork, I’ve decided that using annotations that allow members of a structure or class to be skipped is a good idea. So in my libDwm experiment, I now have a Dwm::skip_io annotation that will cause Read() and Write() functions to skip over data that is marked with this annotation. For example:
#include <sstream>
#include "DwmStreamIO.hh"
struct Foo {
[[=Dwm::skip_io]] int i;
std::string s;
};
int main(int arc, char *argv[]) {
Foo foo1 { 42, "hello" };
Foo foo2 { 99, "goodbye" };
std::stringstream ss;
if (Dwm::StreamIO::Write(ss, foo1)) {
if (Dwm::StreamIO::Read(ss, foo2)) {
std::cout << foo2.i << ' ' << foo2.s << '\n';
}
}
return 0;
}Would produce:
99 helloThe i member of Foo is neither written nor read, while the s member is written and read. Hence we see that foo2.i remains unchanged after Read(), while foo2.s is changed.
Chasing pointers
A significant issue with trying to use reflection for generic serialization/deserialization: what to do with pointers (raw, std::unique_ptr, std::shared_ptr, et. al.). One of the big issues here is rooted in the fact that for my own use, reflection for serialization/deserialization is desired for structures that come from the operating system environment (POSIX, etc.). Those structures were created for C APIs, not C++ APIs, and pointers within them are always raw. Generically, there’s no way to know what they point to. A single element on the heap? An array on the heap? A static array? In other words, I don’t know if the pointer points to one object or an unbounded number of objects, and hence I don’t know how many objects to write, allocate or read.
Even with allocations performed via operator new[], we don’t have a good means of determining how many entries are in the array.
For now, I deny serialization / deserialization of pointers, with one experimental exception: std::unique_ptr whose deleter is std::default_delete<T> (implying a single object).
Reflection-based code is easier to read
Let’s say we have a need to check that all the types in a std::tuple satisfy a boolean predicate named IsStreamable. Before C++26 reflection, I’d wind up writing something like this:
template <typename TupleType>
consteval bool TupleIsStreamable()
{
auto l = []<typename ...ElementType>(ElementType && ...args)
{ return (IsStreamable<ElementType>() && ...); };
return std::apply(l, std::forward<TupleType>(TupleType()));
}While this is far from terrible, it’s not much fun to read (especially for a novice), and is still using relatively modern C++ features (lambda expressions with an explicit template parameter list, and consteval). Not to mention that std::apply is only applicable to tuple-like types (std::tuple, std::pair and std::array). And critically, the above requires default constructibility (note the TupleType() construct call). Because of this final requirement, I often have to resort to the old school technique (add a size_t template parameter and use recursion to check all element types) or the almost as old school technique of using make_index_sequence. Also note that real code would have a requires clause on this function template to verify that TupleType is a tuple-like type, I only left it out here for the sake of brevity.
The equivalent with C++26 reflection:
template <typename TupleType>
consteval bool TupleIsStreamable()
{
constexpr const auto tmpl_args =
define_static_array(template_arguments_of(^^TupleType));
template for (constexpr auto tmpl_arg : tmpl_args) {
if constexpr (! IsStreamable<typename[:tmpl_arg:]>()) {
return false;
}
}
return true;
}
While this is more lines of code, it’s significantly easier to reason about once you understands the basics of P2996. And without changes (other than renaming the function), this works with some other standard class templates such as std::variant. And with some minor additions, it will work for other standard templates. The example below will also handle std::vector, std::set, std::multiset, std::map, std::multimap, std::unordered_set, std::unordered_multiset, std::unordered_map and std::unordered_multimap. We only check type template parameters, and if ParamCount is non-zero, we only look at the first ParamCount template parameters:
template <typename TemplateType, size_t ParamCount = 0>
consteval bool IsStreamableStdTemplate()
{
constexpr const auto tmpl_args =
define_static_array(template_arguments_of(^^TemplateType));
size_t numParams = 0, numTypes = 0, numStreamable = 0;
template for (constexpr auto tmpl_arg : tmpl_args) {
++numParams;
if (ParamCount && (numParams > ParamCount)) {
break;
}
if (std::meta::is_type(tmpl_arg)) {
++numTypes;
if constexpr (! IsStreamable<typename[:tmpl_arg:]>()) {
break;
}
++numStreamable;
}
}
return (numTypes == numStreamable);
}
We can use this with std::vector, std::deque, std::list, std::set, std::multiset, std::unordered_set and std::unordered_multiset by using a ParamCount value of 1. We can use this with std::map, std::multimap, std::unordered_map and std::unordered_multimap by using a ParamCount value of 2. Hence the following would be valid calls to this function (at compile time), just as a list of examples:
IsStreamableStdTemplate<std::array<int,42>>()
IsStreamableStdTemplate<std::pair<std::string,int>>()
IsStreamableStdTemplate<std::set<std::string>>()
IsStreamableStdTemplate<std::vector<int>,1>()
IsStreamableStdTemplate<std::deque<int>,1>()
IsStreamableStdTemplate<std::list<int>,1>()
IsStreamableStdTemplate<std::map<std::string,int>,2>()
IsStreamableStdTemplate<std::tuple<int,std::string,bool,char>>()
IsStreamableStdTemplate<std::variant<int,bool,char,std::string>>()Early thoughts
I have some initial thoughts about what we’re getting with reflection in C++26.
I think that the decision to approve P2996 for C++26 was a good decision. Having std::meta::info be an opaque type to the user with a bunch of functions (more can be added later) to access it is a good thing. From just the simple examples here, it’s pretty easy to see how it’s going to make some code much easier to understand, even without doing anything fancy (noting that I only used a single splice in each example).
I’ve done my fair share of template metaprogramming over the years. While it’s satisfying to be able to do it right when it’s the best (or only) option, it’s generally much more work than I’d like. And more than once I’ve found the cognitive load to be much higher than I’d like. I probably can’t count the number of times that I’ve gone to modify some template metacode 3 years after writing it, only to find that I underestimated the time to make a change just due to the difficulty of comprehending the code (regardless of the quality of the comments and names). This is especially true when I haven’t recently been deep in this kind of code. It has only gotten worse on this front over time; standard C++ has never been a small, simple language. But despite many features in C++11 and beyond being very useful, the scope of the language is now so big that there are probably no programmers on the planet that know the whole language and standard library. It’s too much for any one person to hold in their head.
One other thought is a caution for what some people are expecting without yet digging into details. For example, what we’re getting is not a panacea for serialization and deserialization. Yes, it will indeed be useful for such activities. But it doesn’t magically solve concurrency issues, nor the legacy issues with pointers, unbounded arrays, arrays decaying to pointers, etc. Even our smart pointers introduce unsolved issues for serialization and deserialization. But it’s the legacy stuff we often need from our operating system that will remain largely unsolved for the foreseeable future, as well as any other place where we are forced to interact with C APIs that were not designed for security, safety, and concurrency.
I’m optimistic that what we’re getting is going to be incredibly useful. While I’ve only scratched the surface in this post, I’m already at the point in my own code changes where I wish we had all the voted-in papers in our compilers today. The increased number of features I’m going to be able to cleanly add to my libraries is significant. The refactoring I’m going to be able to do is also significant (already underway with conditional compilation via feature macros I can later remove). I’m anxiously awaiting official compiler support!
