Porting mcidle to C++ with variadic templates

I’ve begun porting mcidle to C++ with Boost. And part of my work is getting packets to serialize in nice ways in a space and performant efficient manner.

I was surprised how much I could accomplish with templates. For instance, here is how we define a packet that just contains VarInt members and get a buf object containing all of the data members written to

typedef network::packet::packet_t<VarInt, VarInt> keep_alive_t;
keep_alive_t KeepAlive((VarInt)3230055, (VarInt)5);

// write to a byte_buf with capacity 1024 bytes
byte_buf<1024> buf;
network::packet::write_packet_to_buf(buf, KeepAlive);
// after writing, buf.to_hex() == "E7 92 C5 01 05"

byte_buf is very similar to BytesIO in Python–something I also ported.

So the question is..how did I do all this? The answer is boost::tuple and template overloads. Below is the code for writing to a packet buffer which is simply a template that is evaluated at compile-time to generate a bunch of functions on each tuple’s argument. Of course this comes at the cost of taking up space in the outputted binary, but that’s a price I’m willing to pay for the added efficiency of not having to write a bunch of C style functions for constructing packets from their fields.

The only thing special about this method is the fact that we recursively call types::write on each member, another templated overload.

template<class ...Ts>
using packet_t = boost::tuple<Ts...>;

template <class ...Ts, uint32_t N>
inline byte_buf<N>& write_packet_to_buf(byte_buf<N>& buf, boost::tuple<Ts...>& pkt) {
    return write_packet_to_buf_<0>(buf, pkt);
}

template <size_t k = 0, class ...Ts, uint32_t N>
inline typename std::enable_if<k < boost::tuples::length<packet_t<Ts...>>::value, byte_buf<N>&>::type
    write_packet_to_buf_(byte_buf<N>& buf, packet_t<Ts...> & pkt) {
    auto& val = boost::get<k>(pkt);
    types::write(buf, val);
    return write_packet_to_buf_<k + 1>(buf, pkt);
}

template <size_t k, class ...Ts, uint32_t N>
inline typename std::enable_if<k == boost::tuples::length<packet_t<Ts...>>::value, byte_buf<N>&>::type
    write_packet_to_buf_(byte_buf<N>& buf, boost::tuple<Ts...>& pkt) {
    return buf;
}

The types stuff is really nice to work with too since it’s just overloads on the type. Here’s how we read/write the VarInt data type.

template <uint32_t N>
void write(byte_buf<N>& buf, VarInt val) {
    do {
        byte temp = val & 0b01111111;
        val = static_cast<uint32_t>(val) >> 7;
        if (val != 0) {
            temp |= 0b10000000;
        }
        buf.write(temp);
    } while (val != 0);
}

template <uint32_t N>
VarInt read_varint(byte_buf<N>& buf) {
    int numRead = 0;
    VarInt result = 0;
    byte read;
    do {
        read = buf.read();
        byte value = read & 0b01111111;
        result |= value << (7 * numRead);

        numRead++;
        if (numRead > 5) {
            throw exceptions::bad_varint_read();
        }
    } while ((read & 0b10000000) != 0);

    return result;
}

When I compare this way of doing things it’s ultimately a lot simpler than having an inheritance-based hierarchy on types and then a definitions dictionary which is iterated over.

The port is still in process so I’ll keep you all posted.

Back