How does the messagepack encoding/decoding compare to msgpack-cpp in terms of performance?

[sztomi]

[nlohmann]

I have no idea. Is there a benchmark suite?

[sztomi]

They used to have benchmark suite written in Perl, but apparently it's gone now. This is their main repo: https://github.com/msgpack/msgpack-c

[nlohmann]

I have little knowledge about that library. Would this be a fair benchmark example:

#include <msgpack.hpp>
#include <fstream>
#include <iostream>
#include <sstream>

int main()
{
    for (auto filename : {"sample.json.msgpack", "all_unicode.json.msgpack"})
    {
        std::ifstream f_msgpack(filename, std::ios::binary);
        std::vector<char> packed((std::istreambuf_iterator<char>(f_msgpack)),
                                    std::istreambuf_iterator<char>());

        auto x = msgpack::unpack(packed.data(), packed.size());
        std::stringstream ss;
        ss << x.get();
        std::cout << ss.str().size() << std::endl;
    }
}

vs.

#include <json.hpp>
#include <fstream>
#include <sstream>

using json = nlohmann::json;

int main()
{
    for (auto filename : {"sample.json.msgpack", "all_unicode.json.msgpack"})
    {
        std::ifstream f_msgpack(filename, std::ios::binary);
        std::vector<uint8_t> packed((std::istreambuf_iterator<char>(f_msgpack)),
                                    std::istreambuf_iterator<char>());
        std::stringstream ss;
        ss << json::from_msgpack(packed);
        std::cout << ss.str().size() << std::endl;
    }
}

input files

[sztomi]

Probably, but without the I/O (I mean that only the deserialization should be timed). I/O is unpredictable and will skew the results. Are you using I/O to make sure that the results are not optimized away?

[nlohmann]

Of course, the time reading the files into memory should not be measured. I was rather asking about the semantics, because I was searching for a size() equivalent in msgpack-c to get the number of elements in an object/array.

[sztomi]

Ah, OK. You can use x.via.array.size, but this might not actually make msgpack-c deserialize the object. In fact, I think it's best to deserialize into a vector or something in both cases.

[nlohmann]

This whole benchmark heavily depends on what you want to do... In my case, I create a JSON value from the MessagePack input, so I would like to compare against something similar.

[sztomi]

You are right, that's a valid comparison. However, why not benchmark both? I think both depicts an interesting facet of the performance.

[nlohmann]

If you can provide examples with a description for msgpack-c, I can try to add examples for my library.

[sztomi]

Sure, I will put together some examples of my use cases and add them as a comment here this weekend.

[nlohmann]

That's great! There are also some large JSON files in the benchmarks folder that could be used.

[nlohmann]

Any news on this @sztomi ?

[sztomi]

Sorry @nlohmann I was on a vacation and forgot to update this thread. Will get back to it soon, thanks for your patience!