[Tutorial] Anything to/from a hex string

[MrD.]

This is a re-post of an article I originally posted on my website.

Introduction
I recently needed to be able to convert any instance of an object in C++ to a file so it could be serialised, and then restored later; I did most of this by writing out each member variable of the object individually at the most basic level. This works fine and is easy enough to implement; but then came the time to do the "generic" version, the version that could write out any object (such as a struct) as a whole.

The I/O classes can write out/read in using either binary or text, the binary version of the generic writer is simply a case of writing out the length of the data, and then each byte of the data. The text version however required me to convert the bytes of data making up the object into something that was sensible as text, I decided to do this by converting each byte into a hex value and adding it to a string. As I was working on this I found that the information about how to do this without using some of the old C-style string manipulation functions is quite sparse on the net, so after much searching, trying, and failing, I now have a solution that works both ways using good old C++ string streams.

I should probably point out that I don't think this solution is endian neutral, so you would likely want to add a method of endian detection and switching to ensure consistency across different architectures.

The Code
When a byte is encoded to hex string, each byte becomes two characters in the range 0 to F. For example, the value “10” would be written out as “0A”.

The code works by using a std:tringstream with hex mode set (std::hex), along with using zero's for padding (std:etfill('0')) and a fixed width of two for each byte added as hex (std:etw(2)).

The toHex function works by adding each byte of the passed data into the string stream as an integer with a width of two; because zero's have been set as padding, any numbers which are added to the stream that are only one character long will be automatically prefixed with a zero resulting in the correct two character hex value being added to the string stream.

The fromHex function works by reading out two characters from the hex string into a string stream, these characters are then converted back into the byte value when read out from the string stream into an integer. Finally, the value of the integer is stored as the value of the byte, and the string is moved on to the next set of two characters.

Code:

// ------------------------------------------------------------------
/*!
    Required headers
*/
#include <string>
#include <sstream>
#include <iomanip>


// ------------------------------------------------------------------
/*!
    Convert a block of data to a hex string
*/
void toHex(
    void *const data,                 //!< Data to convert
    const size_t dataLength,         //!< Length of the data to convert
    std::string &dest                 //!< Destination string
    )
{
    size_t                 index         = 0;
    unsigned char         *byteData     = reinterpret_cast<unsigned char*>(data);
    std::stringstream     hexStringStream;
    hexStringStream << std::hex << std::setfill('0');
    for(; index < dataLength; ++index)
        hexStringStream << std::setw(2) << static_cast<int>(byteData[index]);
    dest = hexStringStream.str();
}


// ------------------------------------------------------------------
/*!
    Convert a hex string to a block of data
*/
void fromHex(
    const std::string &in,             //!< Input hex string
    void *const data                 //!< Data store
    )
{
    size_t             strIndex     = 0;
    size_t             dataIndex     = 0;
    size_t             length         = in.length();
    int             tmpValue     = 0;
    unsigned char     *byteData     = reinterpret_cast<unsigned char*>(data);
    std::string     tmpStr         = "";
    std::stringstream hexStringStream; hexStringStream >> std::hex;
    for(; strIndex < length; strIndex += 2, ++dataIndex)
    {
        // Read out and convert the string two characters at a time
        tmpStr =  in[strIndex];
        tmpStr += in[strIndex + 1];
        
        // Reset and fill the string stream
        hexStringStream.clear();
        hexStringStream.str(tmpStr);
        
        // Do the conversion
        hexStringStream >> tmpValue;
        byteData[dataIndex] = tmpValue;
    }
}

Testing
The code below can be used to test the functions. The expected output would be:

SGenericData::anInt0: 10
SGenericData::anInt1: 20
SGenericData::aFloat: 30.1234
Hex string from data: 0a00000014000000b9fcf041
SGenericData::anInt0: 10
SGenericData::anInt1: 20
SGenericData::aFloat: 30.1234
Data restored

Code:

#include <iostream>

struct SGenericData
{
    int     anInt0;
    int     anInt1;
    float     aFloat;
    
    void write()
    {
        std::cout << "SGenericData::anInt0: " << anInt0 << std::endl;
        std::cout << "SGenericData::anInt1: " << anInt1 << std::endl;
        std::cout << "SGenericData::aFloat: " << aFloat << std::endl;
    }
};

int main()
{
    std::string hexString;
    
    SGenericData data;
    data.anInt0 = 10;
    data.anInt1 = 20;
    data.aFloat = 30.1234f;
    
    // Convert "data" to a hex string
    toHex(&data, sizeof(SGenericData), hexString);
    
    // Output
    data.write();
    std::cout << "Hex string from data: " << hexString << "\n" << std::endl;
    
    // Reset "data"
    data.anInt0 = 0;
    data.anInt1 = 0;
    data.aFloat = 0.0f;
    
    // Convert the hex string back to "data"
    fromHex(hexString, &data);
    
    // Output
    data.write();
    std::cout << "Data restored\n" << std::endl;
    
    system("pause");
    return 0;
}

[Gaijin]

And now the reply on the last on my list......
AWSOME......

This wasn't really what I was looking for, but it will point me into the right direction.. I hope...
Thank you for sharing this!

[g4143]

I just quickly scanned your code...does it provide a method/way to handle pointers values?

[MrD.]

I just quickly scanned your code...does it provide a method/way to handle pointers values?

No, it just blindly works with the data it is provided. If the data you provide it contains a pointer to something than the function will just blindly write out the pointer as the memory address it is pointing at.

This is just designed to be used with simple structs, and quite frankly shouldn't be used for anything else.

If you want to serialise/parse a class or anything more complicated (like something with member functions) that I would strongly recommend adding serialise/parse functions to your classes that write out data member variable by member variable since it avoids all sorts of issues you can get with this method such as different compilers packing classes in different ways. This method simply exists for the writeGeneric, and readGeneric functions in my data I/O class (a function which I recommend avoiding, and only really use for writing out GUIDs).