Distributed Information System (DIS)
  • Home
  • The blog
  • Contact

IDR encoding compared to Go language encoding

5/12/2012

0 Comments

 
As the author of the IDR encoding (yet unpublished), I was very curious do see how it compares to the data encoding proposed in the Go language designed by the Google team (gobs of data).

There are two fundamental difference between the two.

Value encoding

Gobs encodes value by a tag byte followed by a compact byte encoding of the value. The tag identifies the type of the value and its encoded byte length. The byte encoding drops trailing 0 bytes of the value.

IDR uses the most common computer internal representation of data as encoding and has thus no marshaling work.

Advantages

Gobs has two major benefits. The first benefit is that the type of data is provided with the value which allows anyone to decode the values of a message without prior knowledge of its content. The second benefit of such encoding is that data can be split in blocs anywhere since decoding is processed byte after byte.

IDR has the advantage of fast and trivial marshaling as in RPC and IIOP.

Disadventages

The price to pay with Gobs is the additional tag byte and the marshaling work. With IDR, it is the code complexity to ensure the atomicity of the base values if a data stream needs to be split and the absence of base value type information with the data.

Type encoding

Gobs provides the maximum type information with the message so that it is self describing. This makes the encoding more complex since conciseness competes with expressiveness.

RPC, IIOP and ICE rely on the context to determine the type of encoded data. The encoding targeting mainly use in communication, this optimization make sense to some extend.

IDR precedes any message with a type reference. The type reference is a key to a distributed database similar to the DNS from which a description of the data contained in the message may be obtained. It is possible to obtain a concise form to efficiently parse the data by a program or a detailed expressive form with comments to be used by humans.

The IDR data type description strategy seems the most efficient because the data type description is written once. But the decoupling of the type description from the data expose to the risk of loosing access to the data description if it gets deleted.

Conclusion

There are some good and bad points on both sides and there is no easy way to merge the good points into a new optimal encoding.

My experience is that the IDR encoding, while simple and efficient on some aspects, was quite complex to develop.

Today I still favor IDR's choice because of the marshaling efficiency. Olivier Pisano managed to translate the C++ IDR library to the D language in a very short time. So maybe it is just the conception and validation of IDR that took so much time.

I like very much the smart encoding of the base values in Go, but not so much to force all floating point values to be encoded into a double precision float (64bit). I hope they'll change that.

There are other differences between IDR and Gob which have not been detailed here. What they have in common is that both may use their encoding to support persistence. IDR may use it with its distributed database.



0 Comments



Leave a Reply.

    Author

    Christophe Meessen is a  computer science engineer working in France.

    Any suggestions to make DIS more useful ? Tell me by using the contact page.

    Categories

    All
    Business Model
    Database
    Dis
    Ditp
    Dvcs
    Git
    Gob
    Idr
    Misc
    Murphys Law
    Programming Language
    Progress Status
    Startup
    Suggested Reading
    Web Site

    Archives

    December 2017
    November 2015
    September 2015
    February 2013
    December 2012
    November 2012
    May 2012
    February 2012
    March 2010
    October 2009
    September 2009
    July 2009
    June 2009
    May 2009
    February 2009
    January 2009
    November 2008
    September 2008
    August 2008
    July 2008
    May 2008
    April 2008
    March 2008
    February 2008
    January 2008
    December 2007
    October 2007
    August 2007
    July 2007
    June 2007
    May 2007

    RSS Feed

    Live traffic feed
    You have no departures or arrivals yet. Wait a few minutes and check again.
    Powered by FEEDJIT
Powered by Create your own unique website with customizable templates.