Distributed Information System (DIS)
  • Home
  • The blog
  • Contact

Bleams explained...

6/12/2007

0 Comments

 

IDR is the data encoding used with DITP. It is to DITP what XML is to SOAP. IDR uses bleams that combines the benefits of block and stream encoding:

   - no need to specify total size in front of it
   - no upper size limit
   - may be encapsulated without depth limit
   - no constrain on data and doesn't rely on markers or tags
   - no need to parse and search payload data to locate the end of bleam

A bleam is encoded as a sequence of byte blocks of at most 16KB. Each block is preceded by an unsigned short value whose 14 less significant bits encodes the number of bytes of data that follow. Its most significant bit is set to one if the block is not first, and its second most significant bit is set to one if the block is not last.

When encapsulating a bleam, its sequence of blocks is simply inserted in the sequence of blocks of the encapsulating bleam. The encapsulated bleam can be stored in the data of the the encapsulating bleam block if it fully fits in one of its block.

The maximum payload size is 16382 so that the biggest block will be 2^14 byte long. The invalid size value 16383 (0x3FFF) is then used as a signal. A signal block has no payload data.

The signal is used in IDR to inform the receiver that the expected sequence of data is interrupted because an exception or an error occured. If the signal block is flagged as end of bleam, the interruption is anonymous. Otherwise subsequent bleam data provides information on the reason of the interruption. In IDR it is the serialized exception object and the objects it may be referencing. Such interruption will eventually propagate to encloding bleams and be encoded as anonymous interruptions.

Encoding and decoding bleams require some care, but the effort is worth it because of its multiple benefits. With small messages DITP is as performant as common inter-object communication protocols. With bigger message, DITP benefits from the stream oriented encoding and reduced latency and memory usage requirements. DITP can thus be used to send huge files, streamed films, etc.

0 Comments

More on stream versus block oriented protocol...

6/12/2007

0 Comments

 

Most inter-object communication protocols encode message into blocks with its size encoded in front. This requires that the message is fully encoded to compute the size of blocks before it is sent.

A stream oriented protocol doesn't need this. The communication process can be pipelined. As shown in the figure below, this reduces the communication latency. Note that in a two way transaction the saving is doubled.

In absence of the block size, one needs a marker to signal the end of the block. For instance SOAP uses xml tags as markers. There are two drawbacks with such encoding: the marker can't show up in regular data and the marker must be searched to locate the end of a block. When the block size is available, locating the end of the block is trivial and very fast.

IDR and DITP combines the benefits of blocks and streams encoding by introducing a new encoding called BLEAM. Here is a short list of the their properties:

   - bleams have no size limits
   - bleams may be encapsulated with unlimited depth
   - bleams impose no constrain on contained data
   - no need to search and parse contained data to locate the end of bleam
 
The last property is what makes the difference with SOAP and its xml encoding.
See next blog note for a description of the bleam encoding.

0 Comments

Why a Garbage Collector ?

5/15/2007

0 Comments

 

With most modern programming languages there is no such question because garbage collector is built-in. This is not the case with C++, and since I develop the first prototype in this language, I had to anwser it. Do I really need a garbage collector ?

IDR needs a garbage collector because it supports object aggregate encoding and IDR should impose minimal constrains on them. Cycles allowed, minimal difference with local objects, user implemented classes, etc.

Another reason result from the reliance on exception handling. This is the price to pay for using the streaming encoding model. If an exception is generated on the encoder side, it has to be propagated to the decoder. And manual memory management with exceptions can become tricky.

This is why I went into the effort of adding garbage collector support to C++. The good news is that it is planned to be added in the next version of the C++ standard. So the effort to implement IDR in C++ with a temporary solution is not a waste of time.

0 Comments

IDR Status

5/14/2007

0 Comments

 

As beeing the most fundamental component of DIS, IDR was designed and implemented first.

IDR is fully functional and passed all the tests I did. It still need throughout testings for qualifying for production readyness.

IDR supports bleam, data and object aggregate encoding, as well as exception handling with exception encoding. Bleam encapsulation required special care to achieve this.

0 Comments
Forward>>

    Author

    Christophe Meessen is a  computer science engineer working in France.

    Any suggestions to make DIS more useful ? Tell me by using the contact page.

    Categories

    All
    Business Model
    Database
    Dis
    Ditp
    Dvcs
    Git
    Gob
    Idr
    Misc
    Murphys Law
    Programming Language
    Progress Status
    Startup
    Suggested Reading
    Web Site

    Archives

    December 2017
    November 2015
    September 2015
    February 2013
    December 2012
    November 2012
    May 2012
    February 2012
    March 2010
    October 2009
    September 2009
    July 2009
    June 2009
    May 2009
    February 2009
    January 2009
    November 2008
    September 2008
    August 2008
    July 2008
    May 2008
    April 2008
    March 2008
    February 2008
    January 2008
    December 2007
    October 2007
    August 2007
    July 2007
    June 2007
    May 2007

    RSS Feed

    Live traffic feed
    You have no departures or arrivals yet. Wait a few minutes and check again.
    Powered by FEEDJIT
Powered by Create your own unique website with customizable templates.