Question

SanjayJadam on Fri, 04 Jan 2013 11:22:43


I want to read data from my TCP Server and generate CEP insert event from data.

My TCP server sends data in below format:

DATA,NAME,VALUE;DATA,NAME,VALUE;DATA,NAME,VALUE;DATA,NAME,VALUE ...so on.

Where , is field separator and ; is row separator.

I want to generate CEP insert event for every row i.e.,

DATA,NAME,VALUE

My Implementation is: I have created TCP input adapter which connect to TCP server and read data by this way:

Stream stream = tcpClient.GetStream();

byte[] byte = new byte[10000];

Stream.Read(byte,0,10000);

then add all the stream data into stringbuilder variable.

After reading stream I split data with , and try to generate CEP events.

The problem is TCP server send data very fast and my TCP input adapter process little late.

Please suggest me what is the good way to generate CEP Insert Event for the same?

Regards,

Sanjay Jadam


Sponsored



Replies

DevBiker on Fri, 04 Jan 2013 15:52:50


Have you done any performance analysis on your implementation? I think that the issue is with the way you are handling the strings and the streams. First, reading a 10000 byte buffer is a little more than you need to process at once (I would think). Second, reading this into a StringBuilder is good ... but splitting on the semi-colon and then the comma is going to be pretty expensive. While it's a good deal more code, it might be better if you looked at using something like a string reader and reading a character at a time until you come to the commas and semicolons. Also, you are reading synchronously ... you may get better throughput asynchronously.

These are just some guesses. And the issue isn't StreamInsight, per se ... it's going to be in your implementation that reads the TCP stream. You do a couple of different variations on the implemenation and test them using a performance profiler to find where the time is being taken and where you can better tune.