CTIs during adapter replay / resilient query recovery period

Category: sql server streaminsight


Nemoren on Thu, 05 Sep 2013 17:50:20

I have this CTI problem when I'm recovering my query after successful checkpoint.

I have my RecoveryManager remembering the Time the event should be inserted, so when I'm recovering, its Enquing those events at that time.

The problem is that I also have my AdvanceTimeSettings also automatically generating CTIs after a small grace period after each event is enqueued. Which works just fine during normal operation, but seems to break things when I'm replaying my events from the High Water Mark.

I have an input adapter similar to:

app.DefineObservable((DateTimeOffset? hwm) => 
Observable.Using(() => new RecoveryManager<Payload>(), 
      rm => rm.RecoverFrom(hwm)
  .ToPointStreamable(evt => evt.IsCti
                        ? PointEvent<T>.CreateCti(new DateTimeOffset(evt.UtcDateTimeOffsetTicks, TimeSpan.FromHours(0)))
                        : PointEvent<T>.CreateInsert(new DateTimeOffset(evt.UtcDateTimeOffsetTicks, TimeSpan.FromHours(0)), evt.Record),
                        new AdvanceTimeSettings(
                        new AdvanceTimeGenerationSettings(TimeSpan.FromMilliseconds(15), TimeSpan.FromMilliseconds(5)),


In this situation, is the AdvanceTimeSettings creating CTI's based off of "Current REAL Time", or as an offset from the Inserts I'm making, and using the CreateInsert's DateTimeOffset as the basis for the CTI's DateTimeOffset?

I'm feeling like I need to somehow shut off the CTI generation while I'm replaying my recovery events. Is this crazy?

Short story, I'm running into CTI violations during recovery.



DevBiker on Fri, 06 Sep 2013 12:37:01

The problem is that you are creating CTIs. You should do one or the other - either create CTIs manually or have them generated. Not both. Generated CTIs are always based on the timestamps from the events that are being enqueued.