Use self dependency TWT to enforce max concurrency

Category: azure data factory


jader3rd on Thu, 28 May 2020 17:19:37

When configuring a Tumbling Window Trigger I know I can set the max concurrency. But max concurrency doesn't apply to trigger reruns, and I want it to. I'm thinking that I can use a self dependency to do this, but I want to make sure that I get it right. If I want a max of three tumbling windows running at a time for an hourly job do I set the Offset to -03:00:00, and a Window Size of 01:00:00?

Should this make it so that if I log in in the morning, see 16 failed tiggered runs, that I can click rerun on all 16 failed triggers, but only three will run at a time?


MartinJaffer-MSFT on Wed, 03 Jun 2020 21:35:43

Hello jader3rd and thank you for your question.

If I understand your situation correctly, your hypothesis makes sense.

I will reach out internally for more information.

jader3rd on Wed, 03 Jun 2020 22:06:53

I did run into a case yesterday where I needed this and thankfully it turns out that it worked as expected.

MartinJaffer-MSFT on Thu, 04 Jun 2020 20:52:19

Thank you for letting us know, and sharing with the community.