AddTaskAsync throw the message "A task was canceled."

Category: azure batch


Art Bkk on Mon, 07 Aug 2017 09:25:36

When call the method AddTaskAsync it throw the message "A task was canceled."

Here is the code  I used to add task.

await batchClient.JobOperations.AddTaskAsync(jobId, tasks);

I was using this code before and it works fine. Any idea what cause the problem or any workaround on this?


DarylMsft on Wed, 09 Aug 2017 23:24:40

An unfortunate behavior of lower level assemblies is that timeouts are surfaced via this cryptic mechanism.

The first step is to mine the BatchException.  An explainer is found in this forum post here.

Look in the exception to see if the actual root cause is disclosed (chances are it is a timeout but the BatchException should disclose something authoritative).

I see that you are using the override that adds tasks in bulk.  The best practices for this pattern are discussed here.

Given that the publics referenced in that forum post are out of date (but not the issues discussed), I will add some thoughts below.

Timeouts when making simultaneous calls with the BatchClient are most often caused by the default value of ServicePointManager.DefaultConnectionLimit.  Be sure to read the forum post on adding tasks in bulk mentioned above; it discusses this property.  The default is 2 and this causes client side timeouts during large bulk task add operations.  Increasing the value is almost always required.  We cannot recommend a value because that choice is a function of hardware, load, networking, etc.. but for testing purposes values of 20 to 100 are fine.

The throughput of a bulk task add can be greatly increased by changing the value of the BatchClientParallelOptions.MaxDegreeOfParallelism property.  Low values are typically fine (5 or so). Tune as appropriate to the issue mentioned above for DefaultConnectionLimit.

Finally, with bulk task add operations, there is an additional complexity around handling errors that occurred for any individual task in a bulk task add operation.  This is controlled by AddTaskCollectionResultHandler.  You may need to write your own in order to diagnose a particularly difficult bulk add operation.