vtsql on Fri, 14 Dec 2018 11:44:10
As part of my ADF pipeline, I need to use custom activity to execute PowerShell script to delete the staging tables for the Azure Table storage
The scripts work fine when executing from the activity. However, before proceding with further steps I need to ensure the stage tables has been deleted or not. As per the document (https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-dotnet-custom-activity) I can query the stdout file to check the status of cutom activity.
In my case the file is empty.
I tried to send the output the outputs.json by adding to the PS script
but the outputs.json not getting created and neither the stdout getting updated
ChiragMishra-MSFT on Mon, 17 Dec 2018 12:14:15
The stdout and stderr of your custom application are saved to the adfjobs container in the Azure Storage Linked Service you defined when creating Azure Batch Linked Service with a GUID of the task.
For more details, please go through this :
Let me know if it helps or we can gladly continue to probe in further.
Johns-305 on Wed, 19 Dec 2018 14:12:53
Maybe use the REST API instead: Table Service REST API
If the delete operation returns success, the table will be deleted shortly. If you need to wait until it's non-existent, to create new version for instance, you can poll after ~1 min to verify.
Either way, Custom Activity/PowerShell seems a bit circuitous compared to the REST API to accomplish the same thing.