Best gaming deals on Amazon | This weeks Xbox deals with gold

Cutom Activity

Category: azure data factory

Question

vtsql on Fri, 14 Dec 2018 11:44:10


Hi,

As part of my ADF pipeline, I need to use custom activity to execute PowerShell script to delete the staging tables for the Azure Table storage

$StorageAccountName = "sdfsdf"
$StorageAccountKey = "sdfsdfsdfsdf/sdfsdfsdfsdf=="
$ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
$aztables=Get-AzureStorageTable -Context $ctx
foreach($table in $aztables)
{
Remove-AzureStorageTable -table $table.Name -Context $ctx -Verbose -force
}

The scripts work fine when executing from the activity. However, before proceding with further steps I need to ensure the stage tables has been deleted or not. As per the document (https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-dotnet-custom-activity) I can query the stdout file to check the status of cutom activity.

In my case the file is empty.

I tried to send the output the outputs.json by adding to the PS script

'{ "stageTablesremoved": "1"}'|Out-File -FilePath "outputs.json"

but the outputs.json not getting created and neither the stdout getting updated

Please help

 

Replies

ChiragMishra-MSFT on Mon, 17 Dec 2018 12:14:15


Hi,

The stdout and stderr of your custom application are saved to the adfjobs container in the Azure Storage Linked Service you defined when creating Azure Batch Linked Service with a GUID of the task. 

For more details, please go through this :

https://docs.microsoft.com/en-us/azure/data-factory/transform-data-using-dotnet-custom-activity#retrieve-execution-outputs

Let me know if it helps or we can gladly continue to probe in further.

Johns-305 on Wed, 19 Dec 2018 14:12:53


Maybe use the REST API instead: Table Service REST API

If the delete operation returns success, the table will be deleted shortly.  If you need to wait until it's non-existent, to create new version for instance, you can poll after ~1 min to verify.

Either way, Custom Activity/PowerShell seems a bit circuitous compared to the REST API to accomplish the same thing.

Latest posts in the category


azure windowsazure_ja

visual studio fsharp