Question

Parthiban Malan on Mon, 29 May 2017 12:29:08


Hi,

I want to ingest a propitiatory data format file into ADLS and process it using U-SQL. Since Python has a package to read that data format, I am trying to leverage Python Extension for U-SQL to process it. But, as that package is not by default available in U-SQL, I am importing it using DEPLOY RESOURCE directives as explained here (https://stackoverflow.com/questions/41045491/add-more-python-libraries and https://blogs.msdn.microsoft.com/azuredatalake/2017/03/10/using-custom-python-libraries-with-u-sql/).

I am also trying to run the U-SQL job locally from Visual Studio before running it in ADLA. I followed the steps as instructed here (https://blogs.msdn.microsoft.com/azuredatalake/2017/02/20/enabling-u-sql-advanced-analytics-for-local-execution/ and  https://github.com/Azure/usql/issues/66) to setup the environment and run the job locally.

With this setup, I am able to successfully build the job. But when I execute, it throws the following error.

An unhandled exception of type 'System.IO.FileLoadException' occurred in Microsoft.Cosmos.ScopeStudio.BusinessObjects.Debugger.dll

Additional information: Could not load file or assembly 'Microsoft.Analytics.LocalRun, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f300afd708cefcd3' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

Has anyone faced this issue? Any suggestion to debug this issue will be very helpful.

BTW, in my laptop with Windows 7 and Python 3.5.2 | Anaconda 4.1.1 64-bit environment, I am able to successfully read the file using the package. Since it is a propitiatory format file, I have not shared the code and package name.

with regards,
Parthiban
 

Sponsored



Replies

Yangyd-MSDN on Wed, 31 May 2017 01:41:42


Please build clean your solutions directory first and have a try. We have updated the assembly strong name sign with newer released ADL Tools version.   You will need to use the same version when building and debugging the script. 

Here are discussions regarding how to find out the root cause of similar issue, you could try  fuslogvw.exe or process monitor to see where and which Microsoft.Analytics.LocalRun.dll version is being tried to load.

https://stackoverflow.com/questions/215026/the-located-assemblys-manifest-definition-does-not-match-the-assembly-reference

Thanks,

Yu Dong

Parthiban Malan on Thu, 01 Jun 2017 07:26:01


Hi Yu Dong,

After I build clean the solution, I am not facing that issue. Thanks.

The job is getting executed now, but is throwing another error which I have given below after masking my project related information. 

Error Message:
Start : 01/06/2017 08:50:08
Initialize : 01/06/2017 08:50:08
GraphParse : 01/06/2017 08:50:08
Run : 01/06/2017 08:50:08
Start 'Root' : 01/06/2017 08:50:08
End 'Root(Success)' : 01/06/2017 08:50:08
Start '1_SV1_Extract' : 01/06/2017 08:50:08
End '1_SV1_Extract(Error)' : 01/06/2017 08:50:09
Completed with 'Error' : 01/06/2017 08:50:09
Execution failed with error '1_SV1_Extract Error : '{"diagnosticCode":195887140,"severity":"Error","component":"RUNTIME","source":"User","errorId":"E_RUNTIME_USER_UNHANDLED_EXCEPTION_FROM_USER_CODE","message":"An unhandled exception from user code has been reported when invoking the method 'Constructor' on the user type 'Extension.Python.Reducer'","description":"Unhandled exception from user code: \"The file 'E:\\Path-To-My-Project-Folder\\bin\\Debug\\7962804CE78D4BF4\\ProcessTDMSWithPython_949DC07AF3C492D3\\bridgeTests.py' already exists.\"

\nThe details includes more information including any innerexceptions and the stack trace where the exception was raised.","resolution":"Make sure the bug in the user code is fixed.","helpLink":"","details":"==== Caught exception System.IO.IOException\n\n   at System.IO.__Error.WinIOError(Int32 err
\n   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Bool
\n   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access
\n   at System.IO.Compression.ZipFileExtensions.ExtractToFile(ZipArchiveEntry so
\n   at System.IO.Compression.ZipFileExtensions.ExtractToDirectory(ZipArchive so
\n   at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFil
\n   at Extension.Python.Reducer..ctor(String pyFile, String pyScript, String pyVersion) in C:\\Users\\shravan\\Source\\Repos\\VSTS\\USqlExtensions\\lang\\pytho
\n   at ___Scope_Generated_Classes___.__OperatorFactory__.Create_Process_3() in e:\\Path-To-My-Project-Folder\\bin\\Debug\\7962804CE78D4BF4\\ProcessTDMSWithPython_949DC07AF3C492D3\\__ScopeCodeGen__.dll.cs:li
\n   at ScopeEngine.ManagedUDO<7>.{ctor}(ManagedUDO<7>* , basic_string<char\\,std::char_traits<char>\\,std::allocator<char> >* argv, Int32 argc) in e:\\Path-To-My-Project-Folder\\bin\\debug\\7962804ce78d4bf4\\processtdmswithpython_949dc07af3c492d3\\__scopecodegenengine__.dll.cpp:line 874","internalDiagnostics":""}
'
'
Execution failed !
Press any key to continue . . .


While going through the error, it says there is a bug in my code. Also, I could see it referring to some local folder (C:\Users\Source\Repos\VSTS\USqlExtensions\lang\pytho) which is definitely not my machine's local folder... Is this anyway related to this error?

with regards,
Parthiban

Freebugs on Tue, 15 Aug 2017 22:09:02


I get the same error (bridgeTests.py already exists) when trying to run the Python/U-SQL tutorial from here:

https://blogs.msdn.microsoft.com/azuredatalake/2017/03/10/using-custom-python-libraries-with-u-sql/