From a C# U-SQL class assembly; Make an external http web request to get Geo Location from IP Address.
Category: azure data lake
Rahul Sinha-Elevondata on Mon, 05 Sep 2016 10:26:33
I have an Azure Data Lake Analytics U-SQL script to process web click-stream data. I have created an Assembly wherein there is a C# user defined function to make an external API call to get Geo Location details (latitude, longitude etc.) based on an input IP Address. This code works fine as a Console App in Visual Studio. But when I reference the Assembly in a U-SQL job, I get the following error: (Pls note – other functions in this assembly work fine with U-SQL).
Inner exception from user expression: The remote name could not be resolved: 'www.freegeoip.net' Current row dump: …
==== Caught exception System.Net.WebException
at DnnHelperClassesUSQL.GeoLocation.getLocationFromIP(String strIPAddress)
at ___Scope_Generated_Classes___.SqlFilterTransformer_6.Process(IRow row, IUpdatableRow output) in d:\data\CCS\jobs\...\__ScopeCodeGen__.dll.cs:line 374
Error while evaluating expression DnnHelperClassesUSQL.GeoLocation.getLocationFromIP(clientIP)
Here is the C# code snippet from the Assembly:
// Create a WebRequest with the IP Address.
WebRequest _objWebRequest = WebRequest.Create("http://xxx.net/json/" + strIPAddress);
_objWebRequest.Timeout = 5000;
// Get the WebResponse
WebResponse _objWebResponse = _objWebRequest.GetResponse();
// Exception is raised at this point.
MRys on Mon, 05 Sep 2016 22:38:08
We currently do not allow your custom code to execute remote resources (IP addresses). The reason for that is that a careless use of that feature could appear to the target end point as a denial of service attack (think about 10 million rows that all need to do a REST call within a few minutes!) and either shut that end point down, or black list the Azure Service.
Local run works, because it does not run in a sandboxed VM like the service.
I would suggest to get the mapping as a file, deploy it as a resource (DEPLOY RESOURCE) and then do a local lookup.
Feel free though to file or vote for a request at http://aka.ms/adlfeedback, if you really need access to a specific webservice.