Question

Franssani on Wed, 11 Jan 2017 15:45:22


I tried setting up the time out through testsetting(when using mstest): <Timeouts runTimeout="23400000" testTimeout="4200000" />

This how I call the testmethod from powershell:

MSTEST:

if (($ENV:Processor_Architecture -eq "x86" -and (test-path env:PROCESSOR_ARCHITEW6432)) -or ($ENV:Processor_Architecture -eq "AMD64"))
        {$mstestPath = '"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\mstest.exe"'
         write-host "This is a 64 bit O/S machine"}
         elseif ($ENV:Processor_Architecture -eq 'x86')
            {$mstestPath = '"C:\Program Files\Microsoft Visual Studio 11.0\Common7\IDE\mstest.exe"'
             write-host "This is a 32 bit O/S machine"}
                    else {
                        write-host "Unable to determine CPU architecture"
                        exit
                        }

Function TestCaseNoPrintOut ($testcaseName)

$starttime  =Get-Date -format G
write-host $testcaseName " - Start time: " $starttime
write-host ""

#run testcase
$script:outputVariable  = cmd.exe /C "start ""$testcaseName"" /B /MIN /wait $mstestPath /testmetadata:HARTDQ1.vsmdi /testsettings:TraceAndTestImpact.testsettings /test:$testcaseName"
write-host $script:outputVariable

write-host ""
$endtime = Get-Date -format G
write-host "- End time: " $endtime

}

or

 VSTEST:

$vstestPath = '"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"

'Function TestCaseNoPrintOut ($testcaseName)

write-host $testcaseName " - Start time: " $starttime

write-host ""

#run testcase
$script:outputVariable  = cmd.exe /C "start ""$testcaseName"" /B /MIN /wait $vstestPath Test.HARTDQ.dll /tests:$testcaseName /Logger:trx"
write-host $script:outputVariable

write-host ""
$endtime = Get-Date -format G
write-host "- End time: " $endtime
write-host "==================================================="
write-host "====================================================="
}

Log sample (the first sample show vstest was running, suddenly non of the vstest print its result until end of powershell script)

:my note: testmethod was running 12x before this

:

Module_deviceChangeParameterHART_FVAL_EU  - Start time:  1/9/2017 5:27:19 PMMicrosoft (R) Test Execution Command Line Tool Version 11.0.61030.0 Copyright (c) Microsoft Corporation.  All rights reserved.  Starting test discovery, please wait... Passed   module_deviceChangeParameterHART_FVAL_EU Results File: D:\01-Automation\bin\TestResults\testagent_DV13_SIS_AUTO 2017-01-09 17_35_05.trx  Total tests: 1. Passed: 1. Failed: 0. Skipped: 0. Test Run Successful. Test execution time: 7.9357 Minutes

 

- End time:  1/9/2017 5:35:16 PM

===================================================


Module_verifyChangeParameterHART_FVAL_EU  - Start time:  1/9/2017 5:35:46 PM

 

 

 

- End time:  1/9/2017 5:35:46 PM

===================================================

=====================================================

 


====================================================

==================================================

AMS_verifyDeviceParameterEU  - Start time:  1/9/2017 5:36:26 PM

 

 

 

- End time:  1/9/2017 5:36:26 PM

===================================================

: my note: none of test method run until end of powershell script




Replies

qing__ on Thu, 12 Jan 2017 06:33:34


Hi Franssani,

According to your following code:

>>$script:outputVariable  = cmd.exe /C "start ""$testcaseName"" /B /MIN /wait $mstestPath /testmetadata:HARTDQ1.vsmdi /testsettings:TraceAndTestImpact.testsettings /test:$testcaseName"

I have found some problems with above code, in my vstest.console from the VS2012, I have not found /testmetadata switch for vstest.console.

And as far as I know, we could use mstest for .vsmdi file with VS2010. After VS2010, we could not test .vsmdi file, we could use order test to implement the multiply test methods.

And before you using powershell script, please using command line first to make sure the test command is right.

Please let us keep in touch to resolve this issue :)

Sincerely,

Oscar

Franssani on Thu, 12 Jan 2017 16:54:26


sorry, I just edited, my mistake to mix b/w mstest and vstest calling. I did both of them with the same result. The "no longer calling the testmethod" happened randomly. And inconsistent after how many times.

qing__ on Fri, 13 Jan 2017 02:11:34


Hi Franssani,

In order to resolved your issue quickly, please following my above suggestion. Please hand-code the command line first, if it run successfully then change it to powershell.

If it also has some problems, please share me your command line code, I will help you check it in my side.

Check the test results file (.trx) to see how many test methods have been run, and whether has error messages.

You are testing vsmdi, in your VS2012, you could create .vsdmi file? In my side, I could not. Please create a new test in your VS2012 to see which file has been generated?

If you test .vsdmi file, please use VS2010, I hope you read my above reply and this reply carefully. :)

Sincerely,

Oscar

qing__ on Wed, 18 Jan 2017 02:34:00


Hi Franssani,

If my reply help you resolved your issue or doubts, please remember to mark it as answer, which could be helpful for others who have the same issue.

Sincerely,

Oscar

Franssani on Wed, 18 Jan 2017 03:09:51


I did exactly that. I tried hand code then do powershell. As the title said, it has run for a while, but no longer after running suddenly. I tried looking at trx file, logs file, nothing. It is not a functional problem since I already tested this scripts many times with success.. Once in a while, the script do this. 

I tried doing this: if vstest started showing this problem. I switch doing with mstest then it will run again through completion; vice versa.

for vsmdi file, I imported the files I created through VS2010 and edit it with VS2012. It is writen in microsoft website on how to do this.

qing__ on Wed, 18 Jan 2017 05:18:04


Hi Franssani,

According to your description, if you could run your test with vstest.console.exe successfully, but when run it with powershell, so this issue is caused by your powershell script.

Please check your powershell script carefully. :)

Sincerely,

Oscar

qing__ on Wed, 18 Jan 2017 06:42:09


Hi Franssani,

According to your description, you create this project with VS2010, so I suggest you could use mstest from VS2010 to run this project in your powershell script. This could help us to exclude the reason caused by compatibility between VS2010 and VS2012.

Sincerely,

Oscar

Franssani on Wed, 18 Jan 2017 07:39:43


Only the vsmdi file that is vs2010, imported to vs2012 and edit to add other test method. All the codes are compiled in vs2012.

both Vstest.console.exe and mstest.exe are from vs2012. Both have the same issue.

qing__ on Wed, 18 Jan 2017 07:45:55


Hi Franssani,

>>both Vstest.console.exe and mstest.exe are from vs2012. Both have the same issue.

No matter whether from VS2012 or not. You could test it with vstest.console.exe successfully, am I right? But when run powershell, it failed.

So I think this reason is due to powershell script, but this forum is not support powershell technology. Whether my understanding is right?

Sincerely,

Oscar