Delay starting spawned processes

Category: visual studio parallelcpp

Question

DonDilworth on Sat, 06 Feb 2016 17:55:28


My code starts several processes:

    imsg.Format( "%d", i );        // is new process number
    jmsg.Format( "%d", j );        // is task number
    kmsg.Format( "%d", k );        // is network flag
    tmsg.Format( "%d", frametop );    // data set by process 0
    lmsg.Format( "%d", frameleft );
       
    l = spawnl( _P_NOWAIT, path, "SYNOPSYS200v14.exe", imsg, jmsg, kmsg, tmsg, lmsg, NULL );

All processes start immediately, as I can see in the Task Manager.  Problem is, they get no CPU time at first.  After about 10 seconds or so, they all start their number crunching.  Since the whole point is to speed up the calculation, this is not good.

The second process always starts running with no delay.  That is the process on the other half of the hyperthreaded first core.  Rarely, one of the higher processes also starts immediately.  So there is no reason to think there is a delay in my code anywhere.

Can anyone think of a reason for this delay with the other processes?

Replies

Brian Muth on Sat, 06 Feb 2016 18:25:20


My code starts several processes:

    imsg.Format( "%d", i );        // is new process number
    jmsg.Format( "%d", j );        // is task number
    kmsg.Format( "%d", k );        // is network flag
    tmsg.Format( "%d", frametop );    // data set by process 0
    lmsg.Format( "%d", frameleft );
       
    l = spawnl( _P_NOWAIT, path, "SYNOPSYS200v14.exe", imsg, jmsg, kmsg, tmsg, lmsg, NULL );

All processes start immediately, as I can see in the Task Manager.  Problem is, they get no CPU time at first.  After about 10 seconds or so, they all start their number crunching.  Since the whole point is to speed up the calculation, this is not good.

The second process always starts running with no delay.  That is the process on the other half of the hyperthreaded first core.  Rarely, one of the higher processes also starts immediately.  So there is no reason to think there is a delay in my code anywhere.

Can anyone think of a reason for this delay with the other processes?

I can understand there may be a short delay as some time is required to load the programs into memory, but 10 seconds is much too long.

What does the launched program do? Most programs don't immediately start to "number-crunch" but requires some setup. Is the program reading in data, for instance?


DonDilworth on Sat, 06 Feb 2016 18:32:12


It does some initializations, but not 10 seconds worth.  That's why I added the note that some processes do not exhibit this delay.  They all have the same initialization code, and some start right up.

Here's a twist:  If I authorize, say, 4 processes, the delay is much shorter.  The more processes, the longer the delay.  My PC can do 16, and that's where I see the longest delay.  Is it something in the OS, trying to manage all the threads that it is already running?

Brian Muth on Sat, 06 Feb 2016 18:45:58


How many processors does your computer have?

Most tabletop computers nowadays have a quad-core Intel chip, so for four processes, one process is assigned to each core and I would expect the startup to be immediate. If you are trying to run more programs than cores, then some of those programs are going to have to wait for a time slice.

DonDilworth on Sat, 06 Feb 2016 19:57:31


My PC has eight hyper-threaded cores, so it will run 16 separate processes simultaneously.  Once they all start cranking, the total CPU usage peaks at 100% and they all update their windows simultaneously.

My program is structured to not start more processes than the PC has cores, except for 2X with hyper threading.