Home>

As the title suggests, parallel processing using multiprocessing.Pool and multiprocessing.Queue does not work well.

What I want to achieve is to run multiple processes using multiprocessing.Pool and output the processing results of each process together in the parent process.
The executed program and the execution result are described below.

In the execution result, at the end

count
1
count
2


Was output, and the execution did not finish.

Execution environment
・ Windows command prompt
・ Python 3.6.3.

Executed source code

import multiprocessing
import numpy as np
def count (a, q):
    print ("count")
    print (a)
    q.put [a]
if __name__ == "__main__":

    p = multiprocessing.Pool (2) # Up to 4 child processes at the same time
    m = multiprocessing.Manager ()
    q = m.Queue ()
    range_array = np.array ([1,2])
    print ("start")
    for k in range_array:
        print ("function:" + str (k))
        p.apply_async (count, args = (k, q))

    print ('Waiting for all subprocesses done ...')
    print (q.get ())
    p.close ()
    p.join ()
    print ('all process is done ...')


Execution result

$>python test.py
start
function: 1
function: 2
Waiting for all subprocesses done ...
count
1
count
2
Trends