I want to create Sudoku grids using Python. As it's pretty slow if I want to create a bunch of grids, I decided to use multiprocessing. My computer has 4 virtual cores (dual core with hyper threading), so I expected the new program to run about 4 times faster, minus a little overhead maybe. But it's only almost exactly two times faster!Here's the snippet where I use a multiprocessing.Pool:worker_pool = Pool(processes)sudokus = list(filter(None, worker_pool.imap_unordered(create_sudoku, range(n), n // processes)))Why does this happen? Am I using the multiprocessing module in a wrong or inefficient way? Please help me to optimize it.Here's the code of the main module, I guess the grid module is not important for multiprocessing performance:import grid  # my own sudoku grid moduleimport subprocessimport randomimport timefrom multiprocessing import Pool, cpu_countdef create_sudoku(*_, level=medium):    numbers = list(123456789)    random.shuffle(numbers)    grid_string = .join(numbers[:3]) + . * 6 + .join(numbers[3:6]) + . * 6 + .join(numbers[6:9]) + . * 6    random.shuffle(numbers)    grid_string += . * 3 + .join(numbers[:3]) + . * 6 + .join(numbers[3:6]) + . * 6 + .join(numbers[6:9])    random.shuffle(numbers)    grid_string += . * 9 + .join(numbers[:3]) + . * 6 + .join(numbers[3:6]) + . * 6 + .join(numbers[6:9])    g = grid.Grid(grid_string)    g.try_to_solve()    return (grid_string, level) if g.is_solved() else Noneif __name__ == __main__:    n = int(input(How many grids to create? ))    use_multiprocessing = None    while use_multiprocessing is None:        answer = input(Use multiprocessing to speed things up? (Y/n) ).strip().lower()        if len(answer) == 1 and answer in yn:            use_multiprocessing = True if answer == y else False    print()    t0 = time.time()    if use_multiprocessing:        processes = cpu_count()        worker_pool = Pool(processes)        print(Creating {} sudokus using {} processes. Please wait....format(n, processes))        sudokus = list(filter(None,                              worker_pool.imap_unordered(create_sudoku, range(n), n // processes)                              ))    else:        progress_bar, progress_bar_length = 0, 10        sudokus = []        print(Creating {} sudokus.format(n), end=, flush=True)        for i in range(n):            p = int((i / n) * progress_bar_length)            if p > progress_bar:                print(. * (p-progress_bar), end=, flush=True)                progress_bar = p            new_sudoku = create_sudoku()            if new_sudoku:                sudokus.append(new_sudoku)        print()    t = time.time() - t0    print(\nSuccessfully created {}/{} grids ({:.1f}%) in {:.3f}s (average {:.3f}s per grid)!.format(        len(sudokus), n, 100*len(sudokus)/n, t, t/n    ))And here are two sample runs:How many grids to create? 100Use multiprocessing to speed things up? (Y/n) yCreating 100 sudokus using 4 processes. Please wait...Successfully created 100/100 grids (100.0%) in 19.507s (average 0.195s per grid)!How many grids to create? 100Use multiprocessing to speed things up? (Y/n) nLoading 100 sudokus.........Successfully created 100/100 grids (100.0%) in 37.675s (average 0.377s per grid)!
Optimal use of Python 3's multiprocessing package to create Sudoku grids
python;performance;python 3.x;sudoku;multiprocessing