即使完成一些工作，也要不断填满四个并行进程的插槽

Question

我有一个脚本，可以一次运行4个进程的批处理，我不在乎获取每个proc的返回码。我永远不想同时运行4个以上的proc。以下方法的问题在于它一次最多不能填充4个进程。例如，如果proc2和proc3提早完成，我希望proc 5和6启动，而不是仅在1-4完成后才启动。我如何在bash中实现这一目标？run_func_1 &run_func_2 &run_func_3 &run_func_4 &waitrun_func_5 &run_func_6 &run_func_7 &run_func_8 &wait

Answer 1

我尝试使用工作人员池和工作队列来执行自定义实现。新工人将在上一个工人完成任务后立即将其从队列中移出。

您可能可以根据需要修改此脚本，但我希望您会明白我的意图。

这是脚本：

#!/bin/bash

f1() { echo Started f1; sleep 10; echo Finished f1; }

f2() { echo Started f2; sleep 8; echo Finished f2; }

f3() { echo Started f3; sleep 12; echo Finished f3; }

f4() { echo Started f4; sleep 14; echo Finished f4; }

f5() { echo Started f5; sleep 7; echo Finished f5; }

declare -r MAX_WORKERS=2

declare -a worker_pids

declare -a jobs=('f1' 'f2' 'f3' 'f4' 'f5')

available_worker_index() {

    # If number of workers is less than MAX_WORKERS

    # We still have workers that are idle

    declare worker_count="${#worker_pids[@]}"

    if [[ $worker_count -lt $MAX_WORKERS ]]; then

        echo "$worker_count"

        return 0

    fi

    # If we reached this code it means

    # All workers are already created and executing a job

    # We should check which of them finished and return it's index as available

    declare -i index=0

    for pid in "${worker_pids[@]}"; do

        is_running=$(ps -p "$pid" > /dev/null; echo "$?")

        if [[ $is_running != 0 ]]; then

            echo "$index"

            return 0

        fi

        index+=1

    done

    echo "None"

}

for job in "${jobs[@]}"; do

    declare worker_index

    worker_index=$(available_worker_index)

    while [[ $worker_index == "None" ]]; do

        # Wait for available worker

        sleep 3

        worker_index=$(available_worker_index)

    done

    # Run the job in background

    "$job" &

    # Save it's pid for later

    pid="$!"

    worker_pids["$worker_index"]="$pid"

done

# Wait all workers to finish

wait

您只需更改MAX_WORKERS变量即可轻松更改工作池的大小。