Making Bash script parallel with xargs

September 16, 2010
Communication

People, who tried to create parallel Bash scripts, are probably aware of -P parameter of xargs utility. In conjunction with find it allows to execute a specified command for each file and distribute these commands among several processes. This scheme is perfect when you need, say, convert all files in the given directory.

When I started to write such scripts, a certain inconvenience is revealed. Commands, that should be given to xargs, usually appear to be more complicated than a single call of some utility with arguments. It's often needed to pass a set of commands, especially when progress indication and error reporting are required. Finally, a separate script for processing a single item is created in addition to the script that has the xargs call.

I analized how to combine these two script into one...

Nikita Melnichenko.