Running Concurrent Bash Scripts and TasksMax Settings

For a long time, I had an Expect script to login network devices, fetch information and put the output together as a Web page. Think of it as a script checking router version, memory etc. It took about 30 seconds to finish checking a single target and would take hours to finish scanning several subnets.

Finally I think it is time to parallel these tasks so the whole scan could take the same time as scan only one device (etc. 30 seconds versus hours).  And is excited to find out the simple and powerful usage of xargs -P option.

Here are man xargs result regarding -P:

    -P max-procs, --max-procs=max-procs
Run  up  to  max-procs processes at a time; the default is 1.  If max-procs is 0, xargs will run as many processes as possible at a time.  Use the -n option or the -L option with -P; otherwise chances are that only one exec will be done.

Put together into my small script:

#!/usr/bin/bash
echo >/tmp/list
subnets="192.168.4 192.168.6 192.168.7 192.168.14 192.168.15"
(for j in $subnets; do for i in {1..254}; do echo $j.$i; done;done) | xargs -P 12000 -n 1  expect checkscripts  >> /tmp/list

And woola, now scanning through all these subnets still only takes 30 seconds.

Then I integrates Google Table (Thanks!)  into the result and now I get an sort-able table with all the results. Great!

Next step, of course, would be add this as a cronjob and also add an function to allow dynamic update via PHP, this actually did not finish running the whole scans, giving me only partial results along with the following error log:

Resource temporarily unavailable

And it took me a while to figure this out. The reason is by default crond and apache2 process could only handle 512 concurrent processes, which you could check from this sample command here:

systemctl status crond
  crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2018-11-20 12:28:28 PST; 5 days ago
 Main PID: 803 (crond)
    Tasks: 1 (limit: 512)
   CGroup: /system.slice/crond.service

And on my Debian server, the command to fix this are:

systemctl set-property apache2.service TasksMax=12000
systemctl set-property cron.service    TasksMax=12000

Finally everything is happy and xargs saves the day!