RSSAmplifier

Hugues Blog · Jan 15, 2022

Parallel bash function call

0
Sign in to vote or save

This page cannot be shown here. You can still read it on the original site — the toolbar below keeps your place in the directory.

Introduction Now that we can execute bash functions remotly , we can scale the execution using GNU Parallel . When one wants to make a change or retrieve one particular piece of information on a whole set of machines, multiple choice are available (ansible, puppet, pssh, consul, ...) each with their own upsides and downsides. Parallel + ssh + func can give a combination of upsides that I haven't…

Introduction

Now that we can execute bash functions remotly, we can scale the execution using GNU Parallel.

When one wants to make a change or retrieve one particular piece of information on a whole set of machines, multiple choice are available (ansible, puppet, pssh, consul, ...) each with their own upsides and downsides.

Parallel + ssh + func can give a combination of upsides that I haven't seen elsewhere:

  • No prerequisite on remote machine
  • Minimum requisite on local machine (bash, ssh, parallel)
  • Blazing fast thanks to parallel
  • Use plain bash, no new DSL or weird escaping
  • Use shell tools that you already know

For example in fifteen seconds we're able to retrieve 1448 host kernel versions:

$ kernel_version() { uname -r | sed 's/\([0-9].[0-9]*\).*/\1/'; }
$ time parallel_host_list_ssh_func hosts.txt kernel_version -j 250 | sort | uniq -c | sort -n
 2 5.17
 467 5.23
 979 5.12

real 0m15,380s
user 0m49,900s
sys 0m26,688s

Where

You can find an helper functions here: shell_utils, you need to copy the functions into your ~/.bash_aliases.

Read on /posts/parallel_and_bash_function/

Comments

Nothing yet. Say the first thing.

    Sign in to join the conversation.