Home Blog Gallery

Shell Lesson 01, Batch Processing

published Fri Mar 13 2026 05:20:51 GMT-0400 (Eastern Daylight Time)

This is the second lesson in a series! Previously, we did a basic intro on the terminal. Now, let's do something convenient with them: Batch Processing.

We're going to be messing a with a bunch of files at once in this section, which should scare you. What if we didn't mean what we wrote? For protection, always have backups, and if the files you're working on are small enough, consider making a copy of the entire folder in the known "good" state so you can easily swap them in, out and compare:

$ cp -r myfolder myfolder_backup

Example: File converting

I have a lot of random images on my computer. Some of them are even, (gasp) in .webp format!

$ echo *.webp
IMG_9532.webp  IMG_9571.webp

The * character is a wildcard, also known as a glob. it means "match any string of characters, including 0 characters". when combined with a suffix (.webp), this means "match any file that ends in .webp". I've looked at these (using xdg-open IMG_9532.webp, remember!), and they are smallish drawings, so I think converting them to png files makes sense.

I already have a package installed for this, ImageMagick. You could technically do this with a lot of programs, including things with full GUIs attached like the unfortunately named GNU Image Manipulation Program. but imagemagick has a nice cli so we'll stick with that.

$ magick IMG_9532.webp IMG_9532.png
$ ls -lh IMG_9532*
-rw------- 1 alloyed alloyed 683K Mar 13 03:35 IMG_9532.png
-rw-r--r-- 1 alloyed alloyed 116K Mar  4 01:01 IMG_9532.webp

ok cool! image made (yeah, i guess it's a little larger but w/e). Imagemagick supports more than just conversions:

$ magick --help | grep -E "flip|rotate|flop"
  -auto-orient         automagically orient (rotate) image
  -flip                flip image vertically
  -flop                flop image horizontally
  -rotate degrees      apply Paeth rotation to the image
  -transpose           flip image vertically and rotate 90 degrees
  -transverse          flop image horizontally and rotate 270 degrees
$ magick IMG_9532.webp -rotate 90 IMG_9532.png

This gives me the old image rotated by 90! Not that I need that right now; my goal is just to get rid of the webp files.

For loops

This helps me with one file, but if you recall, I actually had two. If this weren't a fake artificial example, I could have hundreds. How can I do them all at once?

Time to introduce a for loop:

$ for file in *.webp; do echo $file; done
IMG_9532.webp
IMG_9571.webp

This looks a lot like the first time we did echo *.webp: the main difference is each file is on a different line. This is because, instead of calling echo IMG_9532.webp IMG_9571.webp, all in one command, we are running something that looks more like:

echo IMG_9532.webp
echo IMG_9571.webp

one command for each element in the list. neato. breaking the syntax down:

for <VARIABLE> in <LIST>
do <FIRST COMMAND>
<MORE COMMANDS...>
done

do/done act like parenthesis, they tell the shell where your inner loop starts and stops. do technically belongs on the same line as the first command, and you can have as many commands as you want, but only the first command needs the do. in shell scripts, it's pretty common to use semicolons to combine multiple lines into one. when writing a script into a file, I especially like

for <...>; do
    <...>
done

but for interactive use i just throw everything onto a single line :p

$ is the variable replacer. It says, "replace me with the variable named file", which we defined as part of the loop. You can also define variables other ways; maybe we'll cover those later.

ok, time to put it all together:

$ for f in *.webp; do echo magick $f $f.png; done
magick IMG_9532.webp IMG_9532.webp.png
magick IMG_9571.webp IMG_9571.webp.png

I leave the echo in there so I can see what the command will actually run before it does it. this is a Good Idea so you don't accidentally do damage you didn't mean to. and, this looks ok... but i find it kind of ugly that the files say .webp.png. Here's a magic incantation to fix that:

$ for f in *.webp; do echo magick $f ${f%.webp}.png; done
magick IMG_9532.webp IMG_9532.png
magick IMG_9571.webp IMG_9571.png

(${} is a variable replacer, but with % we're telling it to remove the .webp at the end) Now, time to commit:

$ for f in *.webp; do magick $f ${f%.webp}.png; done
$ echo IMG_*
IMG_9532.png IMG_9532.webp IMG_9571.png IMG_9571.webp

ok, looks good, time to remove the old webp files (after confirming! again, don't do anything destructive until you're sure).

$ rm -v *.webp
removed 'IMG_9532.webp'
removed 'IMG_9571.webp'

Bonus: globstar

*.webp is a pretty useful wildcard, but it only works on the current directory. if you want it to work in the current folder and subfolders, we'll need to enable a non standard extension (scary!). let's edit the file ~/.bashrc, and look for this section (if it's not there, or your bashrc is empty, you can paste this in)

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar

Let's uncomment the line, so it's enabled

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
shopt -s globstar

Save, and then apply to the current shell:

$ source ~/.bashrc

now, this will work, no matter how many folders and folders-in-folders you have!

$ for f in **/*.webp; do echo magick $f ${f%.webp}.png; done

Bonus: other batch commands

Maybe you don't have an irrational hatred of webp files. maybe try these instead? (don't forget to remove echo)

$ for f in **/*.flac; do echo ffmpeg -i $f -ab 320 ./mp3/${f%.*}.mp3; done

$ for f in **/*.mkv; do echo ffmpeg -i $f -codec copy ${f.*}.mp4; done

$ for f in **/*\ *; do echo mv -iv $f ${f// /_}; done

$ mkdir -pv pics vids audio docs; mv -v *.{png,jpeg,webp,avif} pics; mv -v *.{mp4,webm} vids; mv -v *.{mp3,wav,flac,sf2} audio; mv -v *.{csv,pdf,xlsx,docx} docs

rm -v **/.DS_Store
tagged with: #kit techtips #bash