@eptf they should rewrite all programming books in comic form, it's so clear


Huh. I knew about -e but not -u or pipefail.

Also: Julia Evans is awesome.

Also also: be aware that -e won’t always stop the script (for $REASONS) so treat it as a safety net instead of a predictable thing.

@suetanvil @eptf I fixed a bug in production because of that last week!

a script was failing with an error in the condition (not interior) of an `if` block. that's one of the exceptions -- I'm guessing because `set -e` would otherwise interfere with `if`'s intended behavior -- so the broken script exited with success instead of doing its job, which was canceling an incorrect deploy.

@eptf I knew about (and always use) -x, but -u is new to me, probably because I typically don't work much with environment variables. These days, though, python has mostly replaced bash scripts for me.

@eptf I bought this comic from the website because of this toot. :p

@eptf set -o pipefail is an awful idea to set without thinking. It can make a lot of valid programs fail.

Let’s say you’re using curl, but instead of filtering all lines, you just grep -q to find if a pattern is present at all.

Like this: curl 404.com | grep -q 'panda'

All modern programs don’t output text immediately, instead they buffer it into chunks of size n. So, let’s say curl 404.com outputs a gigantic wall of text that can’t fit into a single buffer. It’ll be buffered, and only then sent to grep, at which point grep will search through that and, if it finds it, exit successfully.

But curl can’t know that grep is already done, so it’ll write more text into a buffer and try to send it to grep. But oh no, grep is done, the pipe is closed, so curl will fail because of a broken pipe, even though that’s what we want to happen because we don’t want curl to work more than it needs to, consuming useless cpu cycles and filling memory with useless data.

Not all error codes are created equal, and especially not in pipes. pipefail should only be used locally around statements you want to pipefail, but never ever ever ever globally, as that, contrary to popular belief, will only lead to more failure-prone programs (for example, you might be testing your program on small files that fit into a buffer entirely, but as soon as you’re using real-world data, it just fails).

Also, instead of using set -e, you should trap errors to a function or a command (by doing trap command [optional signal numbers] ERR) (unless your script is made to be sourced instead of executed, in which case you should use any set options either), as that will allow you to print more information about why it exited and perform any needed cleanup.

I prefer to do scripts with #!/bin/sh -xe , -x option is to make it output all commands it's running so you're not left guessing which exact subcommand spewed up this particular error message.

@eptf Julia Evans is a near unstoppable force of awesome.

@eptf ...Okay, that's good to know. I'll be sure to remember and use this is in any serious bash script I write. XD

Thanks! I habe to admit that I've not been aware if -o and -u

@eptf set -e and set -u also work in any Bourne-compatible shell such as the various sh and ksh variants. set -o pipefail is in ksh93, mksh and zsh but not the original Korn shell or the pdksh.

Sign in to participate in the conversation

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!