/contrib/famzah

Enthusiasm never stops

Using flock() in Bash without invoking a subshell

9 Comments

The flock(1) utility on Linux manages flock(2) advisory locks from within shell scripts or the command line. This lets you synchronize your Bash scripts with all your other applications written in Perl, Python, C, etc.

I’ll focus on the third usage form where flock() is used inside a Bash script. Here is what the man page suggests:

#!/bin/bash

(
flock -s 200

# ... commands executed under lock ...

) 200>/var/lock/mylockfile

Unfortunately, this invokes a subshell which has the following drawbacks:

  • You cannot pass values to variables from the subshell in the main shell script.
  • There is a performance penalty.
  • The syntax coloring in “vim” does not work properly. 🙂

This motivated my colleague zImage to come up with a usage form which does not invoke a subshell in Bash:

#!/bin/bash

exec {lock_fd}>/var/lock/mylockfile || exit 1
flock -n "$lock_fd" || { echo "ERROR: flock() failed." >&2; exit 1; }

# ... commands executed under lock ...

flock -u "$lock_fd"

Note that you can skip the “flock -u “$lock_fd” unlock command if it is at the very end of your script. In such a case, your lock file will be unlocked once your process terminates.

Advertisements

Author: Ivan Zahariev

An experienced Linux & IT enthusiast, Engineer by heart, Systems architect & developer.

9 thoughts on “Using flock() in Bash without invoking a subshell

  1. Thanks for this optional usage of flock. What happens if the script crashes and the line ‘flock -u 200’ is never reached? Specifically: will the lock still be released, as it should?

    • The lock will be released upon script termination or server reboot. You can actually skip the line “flock -u 200″ at the very end of the script, as it’s redundant.

  2. I tried this – but it doesn’t actually create a lock as far as I can tell. Here’s how I tested it.

    1. open two bash shell windows

    2. Execute in the first bash shell
    exec 200>/var/lock/mylockfile
    then
    flock -n 200

    3. Do the exact same thing in the second bash shell checking exit codes.

    The first shell does not stop the lock of the second shell and the second shell commands return 0 (success) at each step.

    Am I missing something here?

  3. Your lock fd 200 is going to be inherited by child processes, which, if long lived, will prevent the lock from being released. e.g. restarting a daemon… sadly bash doesn’t allow setting fd to CLOEXEC.

    On newer bash, consider:

    exec {lock_fd} > /var/lock/mylockfile

    then bash will find a spare fd and assign it to $lock_fd

    • Great hint! I didn’t know about this Bash feature. Note that there must be no space after the {lock_fd}, or else the magic won’t work. A little bit more info regarding this matter: http://stackoverflow.com/questions/8297415/in-bash-how-to-find-the-lowest-numbered-unused-file-descriptor

      I’ve updated my example following your suggestion. Thanks.

      • Another improvement for you, to prevent the lock being inherited by child processes:

        #!/bin/bash

        exec {lock_fd}>/var/lock/mylockfile || exit 1
        flock -n “$lock_fd” || { echo “ERROR: flock() failed.” >&2; exit 1; }
        {

        # … commands executed under lock …

        } {lock_fd}>&-

        flock -u “$lock_fd”

        It “closes” {lock_fd} for the code inside the braces — of course i doesn’t really close it, it first dup’s {lock_fd} to one which it will close before exec, and then closes {lock_fd}. Once the code in { … } competes, then this spare fd is dup;d back to {lock_fd} and the spare fd is then closed.

      • Maybe this bash wrap of “flock -o -c …” will be interesting to you:

        It apparently supports flock -o -c … to call a bash function of the same script, same pid, with no forking, in the same way that flock -c calls other external commands.

        Thus your example becomes:

        something() {
        # … commands executed under lock …

        }
        flock -o /var/lock/mylockfile -c do_something || { echo “ERROR: flock() failed.” >&2; exit 1; }

        There is no need to worry about calling flock -n, or hiding the lock pid from child processes.

        Because the command is called while the flock function wrapper is active, it uses some tricks to avoid trampling on BASH variables or declaring local variables that would affect final command.

        http://blog.sam.liddicott.com/2016/02/using-flock-in-bash-without-invoking.html

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s