Updating bash Variables in a Loop

Introduction

If you have the need to update a bash variable inside a loop that reads input from a command, the expected use of a pipe to achieve that fails. This tip will show you how to correctly get the input from the command and update bash variables correctly. 

Background

If you have the need to read input from a command and update a variable inside a loop, you will probably start with something like this:

data=""

cat data.txt | while read text; do
    data+=" $text"
done

echo $data

It will be surprising to many that at the end of the loop, $data is an empty string. Furthermore, adding echo $data before done in the above snippet clearly shows that $data gets appended to while the loop is running. So what's happening here? The answer is that the script fragment after the pipe gets run in a subshell. This means that any variables accessed inside the subshell, in this case inside the while loop, are local to that subshell, and changes to them are not propagated up to the calling shell. You might be tempted to use export to try to get a "global" variable instance, but that doesn't resolve the issue.

Using the Code

The correct way to do this is to put the shell command in its own subshell and use I/O redirection to read from the command output.

data=""

while read text; do
    data+=" $text"
done < <(cat data.txt) # use of subshell and I/O redirection

The construct (cat data.txt) runs the shell command in a subshell, and prepending < redirects stdout to stdin. stdin is then read into the loop with the < after done.

History

• 8^th March, 2023: v 1.0