If you follow It’s FOSS on Facebook, you might be aware of the weekly Bash Challenge. It is a joint effort by Yes I Know It and It’s FOSS to give you a Bash script exercise to test your Linux skills.

We are bringing this Bash Challenge from Facebook to a wider audience on the regular web. This is the 5th installment of this series. The first 4 challenges can be found on our Facebook pages. You can also buy these challenges in book form:

## Bash Challenge 5

We will show you a terminal screenshot, and ask you to explain why the result is not the one we were expecting. Of course, the most amusing, and most creative, part of the challenge will be to find how to fix the command(s) displayed on the screen to obtain the correct result.

Ready to play? So here is this week’s challenge:

## My Bash don’t know how to count [Difficulty level 1]

This week, I have some data file containing integer numbers, one on each line:

```
cat sample.data
102
071
210
153
```

And I want to compute the sum of all those numbers:

```
declare -i SUM=0
while read X ; do
SUM+=$X
done < sample.data
echo "Sum is: $SUM"
```

Unfortunately, the result I obtain is wrong (the expected result was 536):

`Sum is: 522`

### Challenge

Your challenge is to find :

- Why that result was wrong?
- How to fix my commands to obtain the correct result?

★ Bonus unicorn point if you can find a solution using only Bash internal commands and/or shell substitutions.

We’re looking forward to read your solutions in the comment section below! Don’t forget to be creative.

### Few details

To create this challenge, I used:

- GNU Bash, version 4.4.5 (x86_64-pc-linux-gnu)
- Debian 4.8.7-1 (amd64)
- All commands are those shipped with a standard Debian distribution
- No command was aliased

### Solution

### How to reproduce

Here is the raw code we used to produce this challenge. If you run that in a terminal, you will be able to reproduce **exactly** the same result as displayed in the challenge illustration (assuming you are using the same software version as me):

```
rm -rf ItsFOSS
mkdir -p ItsFOSS
cd ItsFOSS
cat > sample.data << 'EOT'
102
071
210
153
EOT
clear
cat sample.data
declare -i SUM=0
while read X ; do
SUM+=$X
done < sample.data
echo "Sum is: $SUM"
```

### What was the problem ?

The problem was caused by the `071`

value. As you noticed, this number is starting by a `0`

— probably to ensure here all data are formatted on three digits. Nothing complicated here, except that … following an unfortunate convention inherited from the C programming language, prefixing an integer by `0`

is a way to specify that number is expressed in **octal**, and not in **decimal**.

Octal numbers are expressed with digits from `0`

to `7`

. Here is a simple conversion table:

Octal | Decimal |
---|---|

0 | 0 |

1 | 1 |

2 | 2 |

3 | 3 |

4 | 4 |

5 | 5 |

6 | 6 |

7 | 7 |

10 | 8 |

11 | 9 |

12 | 10 |

13 | 11 |

14 | 12 |

.. | .. |

71 | 57 |

This last value is what caused the error when evaluating the sum. The Bash read `071`

and interpreted it as an octal number representing the `57`

decimal value. You can check that easily:

```
echo $((071))
57
```

### How to fix that ?

I can see two main strategies to fix that issue. Either removing the leading zeros. Or finding a way to make the shell understand all my numbers are **decimal** values.

#### Removing leading zeros

Here is a simple solution using the `sed`

external command to remove the leading zeros :

```
declare -i SUM=0
while read X ; do
SUM+=$X
done < <(sed -E s/^0+// sample.data)
echo "Sum is: $SUM"
```

(bonus question: why didn’t I used a **pipe** instead of a **process substitution** ?)

### Specifying **explicitly** the base

The previous solution is (mostly) straightforward — but the Bash allows us to make things better. Instead of trying to **fix** the data, we will simply specify **explicitly** our numbers are expressed in base 10 (decimal), instead of base 8 (octal). You can do that by using the `base#value`

syntax.

Compare those three examples:

```
echo $((071)) # The leading `0` specify the number as octal
57
echo $((8#071)) # We *explicitly* specify base 8 (octal)
57
echo $((10#071)) # We *explicitly* specify base 10 (decimal)
71
```

To **fix** my initial command and obtain the correct result, I only have to explicitly specify the base 10 for all my data :

```
declare -i SUM=0
while read X ; do
SUM+=$((10#$X))
done < sample.data
echo "Sum is: $SUM"
```

And here is the correct result. We hope you enjoyed that challenge. Stay tuned for more fun!

**Author Bio:** *I’m Sylvain Leroux, a software engineer by passion, a teacher by vocation. I have 15 years of experience in teaching Computer Science & Information Technologies at all level. I’m a strong advocate of Linux & OpenSource technologies. I founded Yes I Know IT to share that experience with a wider audience through online courses & free videos. Don’t hesitate to reach me on Twitter.*

It calcualte 071 as octal you can fix it with variable substitution: SUM+=${X#0}

... but could you try to insert "008" as part of the input data ?

;)

Great idea to remove offending leading zeros !

export SUM=0 ; while read X ; do SUM=`expr $SUM + $X` ; done< sample.data ; echo $SUM

536

Yes, as far as I know, the `expr` command always thread its numeric argument as decimal.

The only drawback here is you spawn a new process at each loop iteration. Couldn't we do better than that ?

As everybody has noticed, the problem is the leading 0 before 71. By default bash considers numbers starting with 0 as numbers in octal so it converted (071) oct into decimal. What gave (57) dec.

102 + 57 + 210 + 153 = 522. And voilà!

Another solution: while read X ; do SUM+=$X; done < <(sed 's/^0*//' sample.data)

Great explanation Elisée !

I have nothing against process substitution, but it would have felt more natural to me to use a plain old pipe here. Or did I missed something ? ;)

This will work regardless the 071 :

cat sample.data | awk '{ sum += $1 } END { print sum }'

Or..

in the end of the while loop: "done < <(sed 's/0*//' sample.data)" this will remove the leading zero thus eliminating the octal interpretation

I ❤ AWK.

I was once bitten hardly by some purist about the "Useless Use Of Cat" (https://en.wikipedia.org/wiki/Cat_%28Unix%29#Useless_use_of_cat). Couldn't we avoid that to satisfy everybody ?

Thank you VERY much!!!

using the link you provided I came to this:

awk '{ sum +=$1 } END { print sum }' < sample.data

now I believe everybody is satisfied =]

This will work regardless the 071

cat sample.data | awk '{ sum += $1 } END { print sum }'

Or...

in the end of the while loop: "done < <(sed 's/0*//' sample.data)"

It worked for me

SUM=$(expr $X + $SUM)

Yes it works. The only drawback here is we spawn a new process at each loop iteration. Can't we do better that that ?

The command declare, gets the number 071 as binary format in base 8 and interprets it to 57 (7*8 + 1). and that's because of the zero behind 71.

The problem can be solved by changing the command "declare -i SUM=0" to "typeset SUM=0".

It didn't work for me :

bash$ typeset SUM=0

bash$ SUM+=010

bash$ echo $SUM

0010

According to `help typeset`, this is just a synonym for `declare`. And my variable will NOT have the integer attribute, so the + operator will e a concatenation rather than an addition. Or did I missed something here ?

Nice one. 071 is interpreted as Octal, with decimal equivalent 57. The difference is 14 (decimal) which is the difference between the expected 536 and the surprising 522. To force interpretation of 071 as decimal, change the addition line to SUM+=$((10#$X))

Interesting.. can u pls explain sum+= command

Very great solution by Mario ! Specifying explicitly the base is probably the safest way to use numbers read from a file.

@Adithya, concerning the `SUM+=...` syntax, it is only a shorthand to `SUM=SUM+...`

It is quite command in programming language and probably was invented by Ken Thompson for the B language back in the late 60s (it is spelled =+ in B, but was rewritten += starting with the C language to avoid parsing ambiguities)

071 is being interpreted as octal (57). Force it to decimal (base 10): SUM+=$((10#$X))

Very good ! Congratulations.

Numbers startig with a zero are considered octal... (I ran into this little issue too often, in the past.)

Replying to myself - not nice, I know .*G*

As to how I would solve it... I usually do something like this:

SUM+=$((10#$X))

(Forcing bash to take $X as a decimal number.)

Yes, this is a common pitfall when reading external data in your shell scripts.

It is probably way more safe to always specify explicitly the base 10 using the `$((10#...)) syntax when using the Bash to perform calculations with number read from a file or file-like object.

I just added the 3rd line to force base 10.

declare -i SUM=0

while read X ; do

X=$((10#$X)) # force decimal (base 10)

SUM+=$X

done < sample.data

echo "Sum is: $SUM"

Yes, "forcing" the base 10 using the`$((10#...))` syntax is probably the safest solution when performing calculations with the shell from external data.

The problem:

In sample.data there is a number that starts with '0' (zero), so bash interprets the number as octal base 10.

The solution:

Remove the '0' (zero) on the beginning of the number modifying the SUM counter like this: SUM+=${X#0}

There is other solutions that can work better for other cases, like when the line starts with OctalBase#Number (ex.: 64#100) or '0x', that represents hexadecimal values, but I don't think that this is the case.

Nice use of the "Remove matching prefix pattern" syntax. But what if my data contained the `008` value ?

bash$ X=008

bash$ SUM+=${X#0}

bash: 08: 08: value too great for base (error token is "08")

Can't we fix that still using some kind of shell parameter expansion ? Or maybe using some optional shell features ? (see shopt)

you mean using extglob ?

shopt -s extglob

declare -i SUM=0

while read X; do

SUM+=${X##+(0)}

done < sample.data

echo $SUM

536