The standard deviation ? (sigma) is the square root of the average value of (X – ?)^{2}.

In the case where X takes random values from a finite data set x_{1}, x_{2}, …, x_{N}, with each value having the same probability, the standard deviation is

Assume we have an input file foo with f.ex. line number in first column and in the second column ($2 in awk) we have the values of interest.

File: foo

1 2

2 3

3 6

4 8

5 11

Use the one of the following awk commandos to calculate the standard deviation

awk ‘{sum+=$2; array[NR]=$2} END {for(x=1;x<=NR;x++){sumsq+=((array[x]-(sum/NR))^2);}print sqrt(sumsq/NR)}’ foo

awk ‘{sum+=$2;sumsq+=$2*$2} END {print sqrt(sumsq/NR – (sum/NR)^2)}’ foo

The result is

3.28634

#### Average

Here you may find how to calculate the average or arithmetic mean using AWK.

#### Minimum and maximum

Here you may find how to calculate the minimum and maximum values using AWK.

## 7 comments

## Calculate average using AWK says:

March 9, 2010 at 11:02 (UTC 0)

[…] you may see how to calculate the standard deviation using AWK. […]

## Madia says:

April 26, 2012 at 07:12 (UTC 0)

Yes.

Am given this code

#!/usr/bin/awk -f

{sum+=$3; array[NR]=$3}

END {for(x=1;x<=NR;x++){sumsq+=((array[x]-(sum/NR))**2);}

print sum/NR, " ", sqrt(sumsq/NR)," ",2*(sqrt(sumsq/NR))}

and told to Alter the code in the above example so that it calculates the mean and standard deviation for data in columns 3 to 8 and for records 500 to record 1000

The program should create a table of the data used in the computation and it should store the differences (between actual and mean) in the column following the last column of the input data file.

Thanx for your help

## ashi says:

October 14, 2012 at 05:37 (UTC 0)

very handy script, brilliant

## Vinay says:

November 20, 2012 at 03:13 (UTC 0)

I think there is a mistake in the code. You seem to be computing sum (x^2 – mean^2), which is not the same as (x – mean)^2. The correct value of std deviation is 3.6742.

## ove says:

April 23, 2013 at 11:08 (UTC 0)

No, I think it is correct. Try this:

awk ‘{x[NR]=$2; sum+=$2; n++} END{mean=sum/n; for (i in x){ss += (x[i]-mean)^2} sd = sqrt(ss/n); print “SD = “sd}’ foo

## enid says:

January 24, 2014 at 22:25 (UTC 0)

Use Excel, the result I got agrees with Vinay’s 3.6742

## ove says:

February 4, 2014 at 10:16 (UTC 0)

You should check out the page http://en.wikipedia.org/wiki/Standard_deviation, and if you try that example with the awk-code given here you will get the same result.

And, the formula given half way down the page (just prior to

Interpretation and application) is the same as the awk-code.

So, the awk-code given here is correct.