Chi-square test: the best known goodness of fit statistic. Chi-square is a way to numerically compare two sampled distributions:

• Some number of "bins" is selected, each typically covering a different but similar range of values.
• Some much larger number of independent observations are taken. Each is measured and classified in some bin, and a count for that bin is incremented.
• Each resulting bin count is compared to an expected value for that bin, based on the expected distribution. Each expectation is subtracted from the corresponding bin count, the difference is squared, then divided by the expectation:

2 =

The sum of all the squared normalized differences is the chi-square statistic, and the distribution depends upon the number of bins through the degrees of freedom or df. The df value is normally one less than the number of bins (though this will vary with different test structures). Ideally, we choose the number of bins and the number of samples to get at least ten counts in each bin. For distributions which trail off, it may be necessary to collect the counts (and the expectations) for some number of adjacent bins.

The chi-square c.d.f. tells us how often a particular value or lower would be seen when sampling the expected distribution. Ideally we expect to see chi-square values on the same order as the df value, but often we see huge values for which there really is little point in evaluating a precise probability.