When using LinearModelFit[] we can generate a table of parameter values and statistics for the corresponding model using the property “ParameterTable”. Looking in the documentation I can’t find an explicit statement of the hypothesis test being performed for the p-values listed in this table. I would like to make sure I am interpreting these things properly. So for example if I produce the table

EstimateStandard Errort-StatisticP-Value10.01162610.001993625.831623.092`*10∧-8x$2374(1)−0.005730760.00207492−2.761910.0064413x$2374(2)−0.006141320.00209498−2.931440.0038851x$2374(3)−0.0003746030.00240746−0.1556010.87655x$2374(4)−0.002896210.00256897−1.127380.261323\begin{array}{l|llll}

\text{} & \text{Estimate} & \text{Standard Error} & \text{t-Statistic} & \text{P-Value} \\

\hline

1 & 0.0116261 & 0.00199362 & 5.83162 & \text{3.092$\grave{ }$*${10}^{\wedge}$-8} \\

\text{x$\$$2374}(1) & -0.00573076 & 0.00207492 & -2.76191 & 0.0064413 \\

\text{x$\$$2374}(2) & -0.00614132 & 0.00209498 & -2.93144 & 0.0038851 \\

\text{x$\$$2374}(3) & -0.000374603 & 0.00240746 & -0.155601 & 0.87655 \\

\text{x$\$$2374}(4) & -0.00289621 & 0.00256897 & -1.12738 & 0.261323 \\

\end{array}

how do I interpret these p-values? I would think at a 95% confidence level the first three parameters are statistically significant but the last two are not.

=================

=================

1 Answer

1

=================

Here’s how to interpret statistical significance, here illustrated for a χ2\chi^2 distribution:

Such a distribution tells you the expected probability of finding a value of xx as the sum of squares of values chosen from nn univariate Gaussian distributions (where we call nn the degrees of freedom). Of course, the value of xx can never be negative. The below Manipulate shows this distribution for different values of the settable degrees of freedom (denoted dfdf).

The xx corresponding to a particular pp-value (the “significance”) is the value of xx for which a percentage of pp cases lie above (>xx) in the distribution. If you obtain an experimental value xx above that criterion you can reject the null hypothesis, i.e., that the value could arise if the value were zero “by chance”. We write that value for instance χ20.01(3)\chi^2_{0.01(3)} which means the criterion value of xx for which 0.01 of the cases lie above that xx for 33 degrees of freedom. Adjust the pp value to p=0.05p=0.05 in the below Maniplate and see that the criterion value must drop.

Manipulate[Module[{xsol, x},

xsol = x /.

NSolve[1 – CDF[ChiSquareDistribution[df], x] == Significance,

x][[1, 1]] // Quiet;

Plot[

PDF[ChiSquareDistribution[df], x], {x, 0, 20},

PlotRange -> {0, .4},

ColorFunction -> Function[x, If [x > xsol, Pink, Lighter[Green]]],

ColorFunctionScaling -> False,

Filling -> Axis,

TicksStyle -> Italic,

Ticks -> {Range[0, 20,

1] \[Union] {{xsol,

Column[{” “,

Style[Subscript[\[Chi]^2,

ToString[Significance] <> “(” <> ToString[df] <> “)”],

Red, 12]}, Center], {-.05, .0}, Red}},

Range[0, .4, .1]},

AxesLabel -> {Text[Style[“x”, 14 , Italic]],

Text[Style[“probability”, 14 , Italic]]},

PlotLabel -> TextCell[

Row[{Style[

Subscript[\[Chi]^2,

ToString[Significance] <> “(” <> ToString[df] <> “)”]] ,

” = “, Style[ToString[xsol], Italic, 12]}]

]

]],

{{df, 3, Text[Style[“df”, Italic, 14]]}, 1, 30, 1,

AppearanceElements -> All}, {Significance, {0.01, 0.05}}]

Thanks for taking the time to write this code, but this is not quite what I was asking. The p-value in the parameter table is used for some type of hypothesis test. I assume the null hypothesis is the parameter is equal to zero. Based on the p-value we can accept or reject the null hypothesis. @ significance level .05 we would reject the null hypothesis for the first 3 parameters which says they add a statistically significant contribution to the model. I want to know if this interpretation is correct since the mathematica documentation never says what the null hypothesis is.

– Wintermute

Apr 30 ’15 at 0:21

2

@wintermute: The documentation does specifiy this. Read the “How to Get Results for Fitted Models” and “Statistical Model Analysis” tutorials, e.g., linked in the documentation for the fit functions…

– ciao

Apr 30 ’15 at 0:39