How to symbolically solve the minimization problem in simple linear regression

I want to use mathematica to symbolically solve the minimization problem in simple linear regression:

Find argminα,βQ(α,β),for Q(α,β)=n∑i=1(yi−α−βxi)2,{\text{Find }}\text{arg}\min _{\alpha ,\,\beta }Q(\alpha ,\beta ),\qquad {\text{for }}Q(\alpha ,\beta ) =\sum _{i=1}^{n}(y_{i}-\alpha -\beta x_{i})^{2},

where yiy_i, xix_i, and nn are symbolic but not specific numbers.The expected answer would be something like

\hat {\beta }&={\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}}},\\
{\hat {\alpha }}&={\bar {y}}-{\hat {\beta }}\,{\bar {x}}

Can someone show some code example for doing this? Thanks in advance!




Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Take the tour! 3) When you see good questions and answers, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. Also, please remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign!
– Michael E2
Jul 9 at 21:48



Thanks Michael for your suggestions!
– CrossD
Jul 9 at 21:49



See (107426).
– MarcoB
Jul 9 at 21:54



What have you done so far? People are more likely to help if you show effort.
– Feyre
Jul 9 at 21:56



You’re welcome. Maybe someone else can help with your question. My experience has been that Mathematica’s facility with algebraic/symbolic sums is limited.
– Michael E2
Jul 9 at 21:56


1 Answer


The following is an adaptation of my answer to this question, which focused on the 3D linear least-squares problem.

(* Rules to get constants out of sums (or integrals etc) *)
outrules = {
Sum[f_ + g_, it : {x_Symbol, __}] :> Sum[f, it] + Sum[g, it],
Sum[c_ f_, it : {x_Symbol, __}] :> c Sum[f, it] /; FreeQ[c, x],
Sum[c_, it : {x_Symbol, __}] :> c Sum[1, it] /; FreeQ[c, x]

(*Generate the sum of squares*)
Sum[Expand[(y[i] – a x[i] – b)^2], {i, 1, n}];

(*Calculate the derivatives*)
Grad[%, {a, b}];

(*Use the linearity property of sums*)
Distribute /@ %;

(*Pull out any constants from summations*)
% //. outrules;

(*Set the derivatives equal to zero to generate a system of equations*)
Simplify[Thread[% == 0]];

(*Solve for the a, b parameters*)
Solve[%, {a, b}] // FullSimplify

These sums are not yet expressed as a function of (ˉx,ˉy)(\bar{x},\bar{y}), i.e. the average values of the (xi,yi)(x_i,y_i), respectively, as you have them in your question, but that should be a question of some algebraic transformations. Some such transformations are reported e.g. in this MathWorld document of least-squares fitting.