Cost Function (Stanford CS229)

發表於

Hypothesis

$$h_{\theta}(x)=\theta_0+\theta_1x$$

Parameters

$$\theta_0,\theta_1$$

Cost Function

$$J(\theta_0,\theta_1)=\frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2$$

Goal

$$\min_{\theta_0,\theta_1}J(\theta_0,\theta_1)$$

Simplification, let $\theta_0=0$

$$h_{\theta}(x)=\theta_1x$$
$$J(\theta_1)=\frac{1}{2m}\sum_{i=1}^m(\theta_1x^{(i)}-y^{(i)})^2$$

then goal will become

$$\min_{\theta_1}J(\theta_1)$$

  • Given data
    $(x^{(1)},y^{(1)})=(1,1)$
    $(x^{(2)},y^{(2)})=(2,2)$
    $(x^{(3)},y^{(3)})=(3,3)$

that is, $m=3$

  • evaluate $J(1)$

    • $\theta_1=1, h_{\theta}(x) = 1*x=x$
    • $J(1)=\frac{1}{2*3}\sum_{i=1}^3(1*x^{(i)}-y^{(i)})^2$
      $=\frac{1}{6}[(1-1)^2+(2-2)^2+(3-3)^2]$
      $=0$

  • evaluate $J(0.5)$

    • $\theta_1=0.5, h_{\theta}(x) = 0.5*x=0.5x$
    • $J(0.5)=\frac{1}{2*3}\sum_{i=1}^3(0.5*x^{(i)}-y^{(i)})^2$
      $=\frac{1}{6}[(0.5*1-1)^2+(0.5*2-2)^2+(0.5*3-3)^2]$
      $=\frac{7}{12}$
  • evaluate $J(-0.5)$
    • $\theta_1=-0.5, h_{\theta}(x) = -0.5*x=-0.5x$
    • $J(-0.5)=\frac{1}{2*3}\sum_{i=1}^3(-0.5*x^{(i)}-y^{(i)})^2$
      $=\frac{1}{6}[(-0.5*1-1)^2+(-0.5*2-2)^2+(-0.5*3-3)^2]$
      $=\frac{21}{4}$

then we can plot $J(\theta_1)$ out, by observing the figure, we get $\min_{\theta_1}J(\theta_1) = J(\theta_1=1)=0$, which means that when $\theta_1=1$, the squared error is equal to zero, i.e. $J(1)=0$, is perfectly fit to the data source given.