For a general optimization problem, it usually could be rewritten as maximizing or minimizing a certain function with several constrictions. For example, maybe you want the optimized value non-negative. The most basic form of such is called the primal question which looks like this:
$$ \begin{matrix} \underset{x}{min}f(x), x \in \mathbb{R}^n \newline s.t. \newline g_i(x)\leq 0, i=1,2,…,m \newline h_i(x)= 0, i=1,2,…,p \end{matrix} $$
For simple linear regression:
$$ E\left ( Y|X=x \right ) = \alpha +\beta x $$
we have:
$$ \hat{\beta } = \frac{cov\left ( X, Y \right )}{var\left ( X \right )} $$
$$ \hat{\alpha} = \bar{Y} - \hat{\beta}\bar{X} $$
We can all use the NN method to solve the regression problem but that leads to being nearly impossible to locate exactly which layer foreshadows which feature of the data. Thus, maybe the better way is to upscale the dimension of the linear regression method. That, we not only use $ x $ but $x, x^{2}, x^{1/2}… $to approach the true curve.
When I was in the high school learning about AP statistics I learned the formula:
$$ P(A|B)=\frac{P(A\cap B)}{P(B)} , P(B|A)=\frac{P(A\cap B)}{P(A)}$$
$$ P(A\cap B)=P(A)\cdot P(B|A)=P(B)\cdot P(A|B) $$
$P(A|B)$ is called “Conditional probability” which pretty much self-explained itself. For which I only knew the meaning of each element but not the whole idea, what I do is just plug in numbers, because it is kinda abstract to understand from itself: “The probability of event $A$ happens given event $B$ happened = The probability of events $A and B$ happens divided by the probability of event $B$ happens”
Lately, I was into the studying process of machine learning, and outputting(taking notes) is a vital step of it. Here, I am using Andrew Ng’s Stanford Machine Learning course in Coursera with the language of MATLAB.
So the rest of the code I will write in this post by default are based on MATLAB.
“A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” Tom Mitchell