## Exercises

### Exercise 1

```{exercise-start}

:label: ex1

```

Show that, for any linear subspace $S \subset \mathbb R^n$, $S \cap S^{\perp} = \{0\}$.

### Exercise 2

```{exercise-end}

```

```{solution-start} ex1

:class: dropdown

```

If $x \in S$ and $x \in S^\perp$, then we have in particular

that $\langle x, x \rangle = 0$, but then $x = 0$.

```{solution-end}

```

```{exercise-start}

:label: ex2

```

Let $P = X (X' X)^{-1} X'$ and let $M = I - P$. Show that

$P$ and $M$ are both idempotent and symmetric. Can you give any

intuition as to why they should be idempotent?

### Exercise 3

```{exercise-end}

```

```{solution-start} ex2

:class: dropdown

```

Symmetry and idempotence of $M$ and $P$ can be established

using standard rules for matrix algebra. The intuition behind

idempotence of $M$ and $P$ is that both are orthogonal

projections. After a point is projected into a given subspace, applying

the projection again makes no difference (A point inside the subspace

is not shifted by orthogonal projection onto that space because it is

already the closest point in the subspace to itself).

```{solution-end}

```

```{exercise-start}

:label: ex3

```

Using Gram-Schmidt orthogonalization, produce a linear projection of $y$ onto the column space of $X$ and verify this using the projection matrix $P := X (X' X)^{-1} X'$ and also using QR decomposition, where:

$$

y :=

\left(

\right)

$$

## Solutions

### Exercise 1

If $x \in S$ and $x \in S^\perp$, then we have in particular

that $\langle x, x \rangle = 0$, but then $x = 0$.

```{exercise-end}

```

### Exercise 2

Symmetry and idempotence of $M$ and $P$ can be established

using standard rules for matrix algebra. The intuition behind

idempotence of $M$ and $P$ is that both are orthogonal

projections. After a point is projected into a given subspace, applying

the projection again makes no difference. (A point inside the subspace

is not shifted by orthogonal projection onto that space because it is

already the closest point in the subspace to itself.).

### Exercise 3

```{solution-start} ex3

:class: dropdown

```

Here's a function that computes the orthonormal vectors using the GS

algorithm given in the lecture

Again, we obtain the same answer.

```{solution-end}

```