Null space The black hole of Math
28/08/2022Recovered from Medium
Linear algebra is one of my favorite subjects. It is one of the most important branches of mathematics and can be applied to many other applications: solving linear equations, systems of differential equations, and more.
When I was taught this null space concept, it was part of a routine to make sure we covered all the course content. Teachers put a checkmark next to each concept and you’re good to go! There is usually no interest in understanding what null space is, or why it might be useful. We usually are only interested in doing mathematical exercises without having any sense of the reason behind them.However, I enjoyed asking myself what is this for and why we learn about this but is it never used?
But what is null space?
Null space is the set of all vectors which satisfy a given linear equation. In other words, null space is the set of all solutions to a linear system of equations equal to zero.
In other words, all vectors in a null space are mapped to zero by the matrix. This is an information killer and you can’t reverse this information once it falls in null space.
General context
In mathematics, it is very common to use transformations, be they matrices, vectors, derivatives, or equations… to map from one domain to another. For example, the Laplace transform transforms maps functions in power terms to oscillatory terms.
A matrix is ​​not the exception, what it does is make a change of “perspective”, a change of reference, a change of vectors’ basis, or a change of dimensions…
When the determinant of a matrix is ​​different from zero, we have a reversible mapping… this means that we can transform objects without losing information about them since when performing the inverse transformation we obtain the “original” mat value or object again.
The problem is when the determinant is equal to zero, many stop there as soon as they find this. The most common thing is to say: “This cannot be done” or “This has no solution”. But that is wrong, actually, it does have a solution but it is not unique, they are multiple, in fact, they are infinite, but that they are infinite does not mean that they are any you choose. Those solutions are the null space.
What is it for?
The desire usually is to find a single answer and not too many. This is because one of them is more relevant than the others (optimization). But what if you want to understand this having an infinity of answers? At least not stop because the determinant is zero. You should know about null space then.
Technically
...
Optimization
...
What happens when determining is equal to zero?
...
What has this to do with the information?
...
Why do neural networks learn?
...
Conclusion
Now you should know the insights of null space and be able to answer those questions from the beginning: Why do neural networks learn? What does the null space have to do with the information? Why does a system of linear equations have no solution when the determinant is zero? This also could be used as a mind tool for reinterpreting mapping, transformations, and information processing. We could see the learning process not only about memorizing information but ignoring some of it in this black hole of math called null space.