MAP and MLE : Relation to Least Squares

Estimation of parameters is a central problem in various fields of applied science. In robotics and machine learning, literally a good solution to a problem is cast as an optimization problem of some sort. This optimization problem involves a some observations, residue function and some optimization variables (this exact process is sometimes also referred as … Continue reading MAP and MLE : Relation to Least Squares

Marginalization vs Conditioning for Multivariate Gaussian Distribution

Gaussian distribution have these nice property that under linear transformation the resulting distribution is still a Gaussian distribution. This property is successfully exploited by the Kalman Filter. In a previous post I had explored the details of Extended Kalman Filter. Today I am exploring the intuitive meaning of Marginalization vs Conditioning for Gaussian distributions. It … Continue reading Marginalization vs Conditioning for Multivariate Gaussian Distribution

Linearization of Vector Valued Function

We often come across a vector valued function which needs linearization. Depending on the form though, doing it manually is very tedious and prone to error. In this post, let us try to use a computer algebra system, maxima and try to get a linear approximation of a vector valued function. Basic Theory The principle … Continue reading Linearization of Vector Valued Function

Sensor Fusion: Extended Kalman Filter (EKF)

Overview In this post I am going to briefly tell you about Kalman filter and one of its extensions to non-linear cases, ie. the Extended Kalman Filter (EKF). I will give a concrete example from Robotics on sensor fusion of IMU measurements and Odometry estimates from other SLAM algorithm. In robotics literature, this is referred … Continue reading Sensor Fusion: Extended Kalman Filter (EKF)

Convex Hulls of Special Euclidean Groups

Don't get bogged down by the heavy sounding title. Let's dissect the title first. "Special Euclidean Group" refers to the Euclidean transform aka the rotation and translation matrix together. Recall that the rotation matrix is a 3x3 matrix (9 numbers in all) but have special structure where the determinant of matrix need to be 1.0 … Continue reading Convex Hulls of Special Euclidean Groups

Alternating Minimization

Alternating minimization, is a simple and easy to implement method to compute minima of a function of 2 more variables. Although this looks like a heuristic, the convergence can be proved if the function you are trying to optimize follows the 5-point-property. In this post I am using the alternating minimization approach to finding numerical … Continue reading Alternating Minimization

Toy Gaussian Mixture Estimation with EM Algorithm

This terms quite often in computer vision related research papers. I am going to toyify it (a core simple explanation). All the code snippets are to be found along with the post. This is roughly a summarization of an excellent youtube-mini-series by Victor Lavrenko. The code snippets are my works. Preliminaries To understand this, you … Continue reading Toy Gaussian Mixture Estimation with EM Algorithm

List of Significant Probability Puzzles

I am collecting a list of probability/statistics/random processes puzzle problems. Click on the links for the description and solutions of the problem. It is well worth the effort to study, understand and derive the solutions to these. Helps sharpen the mind and increase your analysis skill. Please feel free to add your suggestion if I … Continue reading List of Significant Probability Puzzles

Part-3 : Controlling a Mechanical System

This is the third post in my series on control systems. In part-1 we explored what the differential equations can do for you and how to make use of a differential equation, particular the Euler-Lagrange equations to describe a mechanical system. In part-2, we derived these differential equations for our 'pendulum on a cart' system. … Continue reading Part-3 : Controlling a Mechanical System