Hacker News

I just glossed through for now so might have missed it, but it seemed you pulled the process noise matrix Q out of a hat. I guess it's explained properly in the book but would be nice with some justification for why the entries are what they are.

Firstly I think the clarity in general is good. The one piece I think you could do with explaining early on is which pieces of what you are describing are the model of the system and which pieces are the Kalman filter. I was following along as you built the markov model of the state matrix etc and then you called those equations the Kalman filter, but I didn't think we had built a Kalman filter yet.

Your early explanation of the filter (as a method for estimating the state of a system under uncertainty) was great but (unless I missed it) when you introduced the equations I wasn't clear that was the filter. I hope that makes sense.

I read and enjoyed your book a few months ago when a friend recommened it to me. I've been interested in control theory for a few years, but I'm still definitely a beginner when it comes to designing good control systems and have never done it professionally.

I've been in the process of writing a tutorial on how PID filters work for a much younger audience. As a result, I've been looking back at the original tutorials that made stuff click for me. I had several engineers try to explain PID control to me over the course of about a year, but I don't think I really got it until I ended up watching Terry Davis (yeah, the TempleOS guy) show off how to use PID control in SimStructure using a hovering rocket as an example.

The way he built the concept up was to take each component and build on the control system until he had something that worked. He started off with a simple proportional controller that ended up having a steady state error with the rocket hovering beneath the target height. Once he had that and pointed out the steady state error, he implemented the integral term showed off how it resulted in overshoot. Once that was working, he implemented the derivative control to back the overshoot off until he had something that settled pretty quickly.

I'm not sure how you could do something similar for a Kalman Filter, but I did find it genuinely constructive to see the thought process behind adding each component of the equation.

Tangent but I love the accessibility menu you have. Made it super easy to tweak the page to be more readable for me

I recently (~6 mo ago) made it a goal to understand and implement a useful Kalman filter, but I realized that they are very tightly coupled to their domain and application. I got about half as far as I wanted, and took a pause. I expect your work here will get me to the finish line, so I am psyched! Thank you!

You lead with "Moreover, it is an optimal algorithm that minimizes state estimation uncertainty." By the end of the tutorial I understood what this meant, but "optimal algorithm" is a vague term I am unfamiliar with (despite using Kalman Filters in my work). It might help to expand on the term briefly before diving into the math, since IIUC it's the key characteristic of the method.

You could do a line extension of your product, like "Kalman Filter in Financial Markets" and sell additional copies :)

I feel like people overcomplicate even the "simple" explanations like the OPs and this one.

Basically, a Kalman filter is part of a larger class of "estimators", which take the input data, and run additional processing on top of it to figure out the true measurement.

The very basic estimator a low pass filter is also an "estimator" - it rejects high frequency noise, and gives you essentially a moving average. But is a static filter that assumes that your process has noise of a certain frequency, and anything below that is actual changes in the measured variable.

You can make the estimator better. Say you have some idea of how the process variable should behave.For a very simple case, say you are measuring temperature, and you have a current measurement, and you know that change in temperature is related to current being put through a winding. You can capture that relationship in a model of the process, which runs along side the measurement of the actual temperature. Now you have the noisy temperature reading, the predicted reading (which acts like a mean), and you can compute the covariance of the noise, which then you can use to tune the parameter of low pass filter. So if your noise changes in frequency for some reason, the filter will adjust and take care of it.

The Kalman filter is an enhanced version of above, with the added feature of capturing correlation between process variables and using the measurement to update variables that are not directly measurement. For example, if position and velocity are correlated, a refined measurement on the position from gps, will also correct a refined measurement on velocity even if you are not measuring velocity (since you are computing velocity based of an internal model)

The reason it can be kind of confusing is because it basically operates in the matrix linear space, by design to work with other tools that let you do further analysis. So with restriction to linear algebra, you have to assume gaussian noise profile, and estimate process dependence as a covariance measure.

But Kalman filter isnt the end/all be all for noise rejection. You can do any sort of estimation in non linear ways. For example, I designed an automated braking system for an aircraft that tracks a certain brake force command, by commanding a servo to basically press on a brake pedal. Instead of a Kalman filter, I basically ran tests on the system and got a 4d map of (position, pressure, servo_velocity)-> new_pressure, which then I inverted to get the required velocity for target new pressure. So the process estimation was basically commanding the servo to move at a certain speed, getting the pressure, then using position, existing pressure, and pressure error to compute a new velocity, and so on.

Kalman filters have always been about state estimation. What you consider an exception is the default in the vast majority of state estimation scenarios.

Before I got into control theory, I've read a lot of HN posts about kalman filters being the "sensor fusion" algorithm, which is the wrong mental model. You can do sensor fusion with state estimation, but you can't do state estimation with sensor fusion.

That's a good article. I also like the visual approach there. My goal here was a bit different. I walk through a concrete radar example step by step, and use multiple examples throughout the tutorial to build intuition and highlight common pitfalls.

I agree that Kalman filters are not magic and that having a reasonable model is essential for good performance.

Higher sampling rates can help in some cases, especially when tracking fast dynamics or reducing measurement noise through repeated updates. However, the main strength of the Kalman filter is combining a model with noisy measurements, not necessarily relying on high sampling rates.

In practice, Kalman filters can work well even with relatively low-rate measurements, as long as the model captures the system dynamics reasonably well.

I also agree that it's often something you design into the system rather than applying as a post-processing step.

Thats just a consequence of sample rate as a whole. The entire linear control space is intricately tied to frequency domain, so you have to sample at a rate at least twice higher than your highest frequency event for accurate capture, as per Nyquist theorem.

All of that stuff is used in industry because a lot of regulation (for things like aircraft) basically requires your control laws to be linear so that you can prove stability.

In reality, when you get into non linear control, you can do a lot more stuff. I did a research project in college where we had an autonomous underwater glider that could only get gps lock when it surfaced, and had to rely on shitty MEMS imu control under water. I actually proposed doing a neural network for control, but it got shot down because "neural nets are black boxes" lol.

Yeah, I try to err on the side of not using them unless the accuracy obtained through more robust methods is just a no-go, because there are so many ways they can suddenly and irrecoverably fail if some sensor randomly produces something weird that wasn't accounted for. Which happens all the time in practice.

That's a fair question. My goal with the site was to make as much material available for free as possible, and the core linear Kalman filter content is indeed freely accessible.

The book goes further into topics like tuning, practical design considerations, common pitfalls, and additional examples. But there are definitely many good free resources out there, including the one you linked.

Huge +1 for Roger Labbe's book/jupyter notebooks. They really helped me grok Kalman filters but also the more general problem and the various approaches that approximate the general problem from different directions.

i haven't seen much from other kalman filter resources, but i can say that this book is incredibly detailed and i would highly recommend it

if you dont want to buy the book, most of the linear kalman filter stuff is available for free: https://kalmanfilter.net/kalman-filter-tutorial.html

That link is a classic!

There are not many good resources on Kalman filters. In fact, I have found a single one that I'd consider good. This is someone who has spent a lot of time to newly understand Kalman filters.