Lens Focus Repeatability and Calibration
Many people are under the impression that your camera/lens will auto-focus the same way each time. Nope, nope, nope. Camera designers have to live in the real world of “close enough”, “fast enough”, and “cheap enough”.
The holy grail of focus is to make sure your target gets inside the zone of acceptable focus. If your camera misses focus every time (and it probably will), it doesn’t really matter as long as the target is still in focus.
I’m going to show you some real-world measurements and what kind of compromises you need to make when evaluating and calibrating your focus. I’m assuming you have a camera that supports focus calibration.
It drives me crazy when people make claims about “facts” without the data to back it up and without giving you the tools to repeat the same experiments for yourself. What follows should be reasonably repeatable by anyone, without much expense involved.
As always, it bears repeating that “your mileage may vary”. Measurements are affected by the camera, lens, light level, aperture, target size, alignment accuracy, target distance, and stuff I haven’t even thought of.
I decided to try the experiment with two different cameras and two different lenses. I chose a Nikon D7100 with my Sigma 150-600 at 300 mm and a Nikon D610 with a Nikkor 24-70 at 70 mm. They’re both competent combinations, and should be representative of what an average camera enthusiast might use. I did all tests with the aperture wide-open, since stopping down would only obscure the results.
The Nikon D7100 has a focus sensitivity down to -2 EV, and the D610 has a focus sensitivity of -1 EV. That doesn’t mean you should perform a focus test there. I always use a light level of at least 10 EV for testing. I’m after focus repeatability, and repeatability will go out the window if you shoot in dim light. How far out of the window would be an excellent topic of study for another time.
Keeping with the theme of doing things by the numbers, I’m using my go-to analysis software MTF Mapper (version 0.5.13) by Frans van den Bergh. I used the “focus” option with his “mfperspective_a3.pdf” chart, printed to about 10” by 12” and mounted flat. The chart is oriented 45-degrees to the camera, to capture correct depth information. This arrangement gives me ample accuracy for evaluation. By the way, the camera pixel size doesn't matter for this particular test, but you need to remember to set it for the other measurements in the program options.
All of my photographs are made with un-sharpened RAW format. I de-focused the lens between each shot, and I alternated between too-near and too-far de-focus to exercise both directions of auto-focus. I always use back-button focus. I didn’t want any directional bias in the shots.
What the focus chart looks like.
The image above shows what gets photographed and analyzed. The chart left side is rotated farther away from the camera’s plane of focus by 45 degrees. The camera focus sensor is pointed dead-center on the chart.
Sample measurement from Nikkor 24-70 mm at f/2.8 and 1 meter.
The above picture shows how the MTF Mapper can measure the key elements in the chart and provide focus error measurements. In this picture, the camera missed focus by 2.8 mm at a distance of 1 meter. The camera focus sensor was aimed at the marker that’s under the vertical orange arrows. The depth of sharp focus for this lens/aperture/distance combination is about 15 mm. Anything inside the 15 mm focus window would be “success”.
It’s possible to make the measurements using only the red, blue, or green-sensitive sensor pixels in the photo, if desired. Lenses with significant longitudinal chromatic aberration will have focus peaks that are widely separated. For this experiment, all that is needed is to be consistent in using the same settings each time.
Sample measurement Sigma 150-600 at 300 mm f/5.6 and 4.22 meters.
The depth of sharp focus for this lens/aperture/distance combination is about 45 mm. Anything inside the 45 mm focus window would be “success”. Here, the camera missed focus by 0.7 mm, which is pretty much dead-on.
Sample results for Nikkor 24-70mm f/2.8 Test
The Nikkor 24-70 mm at 70 mm test follows. It should be noted that this lens is notorious for focusing differently at different focal lengths. This means that it is impossible to “fine tune” focus at a single value and have it correctly calibrated throughout the focal range. I have set a fine-tune value (+16) that under-compensates at 70 mm and over-compensates at 24 mm, with a bias toward 70 mm. The Sigma lens has far smarter firmware in it, and I have it calibrated at 4 different focal lengths and at 4 distance settings per focal length, for a total of 16 calibration fine-tune settings.
Measurement Errors Per Photograph (mm):
-8.1, -11.2, -10.6, -1.7, -8.4, -7.6, -5, -7.2, -4.5, -6.2, -7.3, -8.9, -1.9, -7.1, -1.1, -7.0, -4.3, 2.8, -2.6
N = 19, MEAN = -5.68 mm, STDEV = 3.55 mm
Given a “sharp zone” of about 15 mm (plus, minus 7 mm), I’d say that this test showed a focus miss about a third of the time. The mean of -5.68 mm is my “bias” focus error to help minimize the “+” focus error I get when zoomed to 24 mm.
The standard deviation of 3.55 mm is a measure of the typical magnitude of each focus “miss”. Call this repeatability. This is actually an impressive value. That's a typical error of 1 part in 282, or (1000 mm / 3.55 mm).
Bear in mind that the 15 mm “sharp focus depth” criteria is actually being quite picky. Also note that the focus chart target was only 1 meter away and the shots were with a wide-open aperture. Longer distances and/or stopping down would make the target zone quite sharp.
If I were to expect to spend the day shooting at 70 mm, I’d certainly adjust the focus fine-tune up to the maximum +20, and my focus miss rate would go toward zero.
Typical focus error plot, showing it needs more “+” focus fine-tune to drive it toward the vertical blue marker.
Sample results for Sigma 150-600 mm f/5.6 Test
The Sigma 150-600 mm at 300 mm test follows. In contrast to the Nikkor, this lens has focus fine-tune calibration throughout its zoom and focusing range (16 calibration fine-tune settings). It makes all the difference.
Measurement Errors Per Photograph (mm):
-0.7, -9.8, -14.5, -16.6, 1.1, 14.5, 0.6, -0.19, 9.4, 4.9, -15.3, 7.0, 4.9, -8.2, -12.0
N = 15, MEAN = -2.3 mm, STDEV = 9.8 mm
Given a “sharp zone” of about 45 mm (plus, minus 22.5 mm), I’d say that this test showed it NEVER missed the focus zone.
The standard deviation of 9.8 mm is a measure of the typical magnitude of each focus “miss”. Given the focal length and target distance, this is awesome. That's a typical error of 1 part in 431, or (4220 mm / 9.8 mm).
Bravo, Sigma.
Conclusions
If you really, really want to know how your lenses and cameras perform, these tests are representative of how you would do it.
Being an engineer myself, I never fail to be impressed at how far photographic technology has come. Twenty years ago, you couldn’t get this level of camera/lens performance at any price.
Thanks again, Frans, for your incredible MTF Mapper program.
