Lens Testing - Doing your own simple lens tests

This article has been superseded by a longer and more detailed series of articles on how to evaluate a lens. You can read those articles starting HERE

Here is the original article:

There aren't many easy ways for a user to do their own lens testing without getting into a lot of scientific detail. Some people want the detail, and I'll deal with that in a future article, but in this article I'll deal with a non-technical technique for comparing lenses. Comparing lenses is a lot easier than making absolute measurements.

A simple method that doesn't require special equipment or knowledge is a blind comparative test under "normal" conditions for the user. If you aren't shooting digital I would strongly recommed the use of slide film (ISO 100 or slower) for testing. The average standard of printing by most photofinishers is bad enough to mask differences between lenses. If all you ever use is high speed print film, and all you ever get are small prints from the local drugstore, you can use them for your testing, but you are probably wasting your time and effort since you probably won't see much difference between lenses. If you shoot digital then, of course, you can examine your images on a monitor screen.

What is a "blind comparative" test? Well, you take a lot of shots of the same subject using different lenses. Preferably these should be taken to show the main subject the same size in each shot if possible. Clearly you can't do this with a landscape, but you can when photographing an animal (stuffed animals are good, since they don't move around much!), or a person or a flower. If you are testing a 400mm lens against a 100-300mm zoom, you will have to move in and out to keep the main subject the same size when you change focal lengths.

Take shots in full sun, shade, backlight subjects and so on to cover all the situations you normally work in. Maybe indoors with flash too. Always use a tripod. Take careful notes of which lens was used for each shot, and the aperture and shutter speed used. If you want to get semi-scientific about it you can use a resolution test chart as one of your subjects.

Film

If you are shooting film, when you get the slides back, write the taking conditions on each one (i.e. which lens was used for which shot, what aperture and shutter speed were used and so on - now turn the slides upside down and shuffle them! Look at each one in turn with a 5x or 10x loupe. If you don't have such a loupe, use a lens! A 50mm lens is a decent 5x loupe and a 24mm lens makes a decent 10x loupe. Put the slide on a light box, or some other source of even illumination and look at the it though the lens, with the back of the lens to your eye and the front facing the slide.

Digital

If you're shooting digital you can do much the same thing. Compare shots taken with different lenses without looking at the EXIF data to see which one was shot with which lens. Instead of looking at slides with a loupe, you can look at the digital images at 100% magnification. This should be enough to show even small differences.

Blind Testing

The test is "blind" because you don't know which lens was used for each shot while you are judging them. This prevents unconscious bias on your part. No matter how objective you try to be, the fact that you have just spent all your spare cash on a lens does tend to make you want it to be good, and so give it the benefit of any doubt. Sort all the shots of the same subject into order from "best" to "worst". This is the "comparative" part of the test. Do this for all the slides. Now look at the data to see which images/slides were shot with which lens at which aperture. Shots taken "wide open" will probably not be as good as ones taken when stopped down by a couple of stops. This is normal. At any given aperture however, if your new lens took all the "best" shots, you have a winner. If it took all the "worst" shots, you may have a loser, or you may be expecting too much, depending on which lenses you are comparing.

What's Normal?

When analysing the results remember that the quality of the image from a good $100 50mm f1.8 lens stopped down to f5.6 is probably at least as good if not better then that of a $600 400mm f5.6 lens used at f5.6. The extra cost of long lenses goes into bigger glass, not sharper images. The extra cost means you can back off from the subject 8 times further (and still have the same size image), while maintaining quality. You will see the truth in the statement that, when ever possible and whenever ethical, it's often better (from the point of view of image quality) to get closer with a short focal length lens than to back off and use a longer lens. It's certainly cheaper!

Things to look for are sharpness and contrast, as well as lack of flare and even illumination. To test for even illumination, take a shot of the clear blue sky. I'll bet any zoom and a lot of primes, when used wide open, will show darkening in the corners of the frame! Check the image quality in the corners of the frame too. Most lenses are significantly worse in the coners than in the center, so look carefully there. Lenses also differ in color balance. Some give "warmer" images (i.e they have a slight yellow cast), some are "neutral" and some give "cool" images (i.e. have a slight blue cast).

In the end, only you can decide if the lens meets your needs. No lens is perfect, it's just that some are better than others!