Lens Variance vs Bad Copy

This post was heavily influenced by and will reference an excellent two part article that the great Roger Cicala published at DPReview.com in 2020.  

• Difference Between Sample Variation and Bad Copies Part 1
• Difference Between Sample Variation and Bad Copies Part 2

Why Lens Variance vs Bad Copy?

I get the question very regularly, probably about once a week, of which lens a photographer should buy.  I get why that question comes up so much.  Lenses are a big investment and especially when people are closer to the beginning of their photography journey it can be hard to figure out what all the letters and numbers on the lenses mean.  They need help to decide.

Inevitably I see this question answered with a biased, anecdotal answer of something like 

• I would NEVER buy X lens because I got a bad copy and had to send it back
• I would NEVER buy X lens because my friend went through three of them, got their money back, and bought Y lens that worked great

I know I am personally biased towards Tamron.  I am a Tamron fanboy.  I love my Tamron lenses.  Great quality at a very reasonable price!  Speaks right to my nerdy hobbyist heart.  Due to this bias, I am glad there is someone like Roger who can speak to this as more of an expert than almost anyone else on the planet.  He works for lensrentals.com and has an amazing amount of experience and unbiased opinions backed up by science and data.  I love to read Roger’s articles.

Roger starts off his first article by saying he wants to answer the “seventh most common forum war; the ‘lens variation is a big problem!’ vs ‘I don’t believe it exists!’ argument”. I will similarly point to this post when I see this come up in forums and groups going forward.

The science and data really takes the controversy right out of the topic.

What Is Lens Variance?

I am thoroughly amazed at modern manufacturing processes.  I think it is a miracle that the cameras and lenses we have work at all.  Yet, as amazing as modern manufacturing processes are, they have been deliberately built to allow for small variations in the final product.

These variances can be the result of cost cutting measures, or the result of technical limitations where we simply do know how to do it any better.  Whatever the reason for the variation, there is going to be some in the results of every manufacturing process.  With some products those variations are so negligible they really do not matter.  With other products it can be more meaningful.

I love what Roger shared in his articles about lens variation.  Lens manufacturers are often caught between a rock and a hard place.  They have to decide between a good performing lens at a little lower price point or a very high performing lens at a significantly higher price point.  Each of these decisions have pros and cons, but no matter the decisions every lens is going to come off a manufacturing line with at least a little bit of variation.

Manufacturers can also choose to include optical adjustments in their lenses.  These adjustments come at great cost and can help compensate for small variations, but they do not result in perfect lenses. Here is what Roger had to say about the adjustments that are sometimes offered on lenses:

“I want to emphasize that optical adjustments in a modern lens are not there so that the lens can be tweaked to perfection; the adjustments are compensatory. There are trade-offs. Imagine you’re a technician working on a lens. You can correct the tilt on this element, but maybe that messes up the spacing here. Correcting the spacing issue changes centering there. Correcting the centering messes up tilt again. Eventually, in this hypothetical case, after a lot of back-and-forth you would arrive at a combination of trade-offs; you made the tilt a lot better, but not perfect. That’s the best compromise you can get.”

What Is a Bad Lens Copy?

When compared to Roger my own experience with cameras and lenses is very limited, but I have long held a gut feeling that there are far fewer “bad copies” of a lens impacting image sharpness than photographers want to believe.  I know I have hoped it was the gear and not me behind the camera that was to blame for poor images over the years.  I like how Roger defines a bad copy of a lens:

“A bad copy of a lens has one or more elements so out of adjustment that its images are obviously bad at a glance. Such a lens (assuming it is optically adjustable) can usually be made as good as the rest.”

Bad at a glance.  Bad does not mean small flaws you may pick up when you pixel peep at 200%.  When a lens is bad, it is really bad.  Obviously bad.  When it comes from Roger, I take it as gospel, even when others in forums or groups say otherwise.

Roger didn’t say if they are testing every new lens they get and every lens as it comes back from a rental, but he has done thousands of lens performance tests over the years.  The quantity of tests he has done makes his experience and information significantly more valuable than any of those folks in photography groups or forums telling you about their experience with a lens, or two, or ten.

With that said, I pay special attention to Roger’s experience as a result of all that lens performance testing when he talks about bad copy rates:

“…the actual, genuine ‘bad copy’ rate is way lower than I showed in the graph above. For high-quality lenses it’s about 1% out-of-the-box. This explains why I roll my eyes every time I hear ‘I’ve owned 14 Wonderbar lenses and they’re all perfect.” Statistics suggest you’d need to buy over 50 lenses to get a single bad one. The worst lenses we’ve ever seen have a bad copy rate of maybe 3% so even then, the chances are good you wouldn’t get a bad one out of 14.”

How Do I Know If My Lens Is a Bad Copy?

Time for some hard truth.  If you are having trouble with sharpness, you have only about a 3% chance the issue has to do with the lens.  I get it, we are all searching for the easy button.  It is so much easier to think that we got a bad copy of a lens that doesn’t work correctly than to admit it could be the photographer to blame for the lack of sharpness or other image quality issues.

Still, there is a 3% chance your lens is a bad copy.  So how are you going to tell if a sharpness problem is the lens or something else?  With what Roger has said in his article it will be entirely obvious when the lens is the problem.  So obviously bad you should instantly be able to judge by the back of the camera when you review your shots that something is very, very wrong.

If that is not the case, if you have to zoom into your photos 100% or more to see issues, the lens is not a bad copy.  Sharpness issues are most likely the result of technique or camera settings.  Check out my Photographer’s Checklist For Sharper Photos for help on the most common mistakes photographers make that reduce sharpness.

Should I Send My Lens In To The Manufacturer?

What if you suspect your lens is off a little bit?  Not enough to be a bad copy for sure, but enough sharpness issues even after being sure you are nailing all of the checklist for sharpness above? Should you send the lens in to the manufacturer for cleaning/checkup? 

Over the past few months here late in 2020 I have done a lot of testing of my cameras and lenses.  Not to the level of Roger Cicala of course, but the FoCal software from Reikan is a pretty inexpensive and easy to use tool to do a little bit of testing of a lens on your camera.  If you really want to find out how your lens is doing with your camera, this software is really good.

Over the past three months, whenever I am not using my camera I have had it setup in my DIY home studio to test out my lenses.  I have put over 40,000 shutter actuations on my camera as I have done this testing and I will be doing an episode as soon as I am done digging in this rabbit hole, but the message to share right now is that I decided I needed to send my Tamron 24-70 G2 lens in to Tamron to have them take a look at it.

I wasn’t concerned my lens was a bad copy.  Image quality has never been so obviously bad that could be the case.  What I was getting from my testing using the FoCal software was that my lens was significantly softer at the 70mm end than it was at 24mm.  Significantly here meaning 20% softer.

I know this softness is not an issue with autofocus or needing to do the autofocus micro adjustment (AFMA) process to calibrate my specific Tamron 24-70 G2 lens to my specific Canon 80D camera because my testing to find this was done using live view for focus.  The challenges with a focus sensor not being calibrated to a lens needing AFMA adjustment only comes when you use the viewfinder (the little eyepiece you put up to your eye when shooting) on a DSLR camera.

I had never heard of increased softness on one end of a lens before.  Not having enough experience with a lot of lenses, I decided that since the lens is still under warranty I would send it in to Tamron and see if having them take a look at it might improve this discrepancy.  Maybe a small adjustment made by Tamron would make the 70mm end of the lens come closer to the sharpness I had been getting at the 24mm end.

While I had the lens out to Tamron, I had a shoot come up, so I decided this would be a good time to get some more experience with another lens.  I rented a Sigma 24-70 Art lens to see how it tested on the same camera and made sure I had a high quality lens for my shoot.  The Sigma tested very similarly to the Tamron lens.  Softer at 70mm than at 24mm.  Now that I had seen this with two lenses from different manufacturers, I felt confident this increased softness would remain when I got my Tamron lens back.

I have done enough testing since getting my Tamron lens back now that I can confirm the increased softness at 70mm is still there.  In fact, the sharpness numbers I am getting in my testing using FoCal software is the same after having sent it in as it was before.

If your lens is under warranty and you can afford to be without it for a few weeks (took 4 weeks for me during a COVID year that I am pretty sure slowed things down), other than the cost of shipping there is no downside to sending it in to see if the manufacturer might be able to do something with a lens you suspect is having issues.  Just don’t expect that to work miracles and really take a look at that sharpness checklist.

Reminders

• Email the show phototacopodcast@gmail.com
• Join the Photo Taco Podcast Facebook group
• Find Jeff’s work at https://www.jsharmonphotos.com.  Check out his Photo Taco podcast over at https://phototacopodcast.com where you can search all kinds of topics and find shows discussing the details.  He is on Facebook: https://www.facebook.com/harmon.jeff, Instagram: https://www.instagram.com/harmonjeff/ (@harmonjeff), and Twitter: https://twitter.com/harmon_jeff (@harmon_jeff)   

Podcast: Download (Duration: 37:08 — ) | Embed

Subscribe: TuneIn | RSS