ULTIMATE Guide to Backup for Photographers

Jeff refreshes information from his 2015 article, the ULTIMATE Backup Workflow for Photographers, with information from 2018 including the two “storage walls” photographers slam into in their photographic journey and how to recover from hitting both of them.

First Advertiser – Udemy: This episode is brought to you by Udemy*, the largest marketplace for online learning. Whether you’re looking to learn something new or just sharpen your skills, Udemy has over 65,000 courses starting at just $11.99. Students around the world choose Udemy to discover new passions, expand their skills, and even change careers.  Visit Ude.my/IMPROVE or download the Udemy app to learn anytime, anywhere.

Please subscribe to the Photo Taco feed if you aren’t already because by somewhere in June 2018 this podcast will show up far less frequently here on the Improve Photograpy podcast feed.  You can go to https://phototacopodcast.com to find the links you can use to subscribe using the most popular podcatchers.

To add a little incentive to subscribe, I am running a contest!  The prize is a 30 minute Skype session with me where I will process as many of the winner’s photos as we can get through in those 30 minutes.  The winner will send me a few of their raw files and I will process them in Lightroom and Photoshop and explain my full thought process behind why I am doing the adjustments and edits that I am doing.  Prove that you have subscribed to the Photo Taco podcast in your favorite podcatcher by emailing a screenshot showing your podcatcher saying you are subscribed to phototacopodcast@gmail.com.  That will enter you once into the contest.  Only one entry per subscription, so please do not send one subscription multiple times or I will have to disqualify you from the contest.  You can get two more entries into the contest by having a friend do the same things, emailing a screen shot of their subscription to phototacpodcast@gmail.com along with a note in the email that you got them to do that.  Contest starts May 14th, 2018 and goes through 11pm Mountain on July 6th, 2018.  You can find all of the information about how to enter the contest at https://phototacopodcast.com/contest.

Let’s get into the real reason you are listening today, I want to go through the ULTIMATE guide to backup for photographers today.  Next to questions about new updates to Lightroom, Photoshop, Windows, and macOS being safe, this is the second most prevalent topic for questions I get from listeners.  Incidentally, if you use both Windows and Wacom tablets and have noticed that the Select and Mask workspace keeps closing out on you before you are done changing your selection, I have a bug into both Adobe and Wacom on the issues.  Adobe has told me that they have confirmed with Wacom engineers the issue lies in the Wacom driver for Windows and that they are testing out a fix. Everything works as it should on macOS, but in Windows if you zoom in or out using the touchpad part of the tablet or the ring to the left of the tablet the driver sends the ESC keycode and it closes the Select and Mask workspace.  No ETA on the fix, but I will put a link in the show notes to the Photoshop Feedback forum post where I have documented the issue thoroughly. Just a little nugget of info you get from listening to Photo Taco.

So next to those technical questions about updates to software, backup is the next most prevalent question that I get from listeners, and I totally understand why.  Backup isn’t easy. To do it right takes some investment in hardware, and if you have been doing photography for more than a couple of years it means a fairly significant investment.  To make matters worse, you really won’t want to solve this problem once because making that investment as you are just getting started as a photographer doesn’t make any sense.

Storage of your photos in general quickly becomes a problem and for most it is like a race car driving slamming into a wall, not just once but twice.  The first time is when you fill up that hard drive on your computer. Bam, right into the first wall. Many listeners have contacted me in a panic because they aren’t sure how to solve this problem.  Fortunately I have an article I wrote over at http://improvephotography.com called The Ultimate Backup Workflow for Photographers that goes through a lot of the details and offers practical advice on how to deal with the wall they just hit.  What they don’t realize is not only do they have to figure out how to get more storage for their photos, the part of the wall they didn’t see as they hit it was that they need to make sure backup is part of the solution.  I shared that article just this past week here in mid May 2018 with Dawana Caldwell, one of the great listeners who are in the Photo Taco Podcast Listeners Facebook, and she said it was exactly what she needed. Connor Hibbs, the incredible portrait photographer who co-hosts the Portrait Session podcast with Erica Kay here on the Master Photography podcast network, also posted to the Improve Photography Facebook group a very sad story himself this week about how he had not one, but two hard drives fail.  He was fortunate that he figured out a way to get out of the situation but it was a real wakeup call for him that he is not in a good situation with his backup.

You will want to check the show notes for this episode over at https://phototacopodcast.com for a link to the article to the Improve Photography article.  In short, my advice is to start off small and with the least amount of expense to solve this problem of slamming into that wall by buying a good USB hard drive.  By good I mean one that at minimum has a USB 3.0 connection. Thunderbolt would be faster, but it is also going to be more expensive. The article outlines the different connection speeds so that you can figure out for yourself what you want to use but USB 3.0 is a good solution for both PC and Mac users.  My recommendation here in mid-May 2018 is the Western Digital My Passport USB 3.0 drive that runs about $100.  That is the largest size Western Digital offers in that line of drives right now and will be plenty to get you peeled off that wall and back to doing photography.  If you are a PC user and don’t mind getting inside the computer, then I recommend using a 4TB internal HGST drive, but that takes a bit more know-how than I am going to go through in this episode.

Problem is you can’t buy just one of them.  At least not if you want to keep your photos safe.  You will need to buy two of them so you can setup a 3-2-1 backup solution in addition to expanding the storage you need for your photos.  3-2-1 is a memorable way to think about backup done right that was created by Carnegie Mellon and has been adopted pretty universally now, including the United States Computer Emergency Readiness Team, referred to as US-CERT.  When you think of 3-2-1 backup you should think of 3 copies of your photos, kept on 2 different types of media, and 1 of them stored offsite. So let’s walk through how to get 3-2-1 backup with your two new external hard drives.

An unfortunate part of hitting that storage wall is that you are going to have to change your workflow.  Just prior to slamming into that wall you were very happily putting your memory card into your computer and then letting Lightroom pull the photos off the card and put them wherever it decided to have them go on the hard drive inside your computer.  Problem is that filled up your hard drive in your computer and you can’t do that anymore. So you are going to have to move the photos from your computer to one of your new external hard drives – let’s call this your primary drive. I recommend you move those photos using Finder on Mac or Explorer on Windows rather than doing this in Lightroom for both speed and accuracy – Lightroom has been known to lose files.  This might not be too easy if you haven’t done a good job of organizing your photos on the drive in your computer. If that is you, then you will want to check out another Photo Taco episode I did with the Lightroom Queen called Lightroom Organization.

Once the photos are moved to the external drive, you have them in one place that is safe for now.  Next thing to do is get them in a second place so that when that primary drive fails you have them on a secondary drive.  After all, two drives don’t fail at once – oh wait, they did for Connor just this week. This is why the advice is to have 3 copies of the file on two different media types and one being offsite.  We haven’t met all of those requirements yet.

In fact, we haven’t really made sure we will have even two copies of the photo yet.  You could copy them to each yourself, and if you are disciplined enough to do that, then that’s great.  I am not, and I suspect that if you really thought about it you aren’t going to be either. So, you need to find a solution that will automatically copy whatever you put on that primary drive over to the secondary drive.  I am using the free Microsoft SyncToy application to do this for me, has worked flawlessly for many years now. Mac has a free tool called Automator built into macOS that can be setup to do this too, both need a little bit of technical knowledge to make it happen.  There are commercial applications you can buy on both Mac and Windows to help here, though I haven’t personally used any of them to be able to recommend anything. Most USB hard drives you buy will come with a utility you can use to accomplish the same thing, so check that out as an option as well.

The thing you need to accomplish is to have everything you add new to that primary drive automatically get copied over to the secondary drive at least nightly if not more often.  I have SyncToy setup to be run by the Windows 10 Task Scheduler every night at 11pm and each time it runs SyncToy looks for new files that have been added to my primary drive and copies it over to the secondary drive.  No thought required, which is exactly what I need. I am using internal drives directly connected inside my Windows PC, so if you are using external drives you do have to remember to plug them in, which can be a pain if a laptop is your main computer.  I promise the effort is worthwhile.

With the photos automatically being copied from the primary to the secondary drive we can feel good that we have 2 copies of the photos.  Now what about the 3 and the 1 in the 3-2-1 backup strategy? Remember you need three copies of your photos on two different media types with one offsite.  My recommendation for those that hit this first storage wall is to use an online backup service to get your third copy of the file on a different media type (in this case cloud) that is by nature offsite.  I am personally using BackBlaze for this and can very highly recommend their service for this. You get unlimited backups to BackBlaze, including as many external hard drives as you can directly connect to your computer, for $5/mo, $50/year, or $95/2 years.  BackBlaze is not sponsoring this episode and I am not getting their service for free so this is not a paid advertisement. It is simply a product that works very well for me. Not that there can’t be issues because there can. Every time I have been asked by listeners about what cloud backup service I recommend I hear from one or two listeners who have had negative  experiences with the service. Like anything in technology, things can go wrong and with enough people using it something certainly will. This is why cloud backup alone is not sufficient and you really need that local backup with the two external drives.

I did a Photo Taco episode with the head of marketing over at BackBlaze to talk about challenges photographers face with cloud backups and how it is the company plans to stay alive when so many of their competitors have failed.  You can check that out that episode by searching “photo taco cloud backup” or finding the link in the show notes. There are other solutions many listeners have told me about and I have tried. Amazon Prime includes unlimited photo storage, including most kinds of Raw files and there is a client that will watch a folder for new files and automatically sync them to the cloud.  Google offers something similar but to my knowledge does not consider raw files to be photos, so you would be paying to store those and the costs are going to be a lot higher. I also found the Google sync client to be flaky until just recently. They redesigned their sync client and it is working much better now. DropBox is another common solution I have heard talked about, which would work great and has some other benefits like being able to have your photos and your Lightroom catalog synced across multiple computers this way.  It is also going to be quite a bit more expensive and you have to be very careful that you don’t run Lightroom on those multiple computers at the same time or your catalog can get corrupted and you will lose your processing. If you don’t understand what that means, don’t use DropBox that way. I need to do a Photo Taco episode on how to set that up.

Also, be aware that most ISPs will throttle your connection to any backup provider.  BackBlaze, Amazon Drive, Google Drive, DropBox, no matter which one many ISPs will recognize that traffic as being backup and will slow it down.  For me it looked like the initial sync of my photos, which was 3TB at the time, was going to be done in less than a month. Within a couple of days my ISP, Comcast, throttled the connection and it took 8 months for me to sync 3TB of data.  It had nothing to do with BackBlaze as a cloud backup provider. Since that time Comcast has also changed policy for my area here in the Utah that we have bandwidth caps at 1TB/mo and I get charged more if I use more. So you will want to find out if your ISP has any caps like that and configure you backup client that automatically syncs files from your primary drive out to the cloud won’t use more than those limits.

With two external drives and a cloud service you now have a 3-2-1 backup solution in addition to a place to hold 4TB of data.  Your photo will exist in 3 places – once on the primary external drive, once on the secondary external drive, and once in the cloud.  It is on 2 different types of media with one being local hard drives and the second being cloud, which is also what makes the 1 be met because the cloud is offsite.

Whew.  This wouldn’t be an ultimate guide though if we don’t get past that, which is what I am going to do after thanking a sponsor for this episode – WeTransfer 40 million people use WeTransfer to send and receive files every month.  Since day one, they’ve devoted 30% of their ad space to showcasing creative people from around the world–from musicians, to photographers, to illustrators, to robotic textile creators, to podcasters like us. So in that spirit, we’re skipping the rest of this 60-second ad and getting right back into the podcast. WeTransfer.com.  You make. We Transfer.

I know this episode is a little long but I promised the ULTIMATE guide here so we now have to talk about the second storage wall photographers hit.  This one doesn’t come out of nowhere as much as the first one does, after all you hit that first one and so you have an idea when you solved that problem that it was really more like kicking the can down the road.  So what do you do when you outgrow that 4TB solution I have already gone over? You could replace the 4TB drives with 8TB drives and kick the can down the road a little further, which would be fine. You could rotate through multiple drives but I warn against this as it very quickly becomes difficult to manage your catalog and photos when you start down the road of 4, 6, 8 or more external drives.

No, really the solution here is painful because the costs are going to be much higher to solve this problem.  There are two ways to solve it. One is called direct attached storage or DAS. This is connecting a storage array with multiple drives inside it like a Drobo, G-Technology, or LaCie through a USB or Thunderbolt connection.  The other option is something called network attached storage or NAS. This is connecting a storage array with multiple drives inside it to your computer through a network connection. My recommendation there is Synology but there are a lot of different manufacturers of good NAS devices or even a more DIY solution with FreeNAS.

Let’s use Drobo as a DAS solution and Synology as a NAS solution for solving this second storage wall just to give you an idea of the costs.  Starting off with Drobo then, the 5C product offers a USB-C connection and supports USB 3.0 so it can be used by both Mac and PC, though if you have a Mac or your PC offers Thunderbolt you will want to go that route.  The 5C enclosure, with no drives, costs $344. It can hold 5 hard drives in the enclosure and if you put 5 6TB drives in there you would end up with a little over 20GB of usable storage. You might think my math is off there because 5 drives with 6TB each is 30GB, but I’ll go over why it is you don’t have that much storage in a moment.  Like I mentioned earlier, I recommend HGST drives though drives from Seagate and Western Digital will be fine here as well. Some will recommend that you use NAS drives in these arrays, and that is probably the safest route though I have seen evidence published by some independent testing that shows there isn’t a significant difference between NAS and non-NAS drives even when they are used in a storage array like this.  The thing to avoid here is drives labeled as being green. These drives attempt to save energy and are more likely to cause problems in devices like a Drobo. Continuing with our costs, an HGST 6TB non-NAS drive goes for about $135.  You could kick the can down the road and only put 2 or 3 disks in the Droob, but let’s talk about a solution that should last for a while longer and put 5 of them in there for $675.  That brings the total cost to $1,019. Not the couple of hundred dollars it took to recover from that first storage wall.

The reason you only get 20TB and not 30 is that the storage array is doing something to help with both performance and to protect you from drive failures.  With 5 drives in the device when you go to write files to it differently than when there is a single drive. There are various ways this can be setup that I won’t go through here, but the basic concept is that the file is written to more than one drive.  Sort of like we setup manually to solve the first storage wall where we wrote our photos to two different external drives, except smart people figure out how to do it more efficiently so that we get the same effect with giving up as much storage. With the two external drives you had 8TB of storage and you only got 4TB that were usable because we had to have a fully copy of the primary drive to the secondary drive.  You gave up 50% of the storage so you could have a decent local backup. Here the data is written in a way that we only have to give up about 33% of the storage and the system can have one drive fail without losing any data! Pretty cool stuff.

A Synology NAS storage array works in a very similar way, the biggest difference is the way you connect to it being a slower connection through a network.  You can do WiFi but you really won’t want to because wired is going to be much faster. There are ways you can even get a wired network connection to be pretty speedy but that isn’t something we will go into here.  A Synology DiskStation DS1517 also can have 5 hard drives and comes in at about $550. The disks would cost the same $675, so you are looking at $1,225. You will also give up roughly the same amount of data on the NAS as you do a DAS to get the same protection where one drive in the array can fail without losing any data.  It can also be configured so that you can lose two drives without losing any data but you give up more usable storage. The Synology is a little more expensive than the Drobo solution and it isn’t as fast, so why would anyone choose one of these? A NAS can do more than a Drobo. It is capable of being connected to more than one computer at the same time for example, which you can’t do with a DAS but this isn’t a topic for this episode either.  The point is recovering from hitting this second storage wall is going to take some investment and we aren’t actually done yet because we haven’t met the 3-2-1 backup strategy.

Yes, a storage array solution provides some redundancy so that we can have a drive fail without losing any data, but we don’t truly have two copies of the photos.  If we are truly following the 3-2-1 strategy we can only count a copy of the photo being on the storage array as one copy. Which means you really should have a second storage array that you copy everything to in order to truly have a 3-2-1 backup strategy.  Ouch! Now you can make a decision for yourself here and decide of the storage array is providing enough redundancy that you feel like you can count it as physically providing two copies of the photo since it really is on more than one drive inside the array.  I haven’t gone here myself yet, I am using 6TB drives inside my computer here in mid-May 2018. I have kicked the can down the road so far because that $1,000 investment is too much for me to swallow, and if I am going to do it I actually want to invest in something larger in an 8 bay storage array.

Whether you decide a single storage array is enough for you and you’ll take a chance on not having two drives fail at the same time, or you have two arrays, that still leaves the 2 and the 1 requirements of the 3-2-1 backup strategy unsatisfied.  We have to get it on two different mediums and 1 offsite. This is where today in 2018 I don’t think there is really a great solution. You can actually setup BackBlaze to backup either solution, though you will need a different license type to backup the NAS using BackBlaze B2 which for 20TB would be about $100/mo.  Ouch again. Then there is the consideration of how much your ISP is going to hate you if you have to sync 20TB of data or how long it is going to take to sync it. If you are producing video you create content so quickly 20TB doesn’t end up being that much and at least in the United States there aren’t many places where you can get enough speed and stay under bandwidth caps to make cloud backup a solution.  

So what is a photographer to do?  If you aren’t using all of the 20TB  of data and you are adding to the array slowly then BackBlaze may still be a possible solution – especially with Drobo where it looks to me like you can still count a DAS as an “external drive” and stick with the personal licensing – though when I reached out to BackBlaze they said they would prefer people didn’t do this.  Otherwise, I just don’t know of any really good options to get an offsite backup when you have data this large. I have heard of solutions where people put another storage array at a friend’s or relative’s house and then they use some software to sync between the two over the Internet – but that only solves the licensing costs of a service like BackBlaze with B2.  I have heard of people saying that they have a second array that they copy things to once a month and then second array over to that friend’s or neighbor’s house, but those same people usually admit that they aren’t very consistent with it.

The reality is when you get up into the 20TB+ range of data you are storing, the costs of a true 3-2-1 backup are big enough they just aren’t practical for a photographer.  At least not with anything I am aware of as I record this episode here in May 2018. If any of you know of something better, let me know in the Photo Taco Podcast Listeners Facebook group.  I would love to know of some kind of reasonably priced solution that doesn’t take a computer science degree to make it work.

Resources mentioned in this episode:

• Photoshop Feedback Post – Wacom touch ring for zoom in/out closes Photoshop Select and Mask Workspace on Windows
• Improve Photography – The ULTIMATE Backup Workflow for Photographers
• Photo Taco – Lightroom Organization
• Photo Taco – Cloud Backup (BackBlaze)

Other Show Resources:

• Vote for Jeff’s “Cull” module idea to be added to Lightroom: http://bit.ly/cullmodule
• Photo Taco Archive: https://phototacopodcast.com/category/taco/
• Improve Photography Podcast Facebook Group: https://www.facebook.com/groups/ImprovePhotographyListeners/
• Photo Taco Instagram: https://www.instagram.com/phototacopodcast/
• Photo Taco email: phototacopodcast@gmail.com
• Jeff’s Twitter: https://twitter.com/harmon_jeff
• Jeff’s Facebook Page: https://www.facebook.com/jsharmonphotos/
• Jeff’s Portfolio: http://jsharmonphotos.com
• Jeff’s Instagram: https://www.instagram.com/harmonjeff/
• MacPhun Luminar: http://macphun.evyy.net/c/362006/185399/3255

Podcast: Download (Duration: 59:14 — 81.6MB) | Embed

Subscribe: TuneIn | RSS