Not a question of IF but a question of WHEN!
- 22/05/2012
- Posted by Sephi Bergerson
Today I want to talk about storage and back up of digital images. I’m sure you all back up your pictures on a regular basis, but how? Do you still use a simple external hard drive or back up to DVD and think you are safe? If indeed than you should get your act together and quickly. We all think we are safe until something happens, and with digital technology it is not a question of IF but a question of WHEN!
In the past we used to have boxes where we kept the negatives along with a contact sheet. Some photographers had so many of these boxes that they needed extra rooms to store them, but today when we all shoot digital (well almost all), storage of our archive is a completely different issue. Everything we have can fit into a small box on our desk and be accessed in a matter of seconds using simple search tools. All the high resolution images that I have since starting to shoot digital don’t even fill up a 5TB drive. This is very convenient but what drive is safe enough to hold my life’s work like this? My assignments, book projects, my client’s memories, my family pictures?
The question you should ask yourself is “What if I loose it”?
Are you willing to take the risk? I’m sure we all agree that we can never be too careful but we do have some limitations. If I were an agency with a huge budget I’d probably go crazy with the back up system, but I am not. I am just a freelance photographer working alone (more or less) and running my own small business so the options are limited. However, I will do everything within my power to enable myself a good night sleep (which brings me to think that it is high time to buy a good mattress). The reason this post is written now is because I am at the moment in the midst of a situation that could have been a huge catastrophe but is actually being handled quite well. More about this later.
After having lost a 500GB drive a few years ago (I managed to save the data eventually) I took the extra step and bought one of the very first Drobo systems in India. The Drobo-S is not a cheap piece of equipment but is probably the best back up system out there today. The Drobo is a BeyondRAID system built of five separate hard drives (I use Seagate drives) that keeps track of the safety of the drives at all times. The data will still be secure even in an event of a simultaneous crash of two drives out of the five thanks to an advanced mirroring system. Dual disk redundancy will reduce the possible storage capacity of course but you can get up to 10TB of back up space in one Drobo-S. Read more on the Drobo website.
The Drobo is my main back up but I also have two other drives. One is the amazingly useful Apple Time Capsule and the second is a portable Lacie. I also have an online archive with PhotoShelter where I keep not only the things I hope to sell as stock, but also my important work in progress.
My Back Up workflow
On assignment I shoot all RAW images and every night copy the cards to my laptop as well as a second copy to the portable Lacie. I do not erase the cards until I’m home where I convert the RAW images to DNG and load them onto the Drobo. I do all my abracadabra in LR4 and then export the selected images onto a new folder on the Drobo where they go to sleep. Important images are also exported to JPG10 and uploaded to PhotoShelter. At this point I could theoretically delete the original image folder but I usually still keep it for a few months, just in case i want to get back to it for any reason.
To maintain a regular back up of all my files (not only images) I use TimeMachine with the TimeCapsule which is a very sweet thing actually. The real thing however is SuperDuper which is, as they put it, a heroic system recovery for mere mortals. SuperDuper backs up the entire hard disk to the Drobo on a weekly basis.
The files and documents I use on a regular basis; contracts, invoices, writing, templates etc. along some other things I need are all on the cloud with Dropbox. My contacts, emails, notes, calendar and passwords (I use 1Password) are all on my iCloud account so that even if I loose my phone or my computers I will still have my work schedule, address book and passwords to login to my bank account and see if I have enough money to buy a new mattress.
And if worst comes to worst
Even the best system has it’s weaknesses and a hard drive will eventually fail. My UPS had crashed two days ago causing the Drobo drive to develop a fatal error that seems to be impossible to fix. The best and most recommended drive repair tool out there is DiskWarrior which builds a new directory and replaces the damaged one. I tried and had a message that DiskWarrior cannot replace the damaged Drobo directory and that I have to reformat the drive. The data on the drobo is not lost and I can still access it but cannot write to the disk anymore. I opened a case with Drobo service center and got a mail back that . . they are sending me a replacement unit! I am now backing up everything from the drobo to alternative hard drives and continue working as usual. I know my data is safe and it is only a matter of time until my system will be back to normal. Can you beat that?
This image was taken in Goa a few days ago and has gone through all the above. Feel free to have a look at my archive on PhotoShelter to see more travel images of Goa and India.
Sephi Bergerson is a travel and culture photographer based in India since 2002 and is available for assignments worldwide.
May 23, 2012 12:46 am /
Great insights Mr. Sephi. This is valuable information for all photographers. Thanks for sharing your experience about storage.
May 23, 2012 12:46 am /
Great insights Mr. Sephi. This is valuable information for all photographers. Thanks for sharing your experience about storage.
May 23, 2012 1:16 am /
Great insights Mr. Sephi. This is valuable information for all photographers. Thanks for sharing your experience about storage.
May 23, 2012 12:54 am /
I backed everything up in Google Drive 🙂
May 23, 2012 1:14 am /
But how much storage space do you have there? Don’t you want to have a copy with you as well, just in case kind of thing . . ?
May 23, 2012 5:42 pm /
I got 100 Gigs up there. $4.99 a month. 25 gigs is $2.49 a month. I have about 40 Gigs worth of Pics and DVDs from the wedding. The best Part is all the different vid formats i had have can now be shared with people and they can stream them instead of having to download them. (which reminds me i haven’t invited you to my shared folder yet :D)
The much ignored part about backups is how easy it is to restore once you lose information. And when you do system level backups like you mention it can be slightly complicated. So i prefer to just keep things in Google Drive.
Definitely don’t think thats the way for professional photographers to go. But others who just like to backup pictures might want to do that. My work systems are backed up very similar to yours on a Raid-Z partition with hot swappable drives with 2 levels of failsafe using a System called Nexenta. But I always make it a point to have file levels backups on things that are easy to restore. But, you should do a Windows version of your guide for those who dont use Mac’s (do professinal photographers even use Windows in India??)
I was thinking of getting a Lacie for my/Shruti’s Mac Pro but with 100 Gigs on Google, i don’t see the point.
Again, this is the way to go for people who collect pictures 🙂 rather than professional photographers.
May 24, 2012 5:38 pm /
Hey Vaibhav, thanks for the comments. I guess a 100GB google drive is more than enough for many. Unfortunately with a few TB of information it is not an option. I have to disagree with you on the easiness of restoring files from such a system. It seems rather easy to me, but like I said, google drive is easier for a smaller need of space.
A window version? LOL I don’t think so. Maybe someone else will do that one day 🙂
May 24, 2012 9:05 pm /
Google Drive does have plans upto 16 TB.. but they can be pricey. I think the 5TB plan is about Rs 1.2 Lakh per year. But I dont think internet in India is Fast/Reliable enough to take that option.
I am going to look into this Drobo thing. Cause Backup Exec is very complicated. Its really meant for Disaster scenarios. Also i think there maybe copyright issues with most “Cloud Companies” esp. Google.
May 25, 2012 12:10 pm /
Vaibhav, why pay this much money per year if you get a 5TB drobo for half and you get to keep it and upgrade it if you need. I would nto consider this kind of cloud back up when prices are so high. It will surely go down in a few years and with the internet speed improving will probably be a very viable option but not right now. cheers
May 23, 2012 12:54 am /
I backed everything up in Google Drive 🙂
May 23, 2012 1:24 am /
I backed everything up in Google Drive 🙂
May 23, 2012 1:44 am /
But how much storage space do you have there? Don’t you want to have a copy with you as well, just in case kind of thing . . ?
May 23, 2012 6:12 pm /
I got 100 Gigs up there. $4.99 a month. 25 gigs is $2.49 a month. I have about 40 Gigs worth of Pics and DVDs from the wedding. The best Part is all the different vid formats i had have can now be shared with people and they can stream them instead of having to download them. (which reminds me i haven’t invited you to my shared folder yet :D)
The much ignored part about backups is how easy it is to restore once you lose information. And when you do system level backups like you mention it can be slightly complicated. So i prefer to just keep things in Google Drive.
Definitely don’t think thats the way for professional photographers to go. But others who just like to backup pictures might want to do that. My work systems are backed up very similar to yours on a Raid-Z partition with hot swappable drives with 2 levels of failsafe using a System called Nexenta. But I always make it a point to have file levels backups on things that are easy to restore. But, you should do a Windows version of your guide for those who dont use Mac’s (do professinal photographers even use Windows in India??)
I was thinking of getting a Lacie for my/Shruti’s Mac Pro but with 100 Gigs on Google, i don’t see the point.
Again, this is the way to go for people who collect pictures 🙂 rather than professional photographers.
May 23, 2012 3:23 pm /
Good article on an important subjet. I don’t agree on all what you are saying but overall, if one follows your advice, at least one is well on one’s way to safer storage.
– Do you seriously mean that you delete all the original RAWs/DNGs? “At this point I could theoretically delete the original image folder” That’s definitely not a good idea. It’s lika having made prints from negs and then throwing away the negs…
– One fundamental point in backup is to have at least one off-site copy. Didn’t see that in your description or did I miss it? Online backup is theoretically nice but for photography it is faaaaar too slow and to expensive.
– You use Drobo and one of the weaknesses with it is, as I understand it, that it has a proprietary file system which may make it difficult to recover data. As you discovered. Another option is to use a more standard RAID file server.
But, as I said, excellent description. Will have to write my own one day.
May 24, 2012 5:33 pm /
Dear Per, thanks for this comment. When I delete the original RAW file it is only after I have another copy of it (again an original) as a DNG file folder on the drobo. I am not left without a RAW file I only exchange the NEF Nikon file with a new RAW format.
There is no issue with recovering files form the drobo . Even now that the devise itself is faulty ad I am waiting for a replacement unit to come from SIngapore, I can still access all my files and managed to back them up again on a new drive. The drobo BeyondRaid technology is way more advanced than Raid. What ever happens to the drobo itself the data is most likely to be safe. I have yet to find a better and more reliable system out there.
You are of course correct about off-site back up and I’m afraid this I do not have. I should probably consider this, at least for finished long term projects like books etc. However, as I mentioned before, these special images are on my photoshelter archive. It is of course too slow to back up everything online but a select 100-200GB is not that big a deal.
Please keep me informed when you do write a back up description map. I’d love to read it.
cheers
Sephi
May 23, 2012 3:53 pm /
Good article on an important subjet. I don’t agree on all what you are saying but overall, if one follows your advice, at least one is well on one’s way to safer storage.
– Do you seriously mean that you delete all the original RAWs/DNGs? “At this point I could theoretically delete the original image folder” That’s definitely not a good idea. It’s lika having made prints from negs and then throwing away the negs…
– One fundamental point in backup is to have at least one off-site copy. Didn’t see that in your description or did I miss it? Online backup is theoretically nice but for photography it is faaaaar too slow and to expensive.
– You use Drobo and one of the weaknesses with it is, as I understand it, that it has a proprietary file system which may make it difficult to recover data. As you discovered. Another option is to use a more standard RAID file server.
But, as I said, excellent description. Will have to write my own one day.
May 24, 2012 6:03 pm /
Dear Per, thanks for this comment. When I delete the original RAW file it is only after I have another copy of it (again an original) as a DNG file folder on the drobo. I am not left without a RAW file I only exchange the NEF Nikon file with a new RAW format.
There is no issue with recovering files form the drobo . Even now that the devise itself is faulty ad I am waiting for a replacement unit to come from SIngapore, I can still access all my files and managed to back them up again on a new drive. The drobo BeyondRaid technology is way more advanced than Raid. What ever happens to the drobo itself the data is most likely to be safe. I have yet to find a better and more reliable system out there.
You are of course correct about off-site back up and I’m afraid this I do not have. I should probably consider this, at least for finished long term projects like books etc. However, as I mentioned before, these special images are on my photoshelter archive. It is of course too slow to back up everything online but a select 100-200GB is not that big a deal.
Please keep me informed when you do write a back up description map. I’d love to read it.
cheers
Sephi
June 22, 2012 3:06 pm /
Nice article and very useful subject.
June 22, 2012 3:36 pm /
Nice article and very useful subject.