Storing External HD Filled with Images

[marcfs]

Bob,

Do you have recommendations for storing external HD that are filled with images?
Do stored external HD need to be accessed every “X” months to insure they are usable?

Regards,

Marc

[Bob Atkins]

Magnetic data on disks will fade with time, but I don't know the time scales involved. I've heard as little as 1 year or as long as decades. In normal use I think the data gets refreshed by the disk operating system.

As for archiving data on hard disk, I'd never depend on a single data storage element. I'd always have a backup of the data. In fact I have two 1TB drives attached to my system. One is used for primary storage of images, the other is a copy of the first one.

If I had disks which were not connected to a PC, I'd be tempted to refresh them every year or so. In fact I'd have three disks. Data stored on disk A, backed up on disk B. To refresh I'd copy disk B to disk C and then disk A to disk B. On the next refresh I'd copy disk C to disk A and then disc B to disc C and so on. Basically rotating the data around the ring of three drives.

Hard drives are the only real practical way of backing up very large amounts of data but they are not archival over long periods of time. It's hard to say how long the data will last, but if you wanted to be really sure I'd say a refresh every 6 months to a year would probably be very safe.

[bmpress]

That is very interesting information about hard drives. Here is another possibility...cloud storage. There are several companies that will let you upload to their storage facilities and the costs have come down. These companies also use backup storage at different physical locations to protect agains fire and other disasters. Of course, one must also keep one of your hard drives at another location to provide this same kind of protection.

[Bob Atkins]

At this point I don't really trust "the cloud", plus even with a 10Mb/s internet connection it would take around 300 hours to do a 1TB backup. That's two weeks. It would take the same amount of time to recover the data to a local disk. And that doesn't address the issue of how much it costs for 1TB of cloud storage each year or if the external host can actually offer 10Mb/s connectivity speeds for weeks on end.

You can now buy a 1TB drive for under $50 which you can fill up with data in a few hours. So for $100 you get redundant storage of 1TB of data. That's a pretty good deal.

I don't see how "the cloud" (on line storage) can really compete as a backup solution. It's real use is to be able to access your images from any PC anywhere in the world, though I think that's something that very few people will need.

[bmpress]

Yes, it is slow for standard cable modem or DSL speeds, But if fiber-to-home is available then the issues are cost and reliability. Maybe some day, the US will get everyone high speed internet, but from what has been recently reported...not in my lifetime.

[marcfs]

Bob,

Thanks for your insight and perspective.  With archives becoming larger and larger, and hard drives becoming cheaper and cheaper, we need strategies to save and protect images.  I've tried to move to larger hard drives with all images on two external drives (original and copy) to help avoid some of the issues you have identified. 
I still remain concerned about losing images, but I'm not sure what you can reasonably do to further protect the images at a reasonable cost and investment of time.

Regards,

Marc

[klindup]

The life of magnetic storage devices has always been an issue but there is another issue with backups and that is offsite storage.  Sure the most likely thing is failure of a hard drive and that is when and not if.  There is also the possibility of a disaster such as a fire, flood, earthquake and tornado.  Insurance will cover the cost of replacing stuff but an archive of images is pretty much irreplaceable.  Here we can learn from what commercial firms have always done and that is always to have a backup copy offsite.  How far away that is depends on the threat.  Where I live, the risk is that my house catches fire so storing a backup disk in a friend's house is fine.  If I had to worry about flood I would find a friend who lived in a different area that would not flood.  Given that disk drives are so cheap you can afford to have several.  My strategy is to have three backup drives.  One is a Western Digital passport that backs things up in the background.  That stays in my house and covers me for hard drive failures.  Once a week I copy my image archive to another disk overnight and take it to a friend's house and return with the third disk where I repeat the exercise.  The worst case scenario is that I lose a week's images.

There are online services but I live in a rural community and broadband speeds are not good, certainly for uploads.

Ken

[Bob Atkins]

I think it all depends on how much value you place on your images. If you're a pro with a huge stock collection then offsite storage makes sense because you depend on your images to make a living. If you're just shooting for your own pleasure then it may make sense to archive your most valuable (family) images offline and off premises somewhere (bank security box?), possibly on archival CDs, or using a network storsge system.

For most of us, local redundant storage may be enough security. You can place copies of images (and slides/negatives) in some sort of secure housing. I've heard of freezers or refrigerators in the basement being used. They're strong, somewhat fire resistant and cool storage in the dark prolongs the life of most passive media.

[marcfs]

Bob,

Thanks again for your perspective. 

One good part about image storage is the price of external hard drives has significantly come down.  A Western Digital 3 TB My Book Essentials external HD is priced at around $150.  I’m hopeful prices will continue to come down while transfer speeds with USB 3.0 increase.

Regards,

Marc

[Johnf]

You can get external enclosures with e-SATA interface then buy a standard drive.  This is often a cheaper option and transfer speeds that are noticeably faster than USB 2.0.

USB 3.0 will still need to convert data where e-SATA does not, so at 4.8GB/s it still may not be faster than e-SATA at 3GB/s.  USB 2.0 transfer speed is 480MB/s.  Conversely, USB provides power and e-SATA does not.

Just some (geek) facts

John

[KeithB]

If you are a white-collar professional, you may be able to store your offsite storage at your workplace.  I just changed jobs, and at my old job, this was a good option.  Secure and climate-controlled.  (We did have one hydrogen explosion and the building next door caught fire during the twenty years I worked there, but it would not have affected my backup.)  Where I work now will not let me bring in external hard drives due to security concerns.

[deemery]

The running joke is there are 2 kinds of disk drives, those that have failed, and those that are about to fail.  Google has published some interesting/surprising analysis of the (un)reliability of hard drives in the real world (vice manufacturers MBTF numbers.) 

In my case, I keep 3 copies of my photos, one on my (Mac OS X) server (RAID-5 array), the second on another local machine on my network (backed up weekly) and the third on one of a pair of external hard drives that normally sit at a friend's house.  These are supposed to be rotated monthly (but I'm behind, in part because of a drive failure in one of those external enclosures.)  The hard drive on the 2nd machine (in an external drive enclosure) is getting long in the tooth, I'll probably look at a mandatory 'replace after 5 years' policy. 

And for what it's worth, I've had terrible luck lately with Seagate 1tb 3.5" drives, with 4 of 7 failing in less than 2 years.  My RAID enclosure came from OWC (http://www.macsales.com - I've had great service from them) and they recommended Toshiba drives.  But some online polling indicates no clear consensus on which brand (Seagate, Toshiba, Western Digital, Fujitsu) is best, except that Samsung drives generally have a poor reputation. 

The cost of archival DVDs (both media and transfer time) are really hard to justify, and of course there is the concern that in 10 years conventional DVD reader will be hard to find. 

[deemery]

One more comment:  If you're a PC guy, then if your computer supports USB 3.0, that's probably the least expensive enclosure.   For photo backups, I wouldn't worry much about speed of either the enclosure or the drive itself, it's not like you're going to be accessing these all the time. 

For Macs, the cheapest is USB 2.0, but a USB 3.0 enclosure will work just fine on a Mac at USB 2.0 speeds. 

If your machine supports eSATA, that's a much faster interface on Windows machines.  FW800 is the best you can get right now on a Mac ('native', if you have a Mac Pro or an older Macbook Pro you can add an eSATA card.) 

The newest Macs (Macbook Pro, iMac currently, with the other Mac products probably following) now support the Thunderbolt very high speed interface.  I think there's 1 or 2 TBolt drives available right now and they're expensive!  But what I'm hoping for/expecting to see later this year are TBolt to eSATA adapters.  This will not give you the full speed of Thunderbolt native drives, but will provide substantially better transfer rates over either USB 3 or FW800.  For the last 2 years, I've been buying 'quad interface' enclosures that cost me more $$.  These support USB 2 or USB 3, FW400, FW800 and eSATA.  My previous Macbook Pro had an eSATA ExpressCard34 to eSATA adapter (and that was -great- for backups).  I'm expecting the next Mac Mini to come with Thunderbolt, and when I can get a TBolt-eSATA adapter, I'll replace my current Mini that's connected via FW800 to an OWC quad interface/quad drive Qx2.  (This runs RAID-5, and I've been very happy with the price/performance of this enclosure.)