Refreshing Hard Drive Storage

Larry Jordan has recently been warning against the dangers of leaving drives on a shelf for periods over a year. He says that after a year, the data will start to essentially disappear. There's a way to avoid this effect of the vanishing data, and that's to have the computer read the entire disc.

Larry just posted an article on how to do this preventive action on his recent newsletter (not sure if this link will work for long, as these newsletters are only available for a limited time.) Here's the basics of the preventitive action.
-Make sure your user account has administrator access
-Open the Terminal applications, which is in the Utilities directory
-Enter this command: sudo cat /dev/rdisk0> /dev/null
Actually that command will read your entire boot drive, which is probably not the drive that has been sitting on a shelf for a year. You can run this task on different drives, but first you have to figure out the identifier for the drive (hint: it's not the name that you called the drive.)

To get the disk identifier, go to Disk Utility, also in the Utilities folder. Select the drive, not the volume--the drive's name usually starts with the capacity of the disk followed by the drive's manufacturer, e.g., Seagate, Hitachi, LaCie--the volume is what you named the drive or a partition on the drive. After selecting the disk that you want to run the maintenance on, click on the info icon at the top of the application window. This opens a new window with info on your drive, the disk identifer is the word "disk" followed by a number. If the disk identifier was disk2, then your command would be: sudo cat /dev/rdisk2> /dev/null

The command will take a while to run--we're probably talking hours. You can interrupt this procedure by hitting Control + C while in the Terminal.

Disappearing data is only one risk you run by using a hard drive as long-term storage. But this seems like something worth doing if you care about the data that you have left on a drive.

Firewire vs. USB Performance in New Laptops

So Apple has dropped Firewire from the MacBook and don't expect to ever see it againon the consumer-level laptop from Apple. This is too bad, especially since Apple really boosted the performance of the new MacBook--it would have made a decent computer for editors if it had had Firewire.

The lack of Firewire raises two big issues for video folks. First, the performance of hard drives over USB has traditionally been really poor on Mac's--usually half the speed of Firewire 400. Second, there are all kinds of other gear that uses Firewire--decks, cameras, Firestore, video monitoring/digitizing equipment. We layoff to DVCam at Link and get most of our footage from DV and HDV tapes.

One good bit of news about peripherals with the new laptops is that Apple has finally raised the performance of USB. Check out this article at Bare Feats to see the results of a comparison of hard drive performance of USB, FW400, FW800, and Sata drives on the new laptops. USB is almost as fast as Firewire 400 on the new MacBook Pro.

I still wouldn't recommend doing any serious editing with USB drives on these new laptops. I guess it could work, but Firewire was supposed to be a better standard. My guess is that you would run into problems with capturing and laying off if your media was on USB drives. But these are educated guesses at this point. This may change if Apple does a better job of implementing USB on their Mac's.

Drobo: Your Data Robot

Drobo has been around for about a year and when it first came out I judged it as "cool but not ready for primetime." Drobo is a unique RAID setup for hard drive storage. The biggest drawback was the Drobo only had a USB interface, which is not cool for video editing. Well, this summer Drobo 2 came out with Firewire 800. Data Robotics, the manufacturer, is touting read speeds of 52MB/s and writes of 34MB/s--not too shabby. They say it's good for uncompressed SD, DV, and compressed HD. They cite DVCProHD as OK, but at those data rates I'm thinking that Apple's ProRes 422 is supported too. ProRes maxes out at about 26MB/s--generally I think if the drive's speed is double the needed data rate, then you are OK.

So what makes Drobo unique? It's got a great combination of redundant data storage and the easy-to-use factor. The Drobo case holds up to 4 hard drives, and you only need 2 drives to get started. Your data is spread across the drives in such a way that if one drive failed then you wouldn't lose any data. Need more storage? This is the great part. Just slide in another hard drive of any size. Drobo incorporates you new drive into the existing RAID. If you have four drives in the case and run out of space, then no problem. Simply replace the smallest drive with a bigger drive to get more storage.

My experience with RAID has never gone beyond buying a G-Raid. Whenever I looked into setting up a RAID on my own, there were always enough drawbacks to keep me from making the plunge. It just seemed too difficult to manage all the drives. We are in drive overload anyway. Plus the added expense of more drives and SATA gear was always an issue. Granted Drobo is not cheap. At $500 without any drives, that's a hefty expense to swallow. But I think over it's life it will make up for that in the cost saved by not buying drives with cases, plus the peace-of-mind that you get with redundant storage.

There could be some drawbacks to Drobo. One is that you are putting all of your eggs in one basket. Yes, you are protecting yourself against the failure of a hard drive, but what if Drobo fails. Then you could be in real trouble and pray that Data Robotics will come to your rescue (and do so promptly). Also all of the management seems automatic. That's good if you don't want to deal with the hassle of repairing a RAID with a bad drive, but bad if the maintenance takes place in the middle of a big deadline. If a drive dies, then you don't lose any data. But then Drobo has to start shuffling around all of your data between the existing drives that are still working, and this will take a lot of time. While you are supposed to still be able to access Drobo while it does this automatic maintenance who knows what kind of performance you will get as Drobo and the drives are very busy. Will the Drobo still be able to playback and record video while it rebuilds? So while losing a drive could mean no lost data, it doesn't necessarily mean no downtime for us video editors. But overall, I'd prefer losing no data but have a slower system for a day or so, then totally losing the data.

You are also giving up some of the transportability of a G-RAID. While the Drobo can be moved, it will probably end up having all of your storage on it. If you like to put individual project on a drive, which gets moved around, then Drobo is probably too expensive for that. So it's both good and bad that Drobo can be the only drive case that you will need.

Here's an example of a Drobo system once you get the drives.
Drobo...$500
(3) 500GB Hard Drives...$325
Total Cost...$825
Total storage...930GB
That compares to my favorite type of storage a G-Raid, which gives you the same amount of storage for $355. So that's $500 up front is a lot to ask for. But I'll be thinking about his seriously in the future.

Here are some links on Drobo:
Drobo in Action
A video from Data Robotics on how Drobo works.

Drobolator
See how much storage Drobo will give you when you put different drives inside.

Drobo Firewire 800 Review
A short review of the new Drobo that gives more info on the benefits and setup of Drobo.

Maintenance Tips for Your Mac

MacWorld: "Essential Mac Maintenance: Rev up your routines"

This a good article at MacWorld.com. A bunch of tips on how to get your Mac in good condition for the long run. I highly recommend the article.

One useful tip I liked was using SMART tests to regularly check on the health of your hard drives. Apparently these tests can give you a heads-up on ailing hard drives (before you lose data.) It's not bullet-proof, but it's supposed to protect against some of the more common types of hard drive failures. Michael Kinimoto says he has had good luck with doing regular SMART tests. The article recommends installing some freeware that regularly checks your hard drive's SMART status: Smart Reporter. Unfortunately, SMART tests don't work over firewire. It may work on an external hard drive hooked up through SATA, but that depends on what kind of hardware connection you have for your external SATA drive. The Firmtek Seritek PCI and PCI-E cards I've used pass along SMART data. But at the very least, you can be checking your internal drives on your Mac.

Apple Updates the Mac Pro

Apple has refreshed the Mac Pro--the first full turnover of the Mac Pro since it was first announced in August of 2006. The line has switched from being dominated by Quad-core Xeon's to being dominated by Octo-core Xeon's. There was an uptick in price for the recommended system from $2500 to $2800 for an 8-core 2.8 GHz Xeon Mac Pro.

There are some other improvements in the system architecture and early results show good improvements in system performance. While adding 4 more cores doesn't double the speed, these systems look signficantly faster than older Mac Pro's, especiallly in encoding video. Check out this benchmark test from MacWorld for more info:
"From the Lab: Internal changes bolster latest Mac Pro"
I was surprised at what low marks they gave the internal hard drive that ships with the new Mac. I've always assumed that those had decent performance, but MacWorld found a significant speed bump on their multi-app test by putting a hard drive from an older Mac Pro into the current 8-core 2.8 GHz Xeon. For some good specs on hard drive performance, check out Bare Feats. Apple also has some performance comparisons with other Mac towers here, including rendering & encoding with FCP and After Effects.

There had been rumors that Apple was ready to add a Blu-ray drive for playing back high-def movies to the Mac Pro's. For PC's, adding an HD DVD or Blu-ray drive, or both, is now a common option. Nothing from Apple about Blu-ray for these new computers, but an announcement could come at the MacWorld Conference. I'm anxious to see Apple add Blu-ray disc authoring to DVD Studio Pro, as well as decent support for the HD DVD standard. Adobe's Encore already does this. Commentator Bob Cringely suggests that Jobs will ignore high-def discs because he sees them as a threat to iTunes ruling the media world in his latest column (he also thinks Apple will buy Adobe, which seems a bit far-fetched to me.)

Tutorial on Setting up Drives for Editing

Formatting & Partitioning a Hard Drive in OS X - Tiger and Leopard
By Ken Stone

A short tutorial from kenstone.net that covers the basics on setting up a hard drive for editing. Goes over formatting and partitioning. I sometimes will partition a hard drive. A hard drive's performance drops as the drive fills up, so I will often split up a disc in a faster partition and a slower partition. (One reason I like G-Raid's is that you can get good performance out of the drive even when it is full.)

Article also covers good things to know about permissions and journaling.

[via StudioDaily blog]