Compressor Troubleshooting & Tweaking

Compressor error: "Unable to connect to background process"

Good info on Compressor problems and bugs from Digital Rebellion. They also have a good uninstall app for the whole Final Cut Studio that I had to use recently, FCS Uninstaller.

And here's a tip on speeding up rendering for those with 4 and 8 core systems from kenstone.net:
Compressor - Enabling multiple Cores

September Blogwad

Hard Disk Warning!
Larry Jordan says don't leave a hard drive sitting on your shelf for more than a year with data that you want to keep. Explains why and gives some suggestions on how to keep old hard drives from losing data.

The Freesound Project
Sound effects here. They are available under Creative Commons licenses. Not sure if that would include any of Link's work. As I understand it, Creative Commons means that you aren't doing the project for money and your work will also be available to others under the Creative Commons agreement. But I'm not an expert.

DA Font
Some fonts here some are in the public domain, others are available under Creative Commons licenses. Again, I'm not an expert in Creative Commons licensing. But public domain is obviously cool.

Ripple Training
Some good tutorials in the Ripple Training section of the ProVideo Coaltion website.
Remove the Jitters - Covers the SmoothCam feature in Final Cut Pro 6.
Photoshop & Motion - Simple way to do a quick animation on a multi-layered Photoshop document in Motion 3 by using its 3D features.
Color Workflows With Different Video Formats - Covers how to use different codecs with Color. Warning here about working with Long-GOP formats, like HDV.

HDV video to SD DVD Using Compressor 3 and DVD SP
Exporting HDV Video from the Timeline to Standard Definition DVD
Man, working with HDV can be slow. Hopefully these tutorials will save you some time if you need to make a DVD from some HDV footage. They are over at kenstone.net
Also at kenstone.net, a tutorial on making a Blu-ray high-def video disc on a Mac. Takes you from FCP through Compressor to Adobe's Encore:
Blu-ray with Final Cut Pro, Compressor and Encore

ProRes 422 was one of the big features of Final Cut Pro 6. It was essentially a QuickTime codec, but Apple wasn't going to give it away. Too bad, because it's an efficient codec and very useful, especially for high-def footage. Well, Apple is finally letting people playback ProRes 422 footage on systems without FCP 6 installed (but you still can't create ProRes files without FCP 6.) Here are the links to playback ProRes on Mac's and PC's.

Mac OS X 10.5: Creating and maintaining a bootable "universal" external disk
I didn't know you could do this:
You can make a "universally bootable" external hard disk that will work with both Mac OS X 10.5-compatible PowerPC-based Macs and Intel-based Macs.

Bluetooth Earpiece--2-way radio
Do production? Need to buy a bluetooth head set? Then consider the Dragon. It will work as a 2-way radio with another Dragon headset. They even sell a device that will allow you to link up to 5 Dragon headsets together. Probaby not the cheapest way to communicate with a crew, but it's nice that it will do double-duty as the wireless headset for you cellphone.


Making Compressor Multiprocessor Aware
This one of Larry Jordan's Tips of the Day. Uses QMaster to make virtual clusters on multi-processor systems.

Compressor and Conversions

Compressor 2 and 3 have very powerful tools for converting and upscaling video, as well as doing slo-mo. The optical flow technology, which Apple acquired when they bought the powerful compositing application Shake (only later to kill it), is the key to the results that you can get from Compressor. The key to getting the best results is the Frame Controls settings in Compressor.

Compressor is very good at standards conversion or converting video to a different frame rate. People often call this PAL and NTSC conversion although this isn't really accurate, but it will handle the same tasks that systems that do this do--namely the scaling and frame rate conversions. Frame rate conversions in particular are difficult to do really well. There are some standalone products from Teranex that just do this for around $2500 Well Compressor does this really well, but Teranex does it in real-time and Compressor is really slow.

Optical flow can also help in up-conversion of interlaced footage. Interlacing makes it really hard to do upconversion because your vertical resolution is basically split in two. For an example of how to scale things up badly look at what Final Cut Pro does in the timeline. FCP basically throws away half of your vertical resolution as soon as you scale a video clip. So the difference between 100% scale and 99% or 101% scale is crisp video and a blurry messy. When scaled up like this over 200%, not unusual when putting in standard-def footage into a high-def project, fine detail like type also starts shaking up and down from the interlacing--it looks ugly.

Compressor is an improvement on how FCP works in a number of ways. First, it can analyze the image and decide to selectively keep detail from both fields when the video isn't moving--if visuals in a part of a frame is moving, then only that part of the frame will lose resolution. With the optical flow technology, Compressor guesses how a part of the frame is moving and intelligently creates an image with more resolution than other results. These two approaches are handled in the deinterlacing settings in Compressor--the first approach is described as Better while Best uses optical flow.

The Retiming controls are where Compressor handles different frame rates. For example going from 25 fps video from Europe to 30fps video over in North America, usually handled as 60i (interlaced) especially in standard-def. The Better and Best settings in Compressor use optical flow. Like with deinterlacing, optical flow analyzes your video and recognizes object and how they are moving--the optical flow then intelligently creates new frames that move objects smoothly without losing detail. By the way, going from 24fps, which is a progressive format, is not a problem for us folks working in 60i. You can do this right in FCP without any fancy conversion (assuming you want to keep the 24fps filmic look.)

As an example, here's what I recommended to someone doing PAL to NTSC conversion:
Resizing: Better
Deinterlacing: Better (motion adaptive)
Retiming: Better
In this case, the scaling is going down (PAL has more resolution than NTSC, so you should get good results without jumping through a lot of hoops.) I reserved the optical flow use to only retiming at the Better setting. I didn't recommend optical flow on deinterlacing to save time. But if time isn't an issue then using Best for deinterlacing should improve results.

Unfortunately, all this intelligent analysis by optical flow is very processor and time-intensive. So for example using optical flow for retiming and and deinterlacing makes my quad-core G5 slow to a crawl--I'd give a ballpark estimate of 1 minute to process 1 second of video. But if quality is important, then the results can be excellent.

There are some plug-ins that specialize in upconversion, namely InstantHD. Check here for my post on upconversion.

Here are two good articles on using Compressor. Macworld discusses standards conversions here that walks you through the whole process:
Send video abroad
This post at Digital Rebellion gives detailed info on all the settings in Frame Controls.
Using Frame Controls in Compressor

The Macworld article also recommends an interesting workflow using Cinema Tools to convert PAL to NTSC by conforming the PAL to 24fps, scaling and deinterlacing in Compressor, then adding pulldown in FCP. Interesting workflow that I've never seen used.

The Retiming controls in Compressor can also give you some very good slo-mo results. Here are some cool tests done that compares Final Cut Pro versus Compressor at various speeds on two shots.
"Final Cut Studio slow motion demo"
"Final Cut slow motion at various speeds"
The author brings up a good point--if you are shooting a shot to be slo-mo, then shoot at as high a frame rate as you can--60p, 60i, 50p, 50i are good, where 24 fps is bad for slo-mo. The more samples per second of the image you can acquire, the better the slo-mo will work.

24p vs. 30p

So we are moving away (slowly all be it) from interlaced video to progressive video. There are a number of framerates common with progressive. Here's a brief breakdown
24 fps--what's used on film
30 fps--half as many samples per second as regular TV (e.g., 60 interlaced)
25 fps--half the frame rate of regular TV in Europe (e.g., 50 fps)
50 fps--the samples of images per second used in European TV
60 fps--the samples of images per second used in North America TV ( which is usually interlaced)

So people generally prefer the slower frame rates in documentaries and dramatic material because it reminds people of film and the cinema--in short it looks classy. The faster frame rates look like TV and are a must have for sports where things are moving fast.

This post is primarily concerned with the technological differences (and not aesthetic) between different frame rates. (If you want to brush up on your basic knowledge of interlaced versus progressive, check out their definitions from this glossary from Quantel.) So let me outline a couple of the technical advantages of progressive video versus interlaced video:

1) Monitors are rapidly turning progressive--LCD's (both TV's and computers) and plasmas. These monitors can't natively display interlaced video. So some processing has to be done to de-interlace, and very often you end of losing resolution (and lots of it) when the de-interlacing happens. It depends partly on how good the hardware is that does the de-interlacing, but this happens at the consumer's end--so you never quite know how it will look. In short, a cheap TV will do a bad job of this.

2) Better effects work on progressive footage. Lots of effects, including scaling, require the video to be de-interlaced. Final Cut Pro does a lousy job of de-interlacing. If your footage is progressive (and FCP knows it), then it won't de-interlace to do simple things like scaling up the video. This past week, I had to include some letterboxed standard-def video in a high-def show. It looked super soft in FCP. So I used Compressor to do a very slow but very good de-interlacing of the footage, which I then scaled up (but again I didn't even scale this footage in FCP). It looked a lot better with the de-interlacing that Compressor does than the de-interlacing that FCP does.

3) Progressive distribution of video. Computer monitors were never interlaced--so anything for distribution on computers benefits from being progressive. Also DVD's will handle 24 fps. This is great if the video is going to a progressive display, e.g., LCD or Plasma. As mentioned above, interlaced footage needs to de-interlaced to be scaled, but progressive doesn't. So a 24fps DVD will usually look much better on a high-def display than a comparable 60i video.

Important note: DVD doesn't have a standard for 30 fps--it's either 60i or 24 fps. So your DVD player and your TV treat 30 fps just like 60i--if it gets scaled up then you could lose resolution (but not always--again it depends on the display.)

Now let me move on to outline some advantages and drawbacks of different framerates:

24 fps: So if you're going to make a film print, then you should shoot in this framerate. OK .001 per cent of anything shot by professionals will ever end up on film. But because 24fps has been very hot for many years (essentially the Panasonic DVX-100 set that off) there are a lot of post workflows for working with 24 fps. 30 fps is gaining ground though--there are now lots of setups in FCP for it.

24 fps is very close to 25 fps. So a general option for converting for a European master is to just speed up the movie by 4%. No elaborate and/or expensive standards conversion. I think Compressor will handle this 4% speed up for you. I recently did a post of a technology guide from Quantel and they recommend 24fps as a universal format because it is so easy to transfer across the Atlantic.

And it's important that your software knows that you want to work progressive. You can put 24fps or 30fps in a 60i timeline, but then you've lost all of the technological advantages of working progressive. Your editing software, your computer, your DVD player, your TV will treat it like interlaced video. So as a general rule, pick a frame rate and stick to it throughout post and distribution. Obviously this can sometimes be really problematical--what about archival footage that's 60i? Or you want to work with a shooter and they don't own a camera that shoots progressive. You can convert that to 24fps using Compressor, but it's a real pain. No easy answer for this one.

As or 30 fps, one advantage is that is has more samples per second than 24 fps. 24 fps can look kind of choppy--you especially have to be careful with panning quickly. 30 fps is no walk in the park in this regards, but you have 25% more samples of the image per second than with 24 fps--so definitely less choppy.

In general for cameras, especially with CCD chips, it is easier to create 30 fps than 24 fps. CCD's have been doing 60i for so long that a lot of cameras out there are still based around that technology. Namely, the Sony HVR-Z1, which Link owns. The Z1 doesn't do real progressive imaging, but with the Cineframe modes you can get video that has 24, 25, or 30 images per second. Basically the camera blends the interlaced video to get progressive-looking video. Apparently, the Cineframe 24 with the Sony Z1 is supposed to be really crummy. The Cineframe 30 is supposed to be better, as is the Cineframe 25. Basically the camera is capturing video in an interlaced manner 60 time per second and has to figure out how to make progressive frames with it. With all Cineframe modes, this means tossing out some resolution. The Canon XL-H1 also does a similar process to get progressive frame rates. But going to 30 fps from 60i is easier--basically is the image is moving than you throw about every other field. And the Z1 does Cineframe 30 reasonably but fails with Cineframe 24 (Caveat: this is definitely true for HD, but if you are downconverting to SD it might be OK.)

One option I saw discussed over at a great post and discusion, which talks about frame rates, at HD for Indies is to shoot with the Z1 Cineframe 25 and then slow it down to get 24 fps. I think the easiest way to do this would be with Compressor, but these Compressor workflows are a hassle. However, if quality is important than this is an option.

By the way, Sony is moving from CCD's to CMOS, and the CMOS sensors are much more versatile with frame rates. So the HVR-V1 chips natively captures at 24 fps, as does the new EX-1, the HVR-Z7 and HVR-S270.

One problem is that even if the camera can capture images at 24 fps (or 30 fps), the camera often actually records in video that is actually 60i. With 24 fps that is recorded in a signal of 60i, the camera adds pulldown (see the definition in this Quantel post I did). In post you need to remove the pulldown, otherwise your editing software just treats it like 60i and you lose all the technological advantages of working with progressive video. So to post at 24 fps, you may have to convert the footage to remove the pulldown. This can make post-production very complex. FCP has added a setup for version 6.0.2, which is supposed to automatically remove the pulldown from the V1's 24 fps footage. Newer gear from Sony like the Z7 and S270 are doing a native framerate of 24fps recording in HDV. Tapeless cameras like the Panasonic HVX-200 and the Sony PMW EX-1 handle progressive footage in a much more straight-forward manner.

One drawback with 30 fps is that it doesn't convert well to 24 fps or 25 fps. 60i is easier to convert to 25 fps because it has a lot more samples across time to work with. Basically when you are doing a major standards conversion, a computer needs to generate frames that don't exist by analyzing the image information that is available. The more samples of images, the better that this process can be handled. In short if you have to make 25 fps video or 50i video, than 60 images per second is much better than 30 images per second. (Remember that going from 24 fps to 25 fps can be done in a much easier way by simply speeding up the video by 4%.)

UPDATE
I thought I should add a caveat about the above comments on standards conversion. The workflow that I desribe for speeding up 24 fps to make it a European-friendly 25 fps is a workflow that I've only heard discussed as a step done in a computer, e.g., Final Cut Pro or Compressor. So you could convert the footage on your Mac--but you'd still have to make a master on a 25 fps or 50 fps format, usually tape like DigiBeta or if HD, HDCam or DVCProHD. The traditional way to do standards conversion is that folks just make an NTSC master of their program and then send it to a high-end dub house to get a PAL copy done. This is obviously easier than having to make a master on your own for formats designed to work in Europe. I don't know of a dub scenario that exists for taking a 24 fps master and speeding it up during a transfer to make it 25 fps. If you want a standards conversion done during a dub, then 24 fps may not be any better than 30 fps.

In short, there is no easy answer to this question. First you got to know your camera, know your post options, and know your distribution plans. From there, it's a matter of trade-offs, for quality, time, and money.

Blu-ray and Macs

As I've said before on this blog, Apple isn't giving Blu-ray any love. No drives, no playback, no authoring software. Apple is supposed to offer some support for burning Blu-ray data discs now, but Apple has been slow to jump on the next-gen for optical media (remember how Apple was out in front with DVD players in their computers back in the day?).

Right now, 3rd party software is providing support for burning. Toast 8 and 9 let you burn data Blu-ray. Toast 9 is supposed to let you burn high-def video to Blu-ray that will playback on Blu-ray players. But Adobe's Encore, part of Production Premium CS3, is now the best solution for putting high-def video onto Blu-ray. With a Blu-ray burner on the way to Link TV, and a copy of CS3 Production Premium available, we are on our way to burning our first Blu-ray discs. (Now if I can only convince my boss to buy a high-def TV and a PS3, so we could play the darn things.)

Adobe can encode your video for you, but there's a recent blog post Apple's Compressor & Encore Blu-Ray from Adobe that helps you figure out how to encode your high-def in Compressor. Adobe gets kudos for playing well with others. They don't pretend like they are the end-all and be-all of post-production software and that anything you would ever want to do can be done on their software. And I like the concept of an Adobe blog as a personable way to distribute info about software. The closest thing Apple gets to providing updated info on their software is their antiseptic support documents.

Adobe can't find a way to get h.264 encoding from Compressor working with Blu-ray, so they are using MPEG-2 encoding. Adobe will encode your high-def to h.264 for you, but it's nice not to have to do an export of your entire high-def show before you even start to encode.

I found this blog post over at Final Cut User who is also bemoaning Apple's lame response to Blu-ray.

Upscaling Software Comparison

"Integrating DV Source Footage into HDV Projects, Part 2" by Jan Ozer

Need to incorporate some standard-def footage into a high-def project? Then check out this article. It compares results of upscaling standard def footage to HD with 5 different packages, including FCP. The long and short of it is that it looks like spending $99 on Red Giant Software's Instant HD provides some nice improvements over the built-in scaling of FCP and Premiere Pro.

The article has some very handy comparison shots of the results from different packages. Click on the image with this post to see a larger view of a comparison of Instant HD and FCP. The image shows at the top, Instant HD, and below it, FCP. The handy red circles gives you points of comparison. Instant HD doesn't help with every shot though, and it doesn't create miracles, i.e., make your SD look as sharp as if it was actually shot in HD.

Not included in the comparison is a workflow mentioned by the author in another article that I posted about earlier--using Compressor. I think a good deinterlace from Compressor would have helped with some of the jaggies that showed up in all of the FCP examples of up-rezing. On the other hand, Compressor's good deinterlacing is very slow and the workflow is more complicated than just popping on a filter, like Instant HD, right in Final Cut.

The clear winner in the comparison, Algolith by AlgoSuite, is no longer available. The company has decided to discontinue the software to focus on dedicated hardware that does real-time processing. Not tested in this article is hardware-based upscaling like the AJA Kona 3 does or the Terranex.

Scaling Up your Video

Integrating DV Source Footage into HDV Projects, Part 1 By Jan Ozer

This is a good article on the basic issues on scaling up footage for high-definition video. Goes over frame resolution, pixel aspect ratio, and interlace. One good tidbit mentions an advantage to using Compressor with interlaced footage:

On the other hand, when converting from interlaced DV to progressive HD, deinterlacing is a huge component of the quality equation, which could and perhaps should change your workflow, particularly for Final Cut Pro editors. That’s because Final Cut Pro deploys different deinterlacing algorithms than Compressor, which uses the glacial but ultra-high quality Optical Flow algorithm from Shake. If converting from interlaced source to progressive, you may produce noticeably better quality by scaling and particularly deinterlacing in Compressor. This means creating a separate sequence for your SD footage, rendering it out through Compressor, then re-importing the file back into Final Cut Pro. On the other hand, I found scaling quality relatively close between Final Cut Pro and Compressor, so if you don’t need to de-interlace, scaling in Final Cut Pro should work just fine.

This article says that FCP and Compressor do an equivalent job of scaling-up video. But early this year I wrote about a tip for getting better-quality uprez with Photoshop. A tricky workflow, but if quality really matters then it's doable.
[via FreshDV]

My Most-Used Freeware--Bitrate Pro

I don't know why Apple doesn't include something like this in Compressor and/or Final Cut Pro. The free Bitrate Pro will do a lot for you. How much space will 2 hours of DV footage take up? 20.95 GB. If I want to fit 1 hour and 45 minutes of video on a single-layer DVD what is the highest bitrate that I should use? 5.98 Mb/s. These are the kinds of calculations you can do with bitrate pro. Includes all kinds of handy info about how much data is on a DVD single-layer vs. dual-layer and what data rates different codecs use like ProRes, DV, DV50, and DVCPro HD.

AJA has a similar piece of freeware AJA Data Rate Calculator that is more geared toward figuring out storage for editing. They have presets for most of the editing codecs that their hardware has native support for.

I couldn't remember the name of Bitrate Pro, but I was able to track it down on this cool page that Apple has for video software downloads. You can always use versiontracker or macupdate, but I get overwhelmed by all the software on those sites. I think Apple helps make it manageable to explore freeware and shareware (although there's a lot of demo stuff thrown in there too.)

Compressor 3 Tutorial

"Compressors' New Interface"

I just took a glance at this, but it looks like another good article over at kenstone.net. Not only does it go over some basic things about getting around the new version of Compressor, but it looks like it does a good job of talking about how to encode for DVD's.

I had a lot of trouble with variable bit rate encoding with Compressor 2, so I've been just sticking with constant bit rate. But once I upgrade I'm going to have to figure out the average vs. max bit rates for variable bit rate encoding. Some stuff on that in the article, plus it will help you figure out the normalize setting for your audio.

[via Studio Daily Blog]

Multiple Instances on Compressor 3

"How to Make Compressor 3 Encode much, MUCH faster"

HDforIndies.com shows you how to increase the number of instances for rendering in Compressor 3. This can help maximize CPU usage if Compressor isn't using as much CPU as it can. It also refers to a BareFeats article that has some examples of how much multiple instances can improve rendering time.

More Creative Cow on ProRes

"Part 2: ProRes for HDV, with a splash of Color"

More nitty gritty fromt the Creative Cow on using ProRes 422. Here's my big takeaway:

PPPS. Just so there's no question here: most (but not all) of you are certain that native HDV capture is the way to go. Skip AIC. Skip DVCPRO HD. Just render to ProRes.

PPPPS. Of course, some of this changes once we add a little Color.

So basically if you have HDV footage you can capture it as HDV over firewire, and don't need to think about how you would transcode it to ProRes or rig your system to capture it to ProRess, which you can't do while capturing over firewire. Capture to HDV, but render in the timeline to ProRes 422.

Apparently sending HDV to Color and then rendering back to ProRes is totally screwed up. All your clips get out of sync. Seems like Apple didn't put the kind of quality control into Color as they do FCP before releasing.

Also an interesting note about removing pulldown on 24p footage, as an update to my previous post. Compressor 3 has a smart pulldown remover where you don't have to worry about the cadence of the pulldown. The downside is that you lose timecode.

And the article says that capturing to standard-def ProRes 422 is OK on a Quad G5. But for HD ProRes you need at least a quad-core Intel.