Monday, December 10, 2007

Working with Native Digital Video Formats - DV

A companion post to the previous post on HDV native formats and workflows, this one explores DV workflows, which are often much simpler than those of its high-definition cousin.


DV, short for MiniDV, used to be considered a professional format and still is, in many ways. It’s a format, though, that’s used by many more consumers than professionals, thanks to price drops in recent years as HDV and other high-definition and disk-based formats have entered the market.


This post assumes knowledge of the HDV post.


In a blog post last week, I walked through the workflow of capturing raw HDV files and preparing them for transcoding to Flash video formats. Today's post focus on the same workflow for DV.


Before I start, though, an update on the mystery announcement I mentioned: Adobe announced, on December 4, the final configuration and pricing for Flash Media Server 3 (FMS3). With an eye toward streaming and interactive (two-way or multi-way) video, FMS3 is now available in two flavors: the lower-priced, single-box Flash Media Streaming Server and the higher-priced, scaleable Flash Media Interactive Server. Both FMSS and FMIS will be available in January 2008. For more details seehttp://www.streamingmedia.com/article.asp?id=9774 for an article and a companion podcast.


So back to DV; you might remember I’m writing a white paper on Flash Video choices, which includes shooting some clips to compare HDV and DV workflows as well as the perceived qualities of the resultant transcodes to various Flash video codecs. To test workflows, the raw files were captured in their native formats on both Mac and Windows platforms, and then moved the files cross-platform without requiring transcoding to any other video codec.


We talked about this last time, but it bears repeating: why use native formats? Besides the ability to do a bit-for-bit copy, there's also no need to spend additional time transcoding when moving cross-platform, and no surprises when an intermediate codec (like Apple's AIC, which is used for HDV in its iMovie and Final Cut Express products) isn't supported on the other platform. Plus native formats contain a significant amount of helpful metadata that's stripped away when an intermediate codec is used.


DV uses a Motion-JPEG codec, wrappered in a .DV extension, or .AVI or .MOV if you are capturing in QuickTime or on a Windows Type II DV format, respectively).


Methodology

To test DV capture, I used a tape I’d shot for the white paper I’m writing, on which I had recorded both DV and HDV. Every five minutes, a DV recording was interspersed with an HDV recording. In auto-capture mode on most NLE systems, this would result in the auto-capture utility generating an error when it hits the first portion of the tape that had the other format. The NLE tools I used for capture, though, on both the Mac and Windows platform, were chosen because they handily ignored HDV when placed in an DV capture mode. In the DV capture sequence, HDV was rendered as a blank screen, which looks similar to the capture locking up, except the screen's timecode display continues to advance.


Speaking of tools, here’s a list of the overall tools I used for native DV capture, file manipulation and transcoding:

  1. Canopus Edius Neo (Windows)

  2. Canopus DV File Converter (Windows)

  3. iMovie (from iLife 08) (Mac)

  4. MPEG StreamClip (Mac, Windows)

  5. Sorenson Squeeze 4.57 (Mac, Windows)

  6. On2’s FlixPro 8.53 rc3 (Mac, Windows)


After capture, the next step was to immediately attempt a transcode from the captured native format to a one of several video streaming formats, including Flash (On2’s VP6 or Sorenson’s Spark) or H.264, gauging whether the applications were able to understand native DV format extension (.DV) or instead required some form of massaging (but not extra transcoding) to reach a point where a streaming file could be created. No transcodes were attempted from the NLE’s timeline in an attempt to avoid extra renders and the concatenation issues that often plague NLE exports to intermediate files.


Opening the raw file in the transcoding tool worked some times, but other times it didn't work either because of the native file format's original wrapper or the transcoding tool's inability to see particular multiplexing (interleaving) files. Then I needed to do a bit more work so, if a file's extension needed to be changed or a wrapper modified, this was accomplished at the operating system level or through the use of stand-alone software.


DV Capture and file preparation - Windows Workflow

In my previous post, I mentioned that I chose Canopus Edius Neo (the lower-priced cousin of Edius Pro) because its focus is strictly native HDV and DV. The program was adroit at capturing DV clips and ignoring HDV segments of the tape, just as it had ignored DV while capturing HDV. Captures were then broken down into "clip divisions" which corresponded to each time I’d hit the record button to start or stop the recording.


As with the clip divisions for HDV, Neo presents timecode start/stop times in the bin and on the editing timeline, but doesn't carry this over to the naming of clips at the OS level: Windows Explorer are only numbered sequentially - Cap000(001).m2t, Cap000(002).m2t, etc. This failure to correlate file names in any way back to the original tape footage makes manual naming critical for these files before moving to the next step.


The next step, after capturing the clips and renaming them is to close Edius Neo and open another program called Canopus DV File Converter. Canopus' proprietary DV wrapper and extension don't change the DV bit-for-bit content, but the way it interleaves audio and video content does make it nearly impossible to open the files in another program to transcode files to Flash streaming formats.


DV File Converter has one function: convert between Canopus DV format wrapper and other types of DV wrappers, including DV Type I and DV Type II AVI files. All of these formats store the exact same bit for bit copy of the DV video clip, but some of them write the files to disk differently (interleaving audio in different ways, for instance). Focus Enhancements, the company that I think best grasps the vagaries of different DV formats, has a good cross-reference on DV file formats. Some of these files show up at 720x480 while others show up as 640x480 resolution.


The program introduces no quality loss and can save in both Microsoft's DV Type I and Type II wrapper (both in an .AVI extension). As is typical of naming conventions, MSDV Type I is newer (and was used for Movie Maker 2) while MSDV Type II is older and was used in Movie Maker 1. Confused yet? Don't be; just use Type I as that can also be opened in anything that understands QuickTime, whether on the Mac or Windows platforms.


DV File Converter works extremely quickly, rewrappering the Canopus DV files and putting them into the .AVI extension, with no quality loss. The .AVI files can then be used by many transcoding programs on the Windows platform, including Sorenson Squeeze, which uses QuickTime as a portion of its underlying engine.


The files can also be transferred to a Mac and used as is, but a quick note of caution: since the MSDV codec also has its own unique interleaving scheme, don't attempt to change the file extension from .AVI to .DV for either MSDV Type I or Type II. I did tests on both types and was rewarded with digital snow throughout the image, probably due to the need to way audio is interleaved. Fortunately the damage wasn't permanent as I was able to change the extension back to .AVI and the file worked flawlessly again.


DV Capture and File Preparation - Macintosh Workflow

All I can say is that I'm impressed with iMovie 08 (bundled free with new Macs as part of iLife 08). Not only does it do a great job with DV in a native format, it excels in ignoring footage in other formats during capture, just like Canopus' Edius Neo did.


In addition, and to me this is where iMovie 08 really shines, the automatic capture segments everything in the bin (and at the operating system level) by clip name that includes the time (hour;minute;second) of the initial frame of each clip of footage. It’s not the time it was transferred to the computer, which has no bearing on either timecode or the day a segment was recorded; instead, the time stamp is metadata from the video tape itself, and indicates the starting frame of the day and time it was recorded.


This naming convention, coupled with timecode information in the bin, automatic splitting of clips (like Edius Neo) and automatic folder structure creation both in the bin and at the OS level, makes iMovie 08 idealf for that integrates nicely with the OS. And no stupid intermedia codec, such as the AIC I was ranting about in the previous post; these files, output in a .MOV extension, can be used cross platform with no difficulty at all, including no name changes if the transcoding tool is capable of handling QuickTime files.


DV File Preparation - both Mac and Windows

I thought this deserved a separate mention, since it works on both Mac and Windows boxes: MPEG Streamclip, which was discussed in the previous post as a good way to demux (separate) MPEG-2 transport streams into individual elementary streams (separate audio and video files) also has a useful DV feature. One of the options is Export to DV which has its own extra trick: a very good de-interlacer. I've tried it on numerous files and it not only does a good job of deinterlacing DV footage, making it appear sharper as a standalone .DV file, but it also carries that sharpness through into transcodes for Flash video codecs.


On top of that, the file size is reduced in half, thanks to the way MPEG Streamclip handles deinterlacing, but QuickTime views the data rate of the file as if it were the original size. I at first thought my eyes were playing tricks on me, and I would caution additional testing if you're going to use these DV files cross platform or the video is critical, but it's an amazing bit of workI hope the guys at Squared5 (www.squared5.com) can shed light on.


DV Transcoding - Macintosh and Windows

Once you are done capturing and have made any file preparation changes to DV wrappers or file extensions, it's time to transcode the files. MPEG Streamclip can output to On2 VP6 and H.264, but for final clarity, I recommend using On2's FlixPro 2.5 or greater for Flash 8 Video (VP6-E) output and Sorenson Squeeze for all other outputs to Flash (Spark, H.264).


Sorenson Squeeze, one of the premier transcoding tools, with an excellent batch encode setup and a great watch folder capability (it checks a folder for any new files to transcode into one or multiple new streaming files) can access and transcode from an DV file in the .AVI or .MOV or .DV file extension. On2’s Flix Pro can do the same thing and provides On2’s stellar VP6 support (including, soon, VP6-S simple profile for HD 720p files).

No comments: