Test Criteria - Professional Transcoding and Live Encoding comparison testing

As mentioned yesterday, two additional companies expressed interest in the comparative testing for professional transcoding and live encoding workflow systems: Sorenson and a new player in the space, Octasic (or 8 ASIC).

While neither of the companies has enough of a total solution to compete in this year's testing, both may have solutions in the near- to mid-term that would fill the gaps and allow them to compete during our second annual test, to be held in 2011. That still leaves 13 companies, including Ateme, Envivio, Harmonic, Inlet, Media Excel, Telestream, who are weighing their solutions against the test criteria.

What are the testing criteria?  Here's a generic overview of the way Transitions will test the various hardware-software solutions:

1. Transcoding: these tests come first, as a way to establish a baseline of quality, with speed secondary but almost equally important.

Here we'll look at things such as failover, including job completion and workflow steps; ability to transcode a single file to multiple adaptive bitrates.

On the speed side, we'll look at speed to transcode files from M2T (MPEG-2 TS), H.264 and another format back and forth to each of the other formats; whether the solution is capable of simultaneous transcodes; and we'll judge the speed of transcoding multiple files to multiple adaptive bitrates.

Resolutions to transcode to will be: 1080p, 1080i, 720p, several web and mobile rates.

Finally, we'll see the number of concurrent file-based transcodes that can occur, based on three workflows: Real-time, Faster Than Real Time, Real-Time xN

2. Live Encoding: after transcoding, we'll turn the testing criteria on its head and look at what each solution can do within a set period of time - specifically within the confines of live encoding testing.


Both tests will also assess performance, throughput and ease of use in setting up a variety of workflows, taking a holistic approach to pro transcoding and live encoding workflows. For instance, for clustering, we'll look at initial setup (3 total units, plus mgmnt node and 5 total units, plus mgmnt node), replacement of individual node(s) or the management node.

Part of the testing will take place in our lab environment, to allow for a common set of test criteria in a typical workflow. In another part of the testing, we'll jointly assess management, clustering and fail-over response, in your lab or quality control location – recommended, unless you want to ship a hefty rack of gear.

To keep the tests objective but still cover the testing time, a sponsorship model is being used. We've found with large groups of test candidates that a sponsorship with equal amounts paid by each sponsor - so that no one company dominates the report - works best.

That's the model we'll use for these tests, and more details on the approach, as well as examples of previous multi-party tests and test methodologies have been made available on request to all invited companies.