The present and future of post production business and technology

We were talking a couple of night ago and discussing how various of the NLE developers are dealing with the Carbon/Cocoa transition. But before I get to that a disclaimer and some background.

Disclaimer: I have no inside knowledge. If I did it would almost certainly be under NDA and therefore would not be shared. Everything I post here is based on publicly available knowledge, if not that commonly known. (Depends on how much you care about code and most people don’t.) So, I present data points, interpretation and extrapolation: in plain English, guesswork, but intelligent guesswork! PLEASE DO NOT CONSIDER THESE RAMBLING “RUMORS” OR HAVING ANY INSIDE KNOWLEDGE. THEY DO NOT AND ARE NOT RUMORS.

Final Cut Pro, like most software developed originally for OS 9, is built on a type of code called (for short) Carbon. The Carbon APIs (Application Programming Interface – the building blocks of code) allowed Final Cut Pro, Photoshop, Illustrator, Media Composer and thousands of other applications to make the transition from OS 9 to OS X “reasonably painlessly”. (Meaning, hard but not impossible). Carbon code runs just fine on OS X and is no less efficient in and of itself than the more modern Cocoa code.

So-called Cocoa code is the native language for OS X. It is built almost completely on NeXTstep, which Apple acquired when they acquired NeXT (and Steve Jobs) as a replacement for OS 9. Now, originally Apple said (at the 2006 WWDC) that there would be 64 bit versions of the Carbon APIs. This would have meant that Final Cut Pro, Media Composer, After Effects, Photoshop, Illustrator, et al. would be able to move to 64 bit versions without major rewrite. And so it was good.

Until a year later. At WWDC in July 2007 Apple reversed that decision and said that any application that wants to go to 64 bit would have to be rewritten to Cocoa.  Much gnashing of teeth in the ProApps camp and at Avid and Adobe. Not only can’t they go to 64 bit without rewriting but Final Cut Pro cannot start to use OS X technologies like Grand Central Dispatch and OpenCL until that rewrite is done.

I’m much less familiar with Adobe’s code but the current version of Premiere Pro was ported to OS X in the modern code era and is almost certainly Cocoa where it hits the OS X road. It is highly likely that the majority of the code is in a format that is common to both platforms with mostly interface-specific code for each platform.

That’s also likely with the Media Composer code but I have reason to believe that Avid have been progressively rewriting functional blocks of code from Carbon to Cocoa over the last several releases (since mid 2007 probably)!

Most of the ProApps are already written in Cocoa: Soundtrack Pro, Motion, DVD Studio Pro (when it was based on Spruce not Astate’s code) and Compressor. These are already in a form that makes it relatively easy to move forward to take advantage of modern OS X technologies.

Not so Final Cut Pro.  Now we do know that most of what has been added to Final Cut Pro in recent versions has been written in Cocoa. Apple’s Xcode development tool allows a mixture of code types in the one application. I’m uncertain whether Multicam is written in Cocoa but I’d expect it to be. HDV Log and Capture; Log and Transfer; Share, Master Templates etc are clearly also written in Cocoa. (The main evidence is that the interface is using the “ProApps Kit” interface used in Motion, Soundtrack Pro et al.)

So to the question that started the post: “Are Apple rewriting parts of the code as they go?”  I think the answer is yes.

One really strong piece of evidence is the new Speed Change tools in Final Cut Pro 7. The new interface is ProApps kit, not Final Cut Pro’s interface elements, which by itself suggests new Cocoa code. What is stronger evidence is that speed changes in XML files give different results when imported to Final Cut Pro 6 than they do when imported to Final Cut Pro 7. This is very strong evidence that new code is involved. (The old code would give the same result even with a new interface.)

One would have to extrapolate that the new Marker functions (with their new interface) has also been given new code but that’s much less certain as the Marker interface still shows the original Final Cut Pro interface style with new elements added. (Compare the Speed Change dialog and Marker dialog to see the difference.)

The rewrite to Cocoa, even assuming they don’t make fundamental changes* is very time consuming and a lot of hard work to rewrite and test. That there is evidence in the current release of work already complete strongly suggests that the team is hard at work doing what’s necessary to bring Final Cut Pro into the modern Cocoa OS X code era. But don’t expect to see a converted release any time soon. There’s a lot of work that the QuickTime team has to do to add functionality to the underlying QTKit API (The modern QuickTime API for programmers) that an updated Final Cut Pro needs. Right now there’s no support for QuickTime metadata in QTKit, for example.

* Fundamental Changes. We’ve argued this instead of watching TV as well. Most of Final Cut Pro’s functionality is just fine. There’s not a lot to be gained by totally redesigning and redeveloping the Transition Editor, for example. There are, however, two areas that I think it would behove Apple well to rethink: Media Management and Metadata views. Media Management in Final Cut Pro is now reliable, except for a few edge cases (largely to do with dual system and merged clips). Whether or not to spend a lot of effort (and dollars) to improve it that last little bit for the relatively small customer base that would benefit is a management decision I’m glad I don’t have to make!

I do think they need to do something more flexible with metadata support. Non-tape sources come complete with comprehensive metadata that Apple capture and insert into the media file. This support was added in my all-time favorite Final Cut Pro release – 5.1.2! Unfortunately the Bin interface has limited flexibility. While there are a lot of viewable columns there’s no way to add a column unless you’re the Apple engineer that does that!  Other NLEs are much more flexible. Media Composer will add as many columns for whatever metadata you ask it to display. Not so Final Cut Pro where the only simple way to view QuickTime metadata in a QuickTime file is with my company’s miniME. (Go on, download it. The free demo lets you view all QuickTime metadata in a file and export it to an Excel spreadsheet. Buy it and you can remap the metadata into Final Cut Pro’s columns.)

It takes years to make major transitions in software. QuickTime metadata support came in late 2006. With the advent of Log and Transfer, that support became valuable. So my informed guess is that a future release of Final Cut Pro will allow that metadata to be viewed and used in Final Cut Pro. It’s all there in the file and in any XML export. To me that suggests a foundation for some future construction.

Like I said, nothing more than intelligent guesses. NOT RUMORS. NOT INSIDE INFORMATION. Just me joining dots and I’m bound to be wrong about half of it. I just don’t know which half!

Occasionally I do some work for my day job at Intelligent Assistance!, where we’re actively adding to our metadata-based workflow tools. This time taking the speech transcription metadata from the adobe suite and making it accessible to producers who want text or Excel versions, or even into FCP with the transcription placed into colored markers (one color per speaker). With Transcriptize you can also name the speakers, something not possible in the Adobe tools.

Here’s my interview with Larry Jordan where we announced it and the press release is below.

“Announcing Transcriptize on the Digital Production BuZZ”

Transcriptize expands the usefulness of Adobe Speech Transcription

Take transcriptions from Adobe Production Bundle to Media Composer, Excel and Final Cut Pro.

Burbank, CA (February 12, 2010) – Intelligent Assistance, Inc has introduced a new software tool that takes Transcription XML from Adobe Premiere Pro CS4 or Soundbooth CS4 and converts it text, Excel Spreadsheet or Final Cut Pro clip markers.

“Late last year, Larry Jordan asked if we could create something to make the Adobe Speech Transcription more available”, says Intelligent Assistance’ CEO Philip Hodgetts. “We thought that was a great idea and Transcriptize is the result, less than two months later.”

Transcriptize imports the transcription XML from the Adobe Production Bundle and allows editors and producers to name the speakers – something not possible in the Production Bundle. From there users have the option to:

Export a plain text file, suitable for the needs of a producer or to import into Media Composer’s Script Sync engine.

Export an Excel spreadsheet with a variable number of words per row – Perfect for a producer.

Open the XML from a Final Cut Pro clip and add the transcription to Markers where:

There are a variable number of words per Marker (including one Marker per speaker)

The speaker name is placed in the Marker name

Transcription appears in the clip Marker comment

Marker colors are used to identify each speaker (FCP 7 onward).

The transcription can be searched within Final Cut Pro.

Markers can be easily subclipped based on transcription content.

Transcriptize is available now from www.assistedediting.com/Transcriptize/. MSRP is US$149 with an introductory offer of $99 until the end of February 2010. NFR versions for review are available, contact Philip Hodgetts, details below.

In my experience few productions – be they film or television – are well planned from a workflow perspective. It seems that people do what’s apparently cheapest, or what they have done in the past. This is both dangerous – because the production workflow hasn’t been tested – and inefficient.

In a perfect world (oh *that* again!) the workflow would be thoroughly tested: shoot with the proposed camera, test the digital lab if involved; test the edit all the way through to the actual output of the project. Once the proposed workflow is tested it can be checked for improved efficiency at every step. Perhaps there are software solutions for automating parts of the process that require only small changes to the process to be extremely valuable. Perhaps there are alternatives that would save a lot of time and money if they were known about.

Instead of tested and efficient workflows, people tend to do “what they’ve done before”. When there are large amounts of money at stake on a film or TV series it’s understandable that people opt for the tried and true, even if it’s not particularly efficient because “it will work”.

Part of the problem is that people simply do not test their workflows. I’ve been involved with “film projects” (both direct to DVD and back out to cinematic release) where the workflow for post was not set until shooting had started. In one example the shoot format wasn’t known until less than a week before shooting started.

Maybe there was a time when you could simply rely on “what went before” for a workflow, but with the proliferation of formats and distribution outputs, there are more choices than ever to be made.

Which brings me to the other part of the problem. Most people making workflow decisions are producers, with input from their chosen editor. Chances are, unfortunately, that neither group are very likely to truly understand the technology that underpins the workflow – or even why the workflow “works”. They know enough of what they need to know to get by but my experience has been that most working producers and editors do not actively invest time into learning the technology and improving their own value.

And when they’re not working, they’re working on getting more work. Again, not surprising.

But somewhere along the way, we need producers to research and listen to advisors (like myself) who do understand the workflow and do have a working knowledge of changing technology that can be make a particular project much more efficient to produce, but I have no idea how to connect those producers with the people who can help.

We’ve seen, in just a little under two years, how technology can improve workflows, just with our relatively minor contributions:

Rent a couple of LockIt boxes (or equivalent) on set and save days and days synchronizing audio and video from dual system shoots;

Log your documentary material in a specific way, and take weeks off post production finding the stories in the material (Producers can even do a pre-edit);

Understand how to build a spreadsheet of your titles and how to make a Motion Template and automate the production of titles (and changes to same).

If you know you can recut a self contained file into it’s scene components, how does that change color correction for your project;

Import music with full metadata.

These are all examples of currently-available software tools from my company and others that are working to make post production more efficiently. I wrote more about this in my Filling in the NLE Gaps for DV Magazine.

My question though, is how do we encourage producers to “look around and see what’s available” and open up their workflows to a little modern technology. To this end, Intelligent Assistance is looking to work closely with a limited group of producers in 2010 to find ways to streamline, automate and make-more-robust postproduction workflows. So, if you’re a producer and want to save time and money in post, email me or post in the comments.

If you’ve got ideas on how encourage producers move toward more metadata-based workflows? How do we get the message out?

On the Yahoo-based Final Cut Pro list MarkB posted this just a few minutes ago:

Gave my kids (14 & 16) a green screen kit from Cowboy Studio for Xmas. The older one does a sports video blog, the younger one shoots and edits it (Canon HV20 camera, Final Cut Express, 4-year old iMac).
They used their new chroma key trickery today. I helped them set up the green screen, gave the younger one a 5-minute lesson in how to use my DV Garage plugin, then stayed out of it except for a tip or two. This is what a 14-year old kid can do first time:
http://www.youtube.com/user/JLBsportsTV

Watch the video or at least the first couple of minutes. (I’m not that into football/soccer so it doesn’t mean much to me) but look at the work.Not only is the 16 year old good talent, but the way it’s put together is damned nice too. (In this style of presentation I’ll overlook my long-standing distrust of jump-cuts and live with fact that it’s become an acceptable style: heck in this example I think it works fine.)

Mark mentions that the keyer they used was DV Matte Pro, which I’ve also had a lot of success with: using it on A Musical Journey with Richard Sherman, on the 40th and 45th Anniversary Edition Mary Poppins DVD. After testing all that were available that’s what gave us the best results (although it does have a different approach to fine tuning edges than most keyers, which threw me at first).

I’ve long argued that we have to constantly be improving our skills, because those coming up behind us are staring with a whole lot better craft/technical skills that we did. In fact, we have to keep learning to keep up and make sure our experience and people skills are a whole lot better.

But it does make you wonder what these guys will do with their Christmas “green screen kit” if ever they discover 3D. (I suspect that would be the 14 year old’s realm.) Does easy, accessible keying technology really change production forever?

Over recent months Larry and I have spoken regularly on a variety of topics, so I thought I’d post some of the interviews here.

RED Digital Cinema’s latest announcements and more on how we’re going to fund entertainment

http://www.digitalproductionbuzz.com/BuZZ_Audio/Hodgetts_BuZZ_091105.mp3

More of my thoughts on the Democratization of production

http://www.digitalproductionbuzz.com/BuZZ_Audio/Hodgetts_BuZZ_091126.mp3

My Look Back on 2009

http://www.digitalproductionbuzz.com/BuZZ_Audio/Hodgetts_091224.mp3

My thoughts on what to expect in 2010

http://www.digitalproductionbuzz.com/BuZZ_Audio/Hodgetts_091231.mp3

Although I come from a country where we count frame rates in whole numbers and, therefore don’t need to skip frame counts in the timecode, I thought I understood it at least as well, or better, than “NTSC natives”. That is, until I really, truly had to understand it to ensure the reported time counts in Sequence Clip Reporter are accurate.

Well, it took three pieces of additional information before I truly understood it: Rainer Standke’s insistence that “frame is a frame is a frame”; the realization (although known) that 29.97 is actually slower than 30 fps with the consequence that each frame runs slightly longer than at 30 fps; and that the correction can’t be applied evenly.

At one level I knew all of these things, but it really all came together when I got the “right” mental picture. Since I could not find any illustrations that showed why the skip-frame timecode (a better term than drop frame imnsho) needed to skip frames, I decided to create this one. It’s licensed with a Creative Commons – Attribution license. That means you can reproduce it, or use it for any purpose as long at the attribution to me remains.

Not shown on the image is that the 2 frame skip-forward happens at every minute, except ever 10 minutes. There are other parts to the pattern as the timescale gets longer.

Click on the image to load the full size.

When Larry Jordan sent me out on the Exhibit Floor to find out what Panasonic, JVC, Sony and hard drive manufacturers recommended for long-term storage for non-tape media, and the answer surprised us both: Blu-ray.

Here’s the interview from the Digital Production BuZZ where I explain my findings.

Of course, I don’t have any direct link into the mind of Grant Petty, founder of Blackmagic Design and don’t know more about the purchase of daVinci other than what Grant posted, but it’s such an interesting purchase that I can’t help but comment and guess.

Like so many of the industry’s giants of old, daVinci was losing money in the face of lower priced competition (Apple Color) and a reliance on mostly-obsolete 2K-limited hardware. On the other hand, Resolve is software only and resolution independent running on a cluster of Linux machines connected with Infiniband high speed data interconnect. daVinci also have Revival, although I don’t know anything about what advantages it brings.

Clearly, Grant thinks that the company has not been making the most of its opportunities and more focus on marketing and product development will once-again bring the daVinci brand to prominance. (Assuming it ever lost it.)

However, I don’t expect we’ll see Blackmagic Design suddenly want to start competing with Apple Color. I don’t think that’s the market and Grant himself seems to rule out that direction:

DaVinci Resolve is unique because it uses multiple linux computers linked together with InfiniBand connections and multiple GPU cards so you get the real time performance advantage it has. I donʼt think that can be lowered in price much, however over the next few years as technology advances this might happen a little. However, DaVinci is different to a DeckLink card because itʼs a high performance computing based tool. Our focus will really be on adding more features. Thatʼs what we want, and I guess others would too.

Possibly, some time in the future, a network of multiple Linux machines might be replaced by optimized code on some future 8+core Mac with awesome graphics card and an application written with Grand Central Dispatch and  OpenCL in mind. But don’t hold your breath! Combined CPU+GPU power has to increase a lot to replace multiple machines and the market is not that big.

I think the move will allow daVinci to continue developing their modern products and repositioning the company (to be operated independently of BMD) for the mid-size post house: those that have become dissatisfied with Apple Color but who would not have purchased a full daVinci hardware/software package. If the price could be, say $60K instead of $300K (or more) then that has a really good chance of reviving the brand and – in that inevitable trend – make higher quality available at lower price. That has always been Grant Petty’s goal, so it seems this is consistent.

My friend James Gardiner wrote an interesting post “Are large Post Houses a sunset industry?” and it set me thinking. Now James is writing from an Australian perspective and “large post house” and “boutique” post house have quite different expectations of size than the Australian context. (For example, Alpha Dogs in Burbank bill themselves as a “boutique” post house but in Sydney or Melbourne they’d be one of the larger post houses.)

In general principle he’s right. The economics of the large post facilities (really factories) of the size of IVC, FotoKem, Ascent Media’s various facilities are changing. They probably always have been. And certainly there are signs that the very large post-focused facility in New York and Los Angeles are threatened. Long-term post Burbank post factory Matchframe sold a majority stake for just $300,000 (mostly because of long term debt it is presumed). The costs of maintaining the “heavy iron” of a big post facility can be millions a year.

In general principle I agree with James: these large facilities are probably a sunset industry. But he identified one point that I wanted to expand on.

What a big post house bring to the table is more then just services, they bring know how and knowledge.  You KNOW it is going to work.

That alone is the reason that there will (almost certainly) be facilities like these big post factories: at least in LA and NY. These facilities are large enough to be able to experiment and invest in discovering the best workflows (as, indeed, do the people at Alpha Dogs et. al.) and technologies.

But knowledge gets shared. This is one of the absolutely best things about the current Internet Era: knowledge is freely shared in ways it never could be before.

Look at RED workflows. The RED Digital Cinema camera is a big step forward in performance-for-price and a new class of digital cinema camera. When it was first released the tools and workflow where completely unexplored. None of the major NLE companies had native support for the new wavelet codec and working between NLE and color correction caused nightmares.

Two years on and there are established “best practice” workflows across Final Cut Pro, Media Composer and Premiere Pro. Pretty much anyone who does a little research can find a workflow that’s tested. Where did the posts you find when you do that Internet search come from? People who have solved a problem, sharing the solution with other who have the same problem.

Frankly, this information sharing is what made my reputation. As a very early adopter of NLE (specifically a very early adopter of Media 100) I ran into problems earlier than those who purchased later. I also discovered email groups in early 1997 and benefited from the shared experience of the Media 100 Email List of fellow travelers dealing with NLE in the mid 1990’s. (All digital for more than a decade now.)

I don’t know what form the future post-house/factory will be, but what will survive are the “centers of knowledge” because ultimately that’s more important than expensive, but infrequent access to high-priced technology.  The latter will continually get cheaper and people will find smarter, faster ways to do things, that ultimately become best practice and the “norm” again.

I’d be remiss if I didn’t point out two of our own tools that can give a FCP facility and edge: Sync-N-Link synchronizes dual system video and audio in minutes rather than hours, or if you’re working with an edited Sequence replacing camera audio with multi-track in hours instead of weeks. Sync-N-Link is already being used across a lot of Network and Cable series.

Producers have been printing out EDLs and trying to match them to a spreadsheet to report clip usage or music usage: a tedious task for sure, but one that can be automated with Sequence Clip Reporter, which just takes the pain out of creating a video, audio or combined report, including a reel-by-reel report if that’s what you need.

With the release of Snow Leopard (OS X 10.6) this week, we finally get to see QuickTime X.

Simply put, QuickTime X is, as predicted, a simplified media player and simplified architecture optimized for playback of linear video streams. Most of what made QuickTime interesting to interactive authorers back a few years, is not present in QuickTime X.

We gain some new features: 2.2 gamma, screen capture and easily publish to major online video sharing sites. Screen capture is a nice addition and easy sharing probably would have been predictable if we’d seen Final Cut Pro 7 earlier.

The 2.2 gamma will no doubt take some time to get full adoption but at least it provides a way for us to add or change a color profile. Files with color profiles automatically adjust display to look correct on all screen. (At least, that’s the theory.) Within the Final Cut Studio it seems that correct gamma will be maintained *if* conversions are done with Compressor and not QuickTime 7’s Pro Player.

Chapter display has changed from a pop-up text list to thumbnail images. Better for consumer focused movies; less good for professionals.

Fortunately, it’s not an either/or. You can choose to install QuickTime 7.6 in addition to QuickTime X. If you try and access a movie that requires QT 7 features, users will be prompted to install QT 7 (aka “the real QuickTime!). If you want to make sure it’s installed, Apple have instructions on installing it.

So that’s the story of QuickTime X – a simple, consumer-focused player with a modern-looking interface, just as I predicted a little over a year ago.

Added 8/31 Just got this off a QT Apple email list. It’s not an official word from Apple but I think it sums it up well:

Quicktime X at this time isn’t a replacement to Quicktime 7, just allows faster multi-threaded playback of some of the older codecs.

Added 9/1 Ars Technica has a deep article on the difference between QT X and QT 7 and how QTkit negotiates between them,  that confirms I got my “educated guesses” right and provides more depth in how Apple achieves this.