The present and future of post production business and technology

I probably have mentioned that we’re working on a documentary about Bob Muravez/Floyd Lippencotte Jnr in part because we wanted demo footage we “owned” (so we could make tutorials available down the line) but also because I wanted to try it in action on a practical job.

I start work in prEdit shortly – nearly started today, so it looks like Friday now – but already we discovered some ideas that have now been implemented in the 1.1. release.

Along the way I’ve learnt a lot about how well (or not) Adobe’s Speech Analysis works. (Short answer: it can be very, very good, or it can be pretty disappointing.) As prEdit is really designed to be used with transcriptions I also tested the Adobe Story > OnLocation > Premiere method, which always worked.

Well, from that workflow it became obvious that speakers (Interviewer, Subject) were correctly identified so wouldn’t it be nice if prEdit automatically subclipped on speaker changes. And now it does.

If multiple speakers have been identified in a text transcript, prEdit will create a new subclip on import at each change of speaker

It also became obvious as I was planning my workflow that we needed a way to add new material to an existing prEdit project, and to be able to access the work from historic projects to add to a new one.

New Copy Clips… button so you can copy clips from one prEdit project to another open project

Now that I’m dealing with longer interviews than when we tested, I needed search in the Transcript view.

Search popup menu added to Transcript View.

That led to one problem: adding metadata to multiple subclips at a time. Previously I’d advocated adding common metadata to the clip before subclipping in prEdit (by simply adding returns to the transcript) but if it comes in already split into speakers, that wasn’t going to work!

Logging Information and Comments can be added for multiple selected subclips if the field doesn’t already have an entry for any of the selected subclips

Because you never, ever want to run the risk of over-writing work ready done.

And some nice enhancements:

Faster creation of thumbnails

Bugfix for Good checkbox in Story View

prEdit 1.1 is now available. Check for updates from within the App itself. And if you work in documentary, you should have checked it out already.

‘Interoperable Master Format’ Aims to Take Industry Into a File-based World http://bit.ly/bvF6Vk

A group working under the guidance of the Entertainment Technology Center at USC is trying to create specifications for an interoperable set of master files and associated metadata. This will help interchange and automate downstream distribution based on metadata carried in the file. The first draft of the specification is now done based on (no surprises) the MXF wrapper. (Not good for FCP fans, as Apple has no native support for MXF, without third party help).

Main new items: dynamic metadata for hinting pan-scan downstream and “Output Profile List”:

“The IMF is the source if you will, and the OPL would be an XML script that would tell a transcoder or any other downstream device how to set up for what an output is on the other side,” Lukk explained.

The intention is to bring this draft spec to SMPTE, but first, ETC@ USC is urging the industry to get involved. “We need industry feedback and input on the work that the group has done thus far,” said ETC CEO and executive director David Wertheimer. “Anyone who has interest in this topic should download the draft specification and provide feedback to the group.”

The Future of Picture Editing http://bit.ly/aNRLVA

I’ve had the pleasure of meeting Zak Ray when I travelled to Boston. I like people who have an original take on things and Zak’s approach to picture editing – and his tying it to existing technologies (that may ned improvement) – is an interesting one.

And yet, despite such modern wonders as Avid Media Access and the Mercury Playback Engine, modern NLEs remain fundamentally unchanged from their decades-old origins. You find your clip in a browser, trim it to the desired length, and edit it into a timeline, all with a combination of keys and mouse (or, if you prefer, a pen tablet). But is this process really as physically intuitive as it could be? Is it really an integrable body part in the mind’s eye, allowing the editor to work the way he thinks? Though I can only speak for myself, with my limited years of editing experience, I believe the answer is a resounding “no”. In his now famous lecture-turned-essay In the Blink of an Eye, Walter Murch postulates that in a far-flung future, filmmakers might have the ability to “think” their movies into existence: a “black box” that reads one’s brainwaves and generates the resulting photo-realistic film. I think the science community agrees that such a technology is a long way off. But what untilthen? What I intend to outline here is my thoughts on just that; a delineation of my own ideal picture-editing tools, based on technologies that either currently exist, or are on the drawing board, and which could be implemented in the manner I’d like them to be. Of course, the industry didn’t get from the one-task, one-purpose Moviola to the 2,000 page user manual for Final Cut Pro for no reason. What I’m proposing is not a replacement for these applications as a whole, just the basic cutting process; a chance for the editor to work with the simplicity and natural intuitiveness that film editors once knew, and with the efficiency and potential that modern technology offers.

It’s a good article and a good read. Raises the question though – if Apple (or Adobe/Avid) really innovated the interface would people “hate it” because it was “different”?

Powerful new transcript workflow tool – paper cuts without the pain – from Intelligent Assistance (my day job). http://bit.ly/9nQv07

We just launched prEdit, our pre-editing tool for developing paper cuts (a.k.a. radio cut) from transcripts. prEdit:

• Lets producers or editors cut transcripts into selects in seconds
• Adds and updates log notes with auto-complete logging fields
• Previews the video for any clip, subclip, paper cut or section of paper cut
• Exports to Excel spreadsheets and Final Cut Pro, or Premiere Pro Sequences

“prEdit marks a new generation of postproduction tools,”  say I. “Video editing by text is a whole new way of working that will take weeks out of developing a paper cut.”

prEdit is available now from AssistedEditing.com and carries an MSRP of $395, discounted for an introductory special to $295 until August 31st. The prEdit workflow is described at http://assistedediting.com/prEdit/workflow.html and a video overview is available at http://assistedediting.com/prEdit. The first 80 seconds provide an overview.

The video is now available at YouTube  http://youtu.be/3fV388QsVVA?a

Thanks to an upcoming piece of software we’re working on I’ve been spending a lot of time within the CS5 workflow environment, particularly looking at metadata and the Story workflow, and I’ve come to the conclusion that we’ve been so blinded by the Mercury Engine’s performance that we might not have seen where they’re heading. And I like what I see.

Most readers will likely be aware of the Transcription ability introduced with CS4 and updated in CS5. Either in Soudbooth, or in Adobe Media Encoder (AME) via Premiere Pro for batches, the technology Adobe builds on from Autonomy will transcribe the spoken word into text. Our initial testing wasn’t that promising, but we’ve realized we weren’t sending it any sort of fair test. With good quality audio the results are pretty good: not perfect but close, depending on the source, of course.

We first explored this early in the year when we built and released Transcriptize, to port that transcription metadata from the Adobe world across to Apple’s. That’s what set us down our current path to the unreleased software, but more of that in a later post.

Now we’re back in that world, it’s a pretty amazing “story”. There’s three ways they get it that I see:

1. Good metadata tracking at the file level
2. Flexible metadata handling
3. Metadata-based workflows built into the CS applications (and beyond).

Balancing that is the serious miss of not showing source metadata from non-tape media that doesn’t fit into pre-defined schema. At least that seems to be the case: I can’t find a Metadata Panel that shows the Source Metadata from P2, AVCCAM/AVCHD, or RED to display. Some of the source metadata is displayed in existing fields, but they are only the fields that Adobe has built into Premiere Pro, which miss a lot of information from the source. For example, none of the exposure metadata from RED footage is displayed, nor Latitude and Longitude from P2 and AVCCAM footage.

That’s the downside. To be fair, Final Cut Pro doesn’t display any of the Source Metadata either (although you can access it via the XML file.)  Media Composer can show all the Source if desired.

Good Metadata Tracking at the file level

Apple added QuickTime Metadata to Final Cut Pro 5.1.2 where they retain and track any Source Metadata from files imported via Log and Transfer. This is a flexible schema but definitely under supported. Adobe’s alternative is XMP metadata. (Both XMP and QuickTime metadata can co-exist in most media file formats.)

XMP metadata is file based, meaning it is stored in, and read from, the file. There are seven built-in categories, plus Speech Analysis, which is XMP metadata stored in the file (for most formats) but considered as a different category in the Premiere Pro CS5 interface. I believe that the Source metadata should show in the XMP category because it is file-based even if its not XMP.

On the other plus side XMP metadata is very flexible. You don’t need third party applications to write to the XMP metadata. Inside Premiere Pro CS5 you simply set up the schema you want and the data is written to the file transparently. If the data is in a file when it’s added to a project, it’s read into the project and immediately accessible.

This metadata travels with the file to any and all projects. This provide a great way of sending custom metadata between applications. Speed Analysis metadata is also carried in the file, so it can be read by any Adobe application (and an upcoming one from us, see intro paragraph) direct from the file.

Flexible Metadata Handling

Not only is the XMP file-based metadata incredibly flexible, but you can also apply any metadata scheme to a clip within a Premiere Pro project, right into Clip metadata. For an example of how this is useful, let’s consider what we had to do in Final Cut Pro for First Cuts. Since Final Cut Pro doesn’t have a flexible metadata format, we had to co-opt Master Comments 1-4 and Comment A to carry our metadata. In Premiere Pro CS5 we could simply create new Clip-based fields for Story Keywords, Name, Place, Event or Theme and B-roll search keywords.

(Unfortunately this level of customization in Premiere Pro CS5 does not extend to Final Cut Pro XML import or export.)

An infinitely flexible metadata scheme for clips and for media files (independently) is exactly what I’d want an application to do.

Metadata-based Workflows in the CS5 Applications

To my chagrin I only recently discovered how deeply metadata-based workflows have become embedded in the Adobe workflow. (Thanks to Jacob Rosenberg’s demonstration at the June Editor’s Lounge for turning me on to this.) Adobe have crafted a great workflow for scripted productions that goes like this:

1. Collaboratively write your script in Adobe Story, or import a script from most formats, including Final Draft. (Story is a web application.)
   • Adobe Story parses the script into Scenes and Shots automatically.
2. Export from Adobe Story to a desktop file that is imported into OnLocation during shooting.
   • In OnLocation you have access to all the clips generated out of the Adobe Story file. Clips can be duplicated for multiple takes.
   • Clips are named after Scene/Shot/Take.
3. During shooting you do not need to have a connection to the camera because some genius at Adobe realized that metadata could solve that problem. All that needs to be done during shooting of any given shot/take is for a time stamp to be marked against the Clip:
   • i.e. this clips was being taken “now”.
   • Marking a time stamp is a simple button press with the clip selected.
4. After footage has been shot, the OnLocation project is “pointed” to the media where it automatically matches the shot with the appropriate media file, based on the time stamp metadata in the media file with the time mark in the OnLocation Clip.
   • The media file is renamed to match the clip. Ready for import to Premiere Pro CS5.

Now here’s the genius part in my opinion (other than using the time stamp to link clips). The script from Adobe Story has been embedded in those OnLocation clips, and is now in the clip. Once Speech Analysis is complete for each clip, the script is laid-up against the analyzed media file so each word is time stamped. The advantage of this workflow over using a guide script directly imported is that the original formatting is used when the script comes via Story.

All that needs to be done is to build the sequence based on the script, with the advantage that every clip is now searchable by word. Almost close to, but not quite, Avid’s ScriptSync based on an entirely different technology (Nexidia).

It’s a great use of script and Speech Analysis and a great use of time-stamp metadata to reduce clip naming, linking and script embedding. A hint of the future of metadata-based workflows.

All we need now, Adobe, is access to all the Source Metadata.

During a recent thread here where I “infamously” suggested Apple should drop Log and Capture for the next version of FCP, one of the topics that came up was the use of metadata. Most commenters (all?) appeared – to my interpretation – to feel that reel name and TC were the “essence” of metadata.

And yet, if we look at the most recent work of the Chief Video Architect (apparently for both pro and consumer applications) Randy Ubilos we see that Location metadata is a requirement for the application. According to Apple’s FAQ for iMovie for IPhone if you don’t allow iMovie for iPhone to access your location metadata:

Because photos and videos recorded on iPhone 4 include location information, you must tap OK to enable iMovie to access photos and videos in the Media Library.

If you do not allow iMovie to use your location data, then the app is unable to access photos and videos in the Media Browser.

You can still record media directly from the camera to the timeline but, without the Location metadata, you’re pretty much locked out of iMovie for iPhone for all practical purposes.

There is no location metadata from tape capture! There’s not much from non-tape media right now, although some high end Panasonic cameras have an optional GPS board. However P2 media (both DVCPRO HD and AVC-I) as well as AVCCAM all have metadata slots for latitude and longitude.

Now, I’m NOT saying that Apple should force people to use metadata – particularly if it’s non existent – and this type of restriction in a Pro app would be unconscionable. I merely point out that this shows the type of thinking within Apple. In iMovie for iPhone they can create a better user (consumer) experience because they use Location metadata for automatic lower third locations in the themes.

Where I think it’s a little relevant is in counterpoint to some of my commentors: building an app that’s reliant on metadata is a different app than one relying on simple reel name and TC numbers.

As well as showing the software in action, this series of videos show the results from the software. Each “edit” is based on a set of story keywords (logged with the clips) and a duration. Lower Thirds are automatic; story arc is automatic; b-roll is automatic; audio from b-roll is faded in and out and dropped in volume. All automatically and in seconds.

The project is about a young triple threat – singer, dancer, actor – Tim Draxl, discovered in Sydney when he was just short of his 18th birthday.

He played Rolf in a professional touring production in Australia in 2000 and his career has blossomed from there, and the three CD deal he has with Sony Universal: the first when he was 18!

Remember, these edits were done in seconds, from selects using Assisted Editing’s First Cuts software. And yes, this is my baby (along with Dr Greg Clarke).

The Sound of Music Edit

Without limits – about 13 minutes of material.

Four minute limit set. Edit is tighter and only the best material makes it to the edit.

Growing Up

Tim grew up partly in Australia and partly in Austria as his father worked as a ski instructor. This is the unlimited version of the “Growing Up” edit.

[Update: I forgot the 10 minute limit so one of the movies was too long and YouTube can't distinguish between a 6 minute cut and a 4 minute cut, thinking they're the same. Fortunately the videos are also available on our site. The Growing up unlimited  and six minute versions are available at http://assistedediting.com/FirstCuts/results.html]

And finally with a 4 minute limit.

I was finally watching the Steve Jobs Keynote from WWDC on June 7. (I know, but this was our second try – we get talking about stuff, what can I say?) I got to the iMovie for iPhone 4 demo and was blown away by the creative use of source metadata.

At 58 minutes into the keynote, Randy Ubillos is demonstrating adding a title to the video he’s editing in iMovie and iMovie automatically ads the location into the title. Not magic, but it’s simply reading the location metadata stored with images and videos shot with an iPhone and using that to generate part of the title. This is exactly how metadata should be used: to make life easier and to automate as much of the process as possible.

Likewise the same metadata draws a location pin on the map in one of the different themes. Exactly like the same metadata does in iPhoto.

In a professional application, that GPS data – which is coming to more and more professional and consumer video camcorders – could not only be used to add locations, but also to read what businesses are at the address. From that source and derived metadata (address and business derived from location information) we can infer a lot.

Check out my original article on metadata use in post production and for a more detailed version, with some pie-in-the-sky predictions of where this is going to lead us, download the free Supermeet Magazine number 4 and look for the article (featured on the cover) The Mundane and Magic future of Metadata.

Seven hours from feature request to updated application released: Sync-N-Link now uses log notes from video *or* audio. http://bit.ly/aqcxN7

I love being a small independent software developer: it’s great to be able to respond to customer requests promptly – and it makes the software better. Incidents like this one today make me also appreciative of the communication tools we now have

Some time, overnight our time, we had a new customer buy a copy of Sync-N-Link to sync rushes for 8 episodes of a new drama series: in Belgium! A few hours later he emailed to say that it was doing everything he expected, but their sound guy entered metadata (log notes) into the sound clips and Sync-N-Link (like Final Cut Pro itself) discards audio metadata in favor of the video metadata. (In a merged clip there is only room for one of each type of log note/metadata). The feature request was that the metadata from the audio could be preserved instead of from that from the video.

A good request. The ever efficient Greg Clarke, after morning coffee, got to work. At around 1:30 pm (Pacific) an update was published, ready for download, with the feature added. Not quite seven hours from feature request to released software.

I love that we can do that.

If you use any of our software let us know what more you want it to do. We can be very responsive!

I have recently been scanning in slides and negatives from my young adult days, and I’m really wishing I’d entered more metadata at the time. That would have been too easy. Anyway, I thought it was worthwhile examining the different metadata I found/used in indexing these old slides.

Note: Apparently I believed I’d remember the people in my pictures for ever! While I remember the people, names that are not noted somewhere have evaporated over the 30 or so years in between.

The first, and most common, source of metadata was “Added” metadata: those notes that were made on the slide, a carrier or slide sheet carrier heading. Apparently in my early 20’s I could write in 6 point text, which I’m having trouble reading with much older eyes. Regardless, Added metadata has proven to be the most valuable.

The brief added metadata is usually combined with some “Analytical” metadata. Analytical metadata is metadata we get from analyzing visual content in the image. For example, this picture:

was one of a group of 20 on the same sheet with the single word ‘Burbank’ in the heading. In fact only two of the dozen images were in Burbank. But using Analytical metadata – the general location and the street sign near the traffic lights – the shot is clearly on Burbank Blvd, on the rise to the overpass over the 5 Freeway and railway lines. (The sign is for Front St.)

Co-incidence number 1: this is less than a mile from where i now live.

The other very useful metadata was stamped on the slides themselves: the month and year of processing, which locks down an approximate time scale. Also useful was the fact that I’d numbered all my files sequentially from the beginning of 1973 (my year in Japan). That sequential metadata made it much easier to identify specific times, in combination with the stamped-on date.

A combination of Added and Analytical metadata led me to the discovery that most of my 1976 trip was spent in the West San Fernando Valley. Identified was the location of an awards ceremony (by Church name) and a street shot, that was clearly looking across the SF Valley, was also named. Both turn out to be incredibly close to our Tarzana office and Woodland Hills home (2001-2005 before we moved to Burbank).

This jogged my memory that we had spent a lot of time in a school hall for rehearsal/training and enjoyed a close-by Denny’s. Taft High school (Ventura Blvd at Winnetka) has a Denny’s across the road.

So it is entirely possible that during my 1976 trip to the US, I spent the majority of my time around the area that was to become home a quarter of a century later; and some of the rest of my time near my 30-years-in-the-future home in Burbank.

Crazy. And without metadata I’d have never remembered.