In this attempt to summarize the state of a technology and its application to production and postproduction my focus is on image recognition, including facial detection and recognition. We’re exposed to facial recognition/detection technology in some current apps: Premiere Pro CS5 onward; iPhoto, Final Cut Pro X, Picassa, Facebook, with mixed success.

(more…)

At the San Francisco Supermeet Friday 27th January, Adobe’s Al Mooney revealed a sneak peek at a new application for the Creative Suite called Prelude. (more…)

Although I’ve shamelessly stolen the title from Joe B (@zbutcher on Twitter) I think it does represent a shift in the way we work with our source media.

Now, before I start let me be clear. I am NOT saying timecode is unimportant. I’m NOT saying that timecode is passé and suddenly irrelevant. Timecode remains incredibly important for any tape based access.

What I am saying is that text search – or phonetic search derived from text – is becoming a highly viable, and in many ways superior, way to search and find content. Timecode’s primary role was in being able to identify any given frame from a tape by tape and frame number. There’s nothing wrong with that approach, but as humans we don’t think in “reel and Timecode”, which is why text is a superior option.

(more…)

Artificial Intelligence Predicts What Will Happen Next In A Video http://t.co/TPopRqQq

Many thanks to my friend Don Berube for pointing this out.  While the headline slightly overstates the case, it’s clear we’re heading for an era when computers in general will understand meaning and the content of images.  (more…)

Two stories today that caught my attention are:

Facial Recognition App Detects, Captures Smiles Technology intrudes more & more into “human” territory

Meet Swivl, The Motion Tracking iPhone Dock That Always Keeps You On Camera More and more automatics!

Now, it would be really cool if Swivl tracked you and kept you on camera using facial detection but it does not: instead it uses a hand held transmitter/controller to “know” where to point the camera.  Even with that it will make a great addition to a video blogger, web episode producer as the producer/talent can move and have the camera follow them as they do.

(more…)

The New iPhone’s Face Recognition Capabilities Could Redefine Privacy http://t.co/WayE1Abv

Following on the heels of yesterday’s post about facial recognition in the cloud here’s information on how Apple are applying the technology they gained when they acquired Polar Rose last September, at least within iOS frameworks.

When coders dug through Apple’s beta versions of iOS5 they found what were deemed to be “highly sophisticated” API systems that let an iPhone automatically track eye positions and mouth positions (so the angle to the user, and possibly where their attention is being directed could be calculated) as well as passing key data on to a face recognition algorithm that would be accessible to all apps…not just Apple’s own.

Combine this with the Nuance-licensed voice recognition technology in Siri – also new with iOS 5 and iPhone 4S – and we have the foundation of a very powerful metadata generation system that would automate naming people in clips and form the basis of speech transcription and then keyword extraction.

In my dreams these are technologies that will come to Final Cut Pro X 10.2 or 10.3 in future years.

When I wrote on Saturday about how tracks have evolved from their compositing role, to one where they became defacto metadata I had no idea Apple were about to release the first Final Cut Pro X update (I only found out on Monday). I also wrote in Conquering the metadata foundations of Final Cut Pro X:

Apple teases us with Audio Role metadata that seems to have no current use within Final Cut Pro X.

If you open the Info Pane and select the Info tab, you will see the pop-up menu in Figure 20.1. showing the Roles

One magic way for this to be useful would be as a solution to the missing audio output options. In the near-magic, near-future I expect that a future version of Final Cut Pro X will use this Audio Role metadata to route audio outputs. At version 1 Final Cut Pro X’s audio output options are very basic, and there’s improvement coming, for sure.

So you can imagine how excited I was to open Final Cut Pro X 10.0.1 and see that those fairly limited audio roles (no doubt implemented specifically to enable export to OMF/AAF) had evolved into not only customizable Audio Roles, but to have the same option available for Video clips as well. And a Title Role comes rolled in!

(more…)

While watching the LAFCPUG “X Night” videos I noted that Michael Wohl advises that “all edits should be in V1″ and not doing that is a sign of a failure to commit.  Similarly reviewing some FCP 1 release videos, they once again (and again from Michael) seem to advocate a mostly single track approach.  That would certainly parallel historic ways of working with video or film where only one track, or an A/B configuration was standard.

I think the use of tracks has evolved since then. One of the reasons that people became concerned when Final Cut Pro X took away the traditional track layout was simply because those tracks have become valuable metadata. A typical audio example might be

(more…)

Metadata looms large at IBC http://t.co/gUiIvHX I’ll be teaching Metadata & Asset Mngmnt #dvExpo dvExpo.com code SFC11 to save money.

As expected, metadata is going to be a dominant topic at IBC, whether identified as such, or under the heading of Media Asset Management (MAM), because all media management is done using metadata, which is why I’ve combined them in my Using Metadata For Production and Asset Management day at DV Expo. (Check out my other sessions at DV Expo.)

(more…)

Can a computer Predict a Hit Movie or Song? http://tinyurl.com/3g8ovfk If you mean profitability, yes. Fascinating use of neural networks.

This is a long, and not new, article that rambles through a fascinating story of how a lawyer, “Mr Pink”, “Mr Brown” and “Mr Bootstrap” collectively cracked the code for predicting the profitability of movies, TV shows and (separately) another team shows the likelihood of whether a song is going to be a hit.

The specifics of how they achieved both breakthroughs is interesting: have the computer software (usually some sort of neural net) analyze existing successes – music or movie.  It then analyzes new music or movie proposals to determine whether it is likley be be a hit (music) or how much money it will make at the box office (movie).

(more…)