Larry Jordan got on his (self described) soap box this morning with a thoughtful post about the future of editing in an AI infested world. I think we should all be aware of what’s happening, and I’ve certainly been trying to do my part there, as recent posts attest, but I’m not sure I’m quite as concerned about editing jobs as Larry is. Assistants perhaps.
Larry shared his post with me, asking for feedback, and having written a fairly comprehensive response, I decided to share it here as well. While I mostly address the areas where AI/Machine Learning might be used, and why pervasive automated editing is probably way further in the future than Larry’s concern would indicate, none of that negates Larry’s excellent advice on managing your career.
Here’s what I wrote to Larry.
It’s a topic dear to my heart, and one I’ve written a lot about, and spoken about at a couple of conferences (HPA, Metadata Madness).  the front page of PhilipHodgetts.com is almost entirely posts about AI or Machine Learning.
However, I do NOT see a fully automated editing tool, even for basic work, in the near future. Sure, much smarter templates, continuing the templatorization of production trend that’s been going on for at least the last decade (and a topic Terry Curren and I have discussed on our irregular podcast) but fully automated editing…
Put it this way, when it comes to placing bets with my own money and time, I think the original knowledge patterning that we did for First Cuts in 2008 is probably going to give better results than machine learning based editors.
The reasons I don’t think it’s going to get to the stage you’re worried about, soon, is simply because there is no training set for a neural network (the foundation of all machine learning). Machine learning/Neural networks do their magic by deriving the patterns from examples they are given. These examples have to be pre-graded.
Relatively easy to do if you’re looking over a couple of hundred thousand tax returns. Easy enough to train it to spot skin cancers by showing it the slides and results from millions of examples already reviewed by humans who identify any skin cancer.
These machines need at least hundreds of thousands of examples, and each example must be graded so the machine knows (ultimately) when it’s getting the same result as the human. Ultimately we never know exactly how, or why, the machine is getting the same result as the human. Literally, no-one ever knows how the internals are getting the result,, only that the results the machine gets, get closer and closer to the results the humans got.
Now, I cannot think of a huge, graded set of even simple videos. Even wedding videos, which are relatively simple patterns, would need a couple of hundred thousand examples, all of which have been graded for the “quality†of the result. And since that’s subjective two people may grade the same wedding video differently.
Ditto with corporate or educational video. I think it’s feasible that weddings, corporate, educational and news production *could* be automated if we had a sufficiently powerful neural network and enough examples, but right now the machines aren’t suited for tasks that complex, and the – as previously noted – lack of a training set, is going to delay any implementation well into the future. A future I probably won’t see, and I plan to live quite a while yet. (But never say never).
It might seem like Adobe and Stanford are getting close, but in reality, they’re not deriving style by looking at a body of work, they’re applying style by selecting a few rules of thumb to apply. In that respect, they’ve caught up with the story building part of First Cuts  (and frankly for non-scripted First Cuts  is way more sophisticated).
What will happen, and will take jobs, is the pre-editing stage. Understanding, organizing and presenting massive amounts of media in a form ready for an editor to refine and finesse will happen. Assistant Editor positions may be challenged, but probably not in the TV and Film industry because these niches are very conservative and reluctant to change.
Taking a “glass half full†view, it will empower more producer/editors to fulfill their creativity. People who are very creative, have great story telling abilities and love to shoot, but find the editing process relatively tedious, will be able to have their interviews transcribed with keywords and concepts extracted. The b-roll content will be keyworded automatically.
Based on that derived metadata, building basic story string-outs (using rule-of-thumb ‘knowledge system’ approaches rather than AI/Machine learning) will be part of that.
So while my thinking as evolved to being less concerned, none of that negates the excellent advice for managing careers in Larry’s post, is something everyone should be doing.
Comments
7 responses to “Thought’s about Larry Jordan’s “Worries on the future of Editing””
Great points in response to Larry’s article Philip. But I wonder, Google just announced that its AI DeepMind has taught itself to walk. How is that different than learning to edit? I’m not saying that it is or isn’t.
You say that machine learning requires a training set. But based on watching the video with the DeepMind announcement, it seems like DeepMind started from general principles.
The primary difference between learning to walk and editing is that learning to walk has two principles: be upright, move forward. For relatively simple documentary-style edits, we had to create about 200+ “rules of thumb” for our First Cuts app. Editing is exponentially move complex than walking. Not saying we won’t ever get there – we almost certainly will at some point – but it’s not going to be something that affects this generation of editors.
To some degree this goes along with what I said in my comments to Larry, particularly in regards to which parts of the video industry are more likely to be influenced by AI.
But I don’t agree that “there is no training set for a neural network.” There is research going on right now, in a number of NDA type places, that is connecting editing patterns and styles with hard metadata that is relatively easy to accumulate — things like box office data, or Netflix/YouTube style TV analytics. And while I have some fundamental disagreements with the idea of equating financial success with artistic success, that is besides the point. That data is already out there and it begins to point a path to one form of editing aesthetics.
There are a host of problems with this, of course. Examining footage is never clear and results will vary greatly depending on input of on-set data and how preferences are determined across multiple takes of similar sizes, etc. However, I fully agree with you on First Cut’s basic principle — that AI-style assemblage will only get you to a first cut. If your material works with that level of finesse (or, in other words, LACK of finesse) then you’re already there.
But if it doesn’t then, at the very least, this AI will get you to a first cut that can shave a ton of time off of an editor’s workload. The money people will, of course, use this as a way to shave time off a delivery schedule for any given project. But the opportunity exists for a much deeper editing experience if they avoid that path.
Sounds to me like you’re making my point: that there is no good way to derive suitable “training” materials. Whether a training dataset or a set of rules. Frankly, we did better with First Cuts in 2009 than the “AI editing narrative” with it’s very basic, simple rules of thumb that get applied to decently derived metadata.
I thought it was pretty crude to be honest.
Our goal (Intelligent Assistance, Lumberjack) has always been to get to that first pass quickly so there is more time to do the creative work.
Here’s a good article from Techcrunch on the difficulty of data sets, or appropriate oppositional training.
https://techcrunch.com/2017/07/21/why-the-future-of-deep-learning-depends-on-finding-good-data/
Learning to walk starts with crawling, navigating space first at low angle, then higher– and not always forward!
If the goal is to accumulate and tabulate data from given material only to build a data set to describe it, it is destined to fail as a general automation tool. You could end up with the same sequels greenlighted by suits for decades except with calculated twists and turns. Formulaic offerings are part of our industry, because we enjoy returning to great characters and premises. But sooner or later, it gets old, and the industry renews itself from some new inspiration: where once we had hard core detective heroes, along came the suave superspies. Studios began chroming the superhero universes. New characters for the times, which can be grown and evergreened into a nice entertaining franchises. In those contained products, story editing might achieve some automation which impairs the organizing job.
But if the goal is to create new tools for assistants to expertly program automated organization, the input should be based on requirements derived from discussion between assistant and editor or director, and then entered into a digital sponge with helpers to focus the story treatment (the TurboTax interview system comes to mind.) From deft and informed programming, feeding in character bios, script, footage and track, perhaps a project could be laid out for the editing process it needs in seconds flat, media linked, scene bins created, primary postproduction format applied to scene timelines, etc.
This can only enhance the AE job, not destroy it.
Here is my response to Larry’s column. First, I think Larry got a lot of “details” out of context and wrong. When it comes to AI / Machine Learning, there’s a lot of hype and hysteria that is baseless and needless. That we’ll have automated editing taking our jobs in 2 years has no basis in fact.
“I’ve been editing since 1973 and there is one constant that has never changed. If you go a year and don’t learn a new tool, you’re falling behind. Learning new tools has always been a demand of our industry, this process is nothing new. All that’s new are the new tools, which have ALWAYS been new tools that need to be learned constantly. I think we’re making mountains out of mole hills, honestly.”