Latest posts

Art, Design and Creativity

At the end of every episode of The Terence and Philip Show I challenge our audience to “Do something creative,” but what do I mean by ‘creative’?

Creativity is a mix of randomness and restraint: extreme and editing. If there is no randomness, no sense of doing something different, then it’s hard to be considered creative! It’s equally easy to be completely random without restraint or editing and be unable to decide what is effective design or art. (More on those later.)

A long time ago in a country far away, I employed a graphic designer for some publishing work we were doing. I distinctly remember him generating three versions of an ad where he was unable to decide which was “best.” What was so odd, was that one of the designs clearly communicated content and mood better than the other two, which was immediately obvious (to me). He had random down, but lacked the editorial eye.

Creativity isn’t something exclusive to painters, sculptors and other practitioners of the “fine arts.” While they are a sub-branch of creativity, it is not limited to fine arts. To be really creative one has to be outside common bounds, while fine arts define common bounds!

Design is creativity applied to problem solving, which is why it appeals to me. Whether that problem is an icon design, a garden landscaping challenge or a new iPhone, the same skills of trying something new and different (randomness) combined with an editorial eye, work to solve the challenge.

It’s also beholden on the designer to be aware or changed opportunities: new tools, techniques and material make new opportunities possible. Applying those new solutions is definitely creative.

Art adds another dimension: passion. For about a year I shared a house with a college art student, David Middlebrook, now one of Australia’s more prominent landscape artists. Even then he had a very savvy understanding of “art” and had two working definitions, both equally true. “Art is what the art world will buy” and the much less cynical “Art is Passion.”

It doesn’t matter whether it’s a passion for people, for landscapes or for changing the world, Art is the expression of creativity with passion.

Bottom line: you can’t be conventional and creative.

Filmmaking as a VR “Game”?

According to UploadVR.com Disney’s Jon Favreau directed the photorealistic remake of The Lion King in a VR environment created for the film.

The inevitable merge of VR and filmmaking and one that opens a lot of creative opportunity for photorealistic filmmaking, although I suspect that humans may still suffer from the Uncanny Valley effect, for a couple more iterations of technology.

Micropayments at last?

What’s new in Publishing has an article today about Micropayments “unlocking an entire economy” between ads and subscriptions, which really resonates with me because it’s something I’ve really wanted, and something I’ve invested time and money into developing.

As far back as my earliest days on the Internet on the Media 100 User email Group (still alive and well BTW) I realized I’d make a nice side income if anyone I’d helped could tip me 5c or 10c. That wasn’t possible in the early 1990’s, but come to early this century we tried to bring micropayments to podcasts (via KlickTab).

KlickTab, and the Open TV Network, suffered from the classic bind of not enough content to be attractive for potential users, and not enough users to be attractive to potential content providers.

If we can find a cost effective way of allowing small “tips” (aka Patreon) or charges for an article, or a video or segment of video, that is also frictionless then it might, as the article suggests, open new streams of revenue for content creators.

Subscriptions require a regular, ongoing supply of fresh content, and a long term commitment making them less attractive for creators and consumers, but small – and appropriate – amounts for specific content requires no commitment on either side.

Why I’m Learning Swift

Nothing about my job description requires I code anything! The open secret is that I get credit for the code written by my husband and partner Gregory Clarke, so I don’ have to write code.

Now I have a 50,000′ overview of coding and the technology, enough to be an effective product manager, but I can’t code, or truly understand what Greg has written.

This makes me/us very vulnerable.

Technically I don’t need to write Swift, but I’ve come to the conclusion that I’ll be a more effective manager if I can at least read Swift and gain a glimmer of understanding.

So, starting Sunday, I’m doing the 100 days of Swift course.

Of course, it still won’t help with all the XML knowledge Greg has, nor with the various quirks of FCP Classic, FCP X, and Premiere Pro’s handling of XML or media, but it’s a start!

Metadata is Expensive

As big a fan of all metadata as I am, I do acknowledge that the most useful metadata is also the most expensive to obtain. Technical metadata from the camera comes at almost no cost: set the camera correctly and metadata on frame size, codec, frame rate, Timecode, Time of Shoot, etc comes effectively free.

Not so content metadata about where the shot it, who is in the shot, what people are saying, what people are talking about (requires understanding), or where people are. That metadata takes time (and mostly human involvement) to add, making it quite expensive.

Back in 2008, when we released First Cuts for FCP, we knew the power of metadata to kick start the editing process for non-scripted production. First Cuts didn’t reach its potential because of the expense of the metadata offset some of the benefits.

That’s why I am so interested in the potential for Machine Learning to reduce the cost of acquiring Content Metadata. Once we can derive the metadata affordably, we can use that to kick-start the creative process and avoid the paralysis of an empty timeline!

The Terence & Philip Show Episode 83: Revisiting Machine Learning

After a year we revisit the role Machine Learning applied to production and distribution.

Episode 83: Revisiting Machine Learning and AI

With a Little Bit of Luck

Over the last five years I’ve been learning to sing for my own enjoyment. Over that time I’ve met many talented students of the same vocal coach. It’s a constant reminder that there are way more talented people, than there are parts and fame for them!

The opening montage in A Chorus Line also demonstrates how few opportunities there are for talented dancers. Getting a long running show (on Broadway or TV) is a life changing event, but getting one usually involves a dose of good luck.

For every talented actor, dancer or singer that breaks through, there are dozens – probably hundreds – of equally talented people who were not in the right place at the right time. Who’s parents couldn’t get them to that audition, who couldn’t get time off parenting to make their first single, or however life interferes.

Obviously we prepare for the opportunity (although I doubt I’ll ever be talent scouted for my singing!) because having the opportunity and being unprepared won’t work either. The more prepared you are, and the more ways you are presenting yourself, the more the opportunity might happen.

Luck is morally neutral. Good luck happens to some, and not to others. Bad luck also happens indiscriminately. Like I said yesterday, life is unpredictably good and bad.

There are no small decisions!

We rightly focus on the “big” decisions of life: high school major, college choice, partner, where to live, what job to take, etc. These are important decisions and will affect your future.

The funny thing is, some of the smallest decisions I’ve made, have had the biggest affect on my life. Deciding to leave a fairly boring nightclub to go to another, and overhearing a comment that may have been about me, and therefore changing my mind, turned out to be close to one of the most important decisions of my life, as I met Greg that night.

Some decisions are obviously “big,” but five seconds either way could be the difference between being in the path of an out of control car, or not. A life changing moment one way, barely noticed the other. Five seconds.

A stray bullet could radically change the path of our lives, with zero forethought or planning.

Or a chance meeting in a hallway at NAB might lead to one of the most productive partnerships or radical innovation the industry has seen!

It’s probably best that we don’t realize there are no small decisions, because to give every minor decision the attention it could deserve, would be crippling. I find it much easier not to dwell on it too much.

How to teach a Machine

In a follow up Tweet to my comments about Resolve 16, the Tweeter suggested they could use the Neural Engine to “improve translation of XML from other apps,” which led me to spend a little time (oversimplifying) how we train Machines for Machine learning.

There are three approaches: training sets, oppositional training, simple goal.

Most Machine Learning neural networks are trained using massive amounts of “qualified” data: examples that have been tagged with the desired results to train the machine, then more known examples to test the machine. This has been the approach used for facial recognition, speech-to-text, emotion extraction, color grading, etc. Most of the things we’ve known Machine Learning for so far, have been trained using this type of data set.

Oppositional training is where you have one machine – for example – trying to create a human face, and another machine determining if what is presented is a human face. Because these are machines they iterate very, very quickly and some amazingly realistic human faces have resulted.

Then there’s the Simple goal. The clearest example I’ve seen so far is where a bipedal stick figure in a simulated environment was given the challenge “Stay upright and move forward.” After millions of iterations and experiments the goal was achieved, so they made it more complex with terrain.

Given those gross oversimplifications of some very clever technology, let’s examine the idea of improving XML translation. I’m not aware of any existing training set, or how you’d go about creating a training set that had perfectly matched timelines in two different apps, and the XMLs that they represent. The matching timelines would have to be built manually (or corrected manually) so they were a perfect representation of the other app’s timeline. Not particularly practical.

I don’t see how we can simplify the request to use either oppositional training or boil it down to a simple goal.

Now, another Tweeter suggested they use the Resolve Neural Engine to improve cut detection, and that’s an entirely reasonable, and feasible goal, as we have a substantial body of timelines with EDLs representing the cuts.

Ultimately, what we can do with Machine Learning will come down to how we train it, which is why I am not expecting a Machine-based editing tool for a very long time.

Resolve 16 and Machine Learning Thoughts

While I’m still to see a demo, there are few announced features that I definitely think are in the right direction, particularly those driven by Resolve’s “Neural Engine.” It seems, like Adobe Sensei and Apple’s CoreML, to be a playback engine that implements the Machine Learning models in practical tools.

Improved retiming, facial recognition, color matching, color balance and upscaling are the first implemented features. These are in line with what I have been expecting from ML: smart features that make the process easier for editors. All Resolve’s current Neural Engine driven features are better ways (faster) to do things we’ve been doing for years (other than facial recognition).

Adobe have already implemented ML driven features in their apps, and marketing tools. I’d hope that Avid have taken likely future use of ML into account in their NAB-announced revamp of Media Composer, although I doubt we’ll see any ML driven features there for many years to come, for two reasons. The Media Composer market is largely not ready for it, and Avid will have enough on their plate bringing a newly rewritten version to maturity. I expect we’ll see ML in Media Central before any desktop app.

Apple have made very good use of ML across all their products. It’s why Apple Mail predicts mailboxes for you, among dozens of little features across their devices. The have an excellent playback engine in CoreML, and developers of macOS or iOS have access to some of IBM Watson’s models. The Pro Apps team were also advertising a position for a Machine Learning specialist in late 2017.

Thanks to the Content Auto Analysis (that no-one uses) they even have a pipeline within the app to bring ML derived keywords into the app.

I’m sure we’ll see ML driven tools in FCP X at some future time. It largely depends on the priorities within the Pro Apps Team. I’d love to see a big focus on ML in a future FCP X, but there are those who would rather see collaboration, dupe detection or scrolling timelines.