Twilight star among three authors of paper explaining how ‘neural style transfer’ method was put to use in her directorial debut, the 17-minute short Come Swim 
 
 Twilight and Personal Shopper Kristen Stewart has co-authored a research paper on “neural style transfer”, an arcane technique that uses artificial intelligence to reconfigure an image in the style of another. 
 
 Written with Bhautik J Joshi, a research engineer at Adobe, and producer David Shapiro, Stewart’s paper is related to work done on her short film directing debut Come Swim, which received its world premiere at the Sundance film festival on Thursday. Called Bringing Impressionism to Life with Neural Style Transfer in Come Swim, the paper was submitted on Wednesday on Cornell University library’s open-access arXiv.org website, an online repository for scientific research papers. 
 
 Beyond Twilight: on the set of Personal Shopper with Kristen Stewart Read more 
 
 The paper describes the 17-minute Come Swim as “a poetic, impressionistic portrait of a heartbroken man underwater”, which is “grounded in a painting [by Stewart] of a man rousing from sleep” that “evokes the thoughts an individual has in the first moments of waking”. According to the paper, neural style transfer was then used in the opening and closing scenes to apply the style of the painting to the filmed images. 
 
 The methodology and process of this supposedly pioneering form of visual effects is described in detail over three pages, focussing on the difficulties in obtaining a high-quality moving image. 
 
 Stewart has demonstrated in the past she is not afraid to embrace unconventional creative techniques. In 2014 she revealed a cut-up style poem in Marie Claire called My Heart is a Wiffle Ball/Freedom Pole, whose first lines read: “I reared digital moonlight/You read its clock, scrawled neon/across that black” ||||| Hollywood and the academic artificial intelligence community have an interesting relationship, to say the least. Many people interested in the field, myself included, love science fiction, but regularly end up being the obnoxious commentator on family movie night pointing out that Johnny Depp’s Transcendence is more anti-intelligence than artificial intelligence. So it wasn’t all that surprising that researchers were a bit perplexed when actress and director Kristen Stewart appeared as an author on a machine learning paper. 
 
 To be someone in Hollywood, you've got to put your ML papers on Arxiv and you better use TensorFlow… https://t.co/2Rcg1ccJ36 — François Chollet (@fchollet) January 19, 2017 
 
 I used to tell Twilight jokes in my Intro to AI class. I take it all back. — Mark O. Riedl (@mark_riedl) January 19, 2017 
 
 I used to tell Twilight jokes in my Intro to AI class. I take it all back. — Mark O. Riedl (@mark_riedl) January 19, 2017 
 
 Stewart directed Come Swim, a short film about a man’s day told through both impressionism and realism. The film features the use of the so-called “style transfer” technique to build its story. 
 
 Style transfer is the fun technique that involves using convolutional neural networks to artistically alter video in real time. To do this, you basically show your algorithm an image (say a painting by van Gogh) and then teach it to apply the same painting technique to any regular photo. 
 
 The short, three-page paper, entitled Bringing Impressionism to Life with Neural Style Transfer in Come Swim, is really just a high-level case study about what it’s like to apply style-transfer techniques to cinema. 
 
 In addition to Stewart, Bhautik Joshi and David Shapiro of Adobe and Starlight Studios, respectively, speak to the challenges of trying to reign in style transfer for controlled use. The team’s approach comes off a bit like blindfolded dart-throwing, but, to be fair, the trial and error process and dealing with trade-offs is pretty much what machine learning research is like at all levels. 
 
 One would almost expect that artists and movie-makers would be more comfortable with improvisation than researchers, but this doesn’t seem to be the case. “In a production setting, however, a great deal of creative control is needed to tune the result, and a rigid set of algorithmic constraints run counter to the need for this creative exploration,” note the authors of the paper. ||||| Your browser does not support HTML5 video tag.Click here to view original GIF 
 
 It’s commonly known as the “sports effect,” the “soap opera effect,” “motion smoothing,” or “motion interpolation” and it makes sports on your TV look great, while movies look... weird. TV manufacturers turn this setting on by default and all too often it just gets left that way. Now, James Gunn is getting his Hollywood buddies to help rally against it. 
 
 Advertisement 
 
 On Thursday, the Guardians of the Galaxy director tweeted, “So @rianjohnson, @edgarwright, @mattreevesLA, @chrismcquarrie, @TomCruise & I are all on board the anti-motion-smoothing campaign. Who else?” In order, the people Gunn identified as joining him are the directors of The Last Jedi, Baby Driver, War of the Planet of the Apes, the latest Mission Impossible, and then there’s Tom Cruise. 
 
 Christopher McQuarrie quickly jumped in with some Ice Bucket Challenge-style provocation and nominated James Cameron and Michael Bay to join the campaign. (They haven’t.) 10 Cloverfield Lane director Dan Trachtenberg added his support, and tweeted, “please make this official.” And Reed Morano, director of several episodes of The Handmaid’s Tale, pointed out that she’s been trying to do something about motion smoothing for a long time. 
 
 Advertisement 
 
 If you’re not entirely sure what these folks are going on about, motion smoothing is a feature on most modern TVs that intended to correct hi-def screens’ tendency to make objects in motion appear to be blurry. In order to do this, the TV processes one frame, then the next, and makes a guess on what a new frame that goes between them should look like. This can be very helpful if you’re watching a football game, for example, and you’re attempting to keep track of the ball in a wide landscape shot. It gives everything a crisp edge. The feature can also be good for upping video game frame rates, but it’ll get you killed because it introduces extra lag. 
 
 Most movies are still shot at 24 frames per second these days, and when the motion smoothing is enabled on a TV it causes things to look unnatural, or for some people too natural. The lovely cinematography that the filmmakers worked to craft starts to get that low budget look of television soap operas, and the actors look a little too real. Peter Jackson’s The Hobbit received a lot of criticism for its look and was often said to look like it was being shown on a TV that accidentally had the motion smoothing setting turned on. But in reality, The Hobbit looked that way because it was shot and projected at 48 frames-per-second. Whether you like it or not, that’s the look Peter Jackson wanted. 
 
 The filmmakers that take issue with motion smoothing are primarily annoyed that televisions almost always have this setting turned on by default. Manufacturers have their reasons for this, it makes a TV in the showroom extra stimulating for one. Many people don’t know that they shouldn’t use this setting for every kind of programming, and the fact that different manufacturers have different terminology for their settings makes it more difficult to educate people. As Rian Johnson pointed out to one Twitter commenter who was saying people can just turn it off, the menu settings can be pretty difficult to navigate for a novice. “MENU>PICTURE>ADVANCED CONTROLS>REALITY AUGMENTATION>MOTION LIQUIDITY>FLUID FRAME RESTORATION,” he explained is not a simple series of steps, just try saying “reality augmentation” to your grandmother. “You want movies to look like liquid diarrhea, fine,” Johnson wrote. “But it should be a choice you make, not a hoop everyone has to jump through to unmake.” 
 
 Advertisement 
 
 Other famous people like Peyton Reed, director of Ant-Man, have been jumping into the thread to show their support. And when someone suggested Christopher Nolan, McQuarrie replied that he shares a producer with the filmmaker and “It’s tomorrow’s first bit of business.” Nolan is a good choice because he’s such an old-school guy who still insists on shooting with film and is a notorious stickler for having his films shown as they were intended. It would be even better to gather the support of someone with a broader influence in business—someone like Steven Spielberg. And executives at Sony Films could have a direct influence on their company’s TV biz. 
 
 All that really needs to be done is simplifying the ways that modes can be changed. And when the TV is first booted up, it should explain what the differences are. These industry folks are in a position to make your trip to grandpa’s house less frustrating if they take their efforts beyond clicktivism. 
 
 [James Gunn via Polygon]
– If she ever abandons acting, Kristen Stewart might have a gig in artificial intelligence. The Twilight actress has just published a research paper on a machine learning technique called "neural style transfer" and its use in film, reports Quartz. Style transfer is a "fun technique," according to Tech Crunch, though it's a bit complicated to explain. Essentially, an algorithm is used to apply the style of one thing to something else. In this case, Stewart—with help from Starlight Studios producer David Shapiro and Adobe research engineer Bhautik J Joshi—sought to transfer the look of one of her own paintings to scenes from her short film Come Swim. Using computer programs known as neural networks, the group was able to add "blocks of color and texture" to a test frame until it matched the look of the painting, which depicts "a man rousing from sleep," reports the Verge. Once they had achieved the look they wanted on the test frame, they applied the same technique to the opening and closing scenes of Come Swim, described as "a poetic, impressionistic portrait of a heartbroken man underwater," which premiered at the Sundance Film Festival on Thursday, per the Guardian. The Verge notes you might have used this same technique yourself: Facebook introduced a style transfer filter last year.