Return to site

Lights, Camera, and … Text! Novel Video Editing Tool for User-Friendly, Predictive Text-to-Video Production

· Technical Papers

SIGGRAPH Asia 2019, Brisbane Queensland, Australia, 17–20 November 2019

Brisbane, Australia, 14 November 2019—In a world of novice photographers and videographers, capturing a deluge of content via their smartphones and handheld devices, there is a need for an intelligent, easy-to-use tool for automating the creation of movies and video montages. To date, many quality videos still rely on professional frame-based editing tools to manipulate raw footage and produce a coherent video with a captivating storyline.

A global team of computer scientists, from Tsinghua and Beihang Universities in China, Harvard University in the US and IDC Herzliya in Israel, have developed “Write-A-Video”, a new tool that generates videos from themed text. Using words and text editing, the tool automatically determines which scenes or shots are chosen from a repository to illustrate the desired storyline. The tool enables novice users to produce quality video montages in a simple and user-friendly manner that doesn’t require professional video production and editing skills.

The team is set to present their work at ACM SIGGRAPH Asia, held Nov. 17 to 20 in Brisbane, Australia. SIGGRAPH Asia, now in its 12th year, attracts the most respected technical and creative people from around the world in computer graphics, animation, interactivity, gaming, and emerging technologies.

While existing video editing tools still demand knowledge in video processing and editing, the researchers’ new method allows novices to create videos that tell stories more naturally. Write-A-Video, say the researchers, allows users to create a video montage by simply editing the text that accompanies the video. For example, adding or deleting text, and moving sentences around convert to video-editing operations, such as finding corresponding shots, cutting and rearranging shots, and creating a final video montage result.

“Write-A-Video uses current advances in automatic video understanding and a unique user interface to allow more natural and simpler video creation,” says Professor Ariel Shamir, Dean of the Efi Arazi School of Computer Science at IDC Herzliya. “With our tool, the user provides input mostly in the form of editing of text. The tool automatically searches for semantically matching candidate shots from a video repository, and then uses an optimization method to assemble the video montage by cutting and reordering the shots automatically.”

“Write-A-Video also allows users to explore visual styles for each scene using cinematographic idioms generating, for example, faster or slower paced movies, less or more content movements, etc.” explains Dr. Miao Wang from Beihang University.

When selecting candidate shots from the video repository, the method also considers the aesthetic appeal of the shots, choosing those that are ideally lit, that are well focused and are not blurry or unstable. “At any point, the user can render the movie and preview the video montage result with an accompanying voice-over narration.” says Professor Shi-Min Hu from Tsinghua University.

The team’s research shows that intelligent digital tools combining the abilities of humans and algorithms together can assist users in the creative process. “Our work demonstrates the potential of automatic visual-semantic matching in idiom-based computational editing, offering an intelligent way to make video creation more accessible to non-professionals,” says Shamir.

For the study, the approach was tested on various pieces of themed text and video repositories, with quantitative evaluation and user studies. Users without any video editing experience could produce satisfactory videos using the Write-A-Video tool, sometimes even faster than professionals utilizing frame-based editing software. At SIGGRAPH Asia, the team will demonstrate the Write-A-Video application and showcase a variety of examples of text-to-video productions.

The team includes Miao Wang (State Key Lab of Virtual Reality Technology and Systems/Beihang University and Tsinghua University); Guo-Wei Yang (BNRist/Tsinghua University); Shi-Min Hu (BNRist/Tsinghua University); Shing-Tung Yau (Harvard) and Ariel Shamir (IDC Herzliya, Israel).

For the team’s paper and video, visit the project page at https://cg.cs.tsinghua.edu.cn/andhttp://www.faculty.idc.ac.il/arik/site/writeVideo.asp.

To register for SIGGRAPH Asia, and for more details of the event, visit http://sa2019.siggraph.org

Notes to Editors

1. View a video about this project at https://vimeo.com/357657704

2. For more information on the Technical Papers Program, click here.

3. SIGGRAPH Asia 2019 takes place at the Brisbane Exhibition and Convention Centre 17-20 November 2019. For more information, please visit https://sa2019.siggraph.org.

4. Keep tabs of updates, download images and more at the SIGGRAPH Asia virtual newsroom

5. Media may apply for media accreditation to SIGGRAPH Asia at bit.ly/sa19accreditation

About SIGGRAPH Asia 2019

The 12th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia 2019) will be held in Brisbane, Australia at the Brisbane Convention and Exhibition Centre (BCEC) from 17 – 20 November 2019. The annual event held in Asia attracts the most respected technical and creative people from all over the world who are excited by computer graphics research, science, art, animation, gaming, interactivity, education and emerging technologies.

The four-day conference will include a diverse range of juried programs, such as the Art Gallery / Art Papers, Computer Animation Festival, Courses, Doctoral Consortium, Emerging Technologies, Posters, Technical Briefs, Technical Papers and XR (Extended Reality). Curated programs include Business & Innovation Symposium, Demoscene and Real-Time Live! A three-day exhibition held from 18 – 20 November 2019 will offer a business platform for industry players to market their innovative products and services to the computer graphics and interactive techniques professionals and enthusiasts from Asia and beyond. For more information, please visit http://sa2019.siggraph.org. Find us on: Facebook, Twitter, Instagram and YouTube with the official event hashtag, #SIGGRAPHAsia and #SIGGRAPHAsia2019.

About ACM SIGGRAPH

The ACM Special Interest Group on Computer Graphics and Interactive Techniques is an interdisciplinary community interested in research, technology, and applications in computer graphics and interactive techniques. Members include researchers, developers, and users from the technical, academic, business, and art communities. ACM SIGGRAPH enriches the computer graphics and interactive techniques community year-round through its conferences, global network of professional and student chapters, publications, and educational activities. For more information, please visit www.siggraph.org.

About Koelnmesse

Koelnmesse Pte Ltd is one of the world's largest trade fair companies. Its more than 80 trade fairs and exhibitions have the broadest international scope in the industry, as 60 percent of the exhibitors and 40 percent of the visitors come from outside Germany. The Koelnmesse events include leading global trade fairs for 25 sectors, such as Imm Cologne, Anuga, IDS, INTERMOT, Interzum Cologne, Photokina, Gamescom, and the International Hardware Fair Cologne. Koelnmesse is ACM SIGGRAPH’s event organizer for the last 11 editions of SIGGRAPH Asia. For more information, please visit www.koelnmesse.com.sg.

Media Contacts

Illka Gobius, PINPOINT PR, illka@pinpointpr.sg | WhatsApp +65 97698370

Jamie Huang, Koelnmesse Pte Ltd, jamie.huang@siggraph.org | WhatsApp +65 92329738

All Posts
×

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OK