Making comic books from movies using computer vision

The automated system, inspired by Japan’s ‘Manga’ comics, will help professionals develop their own books from videos, movies, TV shows or cartoons

(Subscribe to our Today’s Cache newsletter for a quick snapshot of top 5 tech stories. Click here to subscribe for free.)

Producing a comic book is a work of art involving high level of creativity. The whole process of ideating, sketching, colouring, inking and lettering content, can take over eight months. Now that production can be completed in lesser time, and done by artificial intelligence (AI), not humans.

A team of researchers, including those from China’s Dalian University of Technology, has developed an AI system to make comic books from movies and other videos. They have used computer vision, a form of AI, to automate creative tasks, which helps to interpret digital content like images, videos, and graphics.

The team said it derived inspiration from ‘Manga’, a style of comics or graphic novels originating from Japan. The automated system will help professionals develop their own books from videos, movies, TV shows or cartoons.

Also Read | Google ends development of its VR painting app

The approach will first extract informative keyframes by analysing the subtitles of the video, and then curate comic-style images. A separate multi-page layout framework can allocate the images across multiple pages and synthesise visually appealing layouts based on the relationship between the images, the team noted in a study titled ‘Automatic Comic Generation with Stylistic Multi-page Layouts and Emotion-driven Text Balloon Generation’.

Then, instead of using the traditional balloon format prevalent in most comic books, the system will generate different types of word balloons based on the emotions of the subtitles, audio and video.

The team conducted several experiments to compare results to that of existing state-of-the-art techniques. It also used different movies and shows like ‘Titanic’, ‘Friends’ and ‘Up In The Air’ to extract clips.

You have reached your limit for free articles this month.

Subscription Benefits Include

Today’s Paper

Find mobile-friendly version of articles from the day’s newspaper in one easy-to-read list.

Unlimited Access

Enjoy reading as many articles as you wish without any limitations.

Personalised recommendations

A select list of articles that match your interests and tastes.

Faster pages

Move smoothly between articles as our pages load instantly.


A one-stop-shop for seeing the latest updates, and managing your preferences.


We brief you on the latest and most important developments, three times a day.

Support Quality Journalism.

*Our Digital Subscription plans do not currently include the e-paper, crossword and print.

Source link

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button