Mangarise

by Elliot Greenfield & Ahmed Mohamed

Mangarize is an open source facial recognition application that not only tracks the user's face but allows you to become your favourite manga characters. Manga is enjoyed by millions around the world and now you can become one yourself. Choose from popular characters from popular manga such as, Dragon Ball, One Piece and Naruto. With the rise of use from applications like Snapchat we thought there was a gap in the market for an app that incorporates manga and facial recognition. The App uses sophisticated facial recognition software from open frameworks and especially the Ofxfacetracker plugin.  What happens when you select an image is the application makes a copy of your face, overlaying a mask with your features such as eyes, mouth and nose and replaces it with the image of the manga character. You can then go onto save the picture by using the GUI and having saved into the folder of your choice. We wanted to keep Mangarize open to all types of people who enjoy reading manga and want something fun to replicate their favourite characters to people who are looking for something different compared to the more mainstream apps like Snapchat and Instagram. The bootable app can launch on Windows and Mac with that in mind you can see how useful this software is .We also wanted to keep the app as simple and easy to use as possible so you will notice an easy to use interface and instructions on how to use it.

The target audience of Mangarise is really aimed at anime and manga fans, however this app could be used by anyone, as some of the features are like other photo apps. Most people at one point would have heard of or seen manga or anime. This would be a great tool for people who aren’t really into it to explore it more. Our aim in creating this app is to give the user the experience of being a character in manga, seeing what they see, giving them a glimpse of their world. The Users need something that is familiar yet innovative and that is simple to access, so the app will be made with a simple interface.  Our target audience also include creatives, like mangaka’s. Many mangaka’s use referenced photos when doing research on the different backgrounds they have in their panels. For example, cars, neighbourhood shops, crosswalks, building, etc. They do this to add a more real element to their drawings. Our app would be able to give them a better experience at referencing backgrounds if they can view different locations, mangarised, in real time.

Through our research we explored Manga and the different ways we can implement it into our project. Manga are comics created in Japan and have their own unique style that differentiates itself from other works created in Europe and America. It is read by people all over the world spanning many different age groups.One of the things that make manga so unique is that it can have a very dark(tone) story line and involve adult-themes.

We tried to create an app that would be able to 'Mangarise' real life, this could entail manipulating the user face or a background. So instead of being engulfed in the Mangaka's world(which by itself is satisfying enough) you'll be able to bring it to you.

Research 

Snapchat uses a sophisticated facial tracking software from Looksery a ukrainian company for £150 million. It has many different features which allow the use to put said filters in realtime. It starts by mapping the face using an algorithm then allows you input filters on your face. These include a face Swap that allows you to put to swap faces with someone you know. Filters that involve movement from the user such as raising an eyebrow or opening a mouth to get and effect or face manipulation. Snapchat also has the ability to purchase more effects and lenses. Its very simple to use and has great output making it fun. The below image showcases the face swapping effect.

Another that uses filters and facial tracking is MSQRD, looking at this it is more sophisticated facial tracking allows users to record a lot longer as in snapchat you can only record up to 10 seconds of video. The choices of filters within MSQRD are abundant. This app also features paid and sponsored filters. These include The Joker from batman, President Obama and a face swapping feature. Overall trying out the apps i found that MSQRD is the more advanced in its facial recognition as snapchat can be temperamental at points. But when it comes to the social platform behind snapchat it wins hands down as it has millions of users whereas MSQRD only allows the user to export videos to your camera roll.

Gif Art

As we are incorporating gifs as our final output for the program we have had a look at various gif artists. The one on the picture below is by the Gif Connoisseur which features a man standing in front of various animated backgrounds. This would work well for us as we could have the final picture placed in front of backgrounds by the users choice.

Yoshi Sodeoka, who creates both prints and videos, was born in Hiroshima and grew up in Yokohama. In 1989, he moved to NYC to study at the Pratt Institute, armed with years of art education. In fact, his childhood art mentor introduced him to bands like the Rolling Stones and Sex Pistols. Both a musician and visual artist, Sodeoka attributes his loose, psychedelic style to his taste in '70s progressive rock—traces of which can also be found in his GIFs. His videos and prints work can be found in the permanent collection at the Museum of the Moving Image and the San Francisco Museum of Modern Art.

Evan Roth is another gif artists that has seen a lot of success on the internet and in real life having his work featured in the museum of modern art. He creates sculptures and prints based off hacker culture and pop culture.

Manga Artists

There are many different magakas out there with different styles, theses mangaka create manga that fit into specific categories. for example ‘Shounen’ and ‘Seinen’. Manga's that are categorised as shounen are more light at heart and are aimed at the teenage demographic.In contrast manga that are categorised as seinen are more gory, have a deeper story and are aimed at adults. These distinctions play a role in how their art turns out. The following images are panels from popular mangaka’s,i have chosen to research these artists particularly because of their unique art styles and success.

Takehiko inoue is best known for ‘Slam Dunk’ and ‘Vagabond'. Both very popular manga.Takehiko inoue’s style is very complex and has a strong realism that's not very common in other mangas.He places extra emphasis in the 3 dimensional aspect of his drawing. This allows the reader to better connect with the story on a deeper level.

Oda eiichiro has a very simple style when drawing faces but takes extra care in his backgrounds. Due to this he excels in world-building, being able to create beautiful and detailed sceneries. His manga ‘one piece’ is the most popular shounen manga of all time with over 345 million copies in circulation worldwide.

Yasuhisa Hara Is a relatively new mangaka but has gained great success due to his intriguing story and brilliant art style.The manga ‘Kingdom’ is loosely based on china and the wars they were involved in in the past. Yasuhisa Hara pays great attention to detail and this is most noted in his gigantic battle arcs.

Genetic Algorithms

We are also interested in a generative process of creating many variation of one image. The idea is for the user to upload an image, for example of themselves, and from there ‘Mangarise’ it.An artist that uses genetic algorithms  particularly well is kate compton(also known as galaxy Kate).She builds algorithms to create many variations of something that is both random and controlled. An example is her Flower Evolving app.she explains her method as  “.. a way to guide that generator towards more constraint-fulfilling and desirable-property-producing content". This technique can play a significant role for our mangarising project.

Challenges

Building a windows version of our app gave us many issues.Once we had decided on our chosen example to build off of,there wasn’t an available windows version.So we had to build it from the beginning connecting all the required libraries directly.Facesubstitution needed ofxCv and ofxFaceTracker.We first built a working ofxCV windows version with their working examples.We looked through the available examples for a plausible foundation to work off of for our app.

(ofxCV example)

With the ofxFacetraker build we were able to identify and track key features of the face,such as the eyes,nose and chin which help us immensely with our goal of building an app that manipulates the user's face in the likeness of manga characters.

(ofxFaceTracker example)

We decided to use QT for creating our interactivity.QT would enable us to make buttons and sliders which would give the user the ability to further manipulate their faces as they liked.We built a prototype off our design with some functionality.We wasn't able to move further on this because combing the face substitution with QT in a way we had intended proved to be difficult.

From here we had successfully built the examples that will allow us to build the facesubstitution example.with the combination of our addons and their now working library paths and directories,key features of the user's face can be tracked and then have an image of our choosing plant over it and blend successfully.

(Here we're merging with an image of Barack Obama)

An evaluation of our project

This project was challenging on many levels, firstly due to using two different operating systems we was building two completely different codes. After getting a working code on Xcode we went onto convert it into windows. We pulled through the challenges and found that we created a successful project. However we are missing key features and with extra time and knowledge we could have implemented these changes and made a more robust application.

Link to repo - http://egree049@gitlab.doc.gold.ac.uk/egree049/mangarize.git

References

(2017). Arturo Castro and Kyle McDonald . [online] Available at: https://github.com/arturoc/FaceSubstitution/ [Accessed 10 May 2017].

Evan-roth.com. (2017). Evan Roth. [online] Available at: http://www.evan-roth.com/work/one-gif-compositions/ [Accessed 10 May 2017].

 (2017). Yasuhisa Hara . [online]  www.animenewsnetwork.com/encyclopedia/people.php?id=82554

Galaxykate.com. (2017). GalaxyKate. [online] Available at: http://www.galaxykate.com/ [Accessed 10 May 2017].

 (2017). Oda eiichiro . [online]  www.simonandschuster.com/authors/Eiichiro-Oda/47022030

Kate Compton. (2017). So you want to build a generator.... [online] Available at: http://galaxykate0.tumblr.com/post/139774965871/so-you-want-to-build-a-generator [Accessed 10 May 2017].

 (2017). Takehiko inoue. [online] www.itplanning.co.jp/

Sodeoka.com. (2017). Yoshi Sodeoka | 袖岡由英. [online] Available at: http://www.sodeoka.com/ [Accessed 10 May 2017].

Thegifconnoisseur.tumblr.com. (2017). The Gif Connoisseur. [online] Available at: http://thegifconnoisseur.tumblr.com/ [Accessed 10 May 2017].