My next project is about converting comics to my native language. Since manual work is a time consuming, we are discussing about automating this. So took my first comic, Doctor Strange(English), and lets talk about the steps involved in this.

This is the page we are going to translate,

Step 1: Detection of text, conversation: This involves passing the image through a deep learning model to identify all the balloons in it.

Step 2: OCRing, So we have the location of all text boxes with us. So we will take each of them and pass through it via an OCR. The quality of OCR depends on the quality of the source image, we can use Tesseract, Abby, Google or any OCR as needed.

So we will get something like this,

Step 3: Translation is the next step. We can use any kind of language translators available for translation.

At this point, the translation may not be perfect. So we cannot fully depend on a translator and s we need to tune it or we should create our own translation model for each type of comic and then do the translation. Different comics use different own kind of dialogues/phrases and content delivery, like for Amar Chitra Katha the English’s content will be different and for Marvel comics it will be exactly different. So we cannot convert them based on any available conversational models for a professional output, but automated translation works for the time being.

Step 4: Text Masking: So now we know where are out texts, so the next step is to mask all the texts in it. This uses AI models to detect text in the balloons that we detected, and remove them. OR just clear all the balloons which may not be perfect since balloons can be of any shape.

Step 5: Replace original text, we can now paste our translated contents over this bubbles.

So yes we have the comic ready.

Here I’m adding the converted comic (Just a couple of pages only.)

Original: Dr. Strange from Marvel

Translated: Dr. Strange From Marvel – Malayalam Translated

Here is one more,

This I did in a hurry (ignore the imperfect mask removal), but it does the job.

Original: Arjun Unicrystal
Translated: Arjun – Unicrystal – Malayalam Translated

So if you want to convert any comics can P.M me, all we need is a person (manual translator) who can verify the automatic translated content and do the correction.

Thanks for reading.