Toward accessible comics for blind and low vision readers - La Rochelle Université Access content directly
Preprints, Working Papers, ... Year : 2024

Toward accessible comics for blind and low vision readers


This work explores how to fine-tune large language models using prompt engineering techniques with contextual information for generating an accurate text description of the full story, ready to be forwarded to off-the-shelve speech synthesis tools. We propose to use existing computer vision and optical character recognition techniques to build a grounded context from the comic strip image content, such as panels, characters, text, reading order and the association of bubbles and characters. Then we infer character identification and generate comic book script with context-aware panel description including character’s appearance, posture, mood, dialogues etc. We believe that such enriched content description can be easily used to produce audiobook and eBook with various voices for characters, captions and playing sound effects.
Fichier principal
Vignette du fichier
0008.pdf (1.19 Mo) Télécharger le fichier
clustering.png (658.23 Ko) Télécharger le fichier
main.pdf (1.18 Mo) Télécharger le fichier
method.png (39.68 Ko) Télécharger le fichier
strip_p12.jpg (223.11 Ko) Télécharger le fichier
strip_p5.jpg (216.72 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04610678 , version 1 (13-06-2024)
hal-04610678 , version 2 (10-07-2024)


  • HAL Id : hal-04610678 , version 1


Christophe Rigaud, Jean-Christophe Burie, Samuel Petit. Toward accessible comics for blind and low vision readers. 2024. ⟨hal-04610678v1⟩
15 View
11 Download


Gmail Mastodon Facebook X LinkedIn More