Loading...
ENHANCING MULTIMODAL MEME CONTENT MODERATION WITH REINFORCEMENT LEARNING
Citations
Altmetric:
Video URL
Date
2026-06
Author
Collections
Files
Mohamed Kmainasi_ OGS Approved Thesis.pdf
Adobe PDF, 9.69 MB
- Embargoed until 2028-07-01
Abstract
Hateful and propagandistic memes require joint reasoning over image and text, since neither modality alone may reveal harmful intent. Thinking-based multimodal large language models (MLLMs) and reinforcement-learning post-training have not yet been combined for meme moderation. We close this gap with a thinking-based MLLM post-trained via Group Relative Policy Optimization (GRPO). Concretely, we (i) systematically benchmark off-the-shelf MLLMs for hateful and propagandistic meme understanding; (ii) extend the benchmarks with distilled chain-of-thought (CoT) rationales and multi-LLM fine-grained propaganda annotations; and (iii) introduce a composite GRPO reward over label correctness, structural compliance, length, and explanation quality, with a one-sided thinking-length term that mitigates CoT collapse. Our approach sets new state of the art on the English Hateful Memes (82.0% accuracy, 0.80 macro-F1; +2.1 percentage points / +1.0 points over the strongest prior baseline) and Arabic ArMeme (72.8% accuracy, 0.612 macro-F1; +7.6 points macro-F1) benchmarks, while producing high-quality natural-language rationales. We publicly release code, data extensions, and evaluation resources