Loading...
DETECTING POLICY-PREFERENCE CONTRADICTIONS IN PRIVACY POLICIES
Files
Razan Malluhi_ OGS Approved Thesis.pdf
Adobe PDF, 699.62 KB
- Embargoed until 2028-07-01
Abstract
Privacy policies are intended to inform users about how their data is collected and used, yet they remain lengthy, complex, and rarely read. While automated policy analysis tools have improved the extraction of data practices, they do not systematically detect contradictions between policy statements and individual user privacy expectations. Existing approaches either rely on manual analysis or prompt-based large language models (LLMs), which suffer from inconsistency, high latency, and dependence on proprietary APIs. To address these limitations, we introduce a preference-conditioned framework for detecting policy-preference contradictions, formulated as a multi-label classification task. We construct a dataset of 2,384 annotated policy segments from 1,464 real-world privacy policies across eight user-defined preferences, combining real data with validated synthetic augmentation to mitigate class imbalance. Manual validation shows an agreement rate of 87.69%. We fine-tune a DeBERTa-v3-base model that achieves a macro F1 score of 0.8950 and exact match accuracy of 0.8167, outperforming both a TF-IDF + Logistic Regression baseline and a zero-shot instruction-tuned LLM. These results demonstrate that supervised, preference-conditioned learning provides more reliable and consistent detection of policy–preference conflicts than both lexical methods and prompt-based inference. The resulting model is fast, reproducible, and locally deployable, making it suitable for real-time privacy analysis. Overall, this work introduces a new benchmark and establishes a practical framework for preference-aware privacy policy understanding