Supervision : Marie-Jeanne LESOT

Co-supervision : VITTAUT Jean-Noël

Generation of counterfactual texts

The objective of this thesis is to evaluate the possibility of generating counterfactuals in NLP under various forms of constraints such as plausibility, grammatical correctness or goal orientation. Counterfactual generators will be evaluated as a source of interpretability and as a method of strengthening the robustness of the language models handled. Thus, this work will answer the following questions: - How existing post-hoc agnostic methods are suitable for deep learning models applied to NL? deep learning models applied to NLP? - How to interpret deep learning models applied to NLP thanks to the parameters of their structure? Can we derive a method for generating counterfactuals? - How can we integrate the constraints of plausibility, efficiency and goal orientation in NLP to the generation of counterfactuals? To this end, the proposed approaches will be tested on various datasets such as the IMDB Database. State-of-the-art language models such as BERT (Bidirectional Encoder Representation from Transformers) and other derivatives of Transformers architecture will be used to address these issues. In particular, the attention coefficients inherent to Transformers architectures will be investigated. Finally, the use of reinforcement learning algorithms will be considered during the text creation process, which is not necessary for the generation of counterfactual examples. Non-antonymous text generators will be tested to improve the quality of the generated counterfactuals. The counterfactual methods will be systematically tested and used to perform data augmentation and bias detection in order to make the models more robust.

2023 Publications

• 2023
  • M. Bhan, N. Achache, V. Legrand, A. Blangero, N. Chesneau : “Evaluating self-attention interpretability through human-grounded experimental protocol”, Explainable Artificial Intelligence, vol. 1903, Communications in Computer and Information Science, Lisbonne, Portugal, pp. 26-46, (Springer Nature Switzerland), (ISBN: 978-3-031-44070-0) (2023)
  • M. Bhan, J.‑N. Vittaut, N. Chesneau, M.‑J. Lesot : “TIGTEC : Token Importance Guided TExt Counterfactuals”, Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science, vol. 14171 (3), Lecture Notes in Computer Science, Turin, Italy, pp. 496–512, (Springer), (ISBN: 978-3-031-43417-4) (2023)
  • M. Bhan, J.‑N. Vittaut, N. Chesneau, M.‑J. Lesot : “Enhancing textual counterfactual explanation intelligibility through Counterfactual Feature Importance”, Proceedings of the 3^rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), Toronto, Canada, pp. 221-231, (Association for Computational Linguistics) (2023)