Direction de recherche : Patrick GALLINARI

Co-encadrement : PIWOWARSKI Benjamin, LAMPRIER Sylvain

Natural Language Generation with Reinforcement Learning

Natural Language Generation (NLG) is the subfield of Natural Language Processing, where the task is to produce natural language outputs. Despite the important progress fostered by the application of Deep Learning, generated texts are still inconsistent and contain factual inconsistencies. At the root cause, we argue in this thesis that deep learning models in NLG suffer from inherent flaws in algorithms, which limits their efficiency. At training time, the standard training strategy, Teacher Forcing, induces the so called exposure bias, a mismatch with inference time, where the errors accumulate. Moreover, NLG suffers from a second flaw: its the automatic evaluation does not reflect well human judgement.
In this thesis, we explore how to improve both evaluation and training in NLG toward more reliable systems. In particular, we propose a Question Answering based metric. We show how this metric can be used as a reward in a Reinforcement Learning setup to improve NLG models. Toward this objective, we also explore learned rewards that are the discriminators, and introduce several new algorithms that benefit NLG during training and decoding times. In particular, we propose to combine Monte Carlo Tree Search with Generative Adversarial Networks, resulting in state-of-the-art models.

Publications 2019-2022

• 2022
  • Th. Scialom : “Natural Language Generation with Reinforcement Learning”, soutenance de thèse, soutenance 06/07/2022, direction de recherche Gallinari, Patrick, co-encadrement : Piwowarski, Benjamin, Lamprier, Sylvain (2022)
• 2021
  • C. Rebuffel, Th. Scialom, L. Soulier, B. Piwowarski, S. Lamprier, J. Staiano, G. Scoutheeten, P. Gallinari : “Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation”, 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Punta Cana, Dominican Republic, pp. 8029-8036, (Association for Computational Linguistics) (2021)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano, A. Wang, P. Gallinari : “QuestEval: Summarization Asks for Fact-based Evaluation”, 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 6594-6604, (Association for Computational Linguistics) (2021)
  • Th. Scialom, P.‑A. Dray, J. Staiano, S. Lamprier, B. Piwowarski : “To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs”, Advances in Neural Information Processing Systems, vol. 34, Virtual, United States, pp. 26585-26597, (Curran Associates, Inc.) (2021)
• 2020
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “ColdGANs: Taming Language GANs with Cautious Sampling Strategies”, Advances in Neural Information Processing Systems, vol. 33, NeurIPS Proceedings, Virtual, Åland Islands, pp. 18978-18989, (Curran Associates, Inc.) (2020)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “ColdGANs: Taming Language GANs with Cautious Sampling Strategies”, (2020)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “Discriminative Adversarial Search for Abstractive Summarization”, (2020)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “MLSUM: The Multilingual Summarization Corpus”, (2020)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “MLSUM: The Multilingual Summarization Corpus”, 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, France, pp. 8051-8067, (Association for Computational Linguistics) (2020)
  • Th. Scialom, P.‑A. Dray, S. Lamprier, B. Piwowarski, J. Staiano : “Discriminative Adversarial Search for Abstractive Summarization”, 37^th International Conference on Machine Learning, vol. 119, Proceedings of Machine Learning Research, Virtual, Åland Islands, pp. 8555-8564, (PMLR) (2020)
• 2019
  • Th. Scialom, S. Lamprier, B. Piwowarski, J. Staiano : “Answers Unite! Unsupervised Metrics for Reinforced Summarization Models”, 2019 Conference on Empirical Methods in Natural Language Processing and the 9^th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), ACL Anthology, Hong Kong, China, pp. 3237-3247, (Association for Computational Linguistics) (2019)
  • Th. Scialom, B. Piwowarski, J. Staiano : “Self-Attention Architectures for Answer-Agnostic Neural Question Generation”, ACL 2019 - Annual Meeting of the Association for Computational Linguistics, Proceedings of the 57^th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 6027-6032, (Association for Computational Linguistics) (2019)
  • Th. Scialom, B. Piwowarski, J. Staiano : “Architecture basée sur les mécanismes d’attention: le cas de la génération de questions neuronales”, COnférence en Recherche d'Informations et Applications - CORIA 2019, 16^th French Information Retrieval Conference, COnférence en Recherche d'Informations et Applications - CORIA 2019, 16^th French Information Retrieval Conference. Lyon, France, May 25-29, 2019. Proceedings, Lyon, France (2019)