1. Measuring Systematic Generalization in Neural Proof Generation with Transformers Gontier, Nicolas, Sinha, Koustuv, Reddy, Siva, and Pal, Christopher NeurIPS 2020 [Abs] [arXiv] [Code]
  2. Evaluating Logical Generalization in Graph Neural Networks Sinha, Koustuv, Sodhani, Shagun, Pineau, Joelle, and Hamilton, William L arXiv preprint arXiv:2003.06560 2020 [Abs] [arXiv] [Code]
  3. Learning an Unreferenced Metric for Online Dialogue Evaluation Sinha, Koustuv, Parthasarathi, Prasanna, Wang, Jasmine, Lowe, Ryan, Hamilton, William L., and Pineau, Joelle ACL 2020 [Abs] [arXiv] [Code]
  4. Probing Linguistic Systematicity Goodwin, Emily, Sinha, Koustuv, and O’Donnell, Timothy J. ACL 2020 [Abs] [arXiv] [Code]
  5. Ideas for Improving the Field of Machine Learning: Summarizing Discussion from the NeurIPS 2019 Retrospectives Workshop Sodhani, Shagun, Jaiswal, Mayoore S, Baker, Lauren, Sinha, Koustuv, Shneider, Carl, Henderson, Peter, Lehman, Joel, and Lowe, Ryan arXiv preprint arXiv:2007.10546 2020 [Abs] [arXiv]
  6. Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) Pineau, Joelle, Vincent-Lamarre, Philippe, Sinha, Koustuv, Larivière, Vincent, Beygelzimer, Alina, d’Alché-Buc, Florence, Fox, Emily, and Larochelle, Hugo arXiv preprint arXiv:2003.12206 2020 [Abs] [arXiv]
  7. NeurIPS 2019 Reproducibility Challenge Sinha, Koustuv, Pineau, Joelle, Forde, Jessica, Ke, Rosemary Nan, and Larochelle, Hugo ReScience C 2020 [PDF] [Code]


  1. CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text Sinha, Koustuv, Sodhani, Shagun, Dong, Jin, Pineau, Joelle, and Hamilton, William L. Empirical Methods of Natural Language Processing (EMNLP) 2019 [Abs] [arXiv] [Code]
  2. ICLR Reproducibility Challenge 2019 Pineau, Joelle, Sinha, Koustuv, Fried, Genevieve, Ke, Rosemary Nan, and Larochelle, Hugo 2019 [PDF]


  1. Compositional Language Understanding with Text-based Relational Reasoning Sinha, Koustuv, Sodhani, Shagun, Hamilton, William L., and Pineau, Joelle Relational Representation Learning Workshop, NIPS 2018 [Abs] [arXiv] [Code]
  2. Adversarial Gain Henderson, Peter, Sinha, Koustuv, Ke, Rosemary Nan, and Pineau, Joelle Arxiv Pre-print 2018 [Abs] [arXiv]
  3. The RLLChatbot: a solution to the ConvAI Challenge Gontier, Nicolas A., Sinha, Koustuv, Henderson, Peter, Serban, Iulian, Noseworthy, Michael, Parthasarathi, Prasanna, and Pineau, Joelle under review at Dialog & Discourse Journal (D&D) 2018 [Abs] [arXiv]
  4. A Hierarchical Neural Attention-based Text Classifier Sinha, Koustuv, Dong, Yue, Chi-kit Cheung, Jackie, and Ruths, Derek Empirical Methods of Natural Language Processing (EMNLP) 2018 [Abs] [PDF]
  5. Ethical Challenges in Data-Driven Dialogue Systems Henderson, Peter, Sinha, Koustuv, Angelard-Gontier, Nicolas, Ke, Nan Rosemary, Fried, Genevieve, Lowe, Ryan, and Pineau, Joelle AAAI/ACM AI Ethics and Society Conference 2018 [Abs] [arXiv] [Code]