Skip to the content.

About

Recently, artificial intelligence has seen the explosion of deep learning models, which are able to reach super-human performance in several tasks, finding application in many domains. These performance improvements, however, come at a cost: DL models are uninterpretable black boxes, where one feeds an input and obtains an output without understanding the motivations behind that prediction or decision.

To address this problem, two research areas are particularly active: the eXplainable AI (XAI) field and the visual analytics community. The eXplainable XAI field tries to address such problems by proposing algorithmic methods that can explain, at least partially, the behavior of these networks. Their works also try to define the limits of interpretability, the definition of valid metrics, the study of users, and the study of the effectiveness of these solutions. Conversely, visual analytics systems target users helping them to understand and interact with machine learning models providing visualizations and systems that facilitate the exploration, analysis, interaction, and understanding of machine learning models. In the last few years, the usage of methodologies that explain deep learning models has become central in these systems. As a result, the interaction between the XAI and visual analytics communities is becoming more and more important.

The workshop aims at advancing the discourse by collecting novel methods and discussing challenges, issues, and goals around the usage of XAI approaches to debug and improve current deep learning models. To achieve this goal, the workshop aims at bringing researchers and practitioners from both fields, strengthening their collaboration. In particular, we narrow the XAI focus to the specific case in which developers or researchers need to debug their models and diagnose system behaviors. Therefore we start from the assumption that this type of user typically has substantial knowledge about the models themselves but needs to validate, debug, and improve them. The workshop is scheduled for December 14, 2021.

Topics

The topics include but are not limited to:


Schedule

Central European Time (CET)

Tue 2:00 p.m. - 2:09 p.m. (CET)
Welcome (Opening)
Roberto Capobianco
Tue 2:10 p.m. - 2:13 p.m. (CET)
Speaker Introduction (Introduction)
Wen Sun
Tue 2:14 p.m. - 2:52 p.m. (CET)
[IT1] Visual Analytics for Explainable Machine Learning (Invited Talk)
Shixia Liu
Tue 2:53 p.m. - 3:03 p.m. (CET)
Q/A Session (Live Q/A)
Wen Sun, Shixia Liu
Tue 3:04 p.m. - 3:05 p.m. (CET)
Speaker Introduction (Introduction)
Biagio La Rosa
Tue 3:05 p.m. - 3:19 p.m. (CET)
[O1] Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation (Oral)  
Théo Jaunet, Christian Wolf
Tue 3:20 p.m. - 3:25 p.m. (CET)
Q/A Session (Live Q/A)
Biagio La Rosa
Tue 3:25 p.m. - 3:35 p.m. (CET)
Break (10min) (Break)
Tue 3:35 p.m. - 3:37 p.m. (CET)
Speaker Introduction (Introduction)
Biagio La Rosa
Tue 3:37 p.m. - 4:21 p.m. (CET)
[IT2] Explainability and robustness: Towards trustworthy AI (Invited Talk)
Andreas Holzinger
Tue 4:22 p.m. - 4:32 p.m. (CET)
Q/A Session (Live Q/A)
Biagio La Rosa, Andreas Holzinger
Tue 4:33 p.m. - 4:34 p.m. (CET)
Speaker Introduction (Introduction)
Leilani H Gilpin
Tue 4:34 p.m. - 4:49 p.m. (CET)
[O2] Not too close and not too far: enforcing monotonicity requires penalizing the right points (Oral)  
Joao Monteiro, mohamed.o.ahmed, Hossein Hajimirsadeghi, Greg Mori
Tue 4:50 p.m. - 4:55 p.m. (CET)
Q/A Session (Live Q/A)
Leilani H Gilpin
Tue 4:55 p.m. - 5:05 p.m. (CET)
Break (10min) (Break)
Tue 5:05 p.m. - 5:07 p.m. (CET)
Speaker Introduction (Introduction)
Biagio La Rosa
Tue 5:07 p.m. - 5:20 p.m. (CET)
[G] Empowering Human Translators via Interpretable Interactive Neural Machine Translation (A glimpse of the future Track)
Gabriele Sarti
Tue 5:21 p.m. - 5:26 p.m. (CET)
Q/A Session (Live Q/A)
Biagio La Rosa, Gabriele Sarti
Tue 5:27 p.m. - 5:28 p.m. (CET)
Speaker Introduction (Introduction)
Biagio La Rosa
Tue 5:28 p.m. - 5:41 p.m. (CET)
[O3] Reinforcement Explanation Learning (Oral)  
agarwalsiddhant10, OWAIS IQBAL, Sree Aditya Buridi, Madda Manjusha, Abir Das
Tue 5:42 p.m. - 5:47 p.m. (CET)
Q/A Session (Live Q/A)
Biagio La Rosa
Tue 5:48 p.m. - 5:50 p.m. (CET)
Spotlight Introduction (Introduction)
Biagio La Rosa
Tue 5:50 p.m. - 5:53 p.m. (CET)
[S1] Interpreting BERT architecture predictions for peptide presentation by MHC class I proteins (Spotlight)  
Hans-Christof Gasser
Tue 5:53 p.m. - 5:57 p.m. (CET)
[S2] XC: Exploring Quantitative Use Cases for Explanations in 3D Object Detection (Spotlight)  
Sunsheng Gu, Vahdat Abdelzad, Krzysztof Czarnecki
Tue 5:57 p.m. - 6:00 p.m. (CET)
[S3] Interpretability in Gated Modular Neural Networks (Spotlight)  
Yamuna Krishnamurthy, Chris Watkins
Tue 6:00 p.m. - 6:03 p.m. (CET)
[S4] A Robust Unsupervised Ensemble of Feature-Based Explanations using Restricted Boltzmann Machines (Spotlight)  
Vadim Borisov, Johannes Meier, Johan Van den Heuvel, Hamed Jalali, Gjergji. Kasneci
Tue 6:03 p.m. - 6:06 p.m. (CET)
[S5] Debugging the Internals of Convolutional Networks (Spotlight)  
Bilal Alsallakh, Narine Kokhlikyan, Vivek Miglani, Shubham Muttepawar, Edward Wang, Sara Zhang, Orion Reblitz-Richardson
Tue 6:06 p.m. - 6:09 p.m. (CET)
[S6] Defuse: Training More Robust Models through Creation and Correction of Novel Model Errors (Spotlight)  
Dylan Slack, Nathalie Rauschmayr, Krishnaram Kenthapadi
Tue 6:09 p.m. - 6:12 p.m. (CET)
[S7] DeDUCE: Generating Counterfactual Explanations At Scale (Spotlight)  
Benedikt Höltgen, Lisa Schut, Jan Brauner, Yarin Gal
Tue 6:12 p.m. - 6:22 p.m. (CET)
Break (10min) (Break)
Tue 6:22 p.m. - 6:25 p.m. (CET)
Speaker Introduction (Introduction)
Alexander Feldman
Tue 6:26 p.m. - 7:04 p.m. (CET)
[IT3] Towards Reliable and Robust Model Explanations (Invited Talk)
Himabindu Lakkaraju
Tue 7:05 p.m. - 7:15 p.m. (CET)
Q/A Session (Live Q/A)
Alexander Feldman, Himabindu Lakkaraju
Tue 7:16 p.m. - 7:17 p.m. (CET)
Speaker Introduction (Introduction)
Roberto Capobianco
Tue 7:17 p.m. - 7:32 p.m. (CET)
[O4] Are All Neurons Created Equal? Interpreting and Controlling BERT through Individual Neurons (Oral)  
Omer Antverg, Yonatan Belinkov
Tue 7:33 p.m. - 7:38 p.m. (CET)
Q/A Session (Live Q/A)
Roberto Capobianco
Tue 7:38 p.m. (CET) (CET) - 7:50 p.m. (CET) (CET)
Break (12min) (Break)
Tue 7:50 p.m. (CET) (CET) - 7:51 p.m. (CET) (CET)
Speaker Introduction (Introduction)
Leilani H Gilpin
Tue 7:51 p.m. (CET) (CET) - 8:23 p.m. (CET) (CET)
[IT4] Detecting model reliance on spurious signals is challenging for post hoc explanation approaches (Invited Talk)
Julius Adebayo
Tue 8:24 p.m. (CET) (CET) - 8:34 p.m. (CET) (CET)
Q/A Session (Live Q/A)
Leilani H Gilpin, Julius Adebayo
Tue 8:35 p.m. (CET) (CET) - 8:36 p.m. (CET) (CET)
Speaker Introduction (Introduction)
Roberto Capobianco
Tue 8:36 p.m. - 9:48 p.m. (CET)
[O5] Do Feature Attribution Methods Correctly Attribute Features? (Oral)  
Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, Julie A Shah
Tue 8:49 p.m. - 8:54 p.m. (CET)
Q/A Session (Live Q/A)
Roberto Capobianco
Tue 8:55 p.m. - 9:10 p.m. (CET)
Break (15min) (Break)
Tue 9:10 p.m. - 9:11 p.m. (CET)
Speaker Introduction (Introduction)
Roberto Capobianco
Tue 9:11 p.m. - 9:27 p.m. (CET)
[O6] Explaining Information Flow Inside Vision Transformers Using Markov Chain (Oral)  
Tingyi Yuan, Xuhong Li, Haoyi Xiong, Dejing Dou
Tue 9:28 p.m. - 9:32 p.m. (CET)
Q/A Session (Live Q/A)
Roberto Capobianco
Tue 9:33 p.m. - 9:36 p.m. (CET)
Speaker Introduction (Introduction)
Alice Xiang
Tue 9:37 p.m. - 10:21 p.m. (CET)
[IT5] Natural language descriptions of deep features (Invited Talk)
Jacob Andreas
Tue 10:22 p.m. - 10:32 p.m. (CET)
Q/A Session
Alice Xiang, Jacob Andreas
Tue 10:33 p.m. - 10:35 p.m. (CET)
Spotlight Introduction (Introduction)
Biagio La Rosa
Tue 10:35 p.m. - 10:39 p.m. (CET)
[S8] Fast TreeSHAP: Accelerating SHAP Value Computation for Trees (Spotlight)  
Jilei Yang
Tue 10:39 p.m. - 10:42 p.m. (CET)
[S9] Simulated User Studies for Explanation Evaluation (Spotlight)  
Valerie Chen, Gregory Plumb, Nicholay Topin, Ameet S Talwalkar
Tue 10:42 p.m. - 10:45 p.m. (CET)
[S10] Exploring XAI for the Arts: Explaining Latent Space in Generative Music (Spotlight)  
Nick Bryan-Kinns, Berker Banar, Corey Ford, Simon Colton
Tue 10:45 p.m. - 10:50 p.m. (CET)
[S11] Interpreting Language Models Through Knowledge Graph Extraction (Spotlight)  
Vinitra Swamy, Angelika Romanou, Martin Jaggi
Tue 10:50 p.m. - 10:53 p.m. (CET)
[S12] Efficient Decompositional Rule Extraction for Deep Neural Networks (Spotlight)  
Mateo Espinosa Zarlenga, Mateja Jamnik
Tue 10:53 p.m. - 10:57 p.m. (CET)
[S13] Revisiting Sanity Checks for Saliency Maps (Spotlight)  
Gal O Yona
Tue 10:57 p.m. - 11:01 p.m. (CET)
[S14] Towards Better Visual Explanations for Deep ImageClassifiers (Spotlight)  
Agnieszka Grabska-Barwinska, Amal Rannen-Triki, Omar Rivasplata, András György
Tue 11:00 p.m. - 11:06 p.m. (CET)
Closing Remarks (Closing)
Biagio La Rosa
Tue 11:06 p.m. - 11:30 p.m. (CET)
Poster Session

Invited Speakers

Adebayo
Julius Adebayo
Detecting model reliance on spurious signals is challenging for post hoc explanation approaches
Jacob Andreas
Natural language descriptions of deep features
Andreas Holzinger
Explainability and robustness: Towards trustworthy AI
Himabindu Lakkaraju
Towards Reliable and Robust Model Explanations
Shixia Liu
Visual Analytics for Explainable Machine Learning

List of Accepted Papers

A Robust Unsupervised Ensemble of Feature-Based Explanations using Restricted Boltzmann Machines
Vadim Borisov, Johannes Meier, Johan Van den Heuvel, Hamed Jalali, and Gjergji. Kasneci
[Paper][Code]

Are All Neurons Created Equal? Interpreting and Controlling BERT through Individual Neurons
Omer Antverg and Yonatan Belinkov
[Paper]

Do Feature Attribution Methods Correctly Attribute Features?
Yilun Zhou, Serena Booth, Marco Tulio Ribeiro, and Julie Shah
[Paper]

Debugging the Internals of Convolutional Networks
Bilal Alsallakh, Narine Kokhlikyan, Vivek Miglani, Shubham Muttepawar, Edward Wang, Sara Zhang, David Adkins, and Orion Reblitz-Richardson
[Paper]

DeDUCE: Generating Counterfactual Explanations At Scale
Benedikt Höltgen, Lisa Schut, Jan M. Brauner, and Yarin Gal
[Paper][Code]

Defuse: Training More Robust Models through Creation and Correction of Novel Model Errors
Dylan Z Slack, Nathalie Rauschmayr, and Krishnaram Kenthapadi
[Paper]

Efficient Decompositional Rule Extraction for Deep Neural Networks
Mateo Espinosa Zarlenga, Zohreh Shams, and Mateja Jamnik
[Paper][Code]

Explaining Information Flow Inside Vision Transformers Using Markov Chain
Yuan Tingyi, Xuhong Li, Haoyi Xiong, Hui Cao, and Dejing Dou
[Paper][Code]

Exploring XAI for the Arts: Explaining Latent Space in Generative Music
Nick Bryan-Kinns, Berker Banar, Corey Ford, Courtney N Reed, Yixiao Zhang, Simon Colton, and Jack Armitage
[Paper][Code]

Fast TreeSHAP: Accelerating SHAP Value Computation for Trees
Jilei Yang
[Paper]

Interpretability in Gated Modular Neural Networks.
Yamuna Krishnamurthy and Chris Watkins
[Paper][Code]

Interpreting BERT architecture predictions for peptide presentation by MHC class I proteins
Hans-Christof Gasser, Georges Bedran, Bo Ren, David Goodlett, Javier Alfaro, and Ajitha Rajan
[Paper][Code]

Interpreting Language Models Through Knowledge Graph Extraction
Vinitra Swamy, Angelika Romanou, and Martin Jaggi
[Paper][Code]

Not too close and not too far: enforcing monotonicity requires penalizing the right points
Joao Monteiro, Mohamed Osama Ahmed, Hossein Hajimirsadeghi, and Greg Mori
[Paper]

Reinforcement Explanation Learning
Siddhant Agarwal, Owais Iqbal, Sree Aditya Buridi, Madda Manjusha, and Abir Das
[Paper] [Code]

Revisiting Sanity Checks for Saliency Maps.
Gal Yona
[Paper]

Simulated User Studies for Explanation Evaluation.
Valerie Chen, Gregory Plumb, Nicholay Topin, and Ameet Talwalkar
[Paper]

Towards Better Visual Explanations for Deep ImageClassifiers.
Agnieszka Grabska-Barwinska, Amal Rannen-Triki, Omar Rivasplata, and András György
[Paper]

Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation
Théo Jaunet, Guillaume Bono, Romain Vuillemot, and Christian Wolf
[Paper][Code]

XC: Exploring Quantitative Use Cases for Explanations in 3D Object Detection
Sunsheng Gu, Vahdat Abdelzad, and Krzysztof Czarnecki
[Paper][Code]

Organization

Organizers

Advisory Board

Peter Stone (Sony AI) and Daniele Nardi (Sapienza University of Rome)

*Outstanding Reviewers**

Program Committee


FAQ

Call for Papers

The call for papers includes three different tracks: regular, mentorship, and a glimpse of the future track. The first one is the traditional track for presenting novel contributions. The second one is directed to young researchers and aims to pair a mentor and a mentee with the goal of polishing early-submissions. The last one is intended for first-year PhD students and MSc students working on the topic, giving them the possibility to introduce themselves and their research proposal to the research community.

Tracks

Regular track:

Submissions to this track have to be novel contributions covering any topic listed above. We don’t accept work that has been already published, that is concurrently submitted to other venues before the submission deadline, or that is presented at the main NeurIPS conference, including as part of an invited talk. We solicit submission of full papers, position papers, and papers describing open problems on one of the topics listed above. Papers must be submitted through the . Papers submitted to the workshop can be submitted to future conferences (e.g. ICML, ICLR) if the acceptance notification comes after the workshop date (December, 14).

We only allow dual submissions of papers submitted to the main NeurIPS 2021 conference with the following constraints: papers must be withdrawn from this workshop before October 1st if they will be accepted at the main conference; papers must include the declaration of dual submission BEFORE the introduction section; authors must append the reviews received from NeurIPS conference (printed from the OpenReview system) to the main corpus and a brief letter summarizing the improvements made after the review process.

We encourage the authors to link a anonymized repository containing the code to replicate the results inside the corpus of the paper. While this is not a mandatory requirement, it will be positively taken in account during the reviewing process and the selection of the contributed talks. You can use Anonymous Github or you can upload your repository on a service that allows anonymity (e.g. GDrive allows anonymous links).

Submissions must follow the NeurIPS anonymized paper format (see NeurIPS style files), and they are limited to a maximum of 9 pages, excluding references. Shorter papers are welcome. They will undergo double-blind peer review. Authors may append to the paper supplementary material, such as appendices, proofs, and derivations; Like submissions, supplementary material must be anonymized. Looking at supplementary material is at the discretion of the reviewers.

At the end of your paper submission, please indicate whether you would like an extended version of the submission to be considered for publication in a journal special issue. According to the feedback from authors, we will further decide whether to publish selected high-quality papers in proceedings or a journal special issue. Reviewers will nominate papers among them with exemplary scientific rigor for publication. We will ask authors again after the acceptance of the paper whether you would like to proceed to a journal special issue. Note that in this case there can be restrictions regarding dual submissions with future conference, depending on the policy of the chosen journal.

Accepted works will be presented as contributed talks or as posters in a poster session or listed on the workshop site as accepted contributions, depending on schedule constraints. It is mandatory that at least one of the authors will attend the workshop and present its work during the contributed talks and the poster session. We are planning to include an additional virtual channel, like Slack, where each accepted paper will have a dedicated section for attendance and authors to discuss. We strongly encourage authors to be available on their section before, during, and shortly after the workshop.

Important Dates for Regular Track:

Submission system opens: 11:59 PM CET, Aug 15, 2021

Submission deadline: 11:59 PM CET, Sep 20, 2021 (Extended deadline!)

Notification date: 11:59 PM CET, Oct 17, 2021

SlidesLive upload for speaker videos: 11:59 PM CET, ~~Oct 25 Oct 28, 2021~~

NEW: Available the LaTeX template style for the camera-ready version HERE. Simply download the file and replace the “neurips_2021.sty” file in your folder. Use the option “final” when importing the neurIPS package using the following command:
\usepackage[final]{neurips_2021}
in your LaTeX .tex file.

Camera-ready deadline : 11:59 PM CET, Nov 5, 2021

Poster deadline : 11:59 PM CET, Dec 1, 2021

Workshop date: Dec 14, 2021

Note that for papers that include a link to the code, the deadline for the camera-ready version is extended to 11:59 PM CET, Nov 15, 2021

Mentorship track:

The mentorship track pairs authors with experienced researchers who have committed to providing meaningful feedback to help polish papers for general submission. This program helps authors to get support and fast feedback from the research community, by facilitating a private dialogue between the mentee and mentor. The papers of this track have to be non-anonymous, template-free. Note that the goal of this track is to receive fast feedback about the structure of the paper and to polish it. It is not intended for deep review. None of the papers submitted to this track will be presented or included in the workshop schedule.

Important Dates for Mentorship track:

Submission deadline: 11:59 PM CET, Sep 30, 2021 LINK

A glimpse of the future track:

This special track aims at discovering and highlighting the most promising students in the field. The students will have the possibility to present their research plan to the community, describing the problems that they plan to address and possible research directions. Submissions for this track must include a research plan (max. 2 pages) and a cover letter from the current or future supervisor. The plan should briefly highlight the problem that the researcher will try to attack, the current methodologies and the expected outcome of its research agenda. This track is intended for first-year PhD students or prospective PhD students. The accepted students will present their plans during the workshop in a dedicated talk. We plan to accept at least 2 students as oral presentations.

Important Dates for A glimpse of the future track:

Submission deadline: 11:59 PM CET, Sep 20, 2021 LINK

Notification Date: 11:59 PM CET, Oct 10, 2021

SlidesLive upload for speaker videos: 11:59 PM CET, Oct 28, 2021


Contacts

If you have any questions feel free to join our slack channel or contact us at any of the following email addresses: