David Alvarez-Melis

Postdoctoral Researcher, Microsoft Research New England

1 Memorial Drive, Cambridge MA

firstsurname.secondsurname@microsoft.com

About

I'm a Postdoctoral Researcher at Microsoft Research New England, within the Machine Learning and Statstics group. My research seeks to make machine learning more broadly applicable (especially to data-poor applications) and trustworthy (e.g., robust and interpretable). I am particularly interested in the implications of these two directions for applications in the natural and medical sciences.

My approach to the first of these goals draws on ideas from statistics, optimization, and applied mathemtics, especially optimal transport, which I have used to develop methods to mitigate data scarcity by various types of geometric dataset manipulations: alignment, comparison, generation, and transformation. This talk provides a high-level overview of this part of my work.

As for trustworthy machine learning, I have worked on methods for explaining predictions of black box models, showed their lack of robustness, proposed methods to robustify them, and sought inspiration in the social sciences to make them human-centered.

In the past, I worked on various aspects of learning with highly-structured data such as text or graphs, ranging from learning representations of structured objects, to generating them, to interpreting models that operate on them.

Bio

I recently obtained a PhD in computer science from MIT, where I worked on various topics in machine learning and natural language processing under the supervision of Tommi Jaakkola. I also hold BSc (Licenciatura) and MS degrees in mathematics from ITAM and Courant Institute (NYU), respectively. During the latter, I worked on semidefinite programming for domain adaptation under the supervision of Mehryar Mohri. Between Master's and PhD, I spent a year at IBM's T.J. Watson Research Center, working with Ken Church and others in the Speech Recognition Group.

News

Projects

Dataset Distances and Dynamics
A principled framework to compare and transform labeled datasets
Word Translation with Optimal Transport
OT-based approaches to fully unsupervised bilingual lexical induction
Optimal Transport with Local and Global Structure
Generalizing the OT problem to include local structure (or ignore global invariances)
Robustly Interpretable Machine Learning
Bridging the gap between model expressiveness and transparency
Dataset Distances and Dynamics
A principled framework to compare and transform labeled datasets
Word Translation with Optimal Transport
OT-based approaches to fully unsupervised bilingual lexical induction
Optimal Transport with Local and Global Structure
Generalizing the OT problem to include local structure (or ignore global invariances)
Robustly Interpretable Machine Learning
Bridging the gap between model expressiveness and transparency
Towards a Theory of Word Embeddings
A theoretical framework to understand the semantic properties of word embeddings

Publications

Most recent publications on Google Scholar.

  • Select
  • ML
  • NLP
  • OT
  • Interp
  • Theses
  • All

Geometric Dataset Distances via Optimal Transport

David Alvarez-Melis, Nicolò Fusi

NeurIPS'20: Neural Information Processing Systems. 2020.

Earlier version at AutoML @ ICML 2020.

Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces

David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola

AISTATS'20: Artificial Intelligence and Statistics. 2020.

Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.

Probabilistic Bias Mitigation in Word Embeddings

Hailey James-Sorenson, David Alvarez-Melis

HCML @ NeurIPS2019

Optimal Transport in Structured Domains: Algorithms and Applications

David Alvarez-Melis (advisor: Tommi S. Jaakkola)

PhD Thesis, MIT. 2019.

pdf

Learning Generative Models across Incomparable Spaces

Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

ICML'19: International Conference on Machine Learning.

Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.

Towards Optimal Transport with Global Invariances

David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola

AISTATS'19: Artificial Intelligence and Statistics. 2019.

Towards Robust Interpretability with Self-Explaining Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

NeurIPS'18: Neural Information Processing Systems. 2018.

Gromov-Wasserstein Alignment of Word Embedding Spaces

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.

Structured Optimal Transport

David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka

AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.

Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.

A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'17: Empirical Methods in Natural Language Processing. 2017.

Tree-structured Decoding with Doubly-recurrent Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

ICLR'17: International Conference on Learning Representations. 2017.

Word Embeddings as Metric Recovery in Semantic Spaces

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).

Geometric Dataset Distances via Optimal Transport

David Alvarez-Melis, Nicolò Fusi

NeurIPS'20: Neural Information Processing Systems. 2020.

Earlier version at AutoML @ ICML 2020.

Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces

David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola

AISTATS'20: Artificial Intelligence and Statistics. 2020.

Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.

Probabilistic Bias Mitigation in Word Embeddings

Hailey James-Sorenson, David Alvarez-Melis

HCML @ NeurIPS2019

Optimal Transport in Structured Domains: Algorithms and Applications

David Alvarez-Melis (advisor: Tommi S. Jaakkola)

PhD Thesis, MIT. 2019.

pdf

Functional Transparency for Structured Data: a Game-Theoretic Approach,

Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola

ICML'19: International Conference on Machine Learning.

Learning Generative Models across Incomparable Spaces

Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

ICML'19: International Conference on Machine Learning.

Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.

Towards Robust, Locally Linear Deep Networks

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

ICLR'19: International Conference on Learning Representations. 2019.

Towards Optimal Transport with Global Invariances

David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola

AISTATS'19: Artificial Intelligence and Statistics. 2019.

Towards Robust Interpretability with Self-Explaining Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

NeurIPS'18: Neural Information Processing Systems. 2018.

Gromov-Wasserstein Alignment of Word Embedding Spaces

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.

Game-theoretic Interpretability for Temporal Modeling

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).

On the Robustness of Interpretability Methods

David Alvarez-Melis, Tommi S. Jaakkola

Workshop on Human Interpretability in Machine Learning (@ICML 2018).

Structured Optimal Transport

David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka

AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.

Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.

The Emotional GAN: Priming Adversarial Generation of Art with Emotion.

David Alvarez-Melis, Judith Amores

NIPS Workshop on Machine Learning for Creativity and Design. 2017.

Distributional Adversarial Networks

Chengtao Li*, David Alvarez-Melis*, Keyulu Xu, Stefanie Jegelka, Suvrit Sra

ICLR'17: International Conference on Learning Representations (Workshop track). 2017.

A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'17: Empirical Methods in Natural Language Processing. 2017.

Tree-structured Decoding with Doubly-recurrent Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

ICLR'17: International Conference on Learning Representations. 2017.

Topic Modeling in Twitter: Aggregating Tweets by Conversations

David Alvarez-Melis*, Martin Saveski*

ICWSM'16: International AAAI Conference on Web and Social Media. 2016. (Short Paper)

Word, graph and manifold embedding from Markov processes

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.

A translation of 'The characteristic function of a random phenomenon' by Bruno de Finetti

David Alvarez-Melis, Tamara Broderick

Translation. 2015

The Matrix Multiplicative Weights Algorithm for Domain Adaptation

David Alvarez-Melis (advisor: Mehryar Mohri)

MS Thesis, Courant Institute. 2013.

pdf

Probabilistic Bias Mitigation in Word Embeddings

Hailey James-Sorenson, David Alvarez-Melis

HCML @ NeurIPS2019

Gromov-Wasserstein Alignment of Word Embedding Spaces

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.

A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'17: Empirical Methods in Natural Language Processing. 2017.

Tree-structured Decoding with Doubly-recurrent Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

ICLR'17: International Conference on Learning Representations. 2017.

Word Embeddings as Metric Recovery in Semantic Spaces

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).

Word, graph and manifold embedding from Markov processes

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.

Geometric Dataset Distances via Optimal Transport

David Alvarez-Melis, Nicolò Fusi

NeurIPS'20: Neural Information Processing Systems. 2020.

Earlier version at AutoML @ ICML 2020.

Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces

David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola

AISTATS'20: Artificial Intelligence and Statistics. 2020.

Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.

Optimal Transport in Structured Domains: Algorithms and Applications

David Alvarez-Melis (advisor: Tommi S. Jaakkola)

PhD Thesis, MIT. 2019.

pdf

Learning Generative Models across Incomparable Spaces

Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

ICML'19: International Conference on Machine Learning.

Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.

Towards Optimal Transport with Global Invariances

David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola

AISTATS'19: Artificial Intelligence and Statistics. 2019.

Gromov-Wasserstein Alignment of Word Embedding Spaces

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.

Structured Optimal Transport

David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka

AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.

Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.

Weight of Evidence as a Basis for Human-Oriented Explanations

David Alvarez-Melis, Hal Daumé III, Jennifer Wortman Vaughan, Hanna Wallach

HCML @ NeurIPS2019

Functional Transparency for Structured Data: a Game-Theoretic Approach,

Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola

ICML'19: International Conference on Machine Learning.

Towards Robust Interpretability with Self-Explaining Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

NeurIPS'18: Neural Information Processing Systems. 2018.

Game-theoretic Interpretability for Temporal Modeling

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).

On the Robustness of Interpretability Methods

David Alvarez-Melis, Tommi S. Jaakkola

Workshop on Human Interpretability in Machine Learning (@ICML 2018).

A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'17: Empirical Methods in Natural Language Processing. 2017.

Optimal Transport in Structured Domains: Algorithms and Applications

David Alvarez-Melis (advisor: Tommi S. Jaakkola)

PhD Thesis, MIT. 2019.

pdf

The Matrix Multiplicative Weights Algorithm for Domain Adaptation

David Alvarez-Melis (advisor: Mehryar Mohri)

MS Thesis, Courant Institute. 2013.

pdf

Lax-Milgram's Theorem: Generalizations and Applications

David Alvarez-Melis (advisor: Carlos Bosch Giral)

BSc Thesis, ITAM. 2011.

pdf

Geometric Dataset Distances via Optimal Transport

David Alvarez-Melis, Nicolò Fusi

NeurIPS'20: Neural Information Processing Systems. 2020.

Earlier version at AutoML @ ICML 2020.

Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces

David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola

AISTATS'20: Artificial Intelligence and Statistics. 2020.

Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.

Probabilistic Bias Mitigation in Word Embeddings

Hailey James-Sorenson, David Alvarez-Melis

HCML @ NeurIPS2019

Weight of Evidence as a Basis for Human-Oriented Explanations

David Alvarez-Melis, Hal Daumé III, Jennifer Wortman Vaughan, Hanna Wallach

HCML @ NeurIPS2019

Optimal Transport in Structured Domains: Algorithms and Applications

David Alvarez-Melis (advisor: Tommi S. Jaakkola)

PhD Thesis, MIT. 2019.

pdf

Functional Transparency for Structured Data: a Game-Theoretic Approach,

Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola

ICML'19: International Conference on Machine Learning.

Learning Generative Models across Incomparable Spaces

Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka

ICML'19: International Conference on Machine Learning.

Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.

Towards Robust, Locally Linear Deep Networks

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

ICLR'19: International Conference on Learning Representations. 2019.

Towards Optimal Transport with Global Invariances

David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola

AISTATS'19: Artificial Intelligence and Statistics. 2019.

Towards Robust Interpretability with Self-Explaining Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

NeurIPS'18: Neural Information Processing Systems. 2018.

Gromov-Wasserstein Alignment of Word Embedding Spaces

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.

Game-theoretic Interpretability for Temporal Modeling

Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola

Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).

On the Robustness of Interpretability Methods

David Alvarez-Melis, Tommi S. Jaakkola

Workshop on Human Interpretability in Machine Learning (@ICML 2018).

Structured Optimal Transport

David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka

AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.

Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.

The Emotional GAN: Priming Adversarial Generation of Art with Emotion.

David Alvarez-Melis, Judith Amores

NIPS Workshop on Machine Learning for Creativity and Design. 2017.

Distributional Adversarial Networks

Chengtao Li*, David Alvarez-Melis*, Keyulu Xu, Stefanie Jegelka, Suvrit Sra

ICLR'17: International Conference on Learning Representations (Workshop track). 2017.

A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models

David Alvarez-Melis, Tommi S. Jaakkola

EMNLP'17: Empirical Methods in Natural Language Processing. 2017.

Tree-structured Decoding with Doubly-recurrent Neural Networks

David Alvarez-Melis, Tommi S. Jaakkola

ICLR'17: International Conference on Learning Representations. 2017.

Word Embeddings as Metric Recovery in Semantic Spaces

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).

Topic Modeling in Twitter: Aggregating Tweets by Conversations

David Alvarez-Melis*, Martin Saveski*

ICWSM'16: International AAAI Conference on Web and Social Media. 2016. (Short Paper)

Word, graph and manifold embedding from Markov processes

Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola

NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.

A translation of 'The characteristic function of a random phenomenon' by Bruno de Finetti

David Alvarez-Melis, Tamara Broderick

Translation. 2015

The Matrix Multiplicative Weights Algorithm for Domain Adaptation

David Alvarez-Melis (advisor: Mehryar Mohri)

MS Thesis, Courant Institute. 2013.

pdf

Lax-Milgram's Theorem: Generalizations and Applications

David Alvarez-Melis (advisor: Carlos Bosch Giral)

BSc Thesis, ITAM. 2011.

pdf

Teaching

Explaining is understanding.

The following extract is from David Goldstein's book on Feynman:

"Feynman was a truly great teacher. He prided himself on being able to devise ways to explain even the most profound ideas to beginning students. Once, I said to him, "Dick, explain to me, so that I can understand it, why spin one-half particles obey Fermi-Dirac statistics." Sizing up his audience perfectly, Feynman said, "I'll prepare a freshman lecture on it." But he came back a few days later to say, "I couldn't do it. I couldn't reduce it to the freshman level. That means we don't really understand it."

Some current and past courses I have TA'd:

Vitæ

Full CV in PDF (or a shorter Resumé).

  • Microsoft Research, New England 2019 -- now
    Postdoctoral Researcher

  • MIT CSAIL 2014 - 2019
    Ph.D. in Computer Science
    Minor: Mathematical Optimization
    Thesis Advisor: Tommi Jaakkola
  • Microsoft Research, NYC Summer 2018
    Research Intern
    Mentors: H. Wallach, J.W. Vaughan, H. Daume III
  • Microsoft Research, Redmond Summer 2016
    Research Intern
    Mentors: S. Yih, M.W. Chang, K. Toutanova, C. Meek
  • IBM Research 2013 - 2014
    Supplemental Researcher
    Speech Recognition Group
  • Courant Institute, NYU 2011 - 2013
    MS in Mathematics
    Thesis Advisor: Mehryar Mohri
  • ITAM 2006 - 2011
    BSc in Applied Mathematics
    Thesis Advisor: Carlos Bosch

Misc

Outside of research, I enjoy running, brewing beer and playing guitar. I also like quotes. Here's a few more:

"We cannot solve our problems with the same thinking we used when we created them." - A. Einstein
"The real danger is not that computers will begin to think like men, but that men will begin to think like computers" - Syndey J. Harris

Meta

This website was built with jekyll based on a template by my [friend|co-author|ex-roommate] and all-around awesome person, Martin Saveski.