I'm a Senior Researcher at Microsoft Research New England, within the Machine Learning and Statstics group. My research seeks to make machine learning more broadly applicable (especially to data-poor applications) and trustworthy (e.g., robust and interpretable). I am particularly interested in the implications of these two directions for applications in the natural and medical sciences.
My approach to the first of these goals draws on ideas from statistics, optimization, and applied mathemtics, especially optimal transport, which I have used to develop methods to mitigate data scarcity by various types of geometric dataset manipulations: alignment, comparison, generation, and transformation. This talk provides a high-level overview of this part of my work.
As for trustworthy machine learning, I have worked on methods for explaining predictions of black box models, showed their lack of robustness, proposed methods to robustify them, and sought inspiration in the social sciences to make them human-centered.
In the past, I worked on various aspects of learning with highly-structured data such as text or graphs, ranging from learning representations of structured objects, to generating them, to interpreting models that operate on them.
Update: In July 2023, I will be joining Harvard SEAS as an Assistant Professor of Computer Science. I will be looking for motivated PhD students and postdocs to join my group. If you are interested in working with me, send me an email with the subject line "[Harvard] Interested in joining your group" and a brief description of your interests and background.
Before coming to MSR, I obtained a PhD in computer science from MIT, where I worked on various topics in machine learning and natural language processing under the supervision of Tommi Jaakkola, where I also worked closely with Stefanie Jegelka. I also hold BSc (Licenciatura) and MS degrees in mathematics from ITAM and Courant Institute (NYU), respectively. During the latter, I worked on semidefinite programming for domain adaptation under the supervision of Mehryar Mohri. Between Master's and PhD, I spent a year at IBM's T.J. Watson Research Center, working with Ken Church and others in the Speech Recognition Group.
Most recent publications on Google Scholar.
Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks
David Alvarez-Melis, Yair Schiff, Youssef Mroueh
Transactions of Machine Learning Research (TMLR). 2022.
Earlier version at OTML: NeurIPS'21 Workshop on Optimal Transport in Machine Learning .
From Human Explanation to Model Interpretabilty: A Framework Based on Weight of Evidence
David Alvarez-Melis, Harmanpreet Kaur, Hal Daumé III, Hanna Wallach, Jennifer Wortman Vaughan
HCOMP '21: The 9th AAAI Conference on Human Computation and Crowdsourcing. 2021.
Dataset Dynamics via Gradient Flows in Probability Space
David Alvarez-Melis, Nicolò Fusi
ICML'21: International Conference on Machine Learning. 2021.
@InProceedings{alvarez-melis2021dataset, title = {Dataset Dynamics via Gradient Flows in Probability Space}, author = {Alvarez-Melis, David and Fusi, Nicol\`o}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {219--230}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/alvarez-melis21a/alvarez-melis21a.pdf}, url = {https://proceedings.mlr.press/v139/alvarez-melis21a.html}, abstract = {Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.} }
Geometric Dataset Distances via Optimal Transport
David Alvarez-Melis, Nicolò Fusi
NeurIPS'20: Neural Information Processing Systems. 2020.
Earlier version at AutoML @ ICML 2020.
@inproceedings{alvarez-melis2020geometric, author = {Alvarez-Melis, David and Fusi, Nicolo}, booktitle = {Advances in Neural Information Processing Systems}, editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin}, pages = {21428--21439}, publisher = {Curran Associates, Inc.}, title = {Geometric Dataset Distances via Optimal Transport}, url = {https://proceedings.neurips.cc/paper/2020/file/f52a7b2610fb4d3f74b4106fb80b233d-Paper.pdf}, volume = {33}, year = {2020} }
Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces
David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola
AISTATS'20: Artificial Intelligence and Statistics. 2020.
Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.
@InProceedings{alvarez-melis2020unsupervised, title = {Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces}, author = {Alvarez-Melis, David and Mroueh, Youssef and Jaakkola, Tommi}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {1606--1617}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/alvarez-melis20a/alvarez-melis20a.pdf}, url = {http://proceedings.mlr.press/v108/alvarez-melis20a.html}, }
Optimal Transport in Structured Domains: Algorithms and Applications
David Alvarez-Melis (advisor: Tommi S. Jaakkola)
PhD Thesis, MIT. 2019.
Learning Generative Models across Incomparable Spaces
Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka
ICML'19: International Conference on Machine Learning.
Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.
@InProceedings{pmlr-v97-bunne19a, title = {Learning Generative Models across Incomparable Spaces}, author = {Bunne, Charlotte and Alvarez-Melis, David and Krause, Andreas and Jegelka, Stefanie}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {851--861}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/bunne19a/bunne19a.pdf}, url = {http://proceedings.mlr.press/v97/bunne19a.html}, abstract = {Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety. However, in some cases, we may want to only learn some aspects (e.g., cluster or manifold structure), while modifying others (e.g., style, orientation or dimension). In this work, we propose an approach to learn generative models across such incomparable spaces, and demonstrate how to steer the learned distribution towards target properties. A key component of our model is the Gromov-Wasserstein distance, a notion of discrepancy that compares distributions relationally rather than absolutely. While this framework subsumes current generative models in identically reproducing distributions, its inherent flexibility allows application to tasks in manifold learning, relational learning and cross-domain learning.} }
Towards Optimal Transport with Global Invariances
David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola
AISTATS'19: Artificial Intelligence and Statistics. 2019.
@InProceedings{pmlr-v89-alvarez-melis19a, title = {Towards Optimal Transport with Global Invariances}, author = {Alvarez-Melis, David and Jegelka, Stefanie and Jaakkola, Tommi S.}, booktitle = {Proceedings of Machine Learning Research}, pages = {1870--1879}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, address = {}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/alvarez-melis19a/alvarez-melis19a.pdf}, url = {http://proceedings.mlr.press/v89/alvarez-melis19a.html}, abstract = {Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can be directly evaluated. Unfortunately neither requirement is likely to hold when object representations are learned from data. Indeed, automatically derived representations such as word embeddings are typically fixed only up to some global transformations, for example, reflection or rotation. As a result, pairwise distances across two such instances are ill-defined without specifying their relative transformation. In this work, we propose a general framework for optimal transport in the presence of latent global transformations. We cast the problem as a joint optimization over transport couplings and transformations chosen from a flexible class of invariances, propose algorithms to solve it, and show promising results in various tasks, including a popular unsupervised word translation benchmark.} }
Towards Robust Interpretability with Self-Explaining Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
NeurIPS'18: Neural Information Processing Systems. 2018.
@incollection{NIPS2018_8003, title = {Towards Robust Interpretability with Self-Explaining Neural Networks}, author = {Alvarez Melis, David and Jaakkola, Tommi}, booktitle = {Advances in Neural Information Processing Systems 31}, editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett}, pages = {7786--7795}, year = {2018}, publisher = {Curran Associates, Inc.}, url = {http://papers.nips.cc/paper/8003-towards-robust-interpretability-with-self-explaining-neural-networks.pdf} }
Gromov-Wasserstein Alignment of Word Embedding Spaces
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.
@InProceedings{alvarezmelis2018gromov, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {Gromov-Wasserstein Alignment of Word Embedding Spaces}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, publisher = {Association for Computational Linguistics}, pages = {1881--1890}, location = {Brussels, Belgium}, url = {http://aclweb.org/anthology/D18-1214} }
Structured Optimal Transport
David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka
AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.
Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.
@InProceedings{pmlr-v84-alvarez-melis18a, title = {Structured Optimal Transport}, author = {David Alvarez-Melis and Tommi Jaakkola and Stefanie Jegelka}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1771--1780}, year = {2018}, editor = {Amos Storkey and Fernando Perez-Cruz}, volume = {84}, series = {Proceedings of Machine Learning Research}, address = {Playa Blanca, Lanzarote, Canary Islands}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/alvarez-melis18a/alvarez-melis18a.pdf}, url = {http://proceedings.mlr.press/v84/alvarez-melis18a.html}, abstract = {Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation to sentence similarities or deep learning. Yet, its ability to capture frequently occurring structure beyond the "ground metric" is limited. In this work, we develop a nonlinear generalization of (discrete) optimal transport that is able to reflect much additional structure. We demonstrate how to leverage the geometry of this new model for fast algorithms, and explore connections and properties. Illustrative experiments highlight the benefit of the induced structured couplings for tasks in domain adaptation and natural language processing.} }
A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'17: Empirical Methods in Natural Language Processing. 2017.
@InProceedings{alvarezmelis2017causal, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {A causal framework for explaining the predictions of black-box sequence-to-sequence models}, booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing}, month = {September}, year = {2017}, address = {Copenhagen, Denmark}, publisher = {Association for Computational Linguistics}, pages = {412--421}, url = {https://www.aclweb.org/anthology/D17-1042} }
Tree-structured Decoding with Doubly-recurrent Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
ICLR'17: International Conference on Learning Representations. 2017.
@inproceedings{alvarezmelis2017tree, title={Tree-structured decoding with doubly-recurrent neural networks}, author={Alvarez-Melis, David and Jaakkola, Tommi S}, booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)}, year={2017} }
Word Embeddings as Metric Recovery in Semantic Spaces
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).
@article{Hashimoto2016Word, author = {Hashimoto, Tatsunori and Alvarez-Melis, David and Jaakkola, Tommi }, title = {Word Embeddings as Metric Recovery in Semantic Spaces}, journal = {Transactions of the Association for Computational Linguistics}, volume = {4}, year = {2016}, issn = {2307-387X}, url = {https://transacl.org/ojs/index.php/tacl/article/view/809}, pages = {273--286} }
InfoOT: Information Maximizing Optimal Transport
Ching-Yao Chuang, Stefanie Jegelka, David Alvarez-Melis
Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings
Frederike Lübeck^{*}, Charlotte Bunne^{*}, Gabriele Gut, Jacobo Sarabia del Castillo, Lucas Pelkmans, David Alvarez-Melis
Are GANs overkill for NLP?
David Alvarez-Melis^{*}, Vikas Garg^{*}, Adam Tauman Kalai^{*}
NeurIPS 2022 (forthcoming)
Hierarchical Optimal Transport for Comparing Histopathology Datasets
Anna Yeaton, Rahul G. Krishnan, Rebecca Mieloszyk, David Alvarez-Melis, Grace Huynh
Medical Imaging in Deep Learning (MIDL). 2022.
Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks
David Alvarez-Melis, Yair Schiff, Youssef Mroueh
Transactions of Machine Learning Research (TMLR). 2022.
Earlier version at OTML: NeurIPS'21 Workshop on Optimal Transport in Machine Learning .
Dataset Dynamics via Gradient Flows in Probability Space
David Alvarez-Melis, Nicolò Fusi
ICML'21: International Conference on Machine Learning. 2021.
@InProceedings{alvarez-melis2021dataset, title = {Dataset Dynamics via Gradient Flows in Probability Space}, author = {Alvarez-Melis, David and Fusi, Nicol\`o}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {219--230}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/alvarez-melis21a/alvarez-melis21a.pdf}, url = {https://proceedings.mlr.press/v139/alvarez-melis21a.html}, abstract = {Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.} }
Geometric Dataset Distances via Optimal Transport
David Alvarez-Melis, Nicolò Fusi
NeurIPS'20: Neural Information Processing Systems. 2020.
Earlier version at AutoML @ ICML 2020.
@inproceedings{alvarez-melis2020geometric, author = {Alvarez-Melis, David and Fusi, Nicolo}, booktitle = {Advances in Neural Information Processing Systems}, editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin}, pages = {21428--21439}, publisher = {Curran Associates, Inc.}, title = {Geometric Dataset Distances via Optimal Transport}, url = {https://proceedings.neurips.cc/paper/2020/file/f52a7b2610fb4d3f74b4106fb80b233d-Paper.pdf}, volume = {33}, year = {2020} }
Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces
David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola
AISTATS'20: Artificial Intelligence and Statistics. 2020.
Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.
@InProceedings{alvarez-melis2020unsupervised, title = {Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces}, author = {Alvarez-Melis, David and Mroueh, Youssef and Jaakkola, Tommi}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {1606--1617}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/alvarez-melis20a/alvarez-melis20a.pdf}, url = {http://proceedings.mlr.press/v108/alvarez-melis20a.html}, }
Probabilistic Bias Mitigation in Word Embeddings
Hailey James-Sorenson, David Alvarez-Melis
HCML @ NeurIPS2019
Optimal Transport in Structured Domains: Algorithms and Applications
David Alvarez-Melis (advisor: Tommi S. Jaakkola)
PhD Thesis, MIT. 2019.
Functional Transparency for Structured Data: a Game-Theoretic Approach,
Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola
ICML'19: International Conference on Machine Learning.
@InProceedings{pmlr-v97-lee19b, title = {Functional Transparency for Structured Data: a Game-Theoretic Approach}, author = {Lee, Guang-He and Jin, Wengong and Alvarez-Melis, David and Jaakkola, Tommi}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3723--3733}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/lee19b/lee19b.pdf}, url = {http://proceedings.mlr.press/v97/lee19b.html}, abstract = {We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted \emph{predictor} such as a neural network, and a set of \emph{witnesses} chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.} }
Learning Generative Models across Incomparable Spaces
Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka
ICML'19: International Conference on Machine Learning.
Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.
@InProceedings{pmlr-v97-bunne19a, title = {Learning Generative Models across Incomparable Spaces}, author = {Bunne, Charlotte and Alvarez-Melis, David and Krause, Andreas and Jegelka, Stefanie}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {851--861}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/bunne19a/bunne19a.pdf}, url = {http://proceedings.mlr.press/v97/bunne19a.html}, abstract = {Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety. However, in some cases, we may want to only learn some aspects (e.g., cluster or manifold structure), while modifying others (e.g., style, orientation or dimension). In this work, we propose an approach to learn generative models across such incomparable spaces, and demonstrate how to steer the learned distribution towards target properties. A key component of our model is the Gromov-Wasserstein distance, a notion of discrepancy that compares distributions relationally rather than absolutely. While this framework subsumes current generative models in identically reproducing distributions, its inherent flexibility allows application to tasks in manifold learning, relational learning and cross-domain learning.} }
Towards Robust, Locally Linear Deep Networks
Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola
ICLR'19: International Conference on Learning Representations. 2019.
Towards Optimal Transport with Global Invariances
David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola
AISTATS'19: Artificial Intelligence and Statistics. 2019.
@InProceedings{pmlr-v89-alvarez-melis19a, title = {Towards Optimal Transport with Global Invariances}, author = {Alvarez-Melis, David and Jegelka, Stefanie and Jaakkola, Tommi S.}, booktitle = {Proceedings of Machine Learning Research}, pages = {1870--1879}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, address = {}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/alvarez-melis19a/alvarez-melis19a.pdf}, url = {http://proceedings.mlr.press/v89/alvarez-melis19a.html}, abstract = {Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can be directly evaluated. Unfortunately neither requirement is likely to hold when object representations are learned from data. Indeed, automatically derived representations such as word embeddings are typically fixed only up to some global transformations, for example, reflection or rotation. As a result, pairwise distances across two such instances are ill-defined without specifying their relative transformation. In this work, we propose a general framework for optimal transport in the presence of latent global transformations. We cast the problem as a joint optimization over transport couplings and transformations chosen from a flexible class of invariances, propose algorithms to solve it, and show promising results in various tasks, including a popular unsupervised word translation benchmark.} }
Towards Robust Interpretability with Self-Explaining Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
NeurIPS'18: Neural Information Processing Systems. 2018.
@incollection{NIPS2018_8003, title = {Towards Robust Interpretability with Self-Explaining Neural Networks}, author = {Alvarez Melis, David and Jaakkola, Tommi}, booktitle = {Advances in Neural Information Processing Systems 31}, editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett}, pages = {7786--7795}, year = {2018}, publisher = {Curran Associates, Inc.}, url = {http://papers.nips.cc/paper/8003-towards-robust-interpretability-with-self-explaining-neural-networks.pdf} }
Gromov-Wasserstein Alignment of Word Embedding Spaces
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.
@InProceedings{alvarezmelis2018gromov, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {Gromov-Wasserstein Alignment of Word Embedding Spaces}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, publisher = {Association for Computational Linguistics}, pages = {1881--1890}, location = {Brussels, Belgium}, url = {http://aclweb.org/anthology/D18-1214} }
Game-theoretic Interpretability for Temporal Modeling
Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola
Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).
On the Robustness of Interpretability Methods
David Alvarez-Melis, Tommi S. Jaakkola
Workshop on Human Interpretability in Machine Learning (@ICML 2018).
Structured Optimal Transport
David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka
AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.
Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.
@InProceedings{pmlr-v84-alvarez-melis18a, title = {Structured Optimal Transport}, author = {David Alvarez-Melis and Tommi Jaakkola and Stefanie Jegelka}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1771--1780}, year = {2018}, editor = {Amos Storkey and Fernando Perez-Cruz}, volume = {84}, series = {Proceedings of Machine Learning Research}, address = {Playa Blanca, Lanzarote, Canary Islands}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/alvarez-melis18a/alvarez-melis18a.pdf}, url = {http://proceedings.mlr.press/v84/alvarez-melis18a.html}, abstract = {Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation to sentence similarities or deep learning. Yet, its ability to capture frequently occurring structure beyond the "ground metric" is limited. In this work, we develop a nonlinear generalization of (discrete) optimal transport that is able to reflect much additional structure. We demonstrate how to leverage the geometry of this new model for fast algorithms, and explore connections and properties. Illustrative experiments highlight the benefit of the induced structured couplings for tasks in domain adaptation and natural language processing.} }
The Emotional GAN: Priming Adversarial Generation of Art with Emotion.
David Alvarez-Melis, Judith Amores
NIPS Workshop on Machine Learning for Creativity and Design. 2017.
Distributional Adversarial Networks
Chengtao Li*, David Alvarez-Melis*, Keyulu Xu, Stefanie Jegelka, Suvrit Sra
ICLR'17: International Conference on Learning Representations (Workshop track). 2017.
A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'17: Empirical Methods in Natural Language Processing. 2017.
@InProceedings{alvarezmelis2017causal, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {A causal framework for explaining the predictions of black-box sequence-to-sequence models}, booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing}, month = {September}, year = {2017}, address = {Copenhagen, Denmark}, publisher = {Association for Computational Linguistics}, pages = {412--421}, url = {https://www.aclweb.org/anthology/D17-1042} }
Tree-structured Decoding with Doubly-recurrent Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
ICLR'17: International Conference on Learning Representations. 2017.
@inproceedings{alvarezmelis2017tree, title={Tree-structured decoding with doubly-recurrent neural networks}, author={Alvarez-Melis, David and Jaakkola, Tommi S}, booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)}, year={2017} }
Topic Modeling in Twitter: Aggregating Tweets by Conversations
David Alvarez-Melis*, Martin Saveski*
ICWSM'16: International AAAI Conference on Web and Social Media. 2016. (Short Paper)
@inproceedings{alvarezmelis2016toic, author = {David Alvarez{-}Melis and Martin Saveski}, title = {Topic Modeling in Twitter: Aggregating Tweets by Conversations}, booktitle = {Proceedings of the Tenth International Conference on Web and Social Media (ICWSM)}, pages = {519--522}, year = {2016}, url = {http://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13162}, }
Word, graph and manifold embedding from Markov processes
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.
A translation of 'The characteristic function of a random phenomenon' by Bruno de Finetti
David Alvarez-Melis, Tamara Broderick
Translation. 2015
The Matrix Multiplicative Weights Algorithm for Domain Adaptation
David Alvarez-Melis (advisor: Mehryar Mohri)
MS Thesis, Courant Institute. 2013.
Are GANs overkill for NLP?
David Alvarez-Melis^{*}, Vikas Garg^{*}, Adam Tauman Kalai^{*}
NeurIPS 2022 (forthcoming)
Probabilistic Bias Mitigation in Word Embeddings
Hailey James-Sorenson, David Alvarez-Melis
HCML @ NeurIPS2019
Gromov-Wasserstein Alignment of Word Embedding Spaces
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.
@InProceedings{alvarezmelis2018gromov, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {Gromov-Wasserstein Alignment of Word Embedding Spaces}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, publisher = {Association for Computational Linguistics}, pages = {1881--1890}, location = {Brussels, Belgium}, url = {http://aclweb.org/anthology/D18-1214} }
A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'17: Empirical Methods in Natural Language Processing. 2017.
@InProceedings{alvarezmelis2017causal, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {A causal framework for explaining the predictions of black-box sequence-to-sequence models}, booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing}, month = {September}, year = {2017}, address = {Copenhagen, Denmark}, publisher = {Association for Computational Linguistics}, pages = {412--421}, url = {https://www.aclweb.org/anthology/D17-1042} }
Tree-structured Decoding with Doubly-recurrent Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
ICLR'17: International Conference on Learning Representations. 2017.
@inproceedings{alvarezmelis2017tree, title={Tree-structured decoding with doubly-recurrent neural networks}, author={Alvarez-Melis, David and Jaakkola, Tommi S}, booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)}, year={2017} }
Word Embeddings as Metric Recovery in Semantic Spaces
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).
@article{Hashimoto2016Word, author = {Hashimoto, Tatsunori and Alvarez-Melis, David and Jaakkola, Tommi }, title = {Word Embeddings as Metric Recovery in Semantic Spaces}, journal = {Transactions of the Association for Computational Linguistics}, volume = {4}, year = {2016}, issn = {2307-387X}, url = {https://transacl.org/ojs/index.php/tacl/article/view/809}, pages = {273--286} }
Word, graph and manifold embedding from Markov processes
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.
InfoOT: Information Maximizing Optimal Transport
Ching-Yao Chuang, Stefanie Jegelka, David Alvarez-Melis
Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings
Frederike Lübeck^{*}, Charlotte Bunne^{*}, Gabriele Gut, Jacobo Sarabia del Castillo, Lucas Pelkmans, David Alvarez-Melis
Hierarchical Optimal Transport for Comparing Histopathology Datasets
Anna Yeaton, Rahul G. Krishnan, Rebecca Mieloszyk, David Alvarez-Melis, Grace Huynh
Medical Imaging in Deep Learning (MIDL). 2022.
Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks
David Alvarez-Melis, Yair Schiff, Youssef Mroueh
Transactions of Machine Learning Research (TMLR). 2022.
Earlier version at OTML: NeurIPS'21 Workshop on Optimal Transport in Machine Learning .
Dataset Dynamics via Gradient Flows in Probability Space
David Alvarez-Melis, Nicolò Fusi
ICML'21: International Conference on Machine Learning. 2021.
@InProceedings{alvarez-melis2021dataset, title = {Dataset Dynamics via Gradient Flows in Probability Space}, author = {Alvarez-Melis, David and Fusi, Nicol\`o}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {219--230}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/alvarez-melis21a/alvarez-melis21a.pdf}, url = {https://proceedings.mlr.press/v139/alvarez-melis21a.html}, abstract = {Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.} }
Geometric Dataset Distances via Optimal Transport
David Alvarez-Melis, Nicolò Fusi
NeurIPS'20: Neural Information Processing Systems. 2020.
Earlier version at AutoML @ ICML 2020.
@inproceedings{alvarez-melis2020geometric, author = {Alvarez-Melis, David and Fusi, Nicolo}, booktitle = {Advances in Neural Information Processing Systems}, editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin}, pages = {21428--21439}, publisher = {Curran Associates, Inc.}, title = {Geometric Dataset Distances via Optimal Transport}, url = {https://proceedings.neurips.cc/paper/2020/file/f52a7b2610fb4d3f74b4106fb80b233d-Paper.pdf}, volume = {33}, year = {2020} }
Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces
David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola
AISTATS'20: Artificial Intelligence and Statistics. 2020.
Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.
@InProceedings{alvarez-melis2020unsupervised, title = {Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces}, author = {Alvarez-Melis, David and Mroueh, Youssef and Jaakkola, Tommi}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {1606--1617}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/alvarez-melis20a/alvarez-melis20a.pdf}, url = {http://proceedings.mlr.press/v108/alvarez-melis20a.html}, }
Optimal Transport in Structured Domains: Algorithms and Applications
David Alvarez-Melis (advisor: Tommi S. Jaakkola)
PhD Thesis, MIT. 2019.
Learning Generative Models across Incomparable Spaces
Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka
ICML'19: International Conference on Machine Learning.
Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.
@InProceedings{pmlr-v97-bunne19a, title = {Learning Generative Models across Incomparable Spaces}, author = {Bunne, Charlotte and Alvarez-Melis, David and Krause, Andreas and Jegelka, Stefanie}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {851--861}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/bunne19a/bunne19a.pdf}, url = {http://proceedings.mlr.press/v97/bunne19a.html}, abstract = {Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety. However, in some cases, we may want to only learn some aspects (e.g., cluster or manifold structure), while modifying others (e.g., style, orientation or dimension). In this work, we propose an approach to learn generative models across such incomparable spaces, and demonstrate how to steer the learned distribution towards target properties. A key component of our model is the Gromov-Wasserstein distance, a notion of discrepancy that compares distributions relationally rather than absolutely. While this framework subsumes current generative models in identically reproducing distributions, its inherent flexibility allows application to tasks in manifold learning, relational learning and cross-domain learning.} }
Towards Optimal Transport with Global Invariances
David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola
AISTATS'19: Artificial Intelligence and Statistics. 2019.
@InProceedings{pmlr-v89-alvarez-melis19a, title = {Towards Optimal Transport with Global Invariances}, author = {Alvarez-Melis, David and Jegelka, Stefanie and Jaakkola, Tommi S.}, booktitle = {Proceedings of Machine Learning Research}, pages = {1870--1879}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, address = {}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/alvarez-melis19a/alvarez-melis19a.pdf}, url = {http://proceedings.mlr.press/v89/alvarez-melis19a.html}, abstract = {Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can be directly evaluated. Unfortunately neither requirement is likely to hold when object representations are learned from data. Indeed, automatically derived representations such as word embeddings are typically fixed only up to some global transformations, for example, reflection or rotation. As a result, pairwise distances across two such instances are ill-defined without specifying their relative transformation. In this work, we propose a general framework for optimal transport in the presence of latent global transformations. We cast the problem as a joint optimization over transport couplings and transformations chosen from a flexible class of invariances, propose algorithms to solve it, and show promising results in various tasks, including a popular unsupervised word translation benchmark.} }
Gromov-Wasserstein Alignment of Word Embedding Spaces
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.
@InProceedings{alvarezmelis2018gromov, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {Gromov-Wasserstein Alignment of Word Embedding Spaces}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, publisher = {Association for Computational Linguistics}, pages = {1881--1890}, location = {Brussels, Belgium}, url = {http://aclweb.org/anthology/D18-1214} }
Structured Optimal Transport
David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka
AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.
Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.
@InProceedings{pmlr-v84-alvarez-melis18a, title = {Structured Optimal Transport}, author = {David Alvarez-Melis and Tommi Jaakkola and Stefanie Jegelka}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1771--1780}, year = {2018}, editor = {Amos Storkey and Fernando Perez-Cruz}, volume = {84}, series = {Proceedings of Machine Learning Research}, address = {Playa Blanca, Lanzarote, Canary Islands}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/alvarez-melis18a/alvarez-melis18a.pdf}, url = {http://proceedings.mlr.press/v84/alvarez-melis18a.html}, abstract = {Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation to sentence similarities or deep learning. Yet, its ability to capture frequently occurring structure beyond the "ground metric" is limited. In this work, we develop a nonlinear generalization of (discrete) optimal transport that is able to reflect much additional structure. We demonstrate how to leverage the geometry of this new model for fast algorithms, and explore connections and properties. Illustrative experiments highlight the benefit of the induced structured couplings for tasks in domain adaptation and natural language processing.} }
From Human Explanation to Model Interpretabilty: A Framework Based on Weight of Evidence
David Alvarez-Melis, Harmanpreet Kaur, Hal Daumé III, Hanna Wallach, Jennifer Wortman Vaughan
HCOMP '21: The 9th AAAI Conference on Human Computation and Crowdsourcing. 2021.
Weight of Evidence as a Basis for Human-Oriented Explanations
David Alvarez-Melis, Hal Daumé III, Jennifer Wortman Vaughan, Hanna Wallach
HCML @ NeurIPS2019
Functional Transparency for Structured Data: a Game-Theoretic Approach,
Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola
ICML'19: International Conference on Machine Learning.
@InProceedings{pmlr-v97-lee19b, title = {Functional Transparency for Structured Data: a Game-Theoretic Approach}, author = {Lee, Guang-He and Jin, Wengong and Alvarez-Melis, David and Jaakkola, Tommi}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3723--3733}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/lee19b/lee19b.pdf}, url = {http://proceedings.mlr.press/v97/lee19b.html}, abstract = {We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted \emph{predictor} such as a neural network, and a set of \emph{witnesses} chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.} }
Towards Robust Interpretability with Self-Explaining Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
NeurIPS'18: Neural Information Processing Systems. 2018.
@incollection{NIPS2018_8003, title = {Towards Robust Interpretability with Self-Explaining Neural Networks}, author = {Alvarez Melis, David and Jaakkola, Tommi}, booktitle = {Advances in Neural Information Processing Systems 31}, editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett}, pages = {7786--7795}, year = {2018}, publisher = {Curran Associates, Inc.}, url = {http://papers.nips.cc/paper/8003-towards-robust-interpretability-with-self-explaining-neural-networks.pdf} }
Game-theoretic Interpretability for Temporal Modeling
Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola
Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).
On the Robustness of Interpretability Methods
David Alvarez-Melis, Tommi S. Jaakkola
Workshop on Human Interpretability in Machine Learning (@ICML 2018).
A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'17: Empirical Methods in Natural Language Processing. 2017.
@InProceedings{alvarezmelis2017causal, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {A causal framework for explaining the predictions of black-box sequence-to-sequence models}, booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing}, month = {September}, year = {2017}, address = {Copenhagen, Denmark}, publisher = {Association for Computational Linguistics}, pages = {412--421}, url = {https://www.aclweb.org/anthology/D17-1042} }
Optimal Transport in Structured Domains: Algorithms and Applications
David Alvarez-Melis (advisor: Tommi S. Jaakkola)
PhD Thesis, MIT. 2019.
The Matrix Multiplicative Weights Algorithm for Domain Adaptation
David Alvarez-Melis (advisor: Mehryar Mohri)
MS Thesis, Courant Institute. 2013.
Lax-Milgram's Theorem: Generalizations and Applications
David Alvarez-Melis (advisor: Carlos Bosch Giral)
BSc Thesis, ITAM. 2011.
InfoOT: Information Maximizing Optimal Transport
Ching-Yao Chuang, Stefanie Jegelka, David Alvarez-Melis
Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings
Frederike Lübeck^{*}, Charlotte Bunne^{*}, Gabriele Gut, Jacobo Sarabia del Castillo, Lucas Pelkmans, David Alvarez-Melis
Are GANs overkill for NLP?
David Alvarez-Melis^{*}, Vikas Garg^{*}, Adam Tauman Kalai^{*}
NeurIPS 2022 (forthcoming)
Hierarchical Optimal Transport for Comparing Histopathology Datasets
Anna Yeaton, Rahul G. Krishnan, Rebecca Mieloszyk, David Alvarez-Melis, Grace Huynh
Medical Imaging in Deep Learning (MIDL). 2022.
Optimizing Functionals on the Space of Probabilities with Input Convex Neural Networks
David Alvarez-Melis, Yair Schiff, Youssef Mroueh
Transactions of Machine Learning Research (TMLR). 2022.
Earlier version at OTML: NeurIPS'21 Workshop on Optimal Transport in Machine Learning .
From Human Explanation to Model Interpretabilty: A Framework Based on Weight of Evidence
David Alvarez-Melis, Harmanpreet Kaur, Hal Daumé III, Hanna Wallach, Jennifer Wortman Vaughan
HCOMP '21: The 9th AAAI Conference on Human Computation and Crowdsourcing. 2021.
Dataset Dynamics via Gradient Flows in Probability Space
David Alvarez-Melis, Nicolò Fusi
ICML'21: International Conference on Machine Learning. 2021.
@InProceedings{alvarez-melis2021dataset, title = {Dataset Dynamics via Gradient Flows in Probability Space}, author = {Alvarez-Melis, David and Fusi, Nicol\`o}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {219--230}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/alvarez-melis21a/alvarez-melis21a.pdf}, url = {https://proceedings.mlr.press/v139/alvarez-melis21a.html}, abstract = {Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.} }
Geometric Dataset Distances via Optimal Transport
David Alvarez-Melis, Nicolò Fusi
NeurIPS'20: Neural Information Processing Systems. 2020.
Earlier version at AutoML @ ICML 2020.
@inproceedings{alvarez-melis2020geometric, author = {Alvarez-Melis, David and Fusi, Nicolo}, booktitle = {Advances in Neural Information Processing Systems}, editor = {H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin}, pages = {21428--21439}, publisher = {Curran Associates, Inc.}, title = {Geometric Dataset Distances via Optimal Transport}, url = {https://proceedings.neurips.cc/paper/2020/file/f52a7b2610fb4d3f74b4106fb80b233d-Paper.pdf}, volume = {33}, year = {2020} }
Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic spaces
David Alvarez-Melis, Youssef Mroueh, Tommi S. Jaakkola
AISTATS'20: Artificial Intelligence and Statistics. 2020.
Earlier version at OTML: NeurIPS'18 Workshop on Optimal Transport for Machine Learning . Spotlight.
@InProceedings{alvarez-melis2020unsupervised, title = {Unsupervised Hierarchy Matching with Optimal Transport over Hyperbolic Spaces}, author = {Alvarez-Melis, David and Mroueh, Youssef and Jaakkola, Tommi}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {1606--1617}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/alvarez-melis20a/alvarez-melis20a.pdf}, url = {http://proceedings.mlr.press/v108/alvarez-melis20a.html}, }
Probabilistic Bias Mitigation in Word Embeddings
Hailey James-Sorenson, David Alvarez-Melis
HCML @ NeurIPS2019
Weight of Evidence as a Basis for Human-Oriented Explanations
David Alvarez-Melis, Hal Daumé III, Jennifer Wortman Vaughan, Hanna Wallach
HCML @ NeurIPS2019
Optimal Transport in Structured Domains: Algorithms and Applications
David Alvarez-Melis (advisor: Tommi S. Jaakkola)
PhD Thesis, MIT. 2019.
Functional Transparency for Structured Data: a Game-Theoretic Approach,
Guang-He Lee, Wengong Jin, David Alvarez-Melis, Tommi S. Jaakkola
ICML'19: International Conference on Machine Learning.
@InProceedings{pmlr-v97-lee19b, title = {Functional Transparency for Structured Data: a Game-Theoretic Approach}, author = {Lee, Guang-He and Jin, Wengong and Alvarez-Melis, David and Jaakkola, Tommi}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {3723--3733}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/lee19b/lee19b.pdf}, url = {http://proceedings.mlr.press/v97/lee19b.html}, abstract = {We provide a new approach to training neural models to exhibit transparency in a well-defined, functional manner. Our approach naturally operates over structured data and tailors the predictor, functionally, towards a chosen family of (local) witnesses. The estimation problem is setup as a co-operative game between an unrestricted \emph{predictor} such as a neural network, and a set of \emph{witnesses} chosen from the desired transparent family. The goal of the witnesses is to highlight, locally, how well the predictor conforms to the chosen family of functions, while the predictor is trained to minimize the highlighted discrepancy. We emphasize that the predictor remains globally powerful as it is only encouraged to agree locally with locally adapted witnesses. We analyze the effect of the proposed approach, provide example formulations in the context of deep graph and sequence models, and empirically illustrate the idea in chemical property prediction, temporal modeling, and molecule representation learning.} }
Learning Generative Models across Incomparable Spaces
Charlotte Bunne, David Alvarez-Melis, Andreas Krause, Stefanie Jegelka
ICML'19: International Conference on Machine Learning.
Earlier version at R2L: NeurIPS'18 Workshop on Relational Representation Learning. Best Paper Award.
@InProceedings{pmlr-v97-bunne19a, title = {Learning Generative Models across Incomparable Spaces}, author = {Bunne, Charlotte and Alvarez-Melis, David and Krause, Andreas and Jegelka, Stefanie}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {851--861}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, address = {Long Beach, California, USA}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/bunne19a/bunne19a.pdf}, url = {http://proceedings.mlr.press/v97/bunne19a.html}, abstract = {Generative Adversarial Networks have shown remarkable success in learning a distribution that faithfully recovers a reference distribution in its entirety. However, in some cases, we may want to only learn some aspects (e.g., cluster or manifold structure), while modifying others (e.g., style, orientation or dimension). In this work, we propose an approach to learn generative models across such incomparable spaces, and demonstrate how to steer the learned distribution towards target properties. A key component of our model is the Gromov-Wasserstein distance, a notion of discrepancy that compares distributions relationally rather than absolutely. While this framework subsumes current generative models in identically reproducing distributions, its inherent flexibility allows application to tasks in manifold learning, relational learning and cross-domain learning.} }
Towards Robust, Locally Linear Deep Networks
Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola
ICLR'19: International Conference on Learning Representations. 2019.
Towards Optimal Transport with Global Invariances
David Alvarez-Melis, Stefanie Jegelka, Tommi S. Jaakkola
AISTATS'19: Artificial Intelligence and Statistics. 2019.
@InProceedings{pmlr-v89-alvarez-melis19a, title = {Towards Optimal Transport with Global Invariances}, author = {Alvarez-Melis, David and Jegelka, Stefanie and Jaakkola, Tommi S.}, booktitle = {Proceedings of Machine Learning Research}, pages = {1870--1879}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, address = {}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/alvarez-melis19a/alvarez-melis19a.pdf}, url = {http://proceedings.mlr.press/v89/alvarez-melis19a.html}, abstract = {Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can be directly evaluated. Unfortunately neither requirement is likely to hold when object representations are learned from data. Indeed, automatically derived representations such as word embeddings are typically fixed only up to some global transformations, for example, reflection or rotation. As a result, pairwise distances across two such instances are ill-defined without specifying their relative transformation. In this work, we propose a general framework for optimal transport in the presence of latent global transformations. We cast the problem as a joint optimization over transport couplings and transformations chosen from a flexible class of invariances, propose algorithms to solve it, and show promising results in various tasks, including a popular unsupervised word translation benchmark.} }
Towards Robust Interpretability with Self-Explaining Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
NeurIPS'18: Neural Information Processing Systems. 2018.
@incollection{NIPS2018_8003, title = {Towards Robust Interpretability with Self-Explaining Neural Networks}, author = {Alvarez Melis, David and Jaakkola, Tommi}, booktitle = {Advances in Neural Information Processing Systems 31}, editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett}, pages = {7786--7795}, year = {2018}, publisher = {Curran Associates, Inc.}, url = {http://papers.nips.cc/paper/8003-towards-robust-interpretability-with-self-explaining-neural-networks.pdf} }
Gromov-Wasserstein Alignment of Word Embedding Spaces
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'18: Empirical Methods in Natural Language Processing. 2018. Oral Presentation.
@InProceedings{alvarezmelis2018gromov, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {Gromov-Wasserstein Alignment of Word Embedding Spaces}, booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing}, year = {2018}, publisher = {Association for Computational Linguistics}, pages = {1881--1890}, location = {Brussels, Belgium}, url = {http://aclweb.org/anthology/D18-1214} }
Game-theoretic Interpretability for Temporal Modeling
Guang-He Lee, David Alvarez-Melis, Tommi S. Jaakkola
Fairness, Accountability, and Transparency in Machine Learning (@ICML 2018).
On the Robustness of Interpretability Methods
David Alvarez-Melis, Tommi S. Jaakkola
Workshop on Human Interpretability in Machine Learning (@ICML 2018).
Structured Optimal Transport
David Alvarez-Melis, Tommi S. Jaakkola, Stefanie Jegelka
AISTATS'18: Artificial Intelligence and Statistics. 2018. Oral Presentation.
Earlier version at NIPS Workshop on Optimal Transport for Machine Learning, 2017, as Extended Oral.
@InProceedings{pmlr-v84-alvarez-melis18a, title = {Structured Optimal Transport}, author = {David Alvarez-Melis and Tommi Jaakkola and Stefanie Jegelka}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1771--1780}, year = {2018}, editor = {Amos Storkey and Fernando Perez-Cruz}, volume = {84}, series = {Proceedings of Machine Learning Research}, address = {Playa Blanca, Lanzarote, Canary Islands}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/alvarez-melis18a/alvarez-melis18a.pdf}, url = {http://proceedings.mlr.press/v84/alvarez-melis18a.html}, abstract = {Optimal Transport has recently gained interest in machine learning for applications ranging from domain adaptation to sentence similarities or deep learning. Yet, its ability to capture frequently occurring structure beyond the "ground metric" is limited. In this work, we develop a nonlinear generalization of (discrete) optimal transport that is able to reflect much additional structure. We demonstrate how to leverage the geometry of this new model for fast algorithms, and explore connections and properties. Illustrative experiments highlight the benefit of the induced structured couplings for tasks in domain adaptation and natural language processing.} }
The Emotional GAN: Priming Adversarial Generation of Art with Emotion.
David Alvarez-Melis, Judith Amores
NIPS Workshop on Machine Learning for Creativity and Design. 2017.
Distributional Adversarial Networks
Chengtao Li*, David Alvarez-Melis*, Keyulu Xu, Stefanie Jegelka, Suvrit Sra
ICLR'17: International Conference on Learning Representations (Workshop track). 2017.
A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models
David Alvarez-Melis, Tommi S. Jaakkola
EMNLP'17: Empirical Methods in Natural Language Processing. 2017.
@InProceedings{alvarezmelis2017causal, author = {Alvarez-Melis, David and Jaakkola, Tommi}, title = {A causal framework for explaining the predictions of black-box sequence-to-sequence models}, booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing}, month = {September}, year = {2017}, address = {Copenhagen, Denmark}, publisher = {Association for Computational Linguistics}, pages = {412--421}, url = {https://www.aclweb.org/anthology/D17-1042} }
Tree-structured Decoding with Doubly-recurrent Neural Networks
David Alvarez-Melis, Tommi S. Jaakkola
ICLR'17: International Conference on Learning Representations. 2017.
@inproceedings{alvarezmelis2017tree, title={Tree-structured decoding with doubly-recurrent neural networks}, author={Alvarez-Melis, David and Jaakkola, Tommi S}, booktitle = {Proceedings of the International Conference on Learning Representations (ICLR)}, year={2017} }
Word Embeddings as Metric Recovery in Semantic Spaces
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
TACL: Transactions of the Association for Computational Linguistics. 2016. (presented at ACL'16).
@article{Hashimoto2016Word, author = {Hashimoto, Tatsunori and Alvarez-Melis, David and Jaakkola, Tommi }, title = {Word Embeddings as Metric Recovery in Semantic Spaces}, journal = {Transactions of the Association for Computational Linguistics}, volume = {4}, year = {2016}, issn = {2307-387X}, url = {https://transacl.org/ojs/index.php/tacl/article/view/809}, pages = {273--286} }
Topic Modeling in Twitter: Aggregating Tweets by Conversations
David Alvarez-Melis*, Martin Saveski*
ICWSM'16: International AAAI Conference on Web and Social Media. 2016. (Short Paper)
@inproceedings{alvarezmelis2016toic, author = {David Alvarez{-}Melis and Martin Saveski}, title = {Topic Modeling in Twitter: Aggregating Tweets by Conversations}, booktitle = {Proceedings of the Tenth International Conference on Web and Social Media (ICWSM)}, pages = {519--522}, year = {2016}, url = {http://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13162}, }
Word, graph and manifold embedding from Markov processes
Tatsunori B. Hashimoto, David Alvarez-Melis, Tommi S. Jaakkola
NIPS 2015 Workshop on Nonparametric Methods for Large Scale Representation Learning. Oral presentation.
A translation of 'The characteristic function of a random phenomenon' by Bruno de Finetti
David Alvarez-Melis, Tamara Broderick
Translation. 2015
The Matrix Multiplicative Weights Algorithm for Domain Adaptation
David Alvarez-Melis (advisor: Mehryar Mohri)
MS Thesis, Courant Institute. 2013.
Lax-Milgram's Theorem: Generalizations and Applications
David Alvarez-Melis (advisor: Carlos Bosch Giral)
BSc Thesis, ITAM. 2011.
Working with students and junior researchers is undoubtedly the best part of my job. I've been fortunate to advise a number of phenomenal students in the past, including:
Explaining is understanding.
The following extract is from David Goldstein's book on Feynman:
"Feynman was a truly great teacher. He prided himself on being able to devise ways to explain even the most profound ideas to beginning students. Once, I said to him, "Dick, explain to me, so that I can understand it, why spin one-half particles obey Fermi-Dirac statistics." Sizing up his audience perfectly, Feynman said, "I'll prepare a freshman lecture on it." But he came back a few days later to say, "I couldn't do it. I couldn't reduce it to the freshman level. That means we don't really understand it."
Some current and past courses I have TA'd:
Full CV in PDF (or a shorter Resumé).
Outside of research, I enjoy running, brewing beer and playing guitar. I also like quotes. Here's a few more:
"We cannot solve our problems with the same thinking we used when we created them." - A. Einstein
"The real danger is not that computers will begin to think like men, but that men will begin to think like computers" - Syndey J. Harris