I am a Professor at IST Austria. My research focuses on high-performance algorithms for machine learning, and spans from purely theoretical results to practical implementations.
Before IST, I was a researcher at ETH Zurich and Microsoft Research, and a Postdoctoral Associate at MIT CSAIL. I received my PhD from the EPFL, under the guidance of Prof. Rachid Guerraoui. 

In Fall 2023, I was a visiting professor at MIT.

My research is supported by an ERC Starting Grant, the Austrian FWF, and generous support from Amazon and Google.

I am also a Principal Machine Learning Researcher at Neural Magic.

You can find a list of recent projects and publications here.

Recent/future service: 

• Editorship: Distributed Computing Journal and Journal of Machine Learning Research
• DISC 2024 PC Chair
• ICML 2024 Local Chair
• Selected PCs: PODC 2024, NeurIPS 2023 (Area Chair), PODC 2022, PPoPP 2022, NeurIPS 2022, MLSYS 2020-2024, DCC 2021 (Program Chair), ICDCS 2021 (Track Chair), ALGOCLOUD 2017 (PC chair)

News:

• We have four papers accepted to ICML 2024: Sparsity-guided Debugging for DNNs, Extreme LLM Compression, Error Feedback for Compressing Preconditioners, and Robust Adaptation
• Our lab presented two papers at ICLR 2024: Scaling laws for sparse models (Spotlight), and SpQR, a sparse-quantized format for accurate compressed LLM inference.
• We presented three papers at NeurIPS 2023: a state of the art unstructured pruning algorithm (CAP), a Variance-Reduction interpretation of Knowledge Distillation, and a state-of-the-art algorithm for structured pruning (ZipLM)
• Our lab had three papers accepted to ICML 2023 (SparseGPT, SparseProp and QSDP), of which the former two for oral presentation!
• Our work on analyzing the dependency depth of Union-Find was accepted to SPAA23. Congratulations to Alex Fedorov, who led the work!
• Our long-term project on fast, scalable and expressive channels in Kotlin led to papers in PLDI23 and PPoPP23, whereas Nikita Koval’s Lincheck tool was accepted to CAV23.
• Jen and Alex’s work on characterizing bias in pruned models has been accepted to CVPR 2023. 
• Our work on one-shot quantization of GPT-scale models (GPTQ) and compression-aware optimization (CrAM) will appear in the proceedings of ICLR 2023.
• Elias’s work on one-shot compression of deep neural networks appeared in NeurIPS 2022. 
• Our work on compressing large language models using second-order information will appear in EMNLP 2022. 
• Our paper on neural network compression with speedup guarantees (SPDY) appeared at ICML 2022, while our paper on the transferrability of sparse features appeared at CVPR 2022.  
• Our former intern Sasha Voitovych presented our work on leader election in graphical population protocols at PODC 2022. 
• Our group has 5 research papers accepted at NeurIPS 2021, on efficient approximations of second-order information, sparse DNN training with guarantees, decentralized training of DNNs, as well as upper and lower bounds for fundamental problems in distributed optimization. 
• Our paper on Lower Bounds for Leader Election under Bounded Contention won the best paper award at DISC 2021. 
• Together with Torsten Hoefler, I gave a tutorial at ICML 2021 on Sparsity in Deep Learning. The recordings are available here, and the JMLR survey on which the tutorial is based is available here. 
• Our paper on communication-compression for distributed second-order optimization was accepted to ICML 2021.
• Our work on analyzing competitive dynamics in population protocols was accepted to PODC 2021, while our work proposing a first concurrent algorithm for dynamic connectivity was accepted to SPAA 2021.
• We have new work appearing in ICLR 2021 on an optimal compression scheme for distributed variance reduction, and on a state-of-the-art Byzantine-resilient defence for distributed SGD.
• Two new papers accepted to AAAI 2021, on a new consistency condition for distributed SGD, and on the first convergence guarantees for asynchronous SGD over non-smooth, non-convex objectives.
• Three new papers presented at NeurIPS 2020, on model compression, scheduling for concurrent belief propagation, and adaptive gradient quantization for SGD.
• Our work describing the Splay-List (a distributionally-adaptive concurrent Skip-List) was  presented at DISC 2020, and was invited to the Special Issue of Distributed Computing dedicated to the conference.
• Our work on Concurrent Search Trees with Doubly-Logarithmic Running Time won the Best Paper Award at PPoPP 2020. Our work on handling load imbalance in deep learning workloads was also presented at the conference.
• Our work on searching for the fastest concurrent Union-Find algorithm was chosen as Best Paper at OPODIS 2019.
• I gave a Keynote Talk at DISC 2019 on Distributed Machine Learning. The slides are here.

I am extremely lucky to be able to work with the following students and postdocs:

• Joel Rybicki (PostDoc @ IST)
• Janne Korhonen (PostDoc @ IST)
• Giorgi Nadiradze (PhD Student @ IST)
• Alexandra Peste (PhD @ IST, co-advised with Christoph Lampert)
• Aleksandr Shevchenko (PhD @ IST, co-advised with Marco Mondelli)
• Ilya Markov (PhD Student @ IST)
• Jen Iofinova (PhD Student @ IST)
• Elias Frantar (PhD Student @ IST)
• Nikita Koval (PhD Student @ ITMO, Researcher at JetBrains)

Visitors / Collaborators / Friends of the Lab: 

• Prof. Faith Ellen (visiting February-May 2020)
• Prof. Nir Shavit (visiting November 2019)
• Prof. Thomas Sauerwald (visiting Oct 2019)
• Prof. Gregory Valiant (visiting Nov 2018)
• Prof. Robert Tarjan (visiting May 2018)
• Dr. Frank McSherry (visiting May 2018)
• Dr. Aleksandar Prokopec (visiting April 2018)
• Prof. Ce Zhang (ETH)
• Prof. Markus Pueschel (ETH)
• Prof. Torsten Hoefler (ETH)

Alumni:

• Kimia Noorbakhsh (Intern -> PhD Student @ MIT) 
• Kaveh Alimohammadi (Intern -> PhD Student @ MIT) 
• Shayan Talaei (Intern -> PhD Student @ Stanford) 
• Peter Davies (PostDoc @ IST -> Lecturer at University of Surrey)
• Bapi Chatterjee (PostDoc @ IST -> Faculty at IIIT Delhi)
• Vitaly Aksenov (PostDoc @ IST -> Lecturer @ City University of London)
• Trevor Brown (PostDoc @ IST -> Assistant Professor @ Waterloo)
• Amirmojtaba Sabour (Intern @ IST -> PhD Student @ U of Toronto)
• Vijaykrishna Gurunanthan (Intern @ IST -> PhD Student @Stanford)
• Saleh Ashkboos (Intern @ IST -> PhD @ ETH Zurich)
• Antonio Polino (MSc @ ETH -> Google Search)
• Rati Gelashvili (PhD@MIT, now Research Lead at Aptos)
• Justin Kopinsky (PhD@MIT, now Researcher at Jane Street)
• Cédric Renggli (MSc Thesis (ETH Medal), with Torsten Hoefler, now PhD@ETH)
• Nandini Singhal (Intern @ IST -> Microsoft Research)
• Demjan Grubić (MSc, now at Google)
• Syed Kamran Haider (Intern @ MSR, now Researcher @ NVIDIA)
• Hyunjik Kim (Intern @ MSR -> Researcher at DeepMind)