We consider decentralized stochastic optimization with the objective function (eg data samples
for machine learning tasks) being distributed over n machines that can only communicate …

Decentralized stochastic optimization methods have gained a lot of attention recently,
mainly because of their cheap per iteration cost, data locality, and their communication-efficiency. …

Decentralized training of deep learning models is a key element for enabling data privacy and
on-device learning over networks, as well as for efficient scaling to large compute clusters. …

Gradient clipping is a popular modification to standard (stochastic) gradient descent, at every
iteration limiting the gradient norm to a certain value $ c> 0$. It is widely used for example …

Decentralized training of deep learning models enables on-device learning over networks,
as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, …

Decentralized optimization methods enable on-device training of machine learning models
without a central coordinator. In many scenarios communication between devices is energy …

Decentralized and asynchronous communications are two popular techniques to speedup
communication complexity of distributed machine learning, by respectively removing the …

… Anastasia Koloskova Anastasia Koloskova is a postdoctoral researcher at EPFL. She was
a PhD student at EPFL in the Laboratory of Optimization and Machine Learning with Prof. …

Decentralized learning provides an effective framework to train machine learning models with
data distributed over arbitrary communication graphs. However, most existing approaches …

Coordinate descent with random coordinate selection is the current state of the art for many
large scale optimization problems. However, greedy selection of the steepest coordinate on …