Stochastic optimisation algorithms are the de facto standard for machine learning with large amounts of data. Handling only a subset of available data in each optimisation step dramatically reduces the per-iteration computational costs, while still ensuring significant progress towards the solution. Driven by the need to solve large-scale optimisation problems as efficiently as possible, the last decade has witnessed an explosion of research in this area. Leveraging the parallels between machine learning and inverse problems has allowed harnessing the power of this research wave for solving inverse problems. In this survey, we provide a comprehensive account of the state-of-the-art in stochastic optimisation from the viewpoint of inverse problems. We present algorithms with diverse modalities of problem randomisation and discuss the roles of variance reduction, acceleration, higher-order methods, and other algorithmic modifications, and compare theoretical results with practical behaviour. We focus on the potential and the challenges for stochastic optimisation that are unique to inverse imaging problems and are not commonly encountered in machine learning. We conclude the survey with illustrative examples from imaging problems to examine the advantages and disadvantages that this new generation of algorithms bring to the field of inverse problems.

Federated learning (FL) is a distributed learning paradigm that allows several clients to learn a global model without sharing their private data. In this paper, we generalize a primal dual fixed point (PDFP) \citePDFP method to federated learning setting and propose an algorithm called Federated PDFP (FPDFP) for solving composite optimization problems. In addition, a quantization scheme is applied to reduce the communication overhead during the learning process. An O(1/k) convergence rate (where is the communication round) of the proposed FPDFP is provided. Numerical experiments, including graph-guided logistic regression, 3D Computed Tomography (CT) reconstruction are considered to evaluate the proposed algorithm.

The classical synthetic aperture radar (SAR) imaging techniques based on matched filters are limited by data bandwidth, resulting in limited imaging performance with side lobes and speckles present. To address the high-resolution SAR imaging problem, sparse reconstruction has been extensively investigated. However, the state-of-the-art sparse recovery methods seldom consider the complex-valued reflectivity of the scene and only recover an approximated real-valued scene instead. Furthermore, iterative schemes associated with the sparse recovery methods demand a high computational cost, which limits the practical applications of these methods. In this paper, we establish a sphere-constrained magnitude-sparsity SAR imaging model, aiming at enhancing the SAR imaging quality with high efficiency. We propose a non-convex non-smooth optimization method, which can be accelerated by stochastic average gradient acceleration to be scalable with large-scale problems. Numerical experiments are conducted with point-target and extended-target simulations. On the one hand, the point-target simulation showcases the superiority of our proposed method over the classical methods in terms of resolution. On the other hand, the extended-target simulation with random phases is considered to be in line with the practical scenario, and the results demonstrate that our method outperforms the classical SAR imaging methods and sparse recovery without phase prior in terms of PSNR. Meanwhile, owing to the stochastic acceleration, our method is faster than the existing sparse recovery methods by orders of magnitude.