In this work we introduce a new class of gradient-free global optimization methods based on a binary interaction dynamics governed by a Boltzmann type equation. In each interaction the particles act taking into account both the best microscopic binary position and the best macroscopic collective position. For the resulting kinetic optimization methods, convergence to the global minimizer is guaranteed for a large class of functions under appropriate parameter constraints that do not depend on the dimension of the problem. In the mean-field limit we show that the resulting Fokker-Planck partial differential equations generalize the current class of consensus based optimization (CBO) methods. Algorithmic implementations inspired by the well-known direct simulation Monte Carlo methods in kinetic theory are derived and discussed. Several examples on prototype test functions for global optimization are reported including an application to machine learning.

Binary Interaction Methods for High Dimensional Global Optimization and Machine Learning / A. Benfenati, G. Borghi, L. Pareschi. - In: APPLIED MATHEMATICS AND OPTIMIZATION. - ISSN 0095-4616. - (2022). [Epub ahead of print] [10.1007/s00245-022-09836-5]

Binary Interaction Methods for High Dimensional Global Optimization and Machine Learning

A. Benfenati
Primo
;
2022

Abstract

In this work we introduce a new class of gradient-free global optimization methods based on a binary interaction dynamics governed by a Boltzmann type equation. In each interaction the particles act taking into account both the best microscopic binary position and the best macroscopic collective position. For the resulting kinetic optimization methods, convergence to the global minimizer is guaranteed for a large class of functions under appropriate parameter constraints that do not depend on the dimension of the problem. In the mean-field limit we show that the resulting Fokker-Planck partial differential equations generalize the current class of consensus based optimization (CBO) methods. Algorithmic implementations inspired by the well-known direct simulation Monte Carlo methods in kinetic theory are derived and discussed. Several examples on prototype test functions for global optimization are reported including an application to machine learning.
Gradient-free methods; Global optimization; Boltzmann equation; Mean-field limit; Consensus-based optimization; Machine learning
Settore MAT/08 - Analisi Numerica
7-giu-2022
Article (author)
File in questo prodotto:
File Dimensione Formato  
s00245-022-09836-5.pdf

accesso aperto

Tipologia: Publisher's version/PDF
Dimensione 1.01 MB
Formato Adobe PDF
1.01 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

Caricamento pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/2434/930556
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact