Research

Research interests: mathematical optimization, data science, numerical analysis, computing, machine learning, control

• Derivative-free optimization

• AI for Optimization

• Approximation theory and neural network approximation

• Model-based trust-region methods

• Subspace methods for large-scale optimization

• Parallel computing and optimization

• Numerical methods for multiple solutions of PDE

RESEARCH SPOTLIGHTS

Solving large-scale (derivative-free) problems with a model-based subspace method*

We propose the method 2D-MoSub (2-dimensional model-based subspace method), which is a novel subspace method for general unconstrained optimization and especially aims to solve large-scale (DFO) problems. 2D-MoSub combines 2-dimensional quadratic interpolation models and trust-region techniques to iteratively update points and explore 2-dimensional subspace. 2D-MoSub's framework includes initialization, constructing the interpolation set, building the quadratic interpolation model, performing trust-region trial steps, and updating the trust-region radius and subspace. Experimental results demonstrate the effectiveness and efficiency of 2D-MoSub in solving a variety of optimization problems.

Designing better approximation models for optimization methods*

(1) One particular class of derivative-free optimization algorithms is trust-region algorithms, based on the quadratic models given by under-determined interpolation. Different techniques in updating the quadratic model from iteration to iteration give different interpolation models. We propose a new way to update the quadratic model by minimizing the H^2 norm of the difference between neighboring quadratic models. We propose the projection in the sense of H^2 norm and the interpolation error analysis of our model function. We obtain the coefficients of the quadratic model function by using the KKT conditions. Numerical results show the advantages of our model.

(2) In addition to (1), different weight coefficients refer to different models, which are important for the algorithm. We propose the barycenter of the weight coefficient region of the least weighted H^2 norm updating quadratic models with vanishing trust-region radius with theoretical analysis and numerical supports.

Seeking the most important property of a model for optimization*

One important class of DFO algorithms is trust-region algorithms based on quadratic models given by under-determined interpolation. We propose a new derivative-free trust-region method by introducing an improved under-determined quadratic interpolation model. Our work gives a theoretical motivation, computational details, the quadratic model's implementation-friendly formula, and related numerical results.

How is the model affected by the transformed function values?*

The least Frobenius norm updating quadratic model is an essential under-determined model for derivative-free trust-region methods. We propose DFO with transformed objectives (DFOTO) and give a model-based method with the least Frobenius norm model. We prove the existence and a necessary and sufficient condition of model optimality-preserving transformations and analyze the model, interpolation error, and convergence property. Numerical results support our model and method.

Improving line-search methods by quadratic models*

The speeding-up and slowing-down (SUSD) direction is a novel direction, which is proved to converge to the gradient descent direction. We propose the algorithm SUSD-TR, which combines the SUSD direction based on the covariance matrix of interpolation points and the solution of the trust-region subproblem of the quadratic interpolation model at the current iteration step. We give the optimization dynamics and convergence of the algorithm SUSD-TR. Numerical results show SUSD-TR's efficiency.

Parametric resonant control of macroscopic behaviors of multiple oscillators*

Consider a finite collection of oscillators, which a user has limited means to perturb due to physical restrictions. We show that as long as the stiffness parameters of these oscillators can be harmonically perturbed, one can design a single shared perturbation, such that macroscopic trajectory tracking is achieved independently in each oscillator; that is, the oscillation amplitudes of all oscillators will approximate, respectively, an arbitrary collection of target functions. This control mechanism is based on the dynamical phenomenon of parametric resonance, which not only permits both increase and decrease of the oscillation amplitude by design, but also the simultaneous control of multiple oscillators with distinct intrinsic frequencies. A simulated animation of a remotely powered-and-controlled array of circuits illustrates the efficacy of this control. Oscillators that can be controlled by this mechanism are not limited to harmonic ones, but those subject to additional weak damping, noise, and nonlinearity.