Introduction Solving a nonlinear least squares problem consists of minimizing a least squares objective function made up of residuals \(g_1(\boldsymbol{\theta}), \ldots, g_n(\boldsymbol{\theta})\) that are nonlinear functions of the parameters of interest \(\boldsymbol{\theta} = (\theta_1,\ldots, \theta_p)'\):

Introduction Automatic differentiation Automatic differentiation (AD) refers to the automatic/algorithmic calculation of derivatives of a function defined as a computer program by repeated application of the chain rule. Automatic differentiation plays an important role in many statistical computing problems, such as gradient-based optimization of large-scale models, where gradient calculation by means of numeric differentiation (i.

Introduction The new gslnls-package provides R bindings to nonlinear least-squares optimization with the GNU Scientific Library (GSL) using the trust region methods implemented by the gsl_multifit_nlinear module. The gsl_multifit_nlinear module was added in GSL version 2.

Introduction Nonlinear regression model As a model setup, we consider noisy observations \(y_1,\ldots, y_n \in \mathbb{R}\) obtained from a standard nonlinear regression model of the form:
\[ \begin{aligned} y_i &\ = \ f(\boldsymbol{x}_i, \boldsymbol{\theta}) + \epsilon_i, \quad i = 1,\ldots, n \end{aligned} \] where \(f: \mathbb{R}^k \times \mathbb{R}^p \to \mathbb{R}\) is a known nonlinear function of the independent variables \(\boldsymbol{x}_1,\ldots,\boldsymbol{x}_n \in \mathbb{R}^k\) and the unknown parameter vector \(\boldsymbol{\theta} \in \mathbb{R}^p\) that we aim to estimate.