Main Content

srchhyb

1-D minimization using a hybrid bisection-cubic search

Syntax

[a,gX,perf,retcode,delta,tol] = srchhyb(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf)

Description

srchhyb is a linear search routine. It searches in a given direction to locate the minimum of the performance function in that direction. It uses a technique that is a combination of a bisection and a cubic interpolation.

[a,gX,perf,retcode,delta,tol] = srchhyb(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf) takes these inputs,

net

Neural network

X

Vector containing current values of weights and biases

Pd

Delayed input vectors

Tl

Layer target vectors

Ai

Initial input delay conditions

Q

Batch size

TS

Time steps

dX

Search direction vector

gX

Gradient vector

perf

Performance value at current X

dperf

Slope of performance value at current X in direction of dX

delta

Initial step size

tol

Tolerance on search

ch_perf

Change in performance on previous step

and returns

a

Step size that minimizes performance

gX

Gradient at new minimum point

perf

Performance value at new minimum point

retcode

Return code that has three elements. The first two elements correspond to the number of function evaluations in the two stages of the search. The third element is a return code. These have different meanings for different search algorithms. Some might not be used in this function.

 0  Normal
 1  Minimum step taken
 2  Maximum step taken
 3  Beta condition not met
delta

New initial step size, based on the current step size

tol

New tolerance on search

Parameters used for the hybrid bisection-cubic algorithm are

alpha

Scale factor that determines sufficient reduction in perf

beta

Scale factor that determines sufficiently large step size

bmax

Largest step size

scale_tol

Parameter that relates the tolerance tol to the initial step size delta, usually set to 20

The defaults for these parameters are set in the training function that calls them. See traincgf, traincgb, traincgp, trainbfg, and trainoss.

Dimensions for these variables are

Pd

No-by-Ni-by-TS cell array

Each element P{i,j,ts} is a Dij-by-Q matrix.

Tl

Nl-by-TS cell array

Each element P{i,ts} is a Vi-by-Q matrix.

Ai

Nl-by-LD cell array

Each element Ai{i,k} is an Si-by-Q matrix.

where

Ni =net.numInputs
Nl =net.numLayers
LD =net.numLayerDelays
Ri =net.inputs{i}.size
Si=net.layers{i}.size
Vi=net.targets{i}.size
Dij=Ri * length(net.inputWeights{i,j}.delays)

More About

collapse all

Hybrid Bisection Cubic Search

Like Brent’s search, srchhyb is a hybrid algorithm. It is a combination of bisection and cubic interpolation. For the bisection algorithm, one point is located in the interval of uncertainty, and the performance and its derivative are computed. Based on this information, half of the interval of uncertainty is discarded. In the hybrid algorithm, a cubic interpolation of the function is obtained by using the value of the performance and its derivative at the two endpoints. If the minimum of the cubic interpolation falls within the known interval of uncertainty, then it is used to reduce the interval of uncertainty. Otherwise, a step of the bisection algorithm is used.

See [Scal85] for a complete description of the hybrid bisection-cubic search. This algorithm does require derivative information, so it performs more computations at each step of the algorithm than the golden section search or Brent’s algorithm.

Algorithms

srchhyb locates the minimum of the performance function in the search direction dX, using the hybrid bisection-cubic interpolation algorithm described on page 50 of Scales (see reference below).

References

Scales, L.E., Introduction to Non-Linear Optimization, New York Springer-Verlag, 1985

Version History

Introduced before R2006a