Sigmoid function - Definition and Overview

The terms sigmoid function or sigmoid curve often refer to a special case of the logistic function. More generally, they may be applied to any real function of a real variable with a sigmoid ("S"-shaped) graph: i.e. a differentiable function whose first derivative is non-negative and has a single local maximum at a finite argument. Besides the standard function above, examples of sigmoid functions (in this general sense) are the ordinary arc-tangent, the hyperbolic tangent, and the error function.

Sigmoid functions in neural nets

Sigmoid functions are often used in neural networks to introduce nonlinearity in the model and/or to make sure that certain signals remain within a specified range. A popular neural net element computes a linear combination of its input signals, and applies a bounded sigmoid function to the result; this model can be seen as a "smoothed" variant of the classical threshold neuron.