# Single neuron

The neuron has several input channels and only one output channel. The neuron calculates a weighted sum of the input signals, and then converts the resulting sum using a given non-linear function. A set consisting of neurons threshold levels and all weights is called neurons parameters. Here the following notation is used: , , …, is the input pattern, , , …, are weights of the neuron  and b – is the threshold of the neuron.

The activation function of the neuron is a function that calculates the output signal of the neuron. The input of this function is the sum of all the products of the signals and weights of these signals.

First, the neuron calculates a weighted sum

(1) then using the activation function it calculates the output signal .

Consider the most frequently used activation functions.

1. Threshold function. This is a simple piecewise linear function. If the input value is less than the threshold value, then the value of the activation function is equal to the minimum allowable value, otherwise, the maximum allowable value.
2. Linear threshold. This is a simple piecewise linear function. It has two linear sections, where the activation function is identically equal to the minimum allowable and maximum allowable value, and there is a section where the function is strictly monotonously increasing.
3. Sigmoid function or sigmoid (sigmoid). This is a monotonically increasing differentiable S-shaped nonlinear function. Sigmoid allows you to amplify weak signals and not be saturated with strong signals.
4. Hyperbolic tangent (hyperbolic tangent, tanh). This function takes an arbitrary real number at the input, and at the output it gives a real number in the range from –1 to 1. Like a sigmoid, the hyperbolic tangent can be saturated. However, unlike sigmoids, the output of this function is centered around zero. 