The neuron

A neuron is a weighted sum of inputs (plus bias), passed through a nonlinear activation.

Think of a neuron as a tiny decision unit: it listens to several input numbers, mixes them with learned importance weights, adds a bias, then squashes the result with an activation $f$.

Inputs and bias feed a weighted sum, then a nonlinearity f produces y.

$$y=f\Big(w_0+\sum_{k=1}^{K} w_k x_k\Big)=f(w_0+w^\top x)$$

Weights $w_k$ and bias $w_0$ are learned parameters. The sign of a weight hints whether that input pushes the output up or down; magnitude hints strength — though the nonlinearity warps a simple “one unit in ⇒ one unit out” story.

Two inputs, tanh

$w_0=0$, $w_1=w_2=1$, $x=(0,2)$, $f=\\tanh$:

$y=\\tanh(0+1\\cdot 0+1\\cdot 2)=\\tanh(2)\\approx 0.96$.

If you flip $w_2$ to $-1$, the same input yields $\\tanh(-2)\\approx -0.96$ — the surface tilts the other way.

Exam tip

Without $f$, stacking neurons collapses to one linear map. The activation is what makes deep nets interesting.

Key takeaways

Linear part + activation.
Weights are learned.
Bias shifts the decision surface.