A rectified linear unit (ReLU) is an activation function that outputs the input directly if the input is positive, and otherwise outputs zero.

• deep learning
• neural networks
• reference

Back-propagation is an algorithm used during the training process of a feedforward neural network.

• deep learning
• neural networks
• reference

A feedforward neural network (aka multilayer perceptron or deep feedforward network) is a supervised learning network in which information only flows forward.

• deep learning
• neural networks
• reference

Loss functions are used to quantify to what extent a prediction was right/wrong (rather than simply if it was right or wrong.) The purpose of a loss function is to work as part of the optimization process to update a neural network so that it reaches the desired result.

• deep learning
• neural networks
• reference

An activation function is used to determine whether an input in a feedforward neural network is moved from one layer to the next.

• deep learning
• neural networks
• reference