Cross entropy is a concept used to quantify the difference between two probability distributions, typically the predicted probabilities and the actual distribution. In the context of classification tasks, particularly in neural networks, cross entropy loss measures how well the predicted distribution captures the actual target outputs. It's crucial for tasks like image recognition or text classification where multiple outcomes are possible, and it assists in updating model parameters through gradient descent.