The term regression originated with a 19th century English guy. His name was Galton and he loved to measure stuff, for example he measured the height of people who had tall parents and found that their average height was less than the parents' average height. He called this regression to the mean. The name 'regression' stuck.
In machine learning you'll come across two common algorithms that include the term regression in their title.
Linear Regression Example
Linear regression is all about predicting numerical values, for example the number of customers in a restaurant on a given day, the price of some commodity or in the example below, the maximum temperature for a given minimum temperature. Using a dataset of weather observations recorded during the second world war we can use some linear regression to build a predictive model. The dataset contains min and max temperatures for each day, we can plot this:
There is some scatter but the plot is quite linear. So this seems to be a good case for using linear regression. Linear regression has the following relationship between the input x and the output y:
y = mx + b, m is the gradient of the line and b is the intercept
Linear regression is all about predicting a numerical value. Logistic regression however is about predicting which class something belongs to. In the example below I use a list of Titanic passengers to classify which passengers survived and which died. The code uses two thirds of the rows as training data then attempts to predict the Survived column value for the remaining one third.
Linear regression is used to predict numerical values, it can be extended to include non-linear regression for example see here. While logistic regression is used in classification problems, real world examples could include classifying customers into categories, classifying network activity into benign or suspicious activity ...