What Is OLS Regression: Formula, Assumptions, and How It Works

Key takeaways

• Ordinary least squares regression, or OLS, estimates a linear relationship by finding a line that minimizes the sum of the squared residuals.

• The difference between linear regression and OLS is that, while linear regression is the theoretical framework used to describe linear relationships between your variables, OLS is a specific method you can use to find the exact line of best fit.

• The assumptions of OLS include linearity, independence, homoscedasticity, no perfect overlap between predictors, and normal errors in order to make more accurate predictions based on your data.

The OLS regression model formula takes the form y = β₀ + β₁x, which represents the slope (β₁) and the y-intercept (β₀). Learn more about what OLS regression is, how it works, and its assumptions, limitations, and alternatives. Then, if you want to start building your statistical skills, consider pursuing the Google Data Analytics Professional Certificate. In as little as six months, you can learn how to clean and organize data for analysis and complete analyses and calculations using spreadsheets, structured query language (SQL), and R programming.

What is OLS regression?

OLS regression is a statistical method you can use to estimate the coefficients in a linear regression model by finding the “best-fitting line.” In plainer terms, imagine you have a scatterplot with advertising money spent on the x (horizontal) axis and number of sales on the y (vertical) axis. The dots trend upward, with more advertising money appearing to correlate with more sales. You want to draw one straight line that represents the trend so you can more precisely predict how many sales will happen based on different budgets. OLS helps you draw this line.

For any line you draw in the scatterplot, each data point will sit either above or below it in most cases. The distance between the line and the data point is called the residual error. OLS measures how good a line is by squaring all of those residual values and adding them up. Squaring the values turns every error positive, regardless of whether the data point is above or below the line, making it easier to compare different fits. The line with the smallest total is thought to best fit the data [1].

OLS regression model formula

The base formula for a simple linear regression line, including when you’re using OLS, is:

y = β₀ + β₁x, sometimes written as y = b + mx

In this equation, the intercept is represented by β₀ or b), and the slope of the line is represented by β₁ or m. Going back to the previous example, imagine you are again predicting sales based on advertising spend. In this case, your “y” value is the number of sales, and the “x” value is the advertising spend. The formula predicts the sales number based on the intercept (β₀ or b) and the slope of the line (β₁ or m).

How it works

Taking the above formula, imagine your intercept is 20, and your slope is five. So your equation would be:

y = 20 + 5x

When you have zero advertising spend, you can expect 20 sales, because 20 + 5(0) = 20. This is what the intercept represents. If you had $5 of advertising spend, you could expect 45 sales, because 20 + 5(5) = 45.

Using the equation, you can predict sales volume based on your advertising spend, allowing you to generally understand how your variables are likely to move together [2].

Read more: Is Data Analytics Hard? Tips for Rising to the Challenge

What is the difference between linear regression and OLS?

Linear regression is the statistical model or framework that defines a linear relationship between your variables, while OLS is an exact estimation method you can use to find the best-fitting line for this linear model [3].

What are the assumptions of OLS?

In order for OLS to give a more accurate estimate, your data needs to meet certain criteria. Some of the foundational assumptions of this model include [4]:

• Linearity: The relationship between your variables actually is a straight line.

• Independence: The data points do not depend on one another, meaning the value of one will not influence the value of others.

Equal spread of errors (homoscedasticity): The data points scatter around the line relatively equally everywhere, rather than growing wider or narrower as you move along the line.

• Normal errors: The errors follow a typical bell-shaped pattern.

While these are all important assumptions for OLS models, they carry weight for different things. To get an unbiased line, the first four are the most important. However, normal errors (or a large sample) give you more confidence that the significance tests are reliable, meaning the p-value (probability of obtaining observed results if the null hypothesis is true) and confidence intervals are more likely to accurately represent your data.

Interpreting OLS output

When you use statistical software to output regression results, you’re likely to get a range of values. To simplify the process, focus on a few key metrics to get a general sense of what your results mean.

The first numbers to pay attention to are the intercept and coefficient. This is your β₀ and β₁, as described in earlier sections, and represents your predicted y-value when x is zero, and how much your y-value moves for every unit increase in x.

Following this, consider your R² and adjusted R². Your R² value tells you the percentage of the ups and downs in your outcome that the model explains on a scale from zero to one. So an R² of 0.7 would tell you that your model explains about 70 percent of the variation in your outcome. An adjusted R² does the same thing but adjusts for predictors, which can help you compare models more accurately.

Diagnostics and model quality

Fitting a line won't be effective if it's not actually a good match for your data in the first place. To check whether it's a good fit, you can run diagnostic tests to assess model quality.

One of the easiest checks is to create a residual plot, which shows the leftover errors after fitting. If these errors form a curve, then it's a good indication that the relationship between your variables is probably non-linear. If they fan out into a wider and wider spread, then the equal-spread rule is likely not being met.

You can also look for outliers and points called “influential points.” These are data points that are abnormal to the best of your data, like outliers that sit far from the line or points that skew model estimates. Influential observations can tug the line toward them, skewing it away from the rest of your data. Spotting these early can help prevent them from distorting your result [5].

What tools do you use to run OLS regression?

OLS regression is typically run in Python (scikit-learn), R (lm() function), or Excel/Google Sheets (LINEST function).

Typically, you won't do the OLS math by hand. Tools like Python and R provide built-in ways to calculate your OLS output and run diagnostic checks. In Python, you can use packages like scikit-learn, while in R, you can use the built-in lm() function. Even in Excel or Google Sheets, you can use the LINEST function to perform these regressions.

Limitations and alternatives

OLS may not always be the right choice for your data, and it’s important to understand when it’s appropriate to use and when other models might be a better fit. In general, OLS runs into trouble when the assumptions break, such as when the error spread is uneven, when data points correlate, or when the real relationship is curved and not linear.

When these assumptions break, other models like weighted least squares (can handle uneven error spread), generalized least squares (can handle linked points), and ridge regression (can handle overlapping predictors) can work more effectively.

Build your data skills with our resources

Get insights into in-demand skills and career trends by subscribing to our LinkedIn newsletter, Career Chat. Build or refresh your data analytics or computing skills with our other free resources:

• Structure your learning: The Ultimate Python Learning Roadmap: From Beginner to Expert

• Hear from an expert: Reimagining Work and Learning with AI: Expert Insights from Dr. Jules White

• Practice with a tutorial: What Is Google Analytics Cohort Analysis?

Whether you want to develop a new skill, get comfortable with an in-demand technology, or advance your abilities, keep growing with a Coursera Plus subscription. You’ll get access to over 10,000 flexible courses.