Forecasting with Linear Regression and KNN Regression in Python

In the realm of machine learning, making accurate predictions based on historical data is a common and crucial task. Two popular methods for tackling such problems are Linear Regression and K-Nearest Neighbors (KNN) Regression. Both come with their unique strengths and applications. In this post, we'll dive into how to implement these models for forecasting using Python's scikit-learn library, focusing on a simple example without the complexity of graphs to keep our focus on the predictions.

Setting Up

Before we start, make sure you have the scikit-learn library installed in your environment:

pip install scikit-learn

The Dataset

Our dataset is straightforward, consisting of ten points, making it easier to understand how our models will learn and predict. We have an input array X and an output array y, with a simple relationship: y = 2x.

X = np.array([[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]])
y = np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 20])

Linear Regression for Forecasting

Linear Regression works by establishing a linear relationship between the input variables and the target variable. After training our model with the given dataset, we aim to predict the output for a new input, X_new = [[11]].

Implementation

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Splitting dataset into training and testing set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Linear Regression
model = LinearRegression()
model.fit(X_train, y_train)

# Forecasting
X_new = np.array([[11]])  # New sample X
y_new_pred = model.predict(X_new)

print(f"Forecasted value for X_new = {X_new.flatten()}: {y_new_pred.flatten()}")

KNN Regression for Forecasting

KNN Regression predicts the value for a new point based on the closest k neighbors' values. It's particularly useful in scenarios where the relationship between the data points is not linear.

Implementation

from sklearn.neighbors import KNeighborsRegressor

# KNN Regression
knn_model = KNeighborsRegressor(n_neighbors=3)
knn_model.fit(X_train, y_train)

# Forecasting
y_new_pred_knn = knn_model.predict(X_new)

print(f"Forecasted value for X_new = {X_new.flatten()} with KNN: {y_new_pred_knn.flatten()}")

Conclusion

Both Linear Regression and KNN Regression offer valuable tools for forecasting in machine learning. The choice between them depends on the specific characteristics of your data and the nature of the problem at hand. In our simple example, given the linear relationship between X and y, Linear Regression is expected to perform exceptionally well. However, KNN Regression can be particularly useful in cases where the relationship is more complex or nonlinear.

Remember, the key to successful machine learning projects lies in understanding your data and choosing the right model based on that understanding. Happy forecasting!