The Need for Ensemble Methods in Machine Learning

Ensemble methods train multiple models (base models) to solve the same problem and combine them for better performance. This offsets the individual variances and biases of the models. They can be used collectively for improved decision making. Let’s find out how.

- Surabhi Dwivedi

It’s a well-known fact that machine learning techniques can be broadly categorised into supervised, unsupervised and reinforcement learning. If data sets are labelled, supervised learning techniques can be used. If they are not labelled and we need to find some pattern in the data by analysing its characteristics, unsupervised learning techniques are used. Reinforcement learning works on the concept of reward and punishment to the model based on its interaction with the environment. This article reviews the ensemble methods in machine learning, which focus on supervised learning techniques.

Supervised learning can be broadly classified into classification and regression techniques. Classification is the supervised learning approach that categorises a given data into a set of classes. The popular classification techniques are – decision tree, logistic regression, Naïve Bayes, support vector machine (SVM) and KNN. Regression is a supervised learning method that can predict a continuous-valued output for the new data given to the algorithm. Popular regression techniques include linear regression, polynomial regression, lasso regression, etc. The ensemble methods can be used for both regression and classification, without any alterations in the theoretical basis of the underlying algorithms. This helps them to address a broad range of machine learning problems.