Understand SVM: From scratch with a simple application.

Topics covered 

6 min readFeb 27, 2021

What is SVM? 

How SVM works? 

What is a Kernel Function/trick ? 

Simple SVM application for ZOO data-to classify the class to which an animal belongs to, provided with features as an input.

“SVM’s are very tough to understand!!” is what everyone believes but, it’s not. It’s more likely a jack fruit that looks very tough outside but when you break in you get a smooth fleshy fruit inside. Same as that of SVM, which is very hard to understand, but can be mastered if dug deeply. We help you understand SVM from scratch along with graphical representation for your easy understanding and a simple application of SVM to ZOO-data to classify animals to their respective classes. let’s jump into some basic concepts that you need to understand for applying SVM.

What is SVM?

SVM is a supervised Machine Learning model that uses classifier models to classify the testing data given as input based on the previously trained data.

How SVM works? let’s consider a simple example. We have two classes red and green.

Each class has two features X & Y. we simply want to classify a data(X, Y) that is either red or green. Now we directly jump to the 2D graphical representation of the given data for our better understanding.

What SVM does is it generates a hyperplane or boundary line in between the two classes so that whenever a testing data is plotted on the graph, it is assigned with a class.

We know that many lines can be drawn that separates the two classes in the above figure but which hyperplane is the best suitable for classification in an important factor to be taken care of.

This is where SVM is appreciated because the hyperplane or the boundary line is chosen in such a way that it has a larger margin space for a flexible classification of the new unknown data i.e the testing data.

Or let me put it this way, the data point on a 2d graph nearest to the opposite class must have a maximum distance from the hyperplane on both sides.

Then you need to know about linearly separable data and non-linearly separable data.

Linearly separable data:

Like in the above figure, when the data is plotted on a 2d graph we get a cluster of data and when the two classes are easily separable by a linear line then this data is known as linearly separable data.

certainly, not every cluster is easily separable by a linear line like shown in the figure.

Hence a 3d plane comes into picture when we are dealing with many numbers of classes. The main agenda is to choose the best hyperplane or the boundary line when the data is not linearly separable.

The above figure is the 2d plane where the data is not easily separable. So what trick can be used to choose the best hyperplane. Another plane ‘z’ is considered.

Now what SVM does is just draw the best hyperplane that separates the two classes.

From the above figure, we can interpret that in a 3D plane, a hyperplane is just a straight line parallel to the x-axis at a certain Z But we are more likely to 2D plane interpretation. Hence when the above figure is viewed in a 2D plane, the hyperplane is a circle.

But calculating hyperplane for non-separable data is a hardcore process as it has to undergo every possible vector in the plane. This calculation is simplified using a trick known as Kernel Trick.

What is a kernel Trick?

This trick simply generates the dot product of the vectors to find the best hyperplane and cut-down the process.

For example,

z = x² + y²

The dot product in the space is given by, 

a · b = xa · xb + ya · yb + za · zb

a · b = xa · xb + ya · yb + (xa² + ya²) · (xb² + yb²)

This trick is also known as kernel function.

Since we are using a linear kernel, A Linear Classifier is obtained. A non-linear classifier is also obtained by using the non-linear kernel.

There are different types of kernels,

Polynomial Kernel.

Radial Basic function RBF Kernel.

Sigmoidal Kernel.

Great! That you’ve made this far. That means you’re ready to dive into SVM and see a real life example.

APPLICATION OF SVM: real life data

Simple SVM application for ZOO data-to classify the class to which an animal belongs to, provided with features as an input. So, this is a data from ZOO that consists of animal_name with features. There are totally 7 classes as you can see below. 

Mammals 

Birds 

Reptiles 

Fish 

Amphibians 

Bugs 

Invertebrates

We have to build a model to classify an animal provided with features as an input to their respective class.

You can download the data from the below link.

Prior knowledge to build this model: 

PANDAS library. 

NUMPY library. 

MATPLOT library.

I’m using a real life data and also the code which I’ll be providing you the link below:

https://github.com/yathish-debug/SVM.git

Firstly we need to import libraries in header files to use them in our code.

2. We need to read the data file simply by providing the path.

Here we are providing the path of our data file and getting the dataframe. Note: please provide the path of your data file.

3. We get the dataframe.

4. Now we need to check whether all our data is of same data dype or not. i.e all our data should be of int data type only.

As we can see we have all data in int datatype only which reduced half of the work. We need to see all the column names and ignore the unwanted column. (Here: unwanted column and class_type)