Data Science Notes 2: Machine learning and Python

A note first: in this post I will talk about the Python programming language and the libraries I am going to use. If you are only interested in the theoretical side of data science and not in programming, you can skip this one. For anyone who wants to build something with the language, I think it will be useful.

In my previous data science post I wrote something along the lines of “the techniques we call machine learning are applied first of all on computers, or on processors that act like computers.” So where do we write these machine learning algorithms, and how do we hand them to a processor? This is where a short detour into computer science is needed. Computers use a language quite different from ours. They carry out the orders given to them under certain conditions, using mathematical expressions. We write those mathematical expressions for the computer through programming languages. The language then translates what we wrote into a much more complicated language the computer understands and passes it to the processor. That is exactly what we are going to do, and we will do it with Python.

Why Python?

I do not want to bore you with a long explanation. The main thing I am aiming for in this series is to explain the work mathematically first. If you only know the practical side of something, all you can do is imitate it. If you put theory next to your practical skill, you get past that limit. Doing the exercises in Python is an extra for those who want it. I would not want to bore the people who are here for the mathematics and drive them away from the series.

Python is quite easy to write and, at a basic level, easy enough to learn to get our point across. That is why everyone in the world is being encouraged to learn Python right now. Its active user communities, the heavy sponsor support behind it, its wide range of libraries, the fact that it runs on almost every operating system and, most importantly for us, the fact that it is the most used language in data science all draw attention to it. Let us move on to the Python libraries we will be using.

NumPy

If you have taken linear algebra you already know matrices and vectors. If you have not, you can get a basic introduction on YouTube or from this link. Machine learning is a discipline that runs entirely on matrices and vectors. For now that is the easiest way to store data. What the coming decades bring, I do not know.

NumPy is the library that lets us manipulate very large versions of those matrices and vectors in Python, meaning it lets us shape them by running various operations on them. By pushing a matrix into different forms we can put the data in order. We will use this library a lot, so I am leaving you a video series where you can learn it. If you watch the follow ups to the video below, you can get a good grip on the library in an hour or two. If you would rather learn while writing code, that works too. Go with whatever your own method for learning a language is.

Matplotlib

When you are working with hundreds of thousands or even millions of records, you need to visualise the data to understand what it says. We use various chart types. In Python we will use the Matplotlib library to build those charts. This way we will see how well our predictions match the real data and, through that, how correctly our machine learning algorithm works. Unfortunately there is not enough Turkish material for this. In truth every data scientist should know English at close to native level anyway. In any case, below is the first video of the tutorial series I used myself.

Pandas

Pandas is the library that makes datasets usable. It lets us load data saved in various formats (xlsx, csv) into Python. It helps with the basic changes we make on datasets. I can say that before we interpret a dataset, meaning before any machine learning work, we will do the most basic operations with this library. I am leaving a nine video tutorial series for it below.

scikit-learn

This is the most widely used machine learning library in Python. In the classification, regression and clustering work we will cover later, it takes maybe hundreds of lines of code we would otherwise write and reduces them to a few lines. The fact that it is designed to work together with NumPy and Pandas is a big advantage for us. In this series we will essentially be learning to use this library, though for every topic we will refresh the mathematical theory first. That is why I am not sharing a tutorial series for it: a tutorial series for scikit-learn would mean machine learning itself.

Adding the libraries and the dataset to Python

# Adding the libraries to the Python code
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Loading the data in Data.csv, which sits in the same folder
# as our Python code, into Python as a dataset
dataset = pd.read_csv('Data.csv')

#Notes on the Python code

• We add the libraries with the as keyword. The main reason is that we will use them constantly, so shortening the names makes life much easier. Without it we would write matplotlib.pyplot in full every single time instead of plt. CSV (comma separated values) is the most widely used format in the world for storing data, but you can also use Pandas to load xlsx (Excel) files into Python. You can find the code for that in the documentation.

In the next post we will look for answers to questions like “What is data preprocessing? How is it done in Python? What is regression?”

Good luck with it.