Data Science Notes: introduction to machine learning

Plenty of article series have been written about data science. If you ask me what my goal is: writing these notes makes me repeat what I have learned, which is useful to me, and at the same time I would like them to be useful to you. Now that the purpose is out of the way, I want to start. In the meantime, you can look through the topics we will study together in this series.

Machine learning is the science of making machines do things that would be considered intelligent if they were done by people.

Marvin Minsky

First I want to touch on the concept of machine learning. Machine learning can be called a set of methods completely intertwined with statistics, containing more numerical analysis techniques than you can count. We use machine learning to make predictions from data and to reach more varied information. In these methods we will use more developed versions of the theory we learn in a probability and statistics course. If you want to know more, write your questions as comments. As far as I know them, I can share resources and information with you. All machine learning techniques are applied on computers or on processors that act like computers. That part is not very important, so I will start with the basic concepts in machine learning.

Data: uninterpreted information. For example, recording the temperature of a district second by second over a day creates hundreds of thousands of lines of information.

Dataset: a data source formed by combining more than one kind of data, whether or not those kinds are correlated with each other. For instance, if we add the humidity recorded every second in that district on the same day to the example above, we get a dataset. In machine learning applications we split datasets.

Why are datasets split?

Here is one way to explain it. Suppose we have twelve months of weather data and we want to build a forecasting application with it and predict the weather ahead. Machine learning algorithms need data to train themselves. You can think of it like this. Your opinions about a city are shaped by the people from there you have met before. You know what those people are like. It works the same way here. A machine learning application asks us for data about the people of that city so that it can understand them.

That is not all of it. Our data is limited, and we have to test whether the algorithm works correctly. Say we fed the algorithm the data of the people we know who live in Hatay. Now we have no people left to test it with. So we split the data in two. We give part of it to the algorithm so it gets to know the people of Hatay, and we hold the rest back to test it. A dataset is usually divided in two.

Training set: the set created so the machine learning algorithm can get to know the data and make its predictions on it during training. In short, it is the dataset that gives the algorithm its preconceptions.

Test set: the dataset we set aside earlier to test how accurately the algorithm built from the training set works.

In the next post I will start with the basic machine learning libraries and explain how a dataset is loaded into Python.

Have a good day.