To be successful with applying machine learning to a problem, it’s important to take a systematic approach. If not, the results could be way off base.
First of all, you need to go through a data process, which we covered in the prior chapter. When this is finished, it’s a good idea to do a visualization of the data. Is it mostly scattered? Or are there some patterns? If the answer is yes, then the data could be a good candidate for machine learning.
The goal of the machine learning process is to create a model, which is based on one or more algorithms. We develop this by training it. The goal is that the model should provide a high-degree of predictability.
Now let’s take a closer look at this (by the way, this will also be applicable for deep learning, which we’ll cover in the next chapter):
Step #1—Data Order
If your data is sorted, then this could skew the results. That is, the machine learning algorithm may detect this as a pattern! This is why it’s a good idea to randomize the order of the data.
Step #2—Choose a Model
You will need to select an algorithm. This will be an educated guess, which will involve a process of trial and error. In this chapter, we’ll look at the various algorithms available.
Step #3—Train the Model
The training data, which will be about 70% of the complete dataset, will be used to create the relationships in the algorithm. For example, suppose you are building a machine learning system to find the value of a used car. Some of the features will include the year manufactured, make, model, mileage, and condition. By processing this training data, the algorithm will calculate the weights for each of these factors.
Example: Suppose we are using a linear regression algorithm, which has the following format:
y = m * x + b
In the training phase, the system will come up with the values for m (which is the slope on a graph) and b (which is the y-intercept).
Step #4—Evaluate the Model
You will put together test data, which is the remaining 30% of the dataset. It should be representative of the ranges and type of information in the training data.
With the test data, you can see if the algorithm is accurate. In our used car example, are the market values consistent with what’s happening in the real world?
Note
With the training and test data, there must not be any intermingling. This can easily lead to distorted results. Interestingly enough, this is a common mistake.
Now accuracy is one measure of the success of the algorithm. But this can, in some cases, be misleading. Consider the situation with fraud deduction. There are usually a small number of features when compared to a dataset. But missing one could be devastating, costing a company millions of dollars in losses.
This is why you might want to use other approaches like Bayes’ theorem.
Step #5—Fine-Tune the Model
In this step, we can adjust the values of the parameters in the algorithm. This is to see if we can get better results.
When fine-tuning the model, there may also be hyperparameters. These are parameters that cannot be learned directly from the training process.
Leave a Reply