Algorithms - High level overview
We would like to explain briefly (high level overview) the implementation methodology we use for the Machine learning algorithms in the E-commerce Advisor.
We will explain the process of getting answers from Machine learning algorithms by analyzing big data.
The stages of the process are:
- Gathering Data
- Preparing Data
- Model Selection
- Training
- Evaluation
- Parameter Tuning
- Prediction
1) First, we collect the data from the databases of e-shops and put them in a very large database.
2) Then we move on to the phase (preparing the data), data preparation, error correction, deduplication, normalization.
3) Then we choose the model -> select the model e.g., Linear regression model
4) Then we move on to the model training stage. The process of "training the model" is in essence the collection of the above data (cleansed data) and their feeding in an ML algorithm. The model emerges as the result / artifact of the training process.
5) Then we have the evaluation phase. In this phase we test the model with data that was never used in the training of the model.
6) After this phase is over, we can change some parameters to have a better model.
7) In the last stage we have the prediction. This is the stage where we look for answers to our initial questions, answers to the problem we posed. We use the now trained model to make predictions to have better results.