Original Source Here
NLP using DeepLeaning Tutorials: A Sentiment Classifier based on perceptron (Part 4/4)
Natural Language Processing is one of the most complicated fields of machine learning, basically due to the complexity and ambiguity of the language. However, it’s also one of the most successful fields, with many real applications that we use every day, like search engines, translation tools and so more.
Sometimes the most complicated tasks can be solved by the simplest techniques. In this article, I will try to explore this claim. So, I will present a complete solution for Sentiment Analysis based on the simplest neural network “Perceptron”, using a real-world task and dataset: to classify whether restaurant reviews on Yelp is positive or negative.
For this I will split this article into four-part as follows:
- The Yelp DataSet review ( link )
- The Vocabulary and The Vectorizer ( link )
- The Training routine ( link )
- Evaluation and inference (the current article concerns this part)
Thanks for your support in advance. If you decide to register for Medium membership here is my subscription page : https://abdelkader-rhouati.medium.com/membership
Part #4: Evaluation and inference
After the training routine, we explained in the last article, the next steps are to evaluate the model on the test dataset and use it to infer new data.
Evaluation of test data
Evaluation of machine learning is a big field of research. Many measurements and methods are possible for a sentiment classifier model, like mine. This subject is very well explained in this article: link. In this example, and to keep things simple, we will evaluate the model only by its accuracy.
The code for the evaluation is exactly the same as the validation step in the training routine but uses the ‘test’ dataset instead of ‘val’ : dataset.set_split(‘test’).
We also put the model in the “eval” mode using the code: classifier.eval(). In this mode, model weights are not updated during predictions on the test dataset. So we can use test accuracy as an accurate measure of model performance.
We get the following test accuracy :
Test Loss: 0.5374936538511884
Test Acc: 0.930599256292907
Inference using new review data
Another way to evaluate the model is to do inference on new review data.
The inference evaluation is based on a qualitative judgment. We use the model to predict sentiments on very small data points (less than 10 in general), and we evaluate its performance one by one.
The following example is an example of the result of an inference test, which illustrates the good accuracy of the model.
this is a pretty awesome book -> positive
Saving and Loading Model
The possibility of saving and loading the models allows using the model prediction without redoing the training each time, which takes hours or even days. This feature makes possible the deployment of machine learning models in real cases.
The next two sections give a very simple way to save and load a model. However, there are more settings options and parameters, for example, to save a checkpoint in which you can resume the model training. For more details, you can refer to the documentation here.
Save a model:
Load a model:
model = ReviewClassifier(num_features=len(vectorizer.review_vocab))
Code for all project
Finally, you can find all code of the project “sentiment classification model using perceptron” on my git repository :
In this series of articles, I presented some fundamental principles and concepts of supervised neural networks. I chose to do it using a case study of the review’s sentiment classification, and using the most basic neural network, “the perception”. Any other neural network follows the same phases and steps.
As a resume, we covered in the 4 articles :
- Concepts of the simplest neural network model, the perceptron
- Some main concepts of machine learning as activation function and loss function.
- A detailed review of Yelp DataSet, as a good dataset for learning and experimentation on NLP Machine learning models
- Transforming Words to Numerical Vector, which is essential to implement a neural network on natural language.
- The training routine on epochs and using a batch of the dataset
- And finally how to evaluate a model
1. “Natural Language Processing with Pytorch” Book (https://www.amazon.fr/Natural-Language-Processing-Pytorch-Applications/dp/1491978236)
2. Saving and Loading Models (https://pytorch.org/tutorials/beginner/saving_loading_models.html)
3. Evaluation of a machine learning model (https://www.jeremyjordan.me/evaluating-a-machine-learning-model/)
Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot