Investing, with persistence, consistency and method, in artificial intelligence and machine learning, @ PaloServices we can achieve up to 90% accuracy in sentiment analysis. The company, working with dedicated and experienced Data Scientists and Data Analysts, develops and maintains separate customer sentiment prediction models, in the Greek language, considering SentimentAnalysis as a “flagship” service of web and social media monitoring.
The 4 Steps to Successfully Predict Sentiment
Step 1. Retrieving data
Processing millions of mentions published on the Web and on Social Media, we collect the data required according to the needs of each individual customer
Step 2. Creating a customer data set – model training
Our team of analysts reads, annotates and classifies negative, positive or neutral mentions to each customer. In other words, they perform the initial model training and/or model evaluation for a plethora of of-the-shelf
models we maintain. The process is repeated on a monthly and or daily basis, depending on the needs and particularities of each project. In any case, the statistical sample of mentions manually annotated by our analysts does not exceed 20% of total mentions to the client.
Step 3. Create and test multiple models
Our Data Scientists create and test multiple models of sentiment prediction. Indicatively, our tests include:
7000 basic machine learning models
3 basic algorithms, about 50 parameters per algorithm, and 48 variations in input data
600 deep learning models
3 basic architectures, in four variants each, and 48 variants in the input data
90 sentiment analysis generic models
also automatically adjusting them to the data of the individual customer
Deep learning vs Machine learning ?
In the age of deep learning, the choice to test classic machine learning algorithms may create questions. However, large amount of data is a prerequisite for the deep learning to succeed. So, until we have collected large volumes of mentions to each client, classic machine learning models often wins the battle.
Step 4. Combination of models
Out of these approximately 7,690 models, we do not just choose the best: We combine them together, creating Ensemble Model for the best performance by channel (Twitter, Facebook, Instagram, YouTube, News, Blogs, Forums)
Why do we combine sentiment prediction models?
The combination of models is not always the best solution. However, in our case:
- It allows us to shape the kind of errors the model makes. For example, as we provide a negative alert service, it is important that we do not “lose” negative mentions. Even if this technique does not always guarantee higher percentages of total accuracy, the combination of models is useful in collecting the most of the negative mentions (using cumulatively the well-predicted negative mentions by multiple models) and sending them as a real time alert notification.
- It helps us to effectively identify the “problematic” mentions that “raise” a major disagreement between different models of sentiment prediction and that we delegate to our analysts for further study and classification.
- It reduces the dependence of success of the individual models on the volume of “train data”: as our analysts classify a percentage of mentions to each client, on a monthly basis, each project progressively scales from a few available train data to big volumes of data. Individual models of sentiment prediction behave differently depending on the volume of train data. The combination of models helps us to overcome that problem and, of course, the more train data, the better performance is achieved.
info: Partners in the development of our Sentiment Analysis technology are the Research Group of the Department of Cultural Technology and Communication at the University of the Aegean and the Laboratory of Knowledge and Uncertainty of the University of Peloponnese, through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE – INNOVATE.
