{"id":2752,"date":"2021-03-01T05:00:00","date_gmt":"2021-03-01T04:00:00","guid":{"rendered":"https:\/\/ellycode.com\/?p=2752"},"modified":"2021-03-01T12:40:08","modified_gmt":"2021-03-01T11:40:08","slug":"can-we-analyze-emotions-with-artificial-intelligence","status":"publish","type":"post","link":"https:\/\/ellycode.com\/en\/blog-en\/can-we-analyze-emotions-with-artificial-intelligence\/","title":{"rendered":"Can we analyze emotions with Artificial Intelligence?"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In recent years, thanks to the Internet, our opportunities to express opinions and make judgments has increased exponentially, in the form of both \u2018thumbs up\u2019 and longer reviews. <strong>Unlike cocktail hour banter, however, our comments do not float away but remain in digital storage forever<\/strong>. Is it possible to extract information, or even better, something of value from this amount of data?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A few days ago a politician posted a farewell speech on a well-known social network that was commented on by over one hundred thousand users. Or think about a Youtube video by a pop group that can receive hundreds of millions of views but also millions of comments. <strong>It is inconceivable that a person, or even a team, could read all these comments and draw any conclusions from them<\/strong>. Is the politician still beloved? Did fans like the new song? Only an automatic analysis tool can provide these answers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-sentiment-analysis\"><strong>Sentiment Analysis<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Sentiment Analysis is a calculation process that purposefully seeks to identify and categorize the opinions expressed in a text fragment. The main goal is to understand whether the author\u2019s attitude towards that particular subject is <strong>positive<\/strong>, <strong>negative, <\/strong>or <strong>neutral<\/strong>. This type of analysis, which got started as a field of natural language processing (NLP), is therefore specialized in<strong> interpreting and cataloging emotions<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What are the possible usage scenarios? Sentiment Analysis is a great tool for analyzing product reviews, conducting market research, monitoring customer service or a social media account, and also managing the reputation of an organization or person.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"752\" height=\"413\" data-attachment-id=\"2791\" data-permalink=\"https:\/\/ellycode.com\/en\/blog-en\/can-we-analyze-emotions-with-artificial-intelligence\/attachment\/performing-twitter-sentiment-analysis1-2\/\" data-orig-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1.jpg?fit=752%2C413&amp;ssl=1\" data-orig-size=\"752,413\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"performing-twitter-sentiment-analysis1\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1.jpg?fit=300%2C165&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1.jpg?fit=752%2C413&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1.jpg?resize=752%2C413&#038;ssl=1\" alt=\"\" class=\"wp-image-2791\" srcset=\"https:\/\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1.jpg 752w, https:\/\/ellycode.com\/wp-content\/uploads\/2021\/02\/performing-twitter-sentiment-analysis1-480x264.jpg 480w\" sizes=\"auto, (min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) 752px, 100vw\" \/><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">But when using it for statistics, <strong>is it reasonable to think that the sample of people surfing the web is representative of the world, national, or local public opinion?<\/strong> Many companies and organizations think so and rely on this tool to guide their activities. So how do you conduct a sentiment analysis?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s start with the data: a sample of phrases, which could be tweets or reviews of any kind, and then use software libraries to conduct an analysis. The key steps to take are:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>delete words that do not help to determine a mood<\/strong>: articles, prepositions, conjunctions, etc. In jargon, these words are called &#8220;stop word&#8221;;<\/li><li><strong>create a table<\/strong>, called <strong>frequency distributions<\/strong>, to determine how many times each word appears in a text;<\/li><li><strong>extract concordances<\/strong>, that is, understand where a word appears in the text and which words appear before and after it;<\/li><li><strong>extract bins<\/strong>, that is, that set of words that often appear together in a sentence, we can have combinations of 2, 3 or 4 words;<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">But these are just the initial steps that lead us to the point where we need to choose a pre-trained model according to the context of our problem. We could use, for example, <strong>NLTK <\/strong>(<strong>N<\/strong>atural<strong> L<\/strong>anguage<strong> T<\/strong>oolKit), a platform for writing programs in Python that work with human language. This platform offers, among its libraries, <strong>VADER <\/strong>(<strong>V<\/strong>alence <strong>A<\/strong>ware <strong>D<\/strong>ictionary and s<strong>E<\/strong>ntiment <strong>R<\/strong>easoner), a specialized model for the language used in social media, particularly suitable for short and slang phrases, but which loses effectiveness when sentences are longer and more structured.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The next steps are the usual ones: evaluate the efficiency of the model, possibly modify it by adding additional features and finally use it on new data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As solutions become increasingly complex, it becomes convenient to choose a Cloud in order to take advantage of its simplicity when integrating between services. Just think about the infrastructure needed to capture continuous streams of sentences from multiple sources and analyze them in near real time, while channeling the results to other databases.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-let-s-take-an-example\">Let&#8217;s take an example<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Major Cloud providers offer ready-made services for natural language processing as we saw in<a href=\"https:\/\/ellycode.com\/en\/blog-en\/from-text-to-understanding-with-artificial-intelligence\/\" target=\"_blank\" rel=\"noreferrer noopener\"> Salvatore&#8217;s article<\/a>. So these services can identify the language of the text, extract the key phrases, places, people, brands or events, and understand whether the text has a positive or negative connotation, all while giving us near-immediate results. They all offer the ability to create a custom template for a set domain by defining specific custom entities.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"659\" data-attachment-id=\"2821\" data-permalink=\"https:\/\/ellycode.com\/en\/blog-en\/can-we-analyze-emotions-with-artificial-intelligence\/attachment\/image02\/\" data-orig-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02.png?fit=1264%2C814&amp;ssl=1\" data-orig-size=\"1264,814\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"image02\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02.png?fit=300%2C193&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02.png?fit=1024%2C659&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02.png?resize=1024%2C659&#038;ssl=1\" alt=\"\" class=\"wp-image-2821\" srcset=\"https:\/\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02-1024x659.png 1024w, https:\/\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02-980x631.png 980w, https:\/\/ellycode.com\/wp-content\/uploads\/2021\/02\/image02-480x309.png 480w\" sizes=\"auto, (min-width: 0px) and (max-width: 480px) 480px, (min-width: 481px) and (max-width: 980px) 980px, (min-width: 981px) 1024px, 100vw\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In EllyCode we developed some multi-cloud libraries that allow us to do very simple tests on Azure, AWS and GCP. I couldn&#8217;t resist the temptation to try them out by analyzing the lyrics to <strong>Radiohead<\/strong>&#8216;s song <em>&#8220;No Surprises&#8221;<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Why Radiohead? Let&#8217;s say that I wanted to facilitate machine learning algorithms by choosing what is considered by many (but not by me) as one of the most depressing groups in the history of music.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe loading=\"lazy\" class=\"youtube-player\" width=\"1080\" height=\"608\" src=\"https:\/\/www.youtube.com\/embed\/u5CVsCnxyXg?version=3&#038;rel=1&#038;showsearch=0&#038;showinfo=1&#038;iv_load_policy=1&#038;fs=1&#038;hl=en-US&#038;autohide=2&#038;wmode=transparent\" allowfullscreen=\"true\" style=\"border:0;\" sandbox=\"allow-scripts allow-same-origin allow-popups allow-presentation allow-popups-to-escape-sandbox\"><\/iframe><\/span>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The first lines are:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>A heart that&#8217;s full up like a landfill<br>A job that slowly kills you<br>Bruises that won&#8217;t heal<\/p><\/blockquote>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>You look so tired, unhappy<br>Bring down the government<br>They don&#8217;t, they don&#8217;t speak for us<\/p><\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">There&#8217;s little doubt here. The results of the algorithms are very clear:<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table><tbody><tr><td>&nbsp;<\/td><td>AWS<\/td><td>Azure<\/td><td>GCP<\/td><\/tr><tr><td>Mixed<\/td><td>0,05<\/td><td>&#8211;<\/td><td>&#8211;<\/td><\/tr><tr><td>Negative<\/td><td>0,85<\/td><td>0,88<\/td><td>&#8211;<\/td><\/tr><tr><td>Neutral<\/td><td>0,07<\/td><td>0,01<\/td><td>&#8211;<\/td><\/tr><tr><td>Positive<\/td><td>0,03<\/td><td>0,11<\/td><td>&#8211;<\/td><\/tr><tr><td>Score, Magnitude<\/td><td>&#8211;<\/td><td>&#8211;<\/td><td>-0,8&nbsp; 0,8<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In GCP, the score indicates the overall emotion found in the text, while magnitude indicates how much emotional content is present in the document. For example, clearly positive sentiment can be evaluated with a score of 0.8 and a magnitude of 3.0 while a clearly negative one with a score of -0.7 and a magnitude of 4.0.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The next sentence is the following:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>I&#8217;ll take a quiet life<br>A handshake of carbon monoxide<br>No alarms and no surprises<\/p><\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Interestingly, Amazon interprets it clearly positively, while the other two services have less certainty.<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table><tbody><tr><td>&nbsp;<\/td><td>AWS<\/td><td>Azure<\/td><td>GCP<\/td><\/tr><tr><td>Mixed<\/td><td>0,02<\/td><td>&#8211;<\/td><td>&#8211;<\/td><\/tr><tr><td>Negative<\/td><td>0,05<\/td><td>0,37<\/td><td>&#8211;<\/td><\/tr><tr><td>Neutral<\/td><td>0,25<\/td><td>0,45<\/td><td>&#8211;<\/td><\/tr><tr><td>Positive<\/td><td>0,68<\/td><td>0,18<\/td><td>&#8211;<\/td><\/tr><tr><td>Score, Magnitude<\/td><td>&#8211;<\/td><td>&#8211;<\/td><td>-0,3&nbsp; 0,3<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Of course, the sentence is shorter and the real meaning is surrounded by seemingly positive words. However, these systems are incapable of distinguishing sarcasm and have not been able to grasp an even more negative message than the first verse.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-conclusions\">Conclusions<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">It is clear that we are faced with very powerful instruments but also with largely unexplored scenarios. A tweet is different from a poem or a chat in a room or a political rally: <strong>context, as we will see again in upcoming articles, is always essential<\/strong>!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Keep following us!<\/p>\n\n\n[et_pb_section global_module=\"1785\"][\/et_pb_section]\n","protected":false},"excerpt":{"rendered":"<p>In recent years, thanks to the Internet, our opportunities to express opinions and make judgments has increased exponentially, in the form of both \u2018thumbs up\u2019 and longer reviews. Unlike cocktail hour banter, however, our comments do not float away but remain in digital storage forever. Is it possible to extract information, or even better, something [&hellip;]<\/p>\n","protected":false},"author":195423236,"featured_media":2771,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","_crdt_document":"","inline_featured_image":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[688637375],"tags":[],"class_list":["post-2752","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog-en"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/03\/5_1105x656_blog_Sentiment-A.png?fit=1105%2C656&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/pcuDuD-Io","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2752","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/users\/195423236"}],"replies":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/comments?post=2752"}],"version-history":[{"count":19,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2752\/revisions"}],"predecessor-version":[{"id":2823,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2752\/revisions\/2823"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/media\/2771"}],"wp:attachment":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/media?parent=2752"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/categories?post=2752"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/tags?post=2752"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}