{"id":2952,"date":"2021-03-15T05:00:00","date_gmt":"2021-03-15T04:00:00","guid":{"rendered":"https:\/\/ellycode.com\/?p=2952"},"modified":"2021-03-11T15:56:41","modified_gmt":"2021-03-11T14:56:41","slug":"learning-to-understand-the-language","status":"publish","type":"post","link":"https:\/\/ellycode.com\/en\/blog-en\/learning-to-understand-the-language\/","title":{"rendered":"Learning to understand the language"},"content":{"rendered":"\n

To understand what a user is saying when writing with natural language, a simple syntax analysis is often not enough. We need tools that can analyze a phrase in its entirety and extract key concepts, often tailored to our own scope of application. For this we need tools that can provide an initial configuration, then specialized and monitored later, during development<\/strong>.<\/p>\n\n\n\n

Among the many cognitive services provided by Microsoft Azure, LUIS is certainly one of the most interesting because it makes it simple to create a model for understanding the user\u2019s requests, a model which can be developed and improved even by those who don’t have programming experience, thanks to an intuitive web interface.<\/p>\n\n\n\n

A LUIS application allows us to analyze a request written in natural language, called Utterance<\/strong>, recognizing the Intent <\/strong>of the user, and possibly containing parameters called Entities<\/strong>. In practice, a model is created by deciding what are valid Intents, and providing examples for each of them. Examples that may contain one or more entities, either chosen from a prebuilt set or created expressly for our particular application domain.<\/p>\n\n\n\n

An example is worth a thousand words<\/h2>\n\n\n\n

Let\u2019s suppose we run our family\u2019s ice cream shop and we want to deal with customers by chat: let\u2019s create a LUIS application together that recognizes takeaway orders.<\/p>\n\n\n\n

\"\"
Regalo Helado, di Joseph Oland<\/a><\/figcaption><\/figure><\/div>\n\n\n\n

To create a LUIS application both an Azure account and a Language Understanding<\/em> resource are needed. If you do not already have an account, you can sign up for free and take advantage of the free credit. After that, you can create the resource by following the Microsoft documentation<\/a>. <\/p>\n\n\n\n

After creating the resource, we can enter the LUIS console<\/a> to create our application simply by choosing the name and the culture.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

From the configuration panel, we can create our very first Intent: to recognize takeaway orders. We will call it TakeawayOrder. From this point, we can provide examples to instruct LUIS<\/strong>: phrases like \u201cdo you do takeaway?\u201d, \u201cI want some ice cream to pick up\u201d, \u201ccan I order some ice cream?\u201d, are all valid examples. It is important for the example phrases to be varied and written using natural language.<\/p>\n\n\n\n

But an ice cream shop has so many flavours you can choose from! To make LUIS recognize them, we can define a new Feature<\/strong>.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Typing the flavours in the Values <\/strong>box, the suggestions box will be populated with terms related to the one already added. After we finish entering all the available flavours in our shop, let\u2019s confirm them with Create<\/strong>.<\/p>\n\n\n\n

We have the flavours now, but a customer could want to purchase more than one serving or flavor in a single order. To handle these cases correctly, LUIS allows us to create Machine Learned entities, which are basically entities composed of several sub-entities. In our case, we need an entity composed of two Features: flavour and number. Since we already created the former, we can create the latter by choosing it from the available prebuilt entities by clicking Add new prebuilt entity. At this point, we only need to create our IceCreamOrder entity, made up of two sub-entities.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

After defining the entity, let\u2019s return to the TakeawayOrder Intent in order to assign the Feature we’ve just created for it.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Now we just have to add example utterances containing this Feature, that will be not automatically recognized. Selecting them manually is required to allow the system to recognize them in the future.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

Once we finish adding input phrases, we proceed with training and tests. <\/strong>In response to our request, we also receive a list of recognized features.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

To summarize, our application is now capable of recognizing an Intent for takeaway orders, with the relative Entities that describe the quantity and flavour of the ice cream. Ideally, every phrase that can\u2019t be recognized will be filed with the None Intent, but to ensure that the system is functioning correctly, it\u2019s important to provide examples for this Intent, once again being sure that they are written in the most natural way possible.<\/p>\n\n\n\n

Publishing<\/h2>\n\n\n\n

Once our application has been tested, we want to make it available to the world. After publishing, our model will be available at a specific endpoint. We\u2019ll then ask this model to make Predictions <\/strong>according to our users\u2019 requests.<\/p>\n\n\n\n

LUIS permits the simultaneous publication of two different versions of our model by using a mechanism called slot. Basically we have two slots called Production <\/strong>and Staging<\/strong>, that allow us to differentiate the production environment from the environment in which we can develop and test new features before releasing them.<\/p>\n\n\n\n

In the Manage <\/strong>tab we can find the list of application versions. To create a new one, we only need to select a version and click on the Clone <\/strong>button. This way we can leave the production version untouched (published in the Production slot) and work on a new version not yet published in the Staging <\/em>slot.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

LUIS is available through the Microsoft SDKs or REST APIs. With both not only is it possible to predict intents, but also to modify the application in a phase called Authoring<\/strong>, with all the tools made available from the web portal.<\/p>\n\n\n\n

Here are the official resources for further details: https:\/\/docs.microsoft.com\/en-us\/azure\/cognitive-services\/luis\/developer-reference-resource#language-based-sdks<\/a><\/p>\n\n\n\n

Improve the model<\/h2>\n\n\n\n

Once published, the application will be put to the test by the users who could formulate requests that the system doesn\u2019t recognize. It\u2019s important that the system is periodically assessed<\/strong> to ensure that it\u2019s functioning correctly. With LUIS it is possible to review unclear requests, the ones with a low confidence score, in order to add them to the examples.<\/p>\n\n\n\n

This operation is really simple: from the Build <\/strong>tab in LUIS we can enter Review Endpoint Utterances<\/strong>, where we can confirm the requests or assign them to another Intent.<\/p>\n\n\n\n

\"\"<\/figure><\/div>\n\n\n\n

The possibility for human operators to verify the outcomes is just one of the points listed by the Microsoft guidelines for Responsible AI. These principles help us understand the impact that our system has on the life of people using them, and what the common pitfalls are when developing it. If you would like to learn more about that, check out Michele\u2019s post<\/a>.<\/p>\n\n\n\n

Conclusions<\/h2>\n\n\n\n

After we analyzed Natural Language Processing solutions in the previous article<\/a>, we have seen how a tool like LUIS could be the next step after text analysis, essential for the interpretation of utterances and to decide what to do with the user requests.<\/p>\n\n\n\n

Amazon and Google also provide ready-to-use solutions similar to LUIS, respectively called Lex and DialogFlow, although they are more focused on the development of chatbots. LUIS instead presents itself on the market as a more generic solution, usable in any context in which it is necessary to understand what a user wants and decide how to respond.<\/p>\n\n\n\n

Tools like this are fundamental to a solution like the one we are developing at Ellycode, because they place emphasis on the possibility of improving the model and adapting it to the user\u2019s domain. And if I can understand what the user is asking of me, I\u2019m on the right path to answering correctly<\/strong>.<\/p>\n\n\n\n

How? Keep following us!<\/p>\n\n\n[et_pb_section global_module=\"1791\"][\/et_pb_section]\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

To understand what a user is saying when writing with natural language, a simple syntax analysis is often not enough. We need tools that can analyze a phrase in its entirety and extract key concepts, often tailored to our own scope of application. For this we need tools that can provide an initial configuration, then […]<\/p>\n","protected":false},"author":195423238,"featured_media":2965,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","_crdt_document":"","inline_featured_image":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[688637375],"tags":[],"class_list":["post-2952","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog-en"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/ellycode.com\/wp-content\/uploads\/2021\/03\/8_1105x656_blog_luiss.png?fit=1105%2C656&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/pcuDuD-LC","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2952","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/users\/195423238"}],"replies":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/comments?post=2952"}],"version-history":[{"count":7,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2952\/revisions"}],"predecessor-version":[{"id":2973,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/posts\/2952\/revisions\/2973"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/media\/2965"}],"wp:attachment":[{"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/media?parent=2952"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/categories?post=2952"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ellycode.com\/en\/wp-json\/wp\/v2\/tags?post=2952"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}