Computer Science Blog @ HdM Stuttgart

Log in
, , ,

Analyzing text with IBM Watson services on Bluemix

Patrick Kleindienst

You might have already heard of IBM’s artificial intelligence “Watson”, which beat two former champions  of the american television game show “Jeopardy!”  back in 2011. What you probably don’t know is that today lots of predefined Watson services are publicy available on IBM’s cloud platform “Bluemix”. These services cover different aspects of AI-backed applications like Visual Recognition, Language Translation or Text to Speech. This post glances on Natural Language Understanding, which is about analyzing text by extracting different kinds of information, and shows how this can be achieved by using Bluemix services.

Requirements

Before we can get started playing around with Watson, we must make sure to have a valid Bluemix account available. If you don’t already have a Bluemix acoount, head for the IBM Bluemix homepage and get a 30-day trial for free. That’s it, now we’re ready to get started.

 

Launching a Watson service on Bluemix

From your Bluemix console, go to Catalog > Services > Watson, which should give you a list of currently available Watson services. The service we’ll use for our test analysis example is Natural Language Understanding (NLU). Once we have this service up and running, we’ll pass it the URL of a previous blog post and make it examine the post for keywords, categories and metadata.
Selecting the service in the list brings us to another page where we can choose a name as well as a plan for our service. Select the free plan (which limits the amount of operations you can execute per day and won’t cost you anything) and  click the “Create” button. After a short amount of time, your Watson service should be displayed on your Bluemix Dashboard and is therefore ready to do some work.
After making sure that your service has started, open it in the Dashboard. Among other things, this page shows the credentials (username/password) you’ll need for authentication and also the base URL for talking to the previously created service. Since we’ll need this information repeatedly throughout the rest of this tutorial, make sure you keep this page open.

 

Testing our Watson service

To find out what the Watson NLU service can do, select your service on the Bluemix Dashboard (if you haven’t already) and click Manage in the left-side panel. The management page provides you with a rough overview and refers to the Documentation for further information.
This should open the Watson Developer cloud page, which offers detailed information on how to create Watson services (what we just did), how to use the NLU API and much more. Feel free to look around and examine the docs on your own.
In order to interact with our newly created Watson service, we’ll make use of its RESTful HTTP interface. I’ll use the Postman REST client for the rest of this tutorial. As an alternative, consider using the API explorer which can be accessed from the documentation start page. This tools allows you to interactively compose and send HTTP requests by means of a very nice GUI. If you prefer working with the command line, there’re also examples of how to do it with cURL.
As a test scenario, let’s assume we have a blog post and we want to get a rough overview by extracting and skimming the most significant keywords which capture the essence of the post. For demo purposes, I will feed one of my former posts on Docker Security to the NLU service. Feel free to choose any other article for your own experiments.
The Watson API allows us to pass text by either provide a public URL or directly embedding it into an HTTP request. I decided to pass the post by URL, since it can be generally accessed via the internet and is a much less verbose method than copying and pasting the whole text. Beyond sending the data to work with, we also have to specify what we want our service to do with the article (extract keywords and metadata).
All in all, we send the following parameters along with our request:

  • version:  “2017-02-27”  (the API version we want to use)
  • url: “https://blog.mi.hdm-stuttgart.de/index.php/2016/09/13/exploring-docker-security-part-3-docker-content-trust/”  (article’s public URL)
  • features: “keywords, metadata” (features we want Watson to extract)
  • keywords.limit: “5” (we limit the amount of keywords returned to 5)

Don’t forget to include your credentials (create a Basic Authorization header in Postman)  and make sure your API end point is correctly specified. For text analysis, the required end point is https://{API URL}/analyze. You can find your base API URL along with your credentials on your service detail page. The URL will probably match https://gateway.watsonplatform.net/natural-language-understanding/api, so we’ll hit https://gateway.watsonplatform.net/natural-language-understanding/api/analyze.

This is how the entire HTTP request looks in Postman:

Figure 1: Setting up our request with Postman REST client

 

Now we’re finally ready to hand off some work to our Watson NLU service. Dispatch the request and view the results. You should get a JSON response similar to this one:

Figure 2: Result returned by Watson API

 

You can see that – as requested – the response includes five keywords, along with a relevance score computed by the service, as well as some metadata about the post.
If we take a look at the keywords and the associated relevance score, I think the result seems to be very good with respect to the input text. However, one can also see that fetching metadata only went well to some extent. For example, extracting the author name did not work, whereas publication date and feed links look pretty good.

 

Conclusion

By having integrated Watson and Watson APIs into Bluemix and having released them as generally available services, IBM enables anyone to make use of modern AI technology and even enrich existing applications with ML capabilities by communicate with Bluemix services either by language-specific SDKs or an easy-to-use RESTful HTTP API.
As we’ve seen in this short tutorial, e.g. text analysis can be done without requiring deep knowledge of Natural Language Processing. And although there’s still potential for these services to be improved and getting better, it’s still quite impressing what can be done with them today.
I hope you enjoyed this introductional guide to IBM Watson Bluemix services, have fun exploring much more on your own!

 

Sources

Posted

in

, , ,

by

Patrick Kleindienst

Tags:

, , ,

Comments

Leave a Reply