Sequence Data Mining for Health Applications

An often overlooked type of Analysis is  Sequence Data Mining (or Sequential Pattern Mining).


Sequence Data Mining is a type of Analysis which aims in extracting patterns sequences of  Events. We can also see Sequence Data Mining as an Associations Discovery Analysis with a Temporal Element.

Sequence Data Mining has many potential applications (Web Page Analytics, Complaint Events, Business Processes) but here today we will show an application for Health. I believe that this type of Analysis will become even more important as wearable technology will be used even more and therefore more Data of this kind will be generated.

Consider the following hypothetical scenario : 

A 30-year old Male patient complaints about several symptoms which -for simplicity reasons- we will name them as Symptom1, Symptom2, Symptom3,etc.

His Doctor tries to identify what is going on and after the patient takes all necessary Blood work and finds no problems. After thorough evaluation the Doctor believes that his patient suffers from Chronic Fatigue Syndrome. Under the Doctor's supervision the patient will record his symptoms along with different supplements to understand more about his condition. Several events (e.g a Visit to the Gym, a stressful Event) will also be taken under consideration to see if any patterns emerge.

-How Can we easily record Data for the scenario above?

-Can we extract sequences of events that occur more frequently than mere chance?

-Can we identify which sequences of Events / Food / Medication may potentially lead to specific Symptoms or to a lack of Symptoms?

Looking the problem through the eyes of a Data Scientist, We have :

A series of Events that happen during a day : A Stressful event, A sedentary day, Cardio workouts, Weight Lifting, Abrupt Weather Deterioration, etc

A Number of Symptoms : Headaches, "Brain Fog", Mood problems, Insomnia, Arthralgia, etc.

Let's begin with Data Collection. We first suggest to the patient to use an Android app called MyLogsPro (or some other equivalent application) to easily input information as this happens :

  

So if the patient feels a specific Symptom he will press the relevant Symptom button on his  mobile device. The same applies for any events that have happened and any Food or Medication taken. As the day passes we have the following data collected :

The snapshot shows what happened starting on the 20th of August 2014, where our patient has logged the intake of Medication (at 08:22 AM) and/or Supplements upon waking up then a Food entry was added at 08:47. At 11:06 the patient had a Symptom and immediately reached his phone and pressed the relevant Symptom (Symptom No 4) button.

After many days of Data Collection we decide that its time to analyze this information. We export the data from the application as a csv file which looks as follows :

We will use KNIME to read the csv file, change the contents of the entries accordingly so that an Algorithm can read the events and then perform Sequence Data Mining. We have the following layout :

 The File Reader reads the .csv file, then during the Pre-processing block (shown in yellow), a String Manipulation node which removes colon (:) from time field (e.g 12:10 becomes 1210). The Sorter sorts the data according to date then time as the second field and a Java snippet uses replaceAll() function to remove all leading zeros from Time field (e.g 0010 becomes 10).

The R Snippet loads the CSPADE Algorithm and then uses this Algorithm to extract pattern of sequences.

After executing the stream we get the following output :

The information consists of two outputs : The first one is a list of sequences along with their support and the second one contains the output from rule induction which gives us two more useful metrics (namely the lift and the confidence for each rule).

We immediately notice an interesting entry on the first output :

Medication1->Symptom2

and on the second output we see that this particular rule has a lift of 1.4 and 0.8 confidence.

However, as Data Scientists we should always double-check the extracted knowledge and must be aware of pitfalls. Let's see some examples (list not exhaustive) :

1) The algorithm does not account for time as it should : As an example, consider the following entries :

10/09/14,08:00,Medication1
10/09/14,08:05,Symptom2

We assume that Medication1 is taken by mouth and needs 60 minutes to be properly dissolved and that these entries occur frequently enough in that order in our data set. Even though the algorithm might show a statistically significant pattern , it is not logical to hypothesize that Medication1 could be related to Symptom2. The Analyst should first examine each of these entries to see which proportion of the records has a time difference of at least -say- or greater than 60 minutes.

Apart from the example shown above we must consider the opposite effect. Consider this entry :

10/09/14,08:00,Medication1
...
...
...
10/09/14,21:05,Symptom2

In other words : Is it possible that a Medication taken in the morning to generate a Symptom 12 hours later?


2) The algorithm is not able to account for the compounding effect of a Medication. For example, the patient might have low levels of Taurine and for this level to be replenished, an x amount of days of Taurine supplementation is needed. The algorithm cannot account for this possibility.


 3) The patient should also input entries of "No Symptoms". It is not clear however when this should be done (e.g at the end of each day? assess every 6 hours and add 2 entries accordingly?)


However, this does not mean that a Sequence Mining algorithm should not be used under these circumstances. This technique can generate several potentially interesting hypotheses which Doctors and/or Researchers may wish to pursue further.
 

New Insights from Text Analytics

Text Analytics has gained the attention it deserves in the past few years. Sentiment Analysis is perhaps the most frequently discussed type of analysis but there will be always new ways to analyze and gain insights from text data.  

Examples of new types of analysis -and they have a vast potential- are in my opinion two :  Sequence Detection and Concept Mining. I am not aware whether  these types of analysis are currently being implemented by any Text Mining practitioner at the moment and if there is one, feel free to add your comments below.

So what is Sequence Detection and Concept Mining ? Some examples :

Suppose that you receive several similar e-mails sent from customers as the one seen below :

"I have been trying repeatedly to solve my billing problem through customer care. I first talked with someone called  Mrs Jane Doe. She said she should transfer my call to another representative from the sales department. Yet another rep from the sales department informed me that i should be talking with the Billing department instead. Unfortunately my bad experience of being transferred through various representatives was not over because the Billing department informed me that i should speak to the......"

Currently Text Analytics software will identify key elements of the above text but a very important piece of information goes unnoticed. It is the sequence of events which takes place :

 (Jane Doe => Sales Dept =>Billing  Dept =>...)


Being able to detect the sequence of events is an important element in understanding customer interaction. In our example above, imagine the possibility of detecting similar sequences through thousands of e-mails or call center transcripts and running a sentiment analysis, a process which then could correlate sentiment with specific event sequences.

Next,  is the usage of Concept Mining (this is just a phrase i coined for this post) : Being able to analyze information to different conceptual levels. A very powerful technique indeed and let's see why this is so.

People that have attended the 7th annual Text Analytics Summit in Boston had the opportunity to listen to several presentations regarding Semantics. The discussions between experts from the Semantics Panel and the attendees revealed that people could not find Semantics practical for several reasons. Yet, in Semantics lies the power of being able to find patterns on different conceptual levels.

As a -very basic- example, if we use Information Extraction to annotate -say- the Tweets containing mentions of American Telcos we can tag each one as a more general category called TELCOS. We can also tag individual prepaid packages as a more general category called PREPAID_PACKAGES. By doing that we can then search for patterns in a more general conceptual level than searching for patterns only at a Telco Brand level or a specific Telco's prepaid package. As an example we can run  a sentiment analysis on all prepaid packages mentions,  identify patterns of negative or positive sentiment and see which Telco is the winner of positive sentiment at a conceptual level.

The possibilities are endless.

Predictive Analytics and Politics - Part 1

One of the most interesting applications of Data/Text Mining and Information Extraction is Politics. I started collecting information from various blogs, websites and forums and applying Information Extraction and Data/Text Mining techniques to extract potentially useful knowledge in this area. By combining different pieces of information one could come up with trends that may tell us what lies ahead of us.

The latest developments in Greece are more or less known to most of people that read International News. The situation is difficult and the voice of citizens in various blogs and forums could give us the sentiment of Greek Web Users. For example :

- Which are the most frequently occurring words?

- Which are the most frequently occurring thoughts?

- What are the things that have to be changed by Greek politicians?

To answer these questions i have started collecting information found on the top 120 Greek blogs, the OpenGov website (a state-run website where Greek citizens express their opinions) and a couple more Greek sites of economic content. For blogs and forums a Java program scans every 20 minutes for new information :

This information is then sent to an annotation engine which analyzes the textual content. Once the text is analyzed we can -for example- produce a keyword vector that we can later use to understand what citizens are saying on the Web. We can then find out answers to many interesting questions such as :

- With which words is Mr George Papandreou (PM of Greece) associated with?

-When there are some very negative words (such as swearing) what other words are found in the same text?

- What does keyword trending tell us? (For example, we identify an increasingly number of swear words in citizen posts)

First let's see some examples regarding the OpenGov website where thousands of citizens have expressed their opinions on the tax policy of the Greek state. The following chart shows us a number of pairwise correlations between written words in these comments :

Under the red rectangle appear two words (dikigoros,iatros) which in Greek mean "Lawyer" and "Medical Doctor" respectively. This essentially tells us that these two professions are used together frequently in citizen discussions. By looking closely at these messages one can reveal that professionals in these two sectors are said to avoid taxes by not issuing receipts.

Next we could use association rule learning to look for some more -potentially interesting - rules :

The highlighted rule although one of low support it could prove interesting : A subset of citizens are requesting that freelancers and the self-employed should be more closely monitored for tax fraud.

Apart from rule learning, it is interesting to identify the proportion of the total dataset for which each rule holds. That also gives us a sense of order with which different ideas and thoughts exist on the mind of citizens.

In the next post : What is the Voice of the Citizen tells us in Blogs and forums?

Emotions, Beliefs and Analytics

When i first came across Data Mining and Machine Learning in 1998 i had no idea of the kind of applications that this field can have. As time passes by, the knowledge that can be available to a data/text miner becomes more and more a serious business....actually, a very serious one.

Not long time ago i have seen a presentation where a map of emotions from the web was created in real time by aggregating specific keywords from blogs and forum posts. Twistori is an example of such an application. Now, let's take this idea one step further.

Twitter is a "social messaging utility" in which users describe what they are doing -or what they are feeling/thinking- now. Users are able to send "tweets" even through SMS messages. The way that these messages are written is an ideal format for text mining : Short phrases that summarize what a user wants to say are a text miner's paradise.

It is logical to assume that Text mining and Information extraction techniques will become more important, since more data will be generated in the future. It is only a matter of time until the next "killer app" like FaceBook, YouTube and Twitter appears. Data/Text miners will be able to identify common "thought clusters" of people.

Now, consider the following example : By visiting this link you will get a list of people that have written in their "tweets" the phrase "I don't want to....".

Once this textual information is captured, preprocessed and then analyzed through cluster analysis we could end up with the following clusters of "I don't want-er's " :


- The cluster of users that do not want to work again/tomorrow/today (18.5%)

- The cluster of users that do not want to go to sleep (6%)

- The cluster of users that do not want to hurt someone (4.2%)


What is also interesting, is the ability to quantify the proportion of cases belonging to each cluster to the total of tweets. As shown in the example above, the most frequently occurring thought is from people that do not feel like working.


Now in the same way one could perform this type of analysis for :

"I Believe...."
"I wish i...."
"I want to buy..."

Essentially, what we are talking about is the extraction of the values, hopes and beliefs of hundreds of thousands -or even millions- of users...and in descending order. Once a first run is performed and clusters are extracted one could run this process again every month and see the trends of those clusters in time. It would be also interesting to see how these thought clusters change after specific World events.

For some people such as marketeers and social researchers -providing that results are accurate enough- this information is invaluable. Others, might feel that such an analysis is bad practice. Of course, there are companies that already capture brand sentiment across the web : Crimson Hexagon and Twitrratr are just two examples.


This post is the first in a series of posts discussing the application of Analytics to capture the thoughts that -as we speak now- exist on the Web. We will go through ways that one could explore this information and more specifically we will look at :

• How clustering can group people's values, beliefs and emotions.

• Why Ontologies and Natural Language Processing are needed for better results.

• How classification analysis might give us knowledge on what are the common characteristics of various 'categories' of users.
  

When Telecom customers complain-Pt. 2

On the previous post i explained the first steps in deploying Information Extraction, Text Mining and Computational Linguistics to capture the essence of Telecom customers complaints.

We have already discussed about the big picture : Retrieve data (essentially user messages from forums) and then use Information Extraction to transform unstructured information to a structured form. This transformation is done by building a set of matching rules for specific phrases or keywords such as

-signal
-antenna
-customer care

and words of sentiment such as

-worse
-worst
-better
-best
-outraged

among many others.


So here comes the interesting part : Suppose that a telecom company has in its possession an application that is able to search and extract sentiment from unstructured information. Having such a tool means that :

• A user can query directly on user forums for -example- specific network problems and break down those problems by area name.

• A user can directly query for hot phrases such as "canceling my subscription" and cluster keywords around those messages. If the telecom company is also running (and most likely it is running) churn prediction models, then analysts have yet another source to cross-check and/or enhance the conclusions of their churning models with this new information.
  

• Special matching rules can be applied to extract why users prefer company XYZ over company ABC.

• This technology can be applied to e-mails and/or free text complaints to the customer care center, which means that analysts can further enhance their churning models with additional data.

• Matching rules can be built that associate keywords to Telecom companies in terms of their co-occurrence. So telecom company XYZ has the phrase "good signal" associated with its brand whilst company ABC has the phrase "bargain" as the associated keyword.

• Match billing plan keywords and then cluster them with sentiment keywords. In other words, how do customers perceived the new billing plan and what is the sentiment about it?

It is easy to realize that Information Extraction combined with Text Mining and linguistics is a powerful combination that can extract many "knowledge nuggets". The fact that such an application cannot be 100% accurate may arise acceptance problems but its sure worth the effort in the end if potential problems are clearly presented before implementation of this application.

Let us not forget that a complaint given by a customer to the customer center remains there - between the boundaries of the company. A complaint posted on a forum can be seen by hundreds of thousands of others (and it will most likely stay there for a long time ) ,influencing potential and existing customers in a non-positive way.

A Sentiment analysis application may be also used for :

• Banking

• Pharmaceuticals

• Insurance
  

• Consumer Products (Customer Reviews)

.
and of course for capturing the sentiment of citizens for politicians (...)

When Telecom customers complain

Probably one of the best uses of Information Extraction, Text mining and Computational Linguistics combined together, is their ability to show us the sentiment of customers. Today we are going to see an example for capturing the sentiment of Telecom customers.

When a customer writes his/her opinion on a forum, a wealth of information is generated because -more importantly- a customer uses words and phrases that cannot be found during a controlled study. The words, phrases and expressions are far more emotionally powerful than a Likert scale answer of type "Totally disagree / agree".

So let us see the steps required :


First Step : The first thing of course is to actually find the data : User forums where people talk about mobile phones and mobile companies is obviously the place to look and there are lots of those places. Perhaps the volume of the messages is not enough but usually the available information is more than enough. Special code can be written to extract text from posts but without loss of the nature of the posting. As an example, the fact that a post has generated 20 replies is considered valuable information. The more posted replies, the more sentiment exists and this information has to be taken into consideration.

Second Step : Deploy information extraction techniques to identify phrases of good or bad sentiment (and actually many other things) about Telecom keywords such as :

- Signal
- Customer Care
- Billing

....etc

The following screen capture shows an example which is in Greek but i will provide all necessary explanation - Please also note that this is a simplified version of the process :





Notice that on the right hand-side there are some bars that denote the type of keywords found : The first category is called "Characterization" and if it is checked (which on the above screen capture it is) the software will highlight posts that only have some kind of characterization, whether good or bad. Notice also the yellow bar which has the name "Network". Because it is checked, words that are synonyms of "Network" are highlighted and indeed this is the case because

Signal = σήμα (in Greek) and
Flawless = άψογο

so the highlighted phrase άψογο σήμα means "flawless signal", which is a good characterization for the signal of two particular telecom companies. Notice also a line under the "Features" tab which says that between positions 3425 to 3429 there is a mention about signal ("mentionsSignal = true").

Again, i have to point out that this is a simplified version of the process. Text Mining and Information Extraction is actually very hard work but it is also very rewarding for those that ultimately deploy and use it. On the next post we will see the problems (and there are many of them) but also how this unstructured information is turned to "nuggets of gold".