Sequence Data Mining for Health Applications

An often overlooked type of Analysis is  Sequence Data Mining (or Sequential Pattern Mining).


Sequence Data Mining is a type of Analysis which aims in extracting patterns sequences of  Events. We can also see Sequence Data Mining as an Associations Discovery Analysis with a Temporal Element.

Sequence Data Mining has many potential applications (Web Page Analytics, Complaint Events, Business Processes) but here today we will show an application for Health. I believe that this type of Analysis will become even more important as wearable technology will be used even more and therefore more Data of this kind will be generated.

Consider the following hypothetical scenario : 

A 30-year old Male patient complaints about several symptoms which -for simplicity reasons- we will name them as Symptom1, Symptom2, Symptom3,etc.

His Doctor tries to identify what is going on and after the patient takes all necessary Blood work and finds no problems. After thorough evaluation the Doctor believes that his patient suffers from Chronic Fatigue Syndrome. Under the Doctor's supervision the patient will record his symptoms along with different supplements to understand more about his condition. Several events (e.g a Visit to the Gym, a stressful Event) will also be taken under consideration to see if any patterns emerge.

-How Can we easily record Data for the scenario above?

-Can we extract sequences of events that occur more frequently than mere chance?

-Can we identify which sequences of Events / Food / Medication may potentially lead to specific Symptoms or to a lack of Symptoms?

Looking the problem through the eyes of a Data Scientist, We have :

A series of Events that happen during a day : A Stressful event, A sedentary day, Cardio workouts, Weight Lifting, Abrupt Weather Deterioration, etc

A Number of Symptoms : Headaches, "Brain Fog", Mood problems, Insomnia, Arthralgia, etc.

Let's begin with Data Collection. We first suggest to the patient to use an Android app called MyLogsPro (or some other equivalent application) to easily input information as this happens :

  

So if the patient feels a specific Symptom he will press the relevant Symptom button on his  mobile device. The same applies for any events that have happened and any Food or Medication taken. As the day passes we have the following data collected :

The snapshot shows what happened starting on the 20th of August 2014, where our patient has logged the intake of Medication (at 08:22 AM) and/or Supplements upon waking up then a Food entry was added at 08:47. At 11:06 the patient had a Symptom and immediately reached his phone and pressed the relevant Symptom (Symptom No 4) button.

After many days of Data Collection we decide that its time to analyze this information. We export the data from the application as a csv file which looks as follows :

We will use KNIME to read the csv file, change the contents of the entries accordingly so that an Algorithm can read the events and then perform Sequence Data Mining. We have the following layout :

 The File Reader reads the .csv file, then during the Pre-processing block (shown in yellow), a String Manipulation node which removes colon (:) from time field (e.g 12:10 becomes 1210). The Sorter sorts the data according to date then time as the second field and a Java snippet uses replaceAll() function to remove all leading zeros from Time field (e.g 0010 becomes 10).

The R Snippet loads the CSPADE Algorithm and then uses this Algorithm to extract pattern of sequences.

After executing the stream we get the following output :

The information consists of two outputs : The first one is a list of sequences along with their support and the second one contains the output from rule induction which gives us two more useful metrics (namely the lift and the confidence for each rule).

We immediately notice an interesting entry on the first output :

Medication1->Symptom2

and on the second output we see that this particular rule has a lift of 1.4 and 0.8 confidence.

However, as Data Scientists we should always double-check the extracted knowledge and must be aware of pitfalls. Let's see some examples (list not exhaustive) :

1) The algorithm does not account for time as it should : As an example, consider the following entries :

10/09/14,08:00,Medication1
10/09/14,08:05,Symptom2

We assume that Medication1 is taken by mouth and needs 60 minutes to be properly dissolved and that these entries occur frequently enough in that order in our data set. Even though the algorithm might show a statistically significant pattern , it is not logical to hypothesize that Medication1 could be related to Symptom2. The Analyst should first examine each of these entries to see which proportion of the records has a time difference of at least -say- or greater than 60 minutes.

Apart from the example shown above we must consider the opposite effect. Consider this entry :

10/09/14,08:00,Medication1
...
...
...
10/09/14,21:05,Symptom2

In other words : Is it possible that a Medication taken in the morning to generate a Symptom 12 hours later?


2) The algorithm is not able to account for the compounding effect of a Medication. For example, the patient might have low levels of Taurine and for this level to be replenished, an x amount of days of Taurine supplementation is needed. The algorithm cannot account for this possibility.


 3) The patient should also input entries of "No Symptoms". It is not clear however when this should be done (e.g at the end of each day? assess every 6 hours and add 2 entries accordingly?)


However, this does not mean that a Sequence Mining algorithm should not be used under these circumstances. This technique can generate several potentially interesting hypotheses which Doctors and/or Researchers may wish to pursue further.
 

Venture Capitals in an Age of Algorithms (Revisited)

Some time ago i wanted to explore the idea of analyzing several kinds and sources of Information (e.g TechCrunch, TheNextWeb,  News sites and Twitter) to identify promising Investment opportunities in Technology and more specifically Startups. 

Here is a snapshot of a Webpage from TechCrunch :

In many posts in this Blog it was discussed how our Reactions for almost any kind of information are recorded. This was not possible when everyone was reading newspapers in its paper form whereas  now any kind of Text is associated with a number of Views, Re-Tweets, Number of Comments and FaceBook "Likes".

The second important information that is being generated is our Emotions for any Topic as these are expressed within Comments, Twitter and FaceBook posts. The intensity of our emotions is also captured and this information is very important since  whatever we associate with intense emotions really stays within our psyche, fuels our interest and (usually) drives our purchase decisions.

We may then continue with some Exploratory work as follows : We can collect Posts from various Tech sources and their associated Reactions, annotate the text with Sentiment, Events and Topics and analyze this information to understand which Topics and/or Events appear to have an affinity for a high number of Reactions or High Sentiment intensity for Startups or Tech Topics .

As an example, 10K posts from various Tech sources were collected and each one of the posts was marked as generating either HIGH or LOW interest based on the amount of Reactions (Re-Tweets, FB Likes, Comments)  that each post generated. Special filtering is applied for the frequencies of the words that appear in each post :

Then this information is fed to KNIME for further analysis. The implementation which  is shown here is rather naive and simplistic for many reasons :  Only keywords are used as input -as opposed to Topics, Events- and many other parameters that are involved and which will be discussed later but for our example we will keep things simple.

The workflow uses 3 algorithms namely PART (so that some rules are generated), SMO and Random Forests :

 
This -again- is a very naive approach which gave a result of 61.9% (F-Measure) in identifying keywords that commonly appear with posts that generate Interest vs posts that do not. We keep in mind that this knowledge alone is not enough with which a decision can be made but we decide to explore things a little further.

We may find that some words that we expected do appear in posts of High interest (such as Google, Apple, Pinterest). There could be however some words that deserve more of our attention such as Education and Schools which during the analysis appeared to exist more frequently in High Interest posts.

So how can this information be used for a potential investment on a Startup and is there really a way to model new ideas and predict their performance?  Again, it is not suggested here that if you come across a startup aimed in Education you should immediately put your money in but this observation could be one parameter to consider. There are so many other considerations such as whether the idea is novel or not, how many competitors exist, who are the people behind the Startup, whether its founders have created a successful Startup in the past, which people have already invested in the particular Startup, what is the "buzz" that this Startup has generated so far and so on.

Whenever we read about a new startup there are some immediate thoughts going through our minds : Does this sound like a good idea? Is it applicable to me and would it make my life easier? Is this idea truly disruptive or not? What does our "gut feeling" tells us?  

We should always keep in mind that there are limitations to what Predictive Analytics can do but perhaps we can extract some hints that we may then use to make better decisions.

It was also interesting to read this post (hence the use of word "Revisited" in this Post's Title) on Gigaom regarding the same Subject. This is a fascinating area that i started looking at and there will be similar posts in the future on this Subject.

Personal Data Mining - (Part 2)

On the previous post i described the way that i used to capture a 1-year worth of personal data using my Smartphone with the purpose of identifying trends with my immune system which at times gave me perennial conjunctivitis and also swollen lymph nodes. Now it was time to analyze all of this data in hope that some useful knowledge could be found.

 I had to make a decision for which tools to use. I used WEKA and also decided to give KNIME a try so here is an example of a KNIME workflow :

I first use the File Reader to read in my 1-year worth of life data, then an R Script which is used for several data transformations. I then send the streams of Data first to an R Script which runs the FSelector package with which several Feature Selection algorithms (about 10 of them) are applied to get an understanding of what are the important Features for the problem at hand.

Then another stream sends the Data to an R node which creates dummy Variables and then sends the transformed Features to a Linear Correlation node for further inspection.

A third stream (not shown) sends the data to 3 Machine-Learning algorithms (namely an SVM, Decision Tree and Random Forest) and the Scorer shows how each algorithm performed.

I first executed the FSelector node using 10-fold Cross-Validation because i wanted to get a first feel of the features that are important in identifying some patterns about my perennial conjunctivitis.  7 out of 10 of  FSelector algorithms agreed that :

1) Vitamin D3
2) Garlic
3) Yoghurt

..Appear to have the most predictive power. The problem is that at this point we do not know if any of the features actually help or aggravate my condition. However, the output of FSelector gives an idea on which features should be looked at more closely.

Then the second stream was run, namely the one which sends the data to 3 machine learning algorithms so that i could get a first feel of how the algorithms perform. All three algorithms gave an F-Score of around  59 - 62%.

By looking at the results some patterns appeared to arise (Note the word "appeared")

1) A rather large daily dose  (>5200 IU) of Vitamin D3 appears to be associated with smaller incidences of conjunctivitis
2) Garlic consumption appears to increase my conjunctivitis incidences.
3) Yoghurt consumption appears to increase my conjunctivitis incidences.


For Pattern (1) we need to be aware that Vitamin D3 dosage has a compounding effect so it is rather naive to think that boolean logic applies (see previous post for more).

Next i had to look at patterns (2) and (3). One of the things that i realized when searching the web for the effect of various nutrients in functions of the human body is the fact that you can find for any several entries that some times contradict  each other. My very brief web search has found Garlic and Yoghurt to be "immune boosters". Of course caution should be exercised in drawing any conclusions because of the way the data have been collected and also the problematic origin of the analysis. Moreover, i am not a doctor and i cannot possibly know whether Garlic or Yoghurt can aggravate an immune response in such a way. 

 I began taking Vitamin D3 and eliminating Garlic and Yoghurt from my diet. The result was that over a period of one month i stopped getting bouts of conjunctivitis and incidences of swollen lymph nodes. So has Vitamin D3 acted as an "immune response regulator" and Garlic - Yoghurt as "immune boosters"?

Although my bouts of conjunctivitis have ceased, I am not in any position to make any claims because there are a lot of uncontrolled variables :

- It could be a placebo effect.
-There may be unknown hidden variables that are important
- (My) Genetics
- Environment
- Variations in Dosage and Nutrient Content
- Interactions between nutrients

and lots of others that could not possibly be accounted for under these circumstances.

What i can say (and this is the reason for writing this post) is that analytics may help us to identify several patterns that may then be used to guide a sound knowledge discovery process. If people had the ability to collect data on a daily basis (see Quantified Self) and then analyze them on a massive scale, several unknown patterns that call for closer investigation could emerge.