Data Mining with Artificial Neural Networks (ANN)

In this post, I will show how artificial neural networks can be used as a technique for data mining and discovering correlations in seemingly random data sets. To provide a context, I will be conducting a fictitious experiment on lunar fruit bats…

On Moon Base 6, Dr Notion has been assisting a team of exobiologists in performing a study on the recently discovered lunar fruit bat. What puzzles them is that whilst the bats are genetically almost identical, their life spans vary enormously. Adding to the mystery, these strange little creatures eat only once in their short life, so the scientists believe there is a connection between their diet and lifespan.

In a controlled experiment, fifty fruit bats were allowed to eat up to one gram from each of four different fruits. The scientists then observed how long the bats lived after eating. Some dropped dead almost immediately, so clearly something was poisoning them, whilst others lived for nearly a full lunar month. The graph below, sorted by increasing longevity, shows how diet relates to life span.



There appears to be no correlation at all.

Undeterred, Dr Notion decided he would try feeding the data into my neural network published last week. He pasted the experiment’s data set (below) into the patterns.csv file and set the _ inputDims variable to a value of four, representing the four fruits from the experiment.

He ran the program and, almost immediately, the network found a solution, proving there was a correlation between fruit type and life span.

Dr Notion decided to write some code so that he could inspect the value of the weights after training. He noted his observations on the network diagram below.



Using these values, he wrote down the equation of the network, where f is the activation function, and discarding any values approaching zero, which gave:

o = f(1.15 * f(0.87b + 0.47c -0.88d) + 1.30 * f(-0.62b + 0.50c + 0.64d) – 2.00)

He noted that the proportion and sign of the weights were more important than their absolute values, due to the distorting effect of the sigmoid activation function. Therefore, three important features stood out:

• Fruit A did not appear, so has no bearing on life span.
  
• Fruit C appears with positive values at each hidden neuron, so has a positive correlation with life span.
  
• Fruits B and D appear in equal and opposite amounts at each hidden neuron, and always with a value larger than fruit C. This meant that similar proportions of fruit B and D had to be eaten otherwise they would be poisonous. They were mutual antidotes.

Armed with this information, Dr Notion theorised that the relation between fruit type and life span could be stated:

Life Span = C - |B-D|

Alternatively:

Life Span = C - ABS(B-D)

He tested his theory by substituting the values from the experiment’s data set into the equation, and Voilà, it matched. Advice for a lunar fruit bat – eat a diet consisting of just fruit C!

Dr Neuron and the exobiologists were happy and went to the pub…

Data set from the experiment:

Fruit A,Fruit B,Fruit C,Fruit D,Life
0.265,0.914,0.262,0.446,0.000
0.232,0.761,0.205,0.371,0.000
0.481,0.011,0.730,0.774,0.000
0.700,0.812,0.042,0.864,0.000
0.149,0.383,0.190,0.022,0.000
0.462,0.452,0.359,0.067,0.000
0.673,0.939,0.898,0.010,0.000
0.971,0.193,0.135,0.993,0.000
0.217,0.163,0.434,0.874,0.000
0.764,0.594,0.074,0.075,0.000
0.197,0.969,0.607,0.201,0.000
0.749,0.309,0.407,0.647,0.070
0.701,0.594,0.186,0.488,0.080
0.685,0.540,0.460,0.167,0.086
0.205,0.275,0.272,0.447,0.099
0.625,0.007,0.251,0.154,0.105
0.238,0.896,0.542,0.476,0.122
0.641,0.402,0.429,0.681,0.150
0.747,0.758,0.725,0.227,0.194
0.656,0.552,0.446,0.790,0.208
0.305,0.831,0.248,0.801,0.219
0.585,0.130,0.389,0.293,0.227
0.695,0.518,0.515,0.237,0.234
0.070,0.512,0.473,0.739,0.246
0.174,0.608,0.810,0.051,0.253
0.845,0.084,0.513,0.316,0.281
0.898,0.676,0.950,0.018,0.292
0.547,0.244,0.606,0.538,0.312
0.040,0.378,0.426,0.490,0.315
0.941,0.663,0.589,0.932,0.320
0.347,0.723,0.495,0.571,0.343
0.213,0.831,0.576,0.646,0.391
0.235,0.127,0.945,0.673,0.399
0.484,0.320,0.832,0.708,0.445
0.873,0.676,0.732,0.948,0.460
0.808,0.551,0.564,0.630,0.485
0.734,0.446,0.666,0.299,0.519
0.911,0.596,0.719,0.405,0.529
0.334,0.997,0.620,0.928,0.551
0.087,0.503,0.585,0.514,0.573
0.614,0.635,0.933,0.316,0.614
0.797,0.340,0.862,0.575,0.626
0.785,0.572,0.931,0.866,0.637
0.090,0.423,0.771,0.304,0.652
0.525,0.075,0.897,0.281,0.691
0.488,0.569,0.780,0.518,0.729
0.779,0.365,0.793,0.340,0.768
0.886,0.742,0.928,0.618,0.804
0.059,0.496,0.900,0.556,0.840
0.151,0.939,0.997,0.999,0.967