Warning
This page is located in archive.

Assignment #3, Learning with Bayesian Networks

Evaluation of student networks:

The networks were evaluated by 4 distinct criteria:

  1. loglik – the value of log likelihood for 10000 test samples generated by the original network,
  2. size – the number of network parameters,
  3. totVarDist – the direct model match computed as the Manhattan distance of the original and the learned joint probability distribution (http://en.wikipedia.org/wiki/Total_variation_distance_of_probability_measures),
  4. KLdiv – the direct model match computed as the Kullback-Leibler divergence of the original and the learned joint probability distribution (http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence).

The first score shall be maximized, the other scores shall be minimized. In order to obtain the scale, your results are compared with the original network (denoted as orig) and a random network (denoted as random, its structure was learned by K2 having a random variable order with an extremely small sample size, its parameters were randomized). On top of that, there are two straightforwardly learned networks K2Random (K2 algorithm, a random order of variables, properly learned parameters) and K2correct (K2 algorithm, the correct order of variables, properly learned parameters).

I ranked your solutions in descending order, I used a weighted average of loglik, size and totVarDist. The orig network (almost necessarily) wins. At the same time, there are at least 6 proper solutions that approach the original. Other 10 solutions can be concerned as very good. The remaining models score poorly. On one hand, the cause can be a minor inattention, on the other hand, you were supposed to check before submission.

Notes:

  1. Do not take the ranking absolutely, the differences are small in many places. Small perturbations in criteria weighting or the sample set can result in rank changes.
  2. The models with good loglik and poor totVarDist and KLdiv suffer from the variable sort that is not topological (the model occasionally works, nevertheless, the inference engine used to construct the joint probability table failed).
  3. KL divergence was not employed in the final score. First, it strongly correlates with topVarDist. Second, it assigns Inf to all the models that set any of its parameters to 0. It is a reminder that priors shall always be used, but we did not deal with them explicitly in labs.
name loglik size totVarDist KLdiv
orig -76509 72 6.6527E-016 0
pytelma1 -76574 87 0.040128 0.0067762
serycjon -76593 67 0.040122 0.0080113
futscdav -76602 52 0.043166 0.0088752
amricjon -76593 59 0.045995 0.007743
novakli2 -76611 65 0.046004 0.009775
shchegal -76617 64 0.0489 0.0108
gamecjan -76621 53 0.049342 0.010975
rusnaras -76635 64 0.0506 0.0125
slunedan -76592 72 0.0514 0.0091
K2Random -76524 105 0.056714 Inf
marekp14 -76642 53 0.0609 0.0136
vaclajon -77354 86 0.054565 Inf
vavrimat -76689 51 0.0703 0.0176
munchmar -76781 52 0.069711 0.022822
rakadam -76714 46 0.0723 0.0188
ondralad -76786 55 0.072722 0.024152
vyskoto4 -76724 64 0.0784 0.0216
fadrhmar -76815 67 0.086512 0.0254
chmelma3 -79548 71 0.0673 Inf
kasimyur -76922 71 0.10211 0.041701
K2Correct -76953 45 0.10673 0.036691
jencidus -82916 83 0.050769 Inf
kryskmar -77091 56 0.1277 0.0576
pavlisim -77124 47 0.1275 0.0592
malyja16 -77111 61 0.1279 0.0583
ciefomar -77193 67 0.13352 0.068011
jandoka3 -77173 49 0.13429 0.060934
staryja3 -77080 60 0.1359 0.0583
drykjan -77173 61 0.13594 0.064542
vsetepet -77292 84 0.143 0.0764
svachmic -82467 80 0.1651 Inf
chamrtom -78631 410 0.24552 0.21754
dolezmat -79879 114 0.3233 0.3271
dundrja1 -81260 269 0.3865 0.47281
makaral1 -82886 57 0.6499 1.4953
random -130513 85 0.85484 5.403
kuskapet -131449 487 0.85963 5.4962
kruzimic -211340 639 0.13216 Inf
nelibjir -128649 181 0.8969 5.5048
kruzimic -211340 639 0.1322 Inf
horenmar -143239 767 0.1766 Inf
kubesda1 -247375 735 0.1071 Inf
linkaja1 -8827683 54 0.87177 Inf
rindtedu -13051543 76 0.59169 Inf
courses/ae4m33rzn/gpm_eval.txt · Last modified: 2016/02/08 16:52 by klema