jeff bilmes mywktqcubaeewashingtonedu call participation evaluation probabilistic inference systems 22nd conference uncertainty artificial intelligence uai july 13th july 16th 2006 massachusetts institute technology cambridge usa httpsslieewashingtonedubilmesuai06inferenceevaluation years many different algorithms making queries probability distributions represented graphical models developed number algorithms exact inference eg junctiontree based schemes using graph triangulations search conditioning based approaches based satcsp techniques approximation methods including variational generalized belief propagation sampling schemes also aspects software implementation relevant data structures memory management subsidiary algorithms significant effect actual runtime purpose evaluation compare performance variety different software systems single set bayesian network bn problems number different bayesian networks made available participant evaluated according following 1 exact inference approximate inference crucial many problem domains recent results shown exact inference still performed variety interesting thought previously intractable problems therefore first uai competition involve systems perform exact inference meaning everyones probability score numerically identical 2 evaluation involve wallclock run time amount memory used fastest time memory ceiling smallest memory time limit evaluated 3 several forms query possible evaluate including probabilityevidence ie summing hidden variables mpe map resulting scoresassignments queries must correct accurate 4 set complete bayesian network models made available bns strictly positive meaning zeros cpts bns mixtures dense sparse deterministic cpts 5 evaluate single threaded performance ie multithreading parallelism first evaluation use underlying intelclass microprocessor running linux operating system systems process 2gb memory limitation 6 two types bns evaluated static dynamic dynamic bayesian networks placed standard easytoread format determined encourage participant interested party submit bn considered evaluation make guarantees particular bn used resulting bns used competition results listed uai web competition encourage continued standardized evaluation important dates 41506 deadline date express intent participate also indicate previdence mpe andor map wish evaluate 42206 deadline date submit bn query type consideration participation submissions necessarily used 5106 date final list bns format timing details made available 5153006 evaluation period time participants given access compute server may compile evaluate system 53006 final results due along verification script 71406 evaluation results announced uai06 updated information please visit httpsslieewashingtonedubilmesuai06inferenceevaluation wed 06 aug 2008 183207 0800 uai call participation evaluation probabilistic inference systems uai06