BayesianFrequentistvs.vs.Frequentist（頻率主義者）:概率是長期的預期出現頻率P(A)=n/N,wherenisthenumberoftimeseventAoccursinNopportunities.“某事發生的概率是0.10.1是在無窮多樣本的極限發生第三次世界大戰的概率是多少Bayesian:degreeofbelief.Itisameasureoftheplausibility（似然性）ofaneventgivenincompleteProbabilityProbabilityisarigorousformalismforuncertainJointprobabilitydistributionspecifiesprobabilityofeveryatomicQueriescanbeansweredbysummingoveratomicFornontrivialdomains,wemustfindawaytoreducethejointsizeIndependenceandconditionalindependenceprovidethetools/ConditionalAandBareindependentP(A|B)= orP(B|A)= orP(A,B)P(A)AisconditionallyindependentofBgivenP(A|B,C)=P(A|ConditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainProbabilityProbabilitytheorycanbeexpressedintermsoftwosimpleequations概率理論可使用兩個簡單線性方程來SumRule（加法規則ProductRule（乘法規則Graphicalmodels（概率圖模型BayesianInference（推導）inBayesianAlsocalledTheyaugmentanalysisinsteadofusingWhatisaConsistsofnodes(alsocalledvertices)andlinks(alsocallededgesorarcs)每個節點表示一個隨機變量(or一組隨機變GraphicalModelsinacomplexsystemisbuiltbycombiningsimplerparts.WhyareGraphicalModels使每個部分連接起來確保系統作為一個整體是一提供模型到數據的連接方法圖理論方面提供bywhichhumanscanmodelhighly-interactingsetsofvariablesthatlendsitselfnaturallytodesigningefficientgeneral-purpose（通用的）algorithmsGraphicalmodels:mixturemodels（混合模型）factoranalysis（因子分析）,hiddenMarkovmodels,Kalmanfilters（卡爾曼濾波器）,etc.優勢ProvidesnaturalframeworkfordesigningInsightsintopropertiesofmodelConditionalindependencepropertiesbyinspectinggraph

MorepopularinAI

Markovrandom（馬爾科夫隨機場MorepopularinVisionandphysicsBayesianasetofnodes,oneperadirected（有向）acyclic（無環）graph(link"directaconditionaldistributionforeachnodegivenitsP(Xi|Parents(Xi))—Inthesimplestcaseconditionaldistributionrepresentedasaconditionalprobabilitytable條件概率表(CPT)givingthedistributionoverXiforeachcombinationofparentvaluesTopology（拓撲結構）ofnetworkencodesconditionalindependenceassertions:Weather獨立于其他變量ToothacheandCatchareconditionallyindependentgivenCavityVariablesBurglary（入室行竊）Alarm,JohnCalls,網絡拓撲結構反映出因果關系AburglarcansetthealarmAnearthquakecansetthealarmThealarmcancauseMarytoThealarmcancauseJohntoExampleCompactness（緊致性ACPTforBooleanXiwithkBooleanparentshas2krowsforthecombinationsofparentvaluesEachrowrequiresonenumberpforXi=(thenumberforXi=falseisjust1-Ifeachvariablehasnomorethankparents,thecompletenetworkrequiresO(n·2k)numbersI.e.,growslinearlywithn,vs.O(2n)forthefulljointdistributionForburglarynet,1+1+4+2+2=10numbers(vs.25-1=31)GlobalGlobalsemantics（全局語義Thefulljointdistributionisdefinedastheproductofthelocalconditionaldistributions:Thefulljointdistributionisdefinedastheproductofthelocalconditionaldistributions:LocalLocalsemantics:eachnodeisconditionallyindependentofitsnondescendants（非后代）givenitsparentsTheorem:Local globalCausalChainsIsXindependentofZgivenEvidencealongthechain“blocks”theCommonCause另一個基礎的形態:twoeffectsofthesamecauseAreXandZAreXandZindependentgivenObservingthecauseblocksinfluencebetweeneffects.CommonEffect最后一種配置形態twocausesofoneeffect(v-structures)AreXandZYes:remembertheballgameandtheraincausingtraffic,nocorrelation?AreXandZindependentgivenNo:rememberthatseeingtrafficputtherainandtheballgameincompetition?ThisisbackwardsfromtheotherObservingtheeffectenablesinfluence NeedamethodsuchthataseriesoflocallytestableassertionsofconditionalindependenceguaranteestherequiredglobalChooseanorderingofvariablesX1,…Fori=1toaddXitotheselectparentsfromX1,…,Xi-1suchP(Xi|Parents(Xi))=P(Xi|X1,...Xi-該父親選擇保證了全局語義ExampleExample(CausalmodelsandconditionalindependenceseemhardwiredforNetworkislesscompact:1+2+4+2+4=13numbers因果關系當貝葉斯網絡反映真正的因果模式時Oftensimpler(nodeshavefewerOfteneasiertothinkOfteneasiertoelicitfromexperts（專家BNs有時無因果關系的網絡是存在的(especiallyifvariablesaremissing)箭頭的真正含義是什么TopologymayhappentoencodecausalTopologyreallyencodesconditionalInferenceinBayesian簡單查詢計算后驗概率e.g.P(NoGas|Gauge油表=empty聯合查詢P(Xi,Xj|E=e)=P(Xi|E=e)P(Xj|最優決策:decisionnetworksincludeutilityinformationprobabilisticinferencerequiredforP(outcome|action,evidence)EvaluationVariableelimination（變量消元）:carryoutsummationsright-to-leftstoringintermediateresults(factors：因子)toavoidSinglyconnectednetworks單聯通網絡(orpolytrees多樹anytwonodesareconnectedbyatmostone(undirected)timeandspacecostofvariableeliminationareMultiplyconnectednetworks多聯通網絡canreduce3SATtoexactinference?NP-equivalenttocounting3SATmodels?#P-Example:Na?veBayesNa?veBayesTotalnumberofparameters（參數）islinearinExample:Example:一個簡單些簡單的特征來嘗試識別垃圾郵件.我們先考慮兩CapsFreee.g.:amessagewiththesubjectheader“NEWMORTGAGERATE“islikelytobespam.Similarly,for“MoneyforFree”,“FREElunch”,etc.模型的構建基于以下三個隨機變量Caps,FreeandSpam,eachofwhichtakeonthevaluesY(forYes)orN(forNo)Caps=YifandonlyifthesubjectofthemessagedoescontainlowercaseFree=Yifandonlyiftheword`free'appearsinthe(lettercaseisSpam=YifandonlyifthemessageisP(Free,Caps,Spam)=P(Spam)P(Caps|Spam)P(Free,Caps,Spam)=P(Spam)P(Caps|P(Free|Example:Example:Learningtoclassify模型包含先驗概率P(Category)P(wordP(Category=c)isestimatedasthefractionofalldocumentsthatareofcategorycP(wordi=true|Category=c)isestimatedasthefractionofdocumentsofcategorycthatcontainwordiTwentyGiven1000trainingdocumentsfromeachgroup.LearntoclassifynewdocumentsaccordingtowhichnewsgroupitcamefromNa?veBayes:89%classificationLearningCurvefor20Example:ADigitNa?veBayesfor簡單版本一種特征ijforeachgridposition可能的特征值是onoff基于圖像中像素的亮度是否大于或小每一個輸入映射到一個特征向量Here:lotsoffeatures,eachisbinaryNa?veBayes