手寫數字識別文本分類類圖像分割割第八章Uncertainty不確定性性對應教材材第13章本章大綱綱Uncertainty不確定性性Probability概率SyntaxandSemantics語法與語語義Inference推理IndependenceandBayes‘Rule

—獨立性及及貝葉斯斯法則不確定性性智能體幾幾乎從來來無法了了解關于于其環境境的全部部事實。。因此其其必須在在不確定定的環境境下行動動。概率推理理得到了某某一證據據，那么么有多大大的幾率率結論為為真？例如：我我頸部痛痛;我得腦膜膜炎的可可能有多多大?不確定性性假如有如如下規則則:iftoothache（牙疼））then原因是cavity（牙齒有有洞）但并不是是所有牙牙疼的病病人都是是因為牙牙齒有洞洞，所以以我們可可以建立立如下規規則:iftoothacheand?gum--disease（牙齦疾疾病）and?filling（補牙））and.....thenproblem==cavity以上規則則是復雜雜的;更好的方方法:iftoothachethenproblemiscavitywith0..8probabilityorP(cavity||toothache)==0.8theprobabilityofcavityis0..8giventoothacheisobserved不確定性性LetactionAt=離起飛時時間提前前t分鐘動身身去機場場At會使我準準時到達達機場嗎嗎?Problems:1.partialobservability/部分可觀觀察性(roadstate,otherdrivers‘plans))

2..noisysensors((trafficreports)3.行動結果果的不確確定性(flattire,etc..)4.immensecomplexityofmodelingandpredictingtraffic因此一個個純粹的的邏輯描描述方法法：1.risksfalsehood（錯誤風風險）:“A25willgetmethereontime”,or2.leadstoconclusionsthataretooweakfordecisionmaking:““A25willgetmethereontimeifthere’snoaccidentonthebridgeanditdoesn‘‘trainandmytiresremainintactetcetc..”(A1440mightreasonablybesaidtogetmethereontimebutI’’dhavetostayovernightintheairport…))世界與模模型中的的不確定定性Trueuncertainty:rulesareprobabilisticinnature擲骰子，，拋硬幣幣惰性:把所有意意外的規規則都列列舉出來來是很困困難的花費太多多時間來來確定所所有的相相關因素素這些規則則過于繁繁雜而難難以使用用理論的無無知:某些領域域中還沒沒有完整整的理論論(e.g.,medicaldiagnosis)實踐的無無知:掌握了所所有規則則但是并不是所所有的相相關信息息都能被被收集到到處理不確確定性的的方法概率理論論作為一一種正式式的方法法for：不確定知知識的表表示和推推理命題中的的模型信信度(event,,conclusion,,diagnosis,etc.)給定可獲獲得的證證據,A25willgetmethereontimewithprobability0.04概率是不不確定性性的語言言現代AI的中心支支柱Probability概率概率理論論提供了了一種方方法以概概括來自自我們的的惰性和和無知的的不確定定性。ProbabilisticassertionssummarizeeffectsofLaziness（惰性））:failuretoenumerateexceptions（例外））,qualifications（條件））,etc.Ignorance（理論的的無知））:lackofrelevantfacts,,initialconditions,etc.Subjectiveprobability（主觀概率率）:

Probabilitiesrelatepropositions（命題））toagent'sownstateofknowledgee..g.,,P((A25|noreportedaccidents)==0.06Thesearenotassertions（斷言））abouttheworld命題的概概率隨著著新證據據的發現現而改變變:

e..g.,,P((A25|noreportedaccidents,5a.m..)==0..15不確定條條件下的的決策假設下述述概率是是真的:P(A25getsmethereontime||……)==0..04P(A90getsmethereontime||……)==0..70P(A120getsmethereontime||……)==0..95P(A1440getsmethereontime||……)==0..9999Whichactiontochoose?Dependsonmypreferences（偏好））formissingflightvs..timespentwaiting,etc.Utilitytheory（效用理理論）用來對偏偏好進行行表示和和推理Decisiontheory==probabilitytheory++utilitytheory決策理論論=概率理論論+效用理論論Syntax語法基本元素素:randomvariable（隨機變變量）Arandomvariableissomeaspectoftheworldaboutwhichwe((may)haveuncertainty通常大寫寫e.g..,Cavity,,Weather,,Temperature類似于命命題邏輯輯:未知世界界被隨機機變量的的賦值所所定義Booleanrandomvariables（布爾隨隨機變量量）e.g..,Cavity（牙洞））(doIhaveacavity?)Discreterandomvariables（離散隨隨機變量量）e.g..,Weatherisoneof<<sunny,rainy,,cloudy,snow>定義域mustbeexhaustive（窮盡的的）andmutuallyexclusive（互斥的的）Continuousrandomvariables（連續隨隨機變量量）e.g..,Temp=21.6;;alsoallow,,e..g.,,Temp<<22.0SyntaxElementaryproposition（命題））constructedbyassignmentofavaluetoarandomvariable::e..g.,,Weather=sunny,Cavity=false(簡寫為?cavity)Complexpropositionsformedfromelementarypropositionsandstandardlogicalconnectivese.g..,Weather=sunny∨Cavity=falseSyntaxAtomicevent:Acompletespecificationofthestateoftheworldaboutwhichtheagentisuncertain原子事件件：對智能能體無法法確定的的世界狀狀態的一一個完

整的詳細描描述。E.g..,iftheworldconsistsofonlytwoBooleanvariablesCavityandToothache,thenthereare4distinctatomicevents:Cavity==false∧Toothache==falseCavity==false∧Toothache==trueCavity==true∧Toothache==falseCavity==true∧Toothache==trueAtomiceventsaremutuallyexclusiveandexhaustive窮盡和互互斥概率公理理對任意命命題A,B0≤P(A)≤1P(true)=1andP(false)=0P(A∨B)=P(A)+P(B)-P(A∧B)Priorprobability（先驗概概率）Priororunconditionalprobabilities（無條件件概率））ofpropositions在沒有任任何其它它信息存存在的情情況下關關于命題題的信度度e.g..,P(Cavity=true))=0.1andP(Weather=sunny)==0..72correspondtobeliefpriortoarrivalofany((new)evidenceProbabilitydistributiongivesvaluesforallpossibleassignments:概率分布布給出一個個隨機變變量所有有可能取取值的概概率P(Weather)=<<0..72,,0.1,0..08,,0.1>((normalized（歸一化化的）,i..e.,,sumsto1)Jointprobabilitydistributionforasetofrandomvariablesgivestheprobabilityofeveryatomiceventonthoserandomvariables((i..e.,,everysamplepoint)聯合概率率分布給給出一個個隨機變變量集的的值的全全部組合合的概率率P(Weather,Cavity)=a4××2matrixofvalues:Everyquestionaboutadomaincanbeansweredbythejointdistributionbecauseeveryeventisasumofsamplepoints連續變量量的概率率Expressdistributionasaparameterized（參數化化的）functionofvalue:P(X==x))=U[18,26]](x))=uniform（均勻分分布）densitybetween18and26連續變量量的概率率MarginalDistributions（邊緣概概率分布布）Marginaldistributionsaresub--tableswhicheliminatevariablesMarginalization((summingout):CombinecollapsedrowsbyaddingConditionalprobability（條件概概率）Conditionalorposteriorprobabilities（后驗概概率）P(a||b)證據累積積過程的的形式化化和發現現新證據據后的概概率更新新當一個命命題為真真的條件件下，指指定命題題的概率率e.g..,P(cavity|toothache)=0.8

i..e.,,鑒于牙疼疼是已知知證據(Notationforconditionaldistributions（條件概概率分布布）:P(cavity|toothache)=asinglenumberP(Cavity,Toothache)==2x2tablesummingto1P(Cavity|Toothache)=2-elementvectorof2-elementvectorsIfweknowmore,,e..g.,,cavityisalsogiven,thenwehaveP(cavity|toothache,,cavity)=1新證據可可能是不不相關的的，可以以簡化,e..g.,,

P((cavity|toothache,,sunny)=P(cavity|toothache)=0.8條件概率率定義條件概率率為:

P((a||b))=P(a∧b)//P(b))ifP(b))>0Productrule（乘法規規則）givesanalternativeformulation:P(a∧b)==P(a||b)P(b))=P(b|a)P(a)Ageneralversionholdsforwholedistributions,e.g.,P(Weather,Cavity)=P(Weather|Cavity)P(Cavity)(Viewasasetof4×2equations,,notmatrixmultiplication))Chainrule（鏈式法法則）isderivedbysuccessiveapplicationofproductrule:條件概率率條件概率率跟標準準概率一一樣,forexample:0<==P((a||e))<==1conditionalprobabilitiesarebetween0and1inclusiveP(a1|e))+P(a2|e))+.....+P(ak|e))=1conditionalprobabilitiessumto1wherea1,…,,akareallvaluesinthedomainofrandomvariableAP(?a|e)==1-P(a|e)negationforconditionalprobabilities通過枚舉舉的推理理Startwiththejointprobabilitydistribution（全聯合合概率分分布）:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題題的概率率等于所所有當它它為真時時的原子子事件的的概率和和通過枚舉舉的推理理Startwiththejointprobabilitydistribution（全聯合合概率分分布）:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題題的概率率等于所所有當它它為真時時的原子子事件的的概率和和通過枚舉舉的推理理Startwiththejointprobabilitydistribution（全聯合合概率分分布）:Foranypropositionφ,sumtheatomiceventswhereitistrue:一個命題題的概率率等于所所有當它它為真時時的原子子事件的的概率和和通過枚舉舉的推理理Startwiththejointprobabilitydistribution（全聯合合概率分分布）:Normalization（歸一化化）Denominator（分母））canbeviewedasanormalizationconstantαP(Cavity|toothache)=αP(Cavity,,toothache)

=α[P(Cavity,,toothache,catch)+P(Cavity,,toothache,?catch)]==α[<<0.108,,0.016>>+<<0..012,0..064>]==αα<0.12,0..08>>=<<0..6,0.4>>Generalidea::computedistributiononqueryvariablebyfixingevidencevariables（證據變變量）andsummingoverhiddenvariables（未觀測測變量））通過枚舉舉的推理理Typically,,weareinterestedin

theposteriorjointdistributionofthequeryvariables（查詢變變量）Ygivenspecificvaluesefortheevidencevariables（證據變變量）ELetthehiddenvariables（未觀測測變量））beH=X-Y–EThentherequiredsummationofjointentriesisdonebysummingoutthehiddenvariables:P(Y|E=e)=αP(Y,E=e)=αΣhP(Y,E=e,H=h)ThetermsinthesummationarejointentriesbecauseY,EandHtogetherexhaustthesetofrandomvariables（Y,E,H構成了域域中所有有變量的的完整集集合）Obviousproblems:1.Worst-casetimecomplexityO(dn)wheredisthelargestarity2.SpacecomplexityO(dn)tostorethejointdistribution

3..HowtofindthenumbersforO(dn)entries?Independence（獨立性性）AandBareindependentiffP(A|B)=P(A)orP(B|A)=P(B)orP(A,B)==P(A)P(B)E.g::rollof2die::P(({1}},{3})==1/6**1/6=1/36P(Toothache,,Catch,Cavity,,Weather)=P(Toothache,,Catch,Cavity)P(Weather)32entriesreducedto12;;fornindependentbiasedcoins,,O(2n)→O(n))Absoluteindependencepowerfulbutrare絕對獨立立強大但但罕見Dentistry（牙科領領域）isalargefieldwithhundredsofvariables,noneofwhichareindependent.Whattodo?獨立的濫濫用天真的數數學笑話話：一個著名名統計學學家永遠遠不會坐坐飛機旅旅行,因為他研研究了航航空旅行行和估計計,任何給定定的航班班上有炸炸彈的可可能性是是一百萬萬分之一一,他不準備備接受這這些可能能性。有一天，，一位同同時在遠遠離家鄉鄉的會議議上遇到到他。““你怎么么到這里里的？坐坐火車嗎嗎？”“不，我飛飛過來的的”“Whataboutthepossibilityofabomb?”“Well,Ibeganthinkingthatiftheoddsofonebombare1:million,,thentheoddsoftwobombsare((1/1,000,000))x((1//1,000,,000).Thisisavery,,verysmallprobability,,whichIcanaccept.SonowIbringmyownbombalong!!”Conditionalindependence條件獨立立性Randomvariablescanbedependent,,butconditionallyindependentExample:YourhousehasanalarmNeighborJohnwillcallwhenhehearsthealarmNeighborMarywillcallwhenshehearsthealarmAssumeJohnandMarydon’ttalktoeachotherIsJohnCallindependentofMaryCall?No–IfJohncalled,itislikelythealarmwentoff,whichincreasestheprobabilityofMarycallingP(MaryCall|JohnCall))≠P(MaryCall)條件獨立立性But,,ifweknowthestatusofthealarm,JohnCallwillnotaffectwhetherornotMarycallsP(MaryCall|Alarm,,JohnCall)==P((MaryCall||Alarm)WesayJohnCallandMaryCallareconditionally

independentgivenAlarmIngeneral,““AandBareconditionallyindependentgivenC””means:P(A||B,C)==P((A||C))P(B||A,C)==P((B||C))P(A,,B||C)==P((A||C))P((B||C))條件獨立立性P(Toothache,,Cavity,Catch)has23-1=7independententries專業領域域知識:Cavitydirectlycausestoothacheandprobe--catches.IfIhave

acavity,theprobabilitythattheprobecatchesinitdoesn‘tdependonwhether

Ihaveatoothache:((1))P(catch|toothache,,cavity)=P(catch|cavity)ThesameindependenceholdsifIhaven’tgotacavity:((2))P(catch|toothache,,?cavity)=P(catch|?cavity)CatchisconditionallyindependentofToothachegivenCavity:P(Catch|Toothache,,Cavity)=P(Catch|Cavity)Equivalentstatements::P(Toothache|Catch,Cavity)=P(Toothache|Cavity)P(Toothache,,Catch|Cavity)=P(Toothache|Cavity)P(Catch|Cavity)條件獨立立性Writeoutfulljointdistributionusingchainrule:P(Toothache,,Catch,Cavity)==P(Toothache|Catch,Cavity)P(Catch,Cavity)==P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)==P(Toothache|Cavity)P(Catch|Cavity)P(Cavity)I.e..,2+2++1==5independentnumbersInmostcases,theuseofconditionalindependencereducesthesizeoftherepresentationofthejointdistributionfromexponentialinntolinearinn.在大多數數情況下下，使用用條件獨獨立性能能將全聯聯合概率率的表示示由n的指數關關系減為為n的線性關關系。Conditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainenvironments.Bayes’Rule（貝葉斯斯法則））Bayes’Rule（貝葉斯斯法則））乘法原則則?Bayes‘rule:orindistributionform為什么該該法則非非常有用用?將條件倒倒轉通常一個個條件是是復雜的的，一個個是簡單單的許多系統統的基礎礎(e.g.語音識別別)現代AI基礎！Bayes’Rule（貝葉斯斯法則））Usefulforassessingdiagnosticprobability（診斷概率）fromcausalprobability（因果概率）:E.g..,letMbemeningitis（腦膜炎炎）,Sbestiffneck（脖子僵僵硬）:Note:腦膜炎的的后驗概概率依然然非常小小!Note:依然要先先檢測脖脖子僵硬硬!Why?Bayes’RuleinPractice使用貝葉葉斯法則則:IH=““havingaheadache“頭痛F=““comingdownwithFlu”流感P(H))=1//10P((F)==1/40P(H|F))=1//2有一天你你早上醒醒來發現現頭很痛痛，于是是得到以以下結論論：“因因為得了了流感以以后50%的幾率會會引起頭頭痛，所所以我有有50%的幾率得得了流感感”Isthisreasoningcorrect?使用貝葉葉斯法則則:IH=""havingaheadache“F=""comingdownwithFlu"P(H))=1//10P((F)==1/40P(H|F))=1//2TheProblem:P(F|H))=??使用貝葉葉斯法則則:IH=""havingaheadache“F=""comingdownwithFlu"P(H)=1/10P(F))=1//40P((H|F)=1/2TheProblem:P(F|H))=P(H||F)P(F)/P(H))==1//8≠≠P((H|F)使用貝葉葉斯法則則:II在一個包包裹里有有2個信封一個信封封里有一一個紅球球(worth$$100)和一個黑黑球另一個信信封里有有2個黑球.黑球一文文不值然后你隨隨機拿出出一個信信封，并并隨機拿拿出一個個球–it’sblack此時此刻刻給你個個機會換換一個信信封.是換呢還還是換呢呢還是換換呢?使用貝葉葉斯法則則:IIE:envelope,1=((R,B),2=((B,B)B:theeventofdrawingablackballP(E||B)==P(B||E)**P(E)//P((B)WewanttocompareP(E==1|B)vs.P(E=2||B)P(B||E=1)==0..5,P(B|E==2)==1P(E==1)==P(E==2)==0.5P(B))=P(B|E==1)P(E==1)++P(B||E=2)P((E=2)==(..5)((.5))+((1))(.5)==.75P(E==1|B)==P((B|E=1))P(E=1))/P((B)==((.5))(.5)/((.75)==1//3P(E==2|B)==P((B|E=2))P(E=2))/P((B)==((1)((.5))/(..75))=2/3因此在已已發現一一個黑球球后,該信封是是1的后驗概概率(thusworth$$100)比信封是是2的后驗概概率低所以還是是換吧課堂測驗驗一名醫生生做了一一個具有有99%可靠性的的測試：：也就是是說,99%的病人其其檢測呈呈陽性,99%的健康人人士檢測測呈陰性性.該醫生估估計1%的人類病病了。。。。Question:一個患者者檢測呈呈陽性.該患者得得病的幾幾率是多多少?0-25%,25--75%%,75-95%,,or95-100%??課堂測驗驗Adoctorperformsatestthathas99%%reliability,i.e..,99%ofpeoplewhoaresicktestpositive,and99%%ofpeoplewhoarehealthytestnegative.Thedoctorestimatesthat1%ofthepopulationissick.Question:Apatienttestspositive.Whatisthechancethatthepatientissick?0-25%,25--75%%,75-95%,,or95-100%??Intuitiveanswer:99%;;Correctanswer:50%Bayes’rulewith多重證據據和條件件獨立性性P(Cavity|toothache∧catch)

=αP(toothache∧catch|Cavity)P(Cavity)

=αP(toothache|Cavity)P(catch|Cavity)P(Cavity)Thisisanexampleofana?veBayesmodel（樸素貝貝葉斯模模型）:Totalnumberofparameters（參數））islinearinn鏈式法則則全聯合分分布using鏈式法則則:P(Toothache,Catch,,Ca