1、2022-3-181吳吳 振振 強強學(xué)號：學(xué)號：0322310256Email: Chapter 3 NEURONAL DYNAMICS II:ACTIVATION MODELSPart I2022-3-182u生物神經(jīng)網(wǎng)生物神經(jīng)網(wǎng)u人工神經(jīng)元人工神經(jīng)元u人工神經(jīng)網(wǎng)絡(luò)的拓?fù)涮匦匀斯ど窠?jīng)網(wǎng)絡(luò)的拓?fù)涮匦?u神經(jīng)動力學(xué)系統(tǒng)概念神經(jīng)動力學(xué)系統(tǒng)概念 u人工神經(jīng)網(wǎng)絡(luò)通用的信號函數(shù)人工神經(jīng)網(wǎng)絡(luò)通用的信號函數(shù)u神經(jīng)元的存貯模式神經(jīng)元的存貯模式上次課程回顧上次課程回顧2022-3-183胞體胞體(Soma)樹突（樹突（Dendrite）胞體胞體(Soma) 軸突（軸突（Axon）突觸（突觸（Synapse）生物

2、神經(jīng)網(wǎng)絡(luò)生物神經(jīng)網(wǎng)絡(luò)2022-3-184 生物神經(jīng)網(wǎng)的六個基本特征：生物神經(jīng)網(wǎng)的六個基本特征： 1）神經(jīng)元及其聯(lián)接神經(jīng)元及其聯(lián)接； 2）神經(jīng)元之間的聯(lián)接強度決定神經(jīng)元之間的聯(lián)接強度決定信號傳遞信號傳遞的強弱；的強弱； 3）神經(jīng)元之間的聯(lián)接強度是可以隨神經(jīng)元之間的聯(lián)接強度是可以隨訓(xùn)練訓(xùn)練改變的；改變的； 4）信號可以是起信號可以是起刺激刺激作用的，也可以是起作用的，也可以是起抑制抑制作作用的；用的； 5）一個神經(jīng)元接受的信號的一個神經(jīng)元接受的信號的累積效果累積效果決定該神經(jīng)決定該神經(jīng)元的狀態(tài)；元的狀態(tài)； 6) 每個神經(jīng)元可以有一個每個神經(jīng)元可以有一個“閾值閾值”。2022-3-185Cell b

3、ody (soma)DendriteNucleus神經(jīng)元 (Neuron)AxonSynapse樹突(Dendtrite)：輸入端 軸突(Axon)： 輸出端突觸(Synapse)： 不同神經(jīng)元的軸突與樹突的結(jié)合部，不同神 經(jīng)元的相互作用用權(quán)值表示，學(xué)習(xí)就是調(diào)整權(quán)值，胞體(Soma)： 是非線性輸入/輸出的單元，可用閾值、分段、Sigmod函 數(shù)近似從仿生學(xué)角度：從仿生學(xué)角度：2022-3-186人工神經(jīng)網(wǎng)絡(luò)人工神經(jīng)網(wǎng)絡(luò)2022-3-187人工神經(jīng)元人工神經(jīng)元 神經(jīng)元是構(gòu)成神經(jīng)網(wǎng)絡(luò)的最基本單元（構(gòu)神經(jīng)元是構(gòu)成神經(jīng)網(wǎng)絡(luò)的最基本單元（構(gòu)件）。件）。 人工神經(jīng)元模型應(yīng)該具有生物神經(jīng)元的六人工神經(jīng)元模

4、型應(yīng)該具有生物神經(jīng)元的六個基本特性。個基本特性。 2022-3-188人工神經(jīng)元的基本構(gòu)成人工神經(jīng)元的基本構(gòu)成 人工神經(jīng)元模擬生物神經(jīng)元的人工神經(jīng)元模擬生物神經(jīng)元的一階特性一階特性。輸入：輸入：X=（x1，x2，xn）聯(lián)接權(quán)：聯(lián)接權(quán)：W=（w1，w2，wn）T網(wǎng)絡(luò)輸入：網(wǎng)絡(luò)輸入： net=xiwi向量形式：向量形式： net=XWxn wnx1 w1x2 w2net=XW2022-3-189激活函數(shù)激活函數(shù)(Activation Function) 激活函數(shù)激活函數(shù)執(zhí)行對該神經(jīng)元所獲得的網(wǎng)執(zhí)行對該神經(jīng)元所獲得的網(wǎng)絡(luò)輸入的變換，也可以稱為激勵函數(shù)、活絡(luò)輸入的變換，也可以稱為激勵函數(shù)、活化函數(shù)：化

5、函數(shù)： o=f（net） 1、線性函數(shù)（、線性函數(shù)（Liner Function） f（net）=k*net+c netooc2022-3-18102、非線性斜面函數(shù)、非線性斜面函數(shù)(Ramp Function) if netf（net）= k*netif |net|0為一常數(shù)，被稱為飽和值，為該神經(jīng)元為一常數(shù)，被稱為飽和值，為該神經(jīng)元的最大輸出。的最大輸出。 2022-3-1811 - - net o 2022-3-18123、閾值函數(shù)（、閾值函數(shù)（Threshold Function） 或或 階躍函數(shù)階躍函數(shù)if netf（net）=-if net 、均為非負(fù)實數(shù)，均為非負(fù)實數(shù)，為閾值為閾

6、值二值形式：二值形式：1if netf（net）=0if net 雙極形式：雙極形式：1if netf（net）=-1if net 2022-3-1813 -onet02022-3-18144、S形函數(shù)形函數(shù) 壓縮函數(shù)（壓縮函數(shù)（Squashing Function）和邏輯斯特）和邏輯斯特函數(shù)（函數(shù)（Logistic Function）。）。f（net）=a+b/(1+exp(-d*net)a，b，d為常數(shù)。它的飽和值為為常數(shù)。它的飽和值為a和和a+b。最簡單形式為：最簡單形式為：f（net）= 1/(1+exp(-d*net) 函數(shù)的飽和值為函數(shù)的飽和值為0和和1。 S形函數(shù)有較好的增益控制

7、形函數(shù)有較好的增益控制 2022-3-1815a+b o(0,c)netac=a+b/22022-3-1816二值函數(shù)二值函數(shù)雙極函數(shù)雙極函數(shù)S型函數(shù)型函數(shù)線性函數(shù)線性函數(shù)2022-3-1817u空間模式空間模式（Spatial Model）u時空模式時空模式（Spatialtemporal Model）u空間模式三種空間模式三種存儲類型存儲類型1、 RAM方式（方式（Random Access Memory）隨機(jī)訪問方式是將地址映射到數(shù)據(jù)。隨機(jī)訪問方式是將地址映射到數(shù)據(jù)。2、 CAM方式（方式（Content Addressable Memory）內(nèi)容尋址方式是將數(shù)據(jù)映射到地址。內(nèi)容尋址方式

8、是將數(shù)據(jù)映射到地址。3、 AM方式（方式（Associative Memory）相聯(lián)存儲相聯(lián)存儲( (聯(lián)想記憶聯(lián)想記憶) )方式是將數(shù)據(jù)映射到數(shù)據(jù)。方式是將數(shù)據(jù)映射到數(shù)據(jù)。 神經(jīng)元的存貯模型神經(jīng)元的存貯模型2022-3-1818 后兩種方式是人工神經(jīng)網(wǎng)絡(luò)的工作方式。后兩種方式是人工神經(jīng)網(wǎng)絡(luò)的工作方式。 在學(xué)習(xí)在學(xué)習(xí)/訓(xùn)練期間，人工神經(jīng)網(wǎng)絡(luò)以訓(xùn)練期間，人工神經(jīng)網(wǎng)絡(luò)以CAM方方式工作；權(quán)矩陣又被稱為網(wǎng)絡(luò)的式工作；權(quán)矩陣又被稱為網(wǎng)絡(luò)的長期存儲長期存儲（Long Term Memory，簡記，簡記為為LTM）。）。 網(wǎng)絡(luò)在正常工作階段是以網(wǎng)絡(luò)在正常工作階段是以AM方式工作的；方式工作的；神經(jīng)元的狀態(tài)表

9、示的模式為神經(jīng)元的狀態(tài)表示的模式為短期存儲短期存儲（Short Term Memory，簡記為，簡記為STM）。 2022-3-1819 自相自相聯(lián)聯(lián)（Auto-associative）映射映射：訓(xùn)練網(wǎng)絡(luò)訓(xùn)練網(wǎng)絡(luò)的樣本集為向量集合為的樣本集為向量集合為 A1，A2，An 在理想情況下，該網(wǎng)絡(luò)在完成訓(xùn)練后，其在理想情況下，該網(wǎng)絡(luò)在完成訓(xùn)練后，其權(quán)矩陣存放的將是上面所給的向量集合。權(quán)矩陣存放的將是上面所給的向量集合。 2022-3-1820 異相聯(lián)（異相聯(lián)（Hetero-associative）映射）映射（A1，B1），（），（A2，B2），），（，（An，Bn） 該網(wǎng)絡(luò)在完成訓(xùn)練后，其權(quán)矩陣存放

10、的將是上面該網(wǎng)絡(luò)在完成訓(xùn)練后，其權(quán)矩陣存放的將是上面所給的向量集合所蘊含的對應(yīng)關(guān)系。所給的向量集合所蘊含的對應(yīng)關(guān)系。 當(dāng)輸入向量當(dāng)輸入向量A不是樣本的第一的分量時，樣本中不是樣本的第一的分量時，樣本中不存在這樣的元素不存在這樣的元素（Ak，Bk），使得），使得AiAAk kAA或者或者AAAAk kAAj j 且此時有且此時有AiAAAAj j 則向量則向量B是是Bi與與Bj的插值。的插值。 2022-3-1821本節(jié)內(nèi)容安排本節(jié)內(nèi)容安排u神經(jīng)動力學(xué)系統(tǒng)uBAM模型uLypaunov函數(shù)uBivalent(二階) BAM理論2022-3-1822神經(jīng)元激勵函數(shù)隨時間變化，其改變方式依賴于下面的

11、動力學(xué)方程：），（YXFFgx ），（YXFFhy (3-1)(3-2)3.1 神經(jīng)動力學(xué)系統(tǒng)神經(jīng)動力學(xué)系統(tǒng)2022-3-1823Na+K+Cl-Cl-K+Organic ionNa+30 mV-+Na+K+Cl-Na+Cl-K+Organic ion+-70 mV-+細(xì)胞膜內(nèi)外電動勢神經(jīng)動力系統(tǒng)在神經(jīng)元中的體現(xiàn)神經(jīng)動力系統(tǒng)在神經(jīng)元中的體現(xiàn)2022-3-1824在缺乏外部作用和神經(jīng)刺激的情況下，最簡單激勵的動力學(xué)模型是：iixx-jjyy-(3-3)(3-4) 一階被動衰減模型2022-3-1825tiiextx-)0()(在任何有窮狀態(tài)下膜電壓以指數(shù)方式迅速衰減到零電位。2022-3-182

12、6 Passive Membrane Decay（被動膜電壓的衰減）0iA Passive-decay ratescales the rate to the membranes resting potential. solution :Passive-decay rate measures: the cell membranesresistance or “friction” to current flow.iiixAx-t-Aiii)e(x(t)x02022-3-1827iAThe larger the passive-decay rate,the faster the decay-the

13、less the resistance to current flow. Pay attention to propertyproperty2022-3-1828 Membrane Time Constants The membrane time constant scales the time variable of the activation dynamical system.The multiplicative constant model:iCiiiix-AxC（38） 2022-3-1829Solution and propertysolutiontCAiiiext-)0()(xi

14、propertyThe smaller the capacitance ,the faster things change As the membrane capacitance increases toward positive infinity,membrane fluctuation slows to stop.2022-3-1830Membrane Resting PotentialsDefine resting Potential as the activation value to which the membrane potential equilibrates in the a

15、bsence of external or neuronal inputs: iPiiiiiPx-AxC+ Solutions )e-(1AP(0)ex(t)xtCA-iitCA-iiiiii+ Definition(3-11)(3-12)2022-3-1831 Note The capacitance appear in the index of the solution,it is called time-scaling capacitance. It does no affect the steady-state solution and does not depend on the f

16、inite initial condition. In resting case,we can find the solution quickly.2022-3-1832Additive External InputAdd inputApply a relatively constant numeral input to a neuron.iiiI-xx+ solution)e-(1I(0)ex(t)x-ti-tii+(3-13)(3-14)2022-3-1833Meaning of the input Input can represent the magnitude of directly

17、 experiment sensory information or directly apply control information. The input changes slowly,and can be assumed constant value. 2022-3-1834神經(jīng)網(wǎng)絡(luò)模型 神經(jīng)網(wǎng)絡(luò)模型已有40多種，代表性的有：u 自適應(yīng)共振(ART)u 雪崩網(wǎng)絡(luò)u 雙向聯(lián)想記憶(BAM)u BP模型u Boltzman機(jī)/Cauchy機(jī)(BCM)u 盒中腦(BSB)u Counter Propagation(CPN)u Hopfield神經(jīng)網(wǎng)絡(luò)u Madalineu 學(xué)習(xí)矩陣(LRN)

18、u 自組織映射(SOM)u 細(xì)胞神經(jīng)網(wǎng)絡(luò)(CNN)u 交替投影神經(jīng)網(wǎng)絡(luò)(APNN)有關(guān)42種模型的詳細(xì)說明請參見神經(jīng)網(wǎng)絡(luò)計算焦李成編（1996）P36P902022-3-1835Bart.Kosko對ANS領(lǐng)域作出了三大貢獻(xiàn)：n雙向聯(lián)想記憶模型（BAM）n模糊認(rèn)知映射n模糊邏輯與ANS結(jié)合BAM模型將單層的Hebb學(xué)習(xí)器推廣成兩層的模型匹配互聯(lián)想器，BAM是非監(jiān)督學(xué)習(xí)的、能夠?qū)崟r學(xué)習(xí)和回憶的ANS。由于其反饋結(jié)構(gòu)及描述其兩層之間相互作用的動力學(xué)方程的性質(zhì)，BAM能夠在任何給定的矩陣后收斂于一個最小解，使BAM能夠建立一種能夠同時學(xué)習(xí)和回憶的全局穩(wěn)定的動力學(xué)系統(tǒng)。Bart.Kosko對對ANS的

19、貢獻(xiàn)的貢獻(xiàn)2022-3-1836What is Associative Memory and BAM ? Associative memory（聯(lián)想記憶）：是神經(jīng)網(wǎng)絡(luò)的重要應(yīng)用之一，它具有很強的容錯性和抗干擾性，能進(jìn)行大規(guī)模并行處理，適用于模式識別、信號處理、故障診斷和圖像處理等。 Bidirectional Associative Memory (BAM)：中國首次載人航天時間：2003.10.15地點：甘肅九泉發(fā)射場航天員：楊利偉 2022-3-1837BAMBAM結(jié)構(gòu)結(jié)構(gòu) 智力鏈智力鏈從一件事想到另一件事，從一件事想到另一件事，“喚回失去的記憶喚回失去的記憶”。 自相聯(lián)自相聯(lián) 異相聯(lián)異相

20、聯(lián)雙向聯(lián)想記憶雙向聯(lián)想記憶BAMBAM（Bidirectional Associative Memory）。BAMBAM具有一定的具有一定的推廣能力推廣能力它對含有一定缺陷的輸入向量，通過對信號的它對含有一定缺陷的輸入向量，通過對信號的不斷變換、修補，最后給出一個正確的輸出不斷變換、修補，最后給出一個正確的輸出。 2022-3-1838基本的基本的BAMBAM結(jié)構(gòu)結(jié)構(gòu) W第第1層層輸入向量輸入向量第第2層層輸出向量輸出向量WTx1xnymy12022-3-1839The BAM : Network Architectureyx )(xy -)(1xyx-1yOutput layerInput

21、layerFeedback partFeedforwardpart2022-3-1840KoskoKosko的改進(jìn)型的改進(jìn)型BAMBAM u離散BAMu連續(xù)BAMu自適應(yīng)BAMu高階自適應(yīng)BAMu競爭自適應(yīng)BAMu隨機(jī)自適應(yīng)BAMu模糊自適應(yīng)BAM2022-3-1841 Neurons do not compute alone. Neuron modify their state activations with external input and with the feedback from one another. 3.2 ADDITIVE NEURONAL FEEDBACK This

22、feedback takes the form of path-weighted signals from synaptically connected neurons.2022-3-1842Synaptic Connection Matricesn neurons in field p neurons in field XFYFThe ith neuron axon in a synapse jth neurons in ijmijmis constant,can be positive,negative or zero.XFYF2022-3-1843Meaning of connectio

23、n matrix The synaptic matrix or connection matrix M is an n-by-p matrix of real number whose entries are the synaptic efficacies .the ijth synapse is excitatoryif ，inhibitory（抑制） if 0mij0mijijm The matrix M describes the forward projections from neuron field to neuron fieldXFYF The matrix N describe

24、s the feedforward projections from neuron field to neuron fieldYFXF2022-3-1844Bidirectional and Unidirectional connection TopologiesBidirectional networksM and N have the same or approximately the samestructure. Unidirectional networkTNM TMN A neuron field synaptically intraconnects to itself. BAM M

25、 is symmetric, the unidirectional network is BAMTMM 2022-3-1845Augmented field and augmented matrix Augmented fieldM connects to ,N connects to then the augmented field intraconnects to itself by the square block matrix BYXZFFFXFYFZFXFYFYFXF00NMB2022-3-1846Augmented field and augmented matrix In the

26、 BAM case,when then hence a BAM symmetries an arbitrary rectangular（長方形） matrix M. In the general case,P is n-by-n matrix.Q is p-by-p matrix.TMN TBB QNMPCIf and only if, the neurons in are symmetrically intraconnected TMN TPP TQQ ZFTCC 2022-3-18473.3 ADDITIVE ACTIVATION MODELSDefine additive activat

27、ion modeln+p coupled first-order differential equations defines the additive activation model+pjijijjInyS1)(x-Axiii+pjjijiiImxS1)(y-Ayjjj（315）（316） 2022-3-1848additive activation model define The additive autoassociative model correspond to a system of n coupled first-order differential equations （3

28、17）+pjijijjImxS1iii)(x-Ax2022-3-1849additive activation model define A special case of the additive autoassociative model(3-18)(3-19)+-+-jiijijiiiiIrxxRxxC+-jiijjjiiImxSRx)(（3-20）iR+njijiirRR111where measures the cytoplasmic resistance between neurons i and j. ijris2022-3-1850Hopfield circuit and co

29、ntinuous additive bidirectionalassociative memories Hopfield circuit arises from if each neuron has a strictly increasing signal function and if the synaptic connection matrix is symmetric(3-21)+-jiijjjiiiiImxSRxxC)( continuous additive bidirectional associative memories+pjiijjjImyS1iii)(x-Ax+pijiji

30、iImxS1jjj)(y-Ay(3-22)(3-23)2022-3-1851 Discrete additive activation models correspond to neurons with threshold signal function3.4 ADDITIVE BIVALENT FEEDBACK The neurons can assume only two value: ON and OFF. ON represents the signal value +1. OFF represents 0 or 1. Bivalent models can represent asy

31、nchronous and stochastic （隨機(jī)的）behavior.2022-3-1852Bivalent Additive BAM BAM-bidirectional associative memory Define a discrete additive BAM with threshold signal functions, arbitrary thresholds and inputs,an arbitrary but constant synaptic connection matrix M,and discrete time steps k. +pjiijkjjkiIm

32、ySx11)(+pijijkiikjImxSy11)(3-24)(3-25)2022-3-1853Bivalent Additive BAMThreshold binary signal functions For arbitrary real-value thresholds for neurons for neurons(3-26)(3-27)-ikiikikiiikikiiUxifUxifxSUxifxS0)(1)(1-jkjjkjkjjjkjkjjVyifVyifySVyifyS0)(1)(1nUUU,1XFpVVV,1YF2022-3-1854A example for BAM mo

33、delExampleA 4-by-3 matrix M represents the forward synaptic projections from to .A 3-by-4 matrix MT represents the backward synaptic projections from to .XFYFYFXF-112230021203M-120213202013TM2022-3-1855A example for BAM model (cont.)Suppose at initial time k all the neurons in are ON.So the signal s

34、tate vector at time k corresponds toYF)(kYS) 111 ()(kYSInput) 1325 () ,(4321-kkkkkxxxxXSuppose0jiVU2022-3-1856A example for BAM model (cont.) first:at time k+1 through synchronous operation,the result is:) 1101 ()(kXS next:at time k+1 ,these signals pass “forward” through the filter M to affect the

35、activations of the neurons.XFYFThe three neurons compute three dot products,or correlations.The signal state vector multiplies each of the three columns of M.)(kXS2022-3-1857A example for BAM model (cont.) the result is:)(,)(,)()(41i3412i411 iikiiikiiikiikmxSmxSmxSMXS)345(-)(131211+kkkyyy1kY+ synchr

36、onously compute the new signal state vector :)(1+kYS) 110()(1+kYS2022-3-1858A example for BAM model (cont.) the signal vector passes “backward” through the synapticfilter at time k+2: synchronously compute the new signal state vector :)(1+kYS)0522()(1-+TkMYS)(24232221+kkkkxxxx2+kX)()(11012+kXS)(kXS2

37、022-3-1859A example for BAM model (cont.) conclusion)()(kkXSXS+2since)()(13+kkYSYSthenThese same two signal state vectors will pass back and forth in bidirectional equilibrium forever-or until new inputs perturb the system out of equilibrium.2022-3-1860A example for BAM model (cont.) asynchronous st

38、ate changes may lead to different bidirectional equilibrium keep the first neurons ON,only update the second and third neurons. At k,all neurons are ON. YFYFMXSYkk)(+1)(345- new signal state vector at time k+1 equals:)()(1111+kYS2022-3-1861A example for BAM model (cont.) new activation state vector

39、equals: synchronously thresholds passing this vector forward to gives)()(251112-+TkkMYSX)()(01002+kXSYFMXSYkk)(23+)(230)()()(13111+kkYSYSXF2022-3-1862A example for BAM model (cont.) similarly, for any asynchronous state change policy we apply to the neurons the system has reached a new equilibrium,t

40、he binary pair represents a fixed point of the system. XF)()()(010024+kkXSXS)(),(11101002022-3-1863conclusion conclusion Different subset asynchronous state change policies applied to the same data need not product the same fixed-point equilibrium. They tend to produce the same equilibria.All BAM st

41、ate changes lead to fixed-point stability.2022-3-1864The BAM Examplein MATLABx1 = 1 -1 -1 1 -1 1 1 -1 -1 1;y1 = 1 -1 -1 -1 -1 1;x2 = 1 1 1 -1 -1 -1 1 1 -1 -1;y2 = 1 1 1 1 -1 -1;w = y1*x1 + y2*x2Matlab codeExamplar No. 1Examplar No. 2Weight matrix 2 0 0 0 -2 0 2 0 -2 0 0 2 2 -2 0 -2 0 2 0 -2 0 2 2 -2

42、 0 -2 0 2 0 -2 0 2 2 -2 0 -2 0 2 0 -2 -2 0 0 0 2 0 -2 0 2 0 0 -2 -2 2 0 2 0 -2 0 2w =2022-3-1865The BAM : Example (cont.)x0 = -1 -1 -1 1 -1 1 1 -1 -1 1;y0 = 1 1 1 1 -1 -1;xold = x0; yold = y0;for i=1:5 nety = w*xold ynew = (nety0)*1 + (nety=0).*yold - (nety0)*1 + (netx=0).*xold - (netx0)*1 if (xnew=

43、xold) break; end xold = xnew; yold = ynew; pauseendUnknown inputSet initial valuesFeedforward partFeedback partUpdate value forNext roundStop if no change2022-3-1866The BAM : Example (cont.)Iteration 1: 4 -12 -12 -12 -4 12nety = 1 -1 -1 -1 -1 1ynew = 4 -8 -8 8 -4 8 4 -8 -4 8netx = 1 -1 -1 1 -1 1 1 -

44、1 -1 1xnew = Final state : xnew = x12022-3-1867Bidirectional Stability definition A BAM system is Bidirectional stable if all inputs converge to fixed-point equilibria.A denotes a binary n-vector in B denotes a binary p-vector in ),(MFFyx n10, p10,2022-3-1868Bidirectional Stability (cont.)Represent

45、a BAM system equilibrates to bidirectional fixed point as),(ffBAfTfffTTBMABMABMABMABMABMA 2022-3-1869Lyapunov第二方法是建立在客觀事實上：如果一個系統(tǒng)的某個 平衡狀態(tài)是是漸近穩(wěn)定的，即 則隨著系統(tǒng)的運動，其貯存的能量將隨著時間的增長而衰減，直至趨于平衡狀態(tài)而使能量趨于極小值。但對一般而言，并沒有這樣的直觀性，因此Lyapunov引入了一個廣義的能量函數(shù)，稱為Lyapunov函數(shù)。 ，其中p為對稱矩陣，這樣Lyapunov第二方法就歸結(jié)為：在不直接求解X的前提下通過研究V函數(shù)及其導(dǎo)數(shù)就可以

46、給出系統(tǒng)平衡狀態(tài)的穩(wěn)定信息。3.5 Lyapunov Functions(李雅普諾夫函數(shù))xx(t) tlimpxxv(x)T2022-3-1870Lyapunov Functions L maps system state variables to real numbers and decreases with time. In BAM case,L maps the Bivalent product space to real numbers.Suppose L is sufficiently differentiable（充分可微） to apply the chain rule:nii

47、idtdxxLLiiixxL(3-28)2022-3-1871 Lyapunov Functions (cont.) The quadratic choice of L Suppose the dynamical system describes the passive decay system.（3-29）TxIxL21iix221（3-30）iixx- The solutiontiiextx-)0()(（3-31）2022-3-1872Lyapunov Functions (cont.) The partial derivative（偏導(dǎo)數(shù)） of the quadratic L:（3-3

48、2）iixxL（3-33）-iixL2(3-34)-iixL2or(3-35)0LIn either caseAt equilibrium(3-36)0LThis occurs if and only if all velocities equal zero0ix2022-3-1873conclusion A dynamical system is stable if some Lyapunov Functions Ldecreases along trajectories. A dynamical system is asymptotically stable if it strictly

49、decreases along trajectories Monotonicity of a Lyapunov Function provides a sufficient condition for stability and asymptotic stability.2022-3-1874Linear system stabilityFor symmetric matrix A and square matrix B,the quadratic form behaves as a strictly decreasing LyapunovTxAxL xBx function for any

50、linear dynamical system if and only if the matrix is negative definite.BAABT+TTAxxxxAL+TTTxBAxxxAB+TTxBAABx+2022-3-1875The relations between convergence rate and eigenvalue sign （特征值符號） A general theorem in dynamical system theory relates convergence rate and ergenvalue sign: A nonlinear dynamical s

51、ystem converges exponetially（指數(shù)） quickly if its system Jacobian has eigenvalues with negative real parts. Locally such nonlinear system behave as linearly. A Lyapunov Function summarizes total system behavuor. A Lyapunov Function often measures the energy of a physical sysem. 2022-3-1876Potential en

52、ergy function represented by quadratic formConsider a system of n variables and its potential-energy function E. Suppose the coordinate measures the displacement from equilibrium of ith unit.The energy depends on only coordinate ,so since E is a physical quantity,we assume it is sufficiently smooth

53、to permit a multivariable Taylor-series expansion about the origin: ix),(nxxEE1ix2022-3-1877Potential energy function represented by quadratic form (cont.)+ijjijiiiixxxxExxEEE22100),(+ijkkjikjixxxxE331!ijjijixxxxE221TxAx21Where A is symmetric,sincejiijjiijaxxExxEa222022-3-1878The reason of (3-42)fol

54、lowsFirst,we defined the origin as an equilibrium of zero potentialenergy;so Second,the origin is an equilibrium only if all first partialderivatives equal zero.Third,we can neglect higher-order terms for small displacement,since we assume the higher-order products are smaller than the quadratic pro

55、ducts.000),(E2022-3-18793.6 Bivalent BAM theoremThe average signal energy L of the forward pass of the Signal state vector through M,and the backward passOf the signal state vector through : XF)(XSTMYF)(YS2TTXMSYSYMSXSL)()()()(+-TTTTTXSMYSXSMYS)()()()(TYSTMXS)()(sinceTTYSMXSL)()(-ijnipjjjiimySxS-)()

56、(2022-3-1880Lower bound of Lyapunov function The signal is Lyapunov function clearly bounded below.For binary or bipolar,the matrix coefficients（系數(shù)） definethe attainable bound:The attainable upper bound is the negative of this expression.-ijijmL2022-3-1881Lyapunov function for the general BAM system

57、 The signal-energy Lyapunov function for the general BAM system takes the formInputs and andconstant vectors of thresholdsthe attainable bound of this function is.TTTVJYSUIXSYMSXSL)()()()(-,NIII1,PJJJ1,NUUU1,NVVV1-jjjiiiijijVJUImL2022-3-1882Bivalent BAM theoremBivalent BAM theorem.every matrix is bidrectionally stablefor synchronous or asynchronous state changes.Proof consider the signal state changes that occur from time k to time k+1,define the vectors of signal state changes as:, )(,),()()()(nnkkxSxSXSXSXS+111, )(,)