2025年線性代數(shù)反向傳播算法中的矩陣求導(dǎo)試題一、基礎(chǔ)概念與符號定義在反向傳播算法的矩陣求導(dǎo)中，需明確以下基本符號與運算規(guī)則：標量對矩陣的求導(dǎo)：標量(f)對矩陣(\mathbf{X}\in\mathbb{R}^{m\timesn})的導(dǎo)數(shù)定義為(\frac{\partialf}{\partial\mathbf{X}}=\begin{bmatrix}\frac{\partialf}{\partialX_{11}}&\cdots&\frac{\partialf}{\partialX_{1n}}\\vdots&\ddots&\vdots\\frac{\partialf}{\partialX_{m1}}&\cdots&\frac{\partialf}{\partialX_{mn}}\end{bmatrix})，結(jié)果與(\mathbf{X})同型。矩陣乘法求導(dǎo)法則：若(\mathbf{Y}=\mathbf{A}\mathbf{X}+\mathbf)，則(\frac{\partial\mathbf{Y}}{\partial\mathbf{X}}=\mathbf{A}^T)（其中(\mathbf{A})為常數(shù)矩陣，(\mathbf)為常數(shù)向量）。激活函數(shù)導(dǎo)數(shù)：Sigmoid函數(shù)(\sigma(z)=\frac{1}{1+e^{-z}})的導(dǎo)數(shù)為(\sigma'(z)=\sigma(z)(1-\sigma(z)))；ReLU函數(shù)(\text{ReLU}(z)=\max(0,z))的導(dǎo)數(shù)為(\text{ReLU}'(z)=1)（當(dāng)(z>0)時）或(0)（當(dāng)(z\leq0)時）。二、單層神經(jīng)網(wǎng)絡(luò)反向傳播求導(dǎo)（計算題）題目1已知單層神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)如下：輸入層：(\mathbf{x}\in\mathbb{R}^{2})（樣本向量）權(quán)重矩陣：(\mathbf{W}\in\mathbb{R}^{3\times2})，偏置向量(\mathbf\in\mathbb{R}^{3})隱藏層：(\mathbf{z}=\mathbf{W}\mathbf{x}+\mathbf)，激活函數(shù)(\mathbf{a}=\sigma(\mathbf{z}))（Sigmoid）損失函數(shù)：(L=\frac{1}{2}|\mathbf{a}-\mathbf{y}|^2)（均方誤差，(\mathbf{y})為標簽向量）要求：推導(dǎo)損失函數(shù)(L)對權(quán)重矩陣(\mathbf{W})和偏置向量(\mathbf)的梯度(\frac{\partialL}{\partial\mathbf{W}})和(\frac{\partialL}{\partial\mathbf})。解答步驟前向傳播變量定義(\mathbf{z}=\mathbf{W}\mathbf{x}+\mathbf)，(\mathbf{a}=\sigma(\mathbf{z}))，(L=\frac{1}{2}(\mathbf{a}-\mathbf{y})^T(\mathbf{a}-\mathbf{y}))。損失對輸出層激活值的梯度(\frac{\partialL}{\partial\mathbf{a}}=(\mathbf{a}-\mathbf{y})\in\mathbb{R}^{3})。損失對隱藏層線性變換的梯度(\frac{\partialL}{\partial\mathbf{z}}=\frac{\partialL}{\partial\mathbf{a}}\odot\sigma'(\mathbf{z}))，其中(\odot)表示逐元素乘法。代入Sigmoid導(dǎo)數(shù)得：(\frac{\partialL}{\partial\mathbf{z}}=(\mathbf{a}-\mathbf{y})\odot\mathbf{a}\odot(1-\mathbf{a}))。損失對權(quán)重和偏置的梯度(\frac{\partialL}{\partial\mathbf{W}}=\frac{\partialL}{\partial\mathbf{z}}\mathbf{x}^T\in\mathbb{R}^{3\times2})（外積運算）(\frac{\partialL}{\partial\mathbf}=\frac{\partialL}{\partial\mathbf{z}}\in\mathbb{R}^{3})（偏置導(dǎo)數(shù)等于對應(yīng)層線性變換的梯度）。題目2給定具體數(shù)值計算梯度：設(shè)(\mathbf{x}=\begin{bmatrix}1\2\end{bmatrix})，(\mathbf{W}=\begin{bmatrix}0.1&0.2\0.3&0.4\0.5&0.6\end{bmatrix})，(\mathbf=\begin{bmatrix}0.1\0.2\0.3\end{bmatrix})，(\mathbf{y}=\begin{bmatrix}1\0\0\end{bmatrix})。要求：計算(\frac{\partialL}{\partial\mathbf{W}})的具體值。數(shù)值計算過程計算(\mathbf{z})和(\mathbf{a})(\mathbf{z}=\mathbf{W}\mathbf{x}+\mathbf=\begin{bmatrix}0.1\times1+0.2\times2+0.1\0.3\times1+0.4\times2+0.2\0.5\times1+0.6\times2+0.3\end{bmatrix}=\begin{bmatrix}0.6\1.3\2.0\end{bmatrix})(\mathbf{a}=\sigma(\mathbf{z})=\begin{bmatrix}\frac{1}{1+e^{-0.6}}\approx0.6457\\frac{1}{1+e^{-1.3}}\approx0.7858\\frac{1}{1+e^{-2.0}}\approx0.8808\end{bmatrix})計算梯度中間變量(\mathbf{a}-\mathbf{y}=\begin{bmatrix}0.6457-1\0.7858-0\0.8808-0\end{bmatrix}=\begin{bmatrix}-0.3543\0.7858\0.8808\end{bmatrix})(\sigma'(\mathbf{z})=\mathbf{a}\odot(1-\mathbf{a})\approx\begin{bmatrix}0.6457\times0.3543\approx0.2288\0.7858\times0.2142\approx0.1684\0.8808\times0.1192\approx0.1049\end{bmatrix})(\frac{\partialL}{\partial\mathbf{z}}=(\mathbf{a}-\mathbf{y})\odot\sigma'(\mathbf{z})\approx\begin{bmatrix}-0.3543\times0.2288\approx-0.0811\0.7858\times0.1684\approx0.1323\0.8808\times0.1049\approx0.0924\end{bmatrix})計算權(quán)重梯度(\frac{\partialL}{\partial\mathbf{W}}=\frac{\partialL}{\partial\mathbf{z}}\mathbf{x}^T\approx\begin{bmatrix}-0.0811\0.1323\0.0924\end{bmatrix}\begin{bmatrix}1&2\end{bmatrix}=\begin{bmatrix}-0.0811&-0.1622\0.1323&0.2646\0.0924&0.1848\end{bmatrix})三、多層神經(jīng)網(wǎng)絡(luò)反向傳播求導(dǎo)（綜合題）題目2已知兩層神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)：輸入層：(\mathbf{x}\in\mathbb{R}^{2})隱藏層1：(\mathbf{z}_1=\mathbf{W}_1\mathbf{x}+\mathbf_1)，(\mathbf{a}_1=\sigma(\mathbf{z}_1))（(\mathbf{W}_1\in\mathbb{R}^{3\times2})，(\mathbf_1\in\mathbb{R}^{3})）隱藏層2：(\mathbf{z}_2=\mathbf{W}_2\mathbf{a}_1+\mathbf_2)，(\mathbf{a}_2=\sigma(\mathbf{z}_2))（(\mathbf{W}_2\in\mathbb{R}^{2\times3})，(\mathbf_2\in\mathbb{R}^{2})）損失函數(shù)：(L=-\mathbf{y}^T\log\mathbf{a}_2-(1-\mathbf{y})^T\log(1-\mathbf{a}_2))（二分類交叉熵）要求：推導(dǎo)(\frac{\partialL}{\partial\mathbf{W}_1})的表達式（用前向傳播變量及導(dǎo)數(shù)表示）。解答關(guān)鍵步驟輸出層梯度（對(\mathbf{W}_2)的梯度）(\frac{\partialL}{\partial\mathbf{a}_2}=\frac{\mathbf{a}_2-\mathbf{y}}{\mathbf{a}_2(1-\mathbf{a}_2)})（交叉熵損失導(dǎo)數(shù)），(\frac{\partialL}{\partial\mathbf{z}_2}=\mathbf{a}_2-\mathbf{y})（結(jié)合Sigmoid導(dǎo)數(shù)化簡），(\frac{\partialL}{\partial\mathbf{W}_2}=\frac{\partialL}{\partial\mathbf{z}_2}\mathbf{a}_1^T)。反向傳播至隱藏層1(\frac{\partialL}{\partial\mathbf{a}_1}=\mathbf{W}_2^T\frac{\partialL}{\partial\mathbf{z}_2})（鏈式法則：損失→隱藏層2線性變換→隱藏層1激活值）(\frac{\partialL}{\partial\mathbf{z}_1}=\frac{\partialL}{\partial\mathbf{a}_1}\odot\sigma'(\mathbf{z}_1))(\frac{\partialL}{\partial\mathbf{W}_1}=\frac{\partialL}{\partial\mathbf{z}_1}\mathbf{x}^T)四、矩陣求導(dǎo)公式證明（證明題）題目3證明：對于矩陣(\mathbf{X}\in\mathbb{R}^{m\timesn})，向量(\mathbf{a},\mathbf\in\mathbb{R}^{m})，有(\frac{\partial(\mathbf{a}^T\mathbf{X}\mathbf)}{\partial\mathbf{X}}=\mathbf{a}\mathbf^T)。證明過程展開標量表達式設(shè)(f=\mathbf{a}^T\mathbf{X}\mathbf=\sum_{i=1}^m\sum_{j=1}^na_iX_{ij}b_j)（按矩陣乘法展開）。逐元素求導(dǎo)對(X_{ij})求導(dǎo)：(\frac{\partialf}{\partialX_{ij}}=a_ib_j)。構(gòu)造梯度矩陣梯度矩陣(\frac{\partialf}{\partial\mathbf{X}})的第(i)行第(j)列元素為(a_ib_j)，即(\mathbf{a}\mathbf^T)（外積形式）。證畢。五、拓展應(yīng)用：批量數(shù)據(jù)反向傳播（應(yīng)用題）題目4給定批量輸入數(shù)據(jù)(\mathbf{X}\in\mathbb{R}^{N\times2})（(N)個樣本，每行一個樣本），標簽矩陣(\mathbf{Y}\in\mathbb{R}^{N\times3})，神經(jīng)網(wǎng)絡(luò)結(jié)構(gòu)同題目1（單層）。要求：寫出批量梯度(\frac{\partialL_{\text{batch}}}{\partial\mathbf{W}})的表達式（(L_{\text{batch}}=\frac{1}{N}\sum_{i=1}^NL_i)，(L_i)為單個樣本損失）。答案(\frac{\partialL_{\text{batch