1、 實驗題目：Regularization實驗學時：實驗日期： 實驗目的：掌握線性回歸和邏輯回歸的正規化，理解 對參量的范數的影響 軟件環境：Octave-4.2.1 (GUI) 實驗步驟與內容：要求：第一部分：線性回歸的正則化1. 下載數據包 "ex5Linx.dat" 和 "ex5Liny.dat" 。2.本實驗中的x是個標量，只有一個特征，用5階多項式擬合的預測函數：為解決過擬合，引入正規化因子，此時代價函數：3.使用梯度下降法：各自正規化因子對應的預測曲線如下：4.使用正規方程解法：=(XTX)1XTy即：各自正規化因子對應的預測曲線如下

2、：第二部分：邏輯回歸的正規化1.下載'ex5Logx.dat' 和 ex5Logy.dat'2.代價方程：其中：采用牛頓法求解最小代價函數。迭代函數：其中：3. 各自正規化因子對應的預測曲線如下：實驗結果1. 2. 結論分析與體會： 1.對于線性回歸，可以看出，隨著 的增大，參量的范數下降。這是由于大的 補償了原代價函數中大的參數。當 過大時，容易出現欠擬合，且預測曲線的走向與實際的相反。 2.對于邏輯回歸，當 增大時， 參量的范數減小。但是大到一定程度后也存在邊界欠擬合的狀況附錄：程序源代碼1x = load('ex4Linx.dat');y = lo

3、ad('ex4Liny.dat');x_test = -1: 0.01 : 1'x1 = ones(size(x(:, 1), 1), x, x.2, x.3, x.4, x.5; % m * 6x_test1 = ones(size(x_test(:, 1), 1), x_test, x_test.2, x_test.3, x_test.4, x_test.5;% testm, n = size(x1);theta = zeros(n, 1);iter = 2000;alpha = 0.07;lamda = 0, 1, 10; % regularized paramJ

4、_value = zeros(iter, 1); % cost valueE = eye(n, n);E(1, 1) = 0;norm_gradient = zeros(length(lamda), 1);for lamdaTemp = 1 : length(lamda) theta = zeros(n, 1); for iterTemp = 1 : iter h_theta = x1 * theta; % m * 1 J_value(iterTemp) = 1 / 2 / m * (sum(h_theta - y).2). + lamda(lamdaTemp) .* (sum(theta.2

5、) - theta(1).2); theta = theta - alpha ./ m .* (x1' * (h_theta - y) + lamda(lamdaTemp) * E * theta); %iteration function end figure; scatter(x, y, 'o','LineWidth', 2, 'MarkerEdgeColor','k','MarkerFaceColor','r'); hold on; plot(x_test, x_test1 * the

6、ta, '-b','LineWidth',2); legend('Training data','5th order fit, =' num2str(lamda(lamdaTemp); figure; plot(1: iter, J_value); xlabel('iteration'); ylabel('J_value'); theta norm_gradient(lamdaTemp) = norm(theta);endnorm_gradient% Normal equationsnorm_nor

7、mal = zeros(length(lamda), 1);for lamdaTemp = 1 : length(lamda) theta = pinv(x1' * x1 + lamda(lamdaTemp) .* E) * x1' * y norm_normal(lamdaTemp) = norm(theta); figure; scatter(x, y, 'o','LineWidth', 2, 'MarkerEdgeColor','k','MarkerFaceColor','r'

8、); hold on; plot(x_test, x_test1 * theta, '-b','LineWidth',2); legend('Training data','5th order fit, =' num2str(lamda(lamdaTemp);endnorm_normal2.x = load('ex4Logx.dat');y = load('ex4Logy.dat');% Find the indices for the 2 classespos = find(y = 1); neg

9、 = find(y = 0);g = inline('1.0 ./ (1.0 + exp(-z)'); % Usage: To find the value of the sigmoid degree = 6;lamda = 0, 1, 10;x1 = map_feature(x(:,1), x(:,2), degree); % m * nm, n = size(x1);E = ones(n, 1);E(1) = 0;norm_lamda = zeros(length(lamda),1);for lamdaTemp = 1 : length(lamda) theta = zer

10、os(n, 1); J_theta = 0; thetaTemp = zeros(n, 1); J_thetaTemp = 0; while (1) h_theta = g(x1 * thetaTemp); % m * 1 J_thetaTemp = -1 ./ m * (sum(y .* log(h_theta) + (1 - y) .* log(1 - h_theta). - lamda(lamdaTemp) ./ 2 * sum(thetaTemp.2) - thetaTemp(1).2) if (abs(J_theta - J_thetaTemp) < 0.0001) theta

11、 = thetaTemp break; end J_theta = J_thetaTemp; H = 1 ./ m * (x1' * diag(h_theta .*(1 - h_theta) * x1 + lamda(lamdaTemp) .* diag(E); % n * n delta_J = 1 ./ m * (x1' * (h_theta - y) + lamda(lamdaTemp) .* diag(E) * thetaTemp); % n * 1 thetaTemp = thetaTemp - pinv(H) * delta_J; end norm_lamda(la

12、mdaTemp) = norm(theta); figure; plot(x(pos, 1), x(pos, 2), '+', 'MarkerEdgeColor','k','MarkerFaceColor','k','MarkerSize',6); hold on; plot(x(neg, 1), x(neg, 2), 'o', 'MarkerEdgeColor','k','MarkerFaceColor','r',&#

13、39;MarkerSize',6); % % Define the ranges of the grid u = linspace(-1, 1.5, 200); v = linspace(-1, 1.5, 200); % Initialize space for the values to be plotted z = zeros(length(u), length(v); % Evaluate z = theta*x over the grid for i = 1:length(u) for j = 1:length(v) % Notice the order of j, i here! z(j,i) = map_feature(u(i), v(j)*theta; end end % Because of the way that contour plotting works % in Matlab, we need to transpose z, or % else the axis orientation will be flipped! %z = z' % Plot z = 0 by specifying the range 0, 0 hold on; co