dynare/matlab/doc/mr_hessian.html

358 lines
19 KiB
HTML

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
<title>Description of mr_hessian</title>
<meta name="keywords" content="mr_hessian">
<meta name="description" content="Copyright (C) 2004 Marco Ratto">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="generator" content="m2html &copy; 2003 Guillaume Flandin">
<meta name="robots" content="index, follow">
<link type="text/css" rel="stylesheet" href="../m2html.css">
</head>
<body>
<a name="_top"></a>
<div><a href="../index.html">Home</a> &gt; <a href="index.html">.</a> &gt; mr_hessian.m</div>
<!--<table width="100%"><tr><td align="left"><a href="../index.html"><img alt="<" border="0" src="../left.png">&nbsp;Master index</a></td>
<td align="right"><a href="index.html">Index for .&nbsp;<img alt=">" border="0" src="../right.png"></a></td></tr></table>-->
<h1>mr_hessian
</h1>
<h2><a name="_name"></a>PURPOSE <a href="#_top"><img alt="^" border="0" src="../up.png"></a></h2>
<div class="box"><strong>Copyright (C) 2004 Marco Ratto</strong></div>
<h2><a name="_synopsis"></a>SYNOPSIS <a href="#_top"><img alt="^" border="0" src="../up.png"></a></h2>
<div class="box"><strong>function [hessian_mat, gg, htol1, ihh, hh_mat0] = mr_hessian(func,x,hflag,htol0,varargin) </strong></div>
<h2><a name="_description"></a>DESCRIPTION <a href="#_top"><img alt="^" border="0" src="../up.png"></a></h2>
<div class="fragment"><pre class="comment"> Copyright (C) 2004 Marco Ratto
adapted from Michel Juillard original rutine hessian.m
[hessian_mat, gg, htol1, ihh, hh_mat0] = mr_hessian(func,x,hflag,htol0,varargin)
numerical gradient and Hessian, with 'automatic' check of numerical
error
func = name of the function: func must give two outputs:
- the log-likelihood AND the single contributions at times t=1,...,T
of the log-likelihood to compute outer product gradient
x = parameter values
hflag = 0, Hessian computed with outer product gradient, one point
increments for partial derivatives in gradients
hflag = 1, 'mixed' Hessian: diagonal elements computed with numerical second order derivatives
with correlation structure as from outer product gradient;
two point evaluation of derivatives for partial derivatives
in gradients
hflag = 2, full numerical Hessian, computes second order partial derivatives
uses Abramowitz and Stegun (1965) formulas 25.3.24 and 25.3.27
p. 884.
htol0 = 'precision' of increment of function values for numerical
derivatives
varargin: other parameters of func</pre></div>
<!-- crossreference -->
<h2><a name="_cross"></a>CROSS-REFERENCE INFORMATION <a href="#_top"><img alt="^" border="0" src="../up.png"></a></h2>
This function calls:
<ul style="list-style-image:url(../matlabicon.gif)">
</ul>
This function is called by:
<ul style="list-style-image:url(../matlabicon.gif)">
<li><a href="newrat.html" class="code" title="function [xparam1, hh, gg, fval, igg] = newrat(func0, x, hh, gg, igg, ftol0, nit, flagg, varargin)">newrat</a> </li></ul>
<!-- crossreference -->
<h2><a name="_source"></a>SOURCE CODE <a href="#_top"><img alt="^" border="0" src="../up.png"></a></h2>
<div class="fragment"><pre>0001 <span class="comment">% Copyright (C) 2004 Marco Ratto</span>
0002 <span class="comment">% adapted from Michel Juillard original rutine hessian.m</span>
0003 <span class="comment">%</span>
0004 <span class="comment">% [hessian_mat, gg, htol1, ihh, hh_mat0] = mr_hessian(func,x,hflag,htol0,varargin)</span>
0005 <span class="comment">%</span>
0006 <span class="comment">% numerical gradient and Hessian, with 'automatic' check of numerical</span>
0007 <span class="comment">% error</span>
0008 <span class="comment">%</span>
0009 <span class="comment">% func = name of the function: func must give two outputs:</span>
0010 <span class="comment">% - the log-likelihood AND the single contributions at times t=1,...,T</span>
0011 <span class="comment">% of the log-likelihood to compute outer product gradient</span>
0012 <span class="comment">% x = parameter values</span>
0013 <span class="comment">% hflag = 0, Hessian computed with outer product gradient, one point</span>
0014 <span class="comment">% increments for partial derivatives in gradients</span>
0015 <span class="comment">% hflag = 1, 'mixed' Hessian: diagonal elements computed with numerical second order derivatives</span>
0016 <span class="comment">% with correlation structure as from outer product gradient;</span>
0017 <span class="comment">% two point evaluation of derivatives for partial derivatives</span>
0018 <span class="comment">% in gradients</span>
0019 <span class="comment">% hflag = 2, full numerical Hessian, computes second order partial derivatives</span>
0020 <span class="comment">% uses Abramowitz and Stegun (1965) formulas 25.3.24 and 25.3.27</span>
0021 <span class="comment">% p. 884.</span>
0022 <span class="comment">% htol0 = 'precision' of increment of function values for numerical</span>
0023 <span class="comment">% derivatives</span>
0024 <span class="comment">%</span>
0025 <span class="comment">% varargin: other parameters of func</span>
0026 <span class="comment">%</span>
0027
0028 <a name="_sub0" href="#_subfunctions" class="code">function [hessian_mat, gg, htol1, ihh, hh_mat0] = mr_hessian(func,x,hflag,htol0,varargin)</a>
0029 <span class="keyword">global</span> options_ bayestopt_
0030 <span class="keyword">persistent</span> h1 htol
0031
0032 gstep_=options_.gstep;
0033 <span class="keyword">if</span> isempty(htol), htol = 1.e-4; <span class="keyword">end</span>
0034 func = str2func(func);
0035 [f0, ff0]=feval(func,x,varargin{:});
0036 n=size(x,1);
0037 h2=bayestopt_.ub-bayestopt_.lb;
0038 <span class="comment">%h1=max(abs(x),gstep_*ones(n,1))*eps^(1/3);</span>
0039 <span class="comment">%h1=max(abs(x),sqrt(gstep_)*ones(n,1))*eps^(1/6);</span>
0040 <span class="keyword">if</span> isempty(h1),
0041 h1=max(abs(x),sqrt(gstep_)*ones(n,1))*eps^(1/4);
0042 <span class="keyword">end</span>
0043
0044 <span class="keyword">if</span> htol0&lt;htol,
0045 htol=htol0;
0046 <span class="keyword">end</span>
0047 xh1=x;
0048 f1=zeros(size(f0,1),n);
0049 f_1=f1;
0050 ff1=zeros(size(ff0));
0051 ff_1=ff1;
0052
0053 <span class="comment">%for i=1:n,</span>
0054 i=0;
0055 <span class="keyword">while</span> i&lt;n,
0056 i=i+1;
0057 h10=h1(i);
0058 hcheck=0;
0059 dx=[];
0060 xh1(i)=x(i)+h1(i);
0061 [fx, ffx]=feval(func,xh1,varargin{:});
0062 it=1;
0063 dx=(fx-f0);
0064 ic=0;
0065 <span class="comment">% if abs(dx)&gt;(2*htol),</span>
0066 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0067 <span class="comment">% while c</span>
0068 <span class="comment">% h1(i)=h1(i)*0.9;</span>
0069 <span class="comment">% xh1(i)=x(i)+h1(i);</span>
0070 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0071 <span class="comment">% ic=1;</span>
0072 <span class="comment">% end</span>
0073 <span class="comment">% if ic,</span>
0074 <span class="comment">% [fx, ffx]=feval(func,xh1,varargin{:});</span>
0075 <span class="comment">% dx=(fx-f0);</span>
0076 <span class="comment">% end</span>
0077 <span class="comment">% end</span>
0078
0079 icount = 0;
0080 h0=h1(i);
0081 <span class="keyword">while</span> (abs(dx(it))&lt;0.5*htol | abs(dx(it))&gt;(2*htol)) &amp; icount&lt;10 &amp; ic==0,
0082 <span class="comment">%while abs(dx(it))&lt;0.5*htol &amp; icount&lt; 10 &amp; ic==0,</span>
0083 icount=icount+1;
0084 <span class="comment">%if abs(dx(it)) ~= 0,</span>
0085 <span class="keyword">if</span> abs(dx(it))&lt;0.5*htol
0086 <span class="keyword">if</span> abs(dx(it)) ~= 0,
0087 h1(i)=min(0.3*abs(x(i)), 0.9*htol/abs(dx(it))*h1(i));
0088 <span class="keyword">else</span>
0089 h1(i)=2.1*h1(i);
0090 <span class="keyword">end</span>
0091 xh1(i)=x(i)+h1(i);
0092 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0093 <span class="comment">% while c</span>
0094 <span class="comment">% h1(i)=h1(i)*0.9;</span>
0095 <span class="comment">% xh1(i)=x(i)+h1(i);</span>
0096 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0097 <span class="comment">% ic=1;</span>
0098 <span class="comment">% end</span>
0099 [fx, ffx]=feval(func,xh1,varargin{:});
0100 <span class="keyword">end</span>
0101 <span class="keyword">if</span> abs(dx(it))&gt;(2*htol),
0102 h1(i)= htol/abs(dx(it))*h1(i);
0103 xh1(i)=x(i)+h1(i);
0104 [fx, ffx]=feval(func,xh1,varargin{:});
0105 <span class="keyword">while</span> (fx-f0)==0,
0106 h1(i)= h1(i)*2;
0107 xh1(i)=x(i)+h1(i);
0108 [fx, ffx]=feval(func,xh1,varargin{:});
0109 ic=1;
0110 <span class="keyword">end</span>
0111 <span class="keyword">end</span>
0112 it=it+1;
0113 dx(it)=(fx-f0);
0114 h0(it)=h1(i);
0115 <span class="keyword">if</span> h1(i)&lt;1.e-12*min(1,h2(i)),
0116 ic=1;
0117 hcheck=1;
0118 <span class="keyword">end</span>
0119 <span class="comment">%else</span>
0120 <span class="comment">% h1(i)=1;</span>
0121 <span class="comment">% ic=1;</span>
0122 <span class="comment">%end</span>
0123 <span class="keyword">end</span>
0124 <span class="comment">% if (it&gt;2 &amp; dx(1)&lt;10^(log10(htol)/2)) ,</span>
0125 <span class="comment">% [dum, is]=sort(h0);</span>
0126 <span class="comment">% if find(diff(sign(diff(dx(is)))));</span>
0127 <span class="comment">% hcheck=1;</span>
0128 <span class="comment">% end</span>
0129 <span class="comment">% elseif (it&gt;3 &amp; dx(1)&gt;10^(log10(htol)/2)) ,</span>
0130 <span class="comment">% [dum, is]=sort(h0);</span>
0131 <span class="comment">% if find(diff(sign(diff(dx(is(1:end-1))))));</span>
0132 <span class="comment">% hcheck=1;</span>
0133 <span class="comment">% end</span>
0134 <span class="comment">% end</span>
0135 f1(:,i)=fx;
0136 ff1=ffx;
0137 <span class="keyword">if</span> hflag, <span class="comment">% two point based derivatives</span>
0138 xh1(i)=x(i)-h1(i);
0139 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0140 <span class="comment">% ic=0;</span>
0141 <span class="comment">% while c</span>
0142 <span class="comment">% h1(i)=h1(i)*0.9;</span>
0143 <span class="comment">% xh1(i)=x(i)-h1(i);</span>
0144 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0145 <span class="comment">% ic = 1;</span>
0146 <span class="comment">% end</span>
0147 [fx, ffx]=feval(func,xh1,varargin{:});
0148 f_1(:,i)=fx;
0149 ff_1=ffx;
0150 <span class="comment">% if ic,</span>
0151 <span class="comment">% xh1(i)=x(i)+h1(i);</span>
0152 <span class="comment">% [f1(:,i), ff1]=feval(func,xh1,varargin{:});</span>
0153 <span class="comment">% end</span>
0154 ggh(:,i)=(ff1-ff_1)./(2.*h1(i));
0155 <span class="keyword">else</span>
0156 ggh(:,i)=(ff1-ff0)./h1(i);
0157 <span class="keyword">end</span>
0158 xh1(i)=x(i);
0159 <span class="keyword">if</span> hcheck &amp; htol&lt;1,
0160 htol=min(1,max(min(abs(dx))*2,htol*10));
0161 h1(i)=h10;
0162 i=0;
0163 <span class="keyword">end</span>
0164 save hess
0165 <span class="keyword">end</span>
0166
0167 h_1=h1;
0168 xh1=x;
0169 xh_1=xh1;
0170
0171 <span class="keyword">if</span> hflag,
0172 gg=(f1'-f_1')./(2.*h1);
0173 <span class="keyword">else</span>
0174 gg=(f1'-f0)./h1;
0175 <span class="keyword">end</span>
0176
0177 <span class="keyword">if</span> hflag==2,
0178 gg=(f1'-f_1')./(2.*h1);
0179 hessian_mat = zeros(size(f0,1),n*n);
0180 <span class="keyword">for</span> i=1:n
0181 <span class="keyword">if</span> i &gt; 1
0182 k=[i:n:n*(i-1)];
0183 hessian_mat(:,(i-1)*n+1:(i-1)*n+i-1)=hessian_mat(:,k);
0184 <span class="keyword">end</span>
0185 hessian_mat(:,(i-1)*n+i)=(f1(:,i)+f_1(:,i)-2*f0)./(h1(i)*h_1(i));
0186 temp=f1+f_1-f0*ones(1,n);
0187 <span class="keyword">for</span> j=i+1:n
0188 xh1(i)=x(i)+h1(i);
0189 xh1(j)=x(j)+h_1(j);
0190 xh_1(i)=x(i)-h1(i);
0191 xh_1(j)=x(j)-h_1(j);
0192 <span class="comment">%hessian_mat(:,(i-1)*n+j)=-(-feval(func,xh1,varargin{:})-feval(func,xh_1,varargin{:})+temp(:,i)+temp(:,j))./(2*h1(i)*h_1(j));</span>
0193 <span class="comment">%temp1 = feval(func,xh1,varargin{:});</span>
0194 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0195 <span class="comment">% lam=1;</span>
0196 <span class="comment">% while c,</span>
0197 <span class="comment">% lam=lam*0.9;</span>
0198 <span class="comment">% xh1(i)=x(i)+h1(i)*lam;</span>
0199 <span class="comment">% xh1(j)=x(j)+h_1(j)*lam;</span>
0200 <span class="comment">% %disp( ['hessian warning cross ', num2str(c) ]),</span>
0201 <span class="comment">% c=mr_nlincon(xh1,varargin{:});</span>
0202 <span class="comment">% end</span>
0203 <span class="comment">% temp1 = f0+(feval(func,xh1,varargin{:})-f0)/lam;</span>
0204 temp1 = feval(func,xh1,varargin{:});
0205
0206 <span class="comment">% c=mr_nlincon(xh_1,varargin{:});</span>
0207 <span class="comment">% while c,</span>
0208 <span class="comment">% lam=lam*0.9;</span>
0209 <span class="comment">% xh_1(i)=x(i)-h1(i)*lam;</span>
0210 <span class="comment">% xh_1(j)=x(j)-h_1(j)*lam;</span>
0211 <span class="comment">% %disp( ['hessian warning cross ', num2str(c) ]),</span>
0212 <span class="comment">% c=mr_nlincon(xh_1,varargin{:});</span>
0213 <span class="comment">% end</span>
0214 <span class="comment">% temp2 = f0+(feval(func,xh_1,varargin{:})-f0)/lam;</span>
0215 temp2 = feval(func,xh_1,varargin{:});
0216
0217 hessian_mat(:,(i-1)*n+j)=-(-temp1 -temp2+temp(:,i)+temp(:,j))./(2*h1(i)*h_1(j));
0218 xh1(i)=x(i);
0219 xh1(j)=x(j);
0220 xh_1(i)=x(i);
0221 xh_1(j)=x(j);
0222 j=j+1;
0223 save hess
0224 <span class="keyword">end</span>
0225 i=i+1;
0226 <span class="keyword">end</span>
0227
0228 <span class="keyword">elseif</span> hflag==1,
0229 hessian_mat = zeros(size(f0,1),n*n);
0230 <span class="keyword">for</span> i=1:n,
0231 dum = (f1(:,i)+f_1(:,i)-2*f0)./(h1(i)*h_1(i));
0232 <span class="keyword">if</span> dum&gt;eps,
0233 hessian_mat(:,(i-1)*n+i)=dum;
0234 <span class="keyword">else</span>
0235 hessian_mat(:,(i-1)*n+i)=max(eps, gg(i)^2);
0236 <span class="keyword">end</span>
0237 <span class="keyword">end</span>
0238 <span class="comment">%hessian_mat2=hh_mat(:)';</span>
0239 <span class="keyword">end</span>
0240
0241 gga=ggh.*kron(ones(size(ff1)),2.*h1'); <span class="comment">% re-scaled gradient</span>
0242 hh_mat=gga'*gga; <span class="comment">% rescaled outer product hessian</span>
0243 hh_mat0=ggh'*ggh; <span class="comment">% outer product hessian</span>
0244 A=diag(2.*h1); <span class="comment">% rescaling matrix</span>
0245 igg=inv(hh_mat); <span class="comment">% inverted rescaled outer product hessian</span>
0246 ihh=A'*igg*A; <span class="comment">% inverted outer product hessian</span>
0247 <span class="keyword">if</span> hflag&gt;0 &amp; min(eig(reshape(hessian_mat,n,n)))&gt;0,
0248 hh0 = A*reshape(hessian_mat,n,n)*A'; <span class="comment">%rescaled second order derivatives</span>
0249 hh = reshape(hessian_mat,n,n); <span class="comment">%rescaled second order derivatives</span>
0250 sd0=sqrt(diag(hh0)); <span class="comment">%rescaled 'standard errors' using second order derivatives</span>
0251 sd=sqrt(diag(hh_mat)); <span class="comment">%rescaled 'standard errors' using outer product</span>
0252 hh_mat=hh_mat./(sd*sd').*(sd0*sd0'); <span class="comment">%rescaled inverse outer product with 'true' std's</span>
0253 igg=inv(hh_mat); <span class="comment">% rescaled outer product hessian with 'true' std's</span>
0254 ihh=A'*igg*A; <span class="comment">% inverted outer product hessian</span>
0255 hh_mat0=inv(A)'*hh_mat*inv(A); <span class="comment">% outer product hessian with 'true' std's</span>
0256 sd=sqrt(diag(ihh)); <span class="comment">%standard errors</span>
0257 sdh=sqrt(1./diag(hh)); <span class="comment">%diagonal standard errors</span>
0258 <span class="keyword">for</span> j=1:length(sd),
0259 sd0(j,1)=min(bayestopt_.pstdev(j), sd(j)); <span class="comment">%prior std</span>
0260 sd0(j,1)=10^(0.5*(log10(sd0(j,1))+log10(sdh(j,1))));
0261 <span class="comment">%sd0(j,1)=0.5*(sd0(j,1)+sdh(j,1));</span>
0262 <span class="keyword">end</span>
0263 ihh=ihh./(sd*sd').*(sd0*sd0'); <span class="comment">%inverse outer product with modified std's</span>
0264 igg=inv(A)'*ihh*inv(A); <span class="comment">% inverted rescaled outer product hessian with modified std's</span>
0265 hh_mat=inv(igg); <span class="comment">% outer product rescaled hessian with modified std's</span>
0266 hh_mat0=inv(A)'*hh_mat*inv(A); <span class="comment">% outer product hessian with modified std's</span>
0267 <span class="comment">% sd0=sqrt(1./diag(hh0)); %rescaled 'standard errors' using second order derivatives</span>
0268 <span class="comment">% sd=sqrt(diag(igg)); %rescaled 'standard errors' using outer product</span>
0269 <span class="comment">% igg=igg./(sd*sd').*(sd0*sd0'); %rescaled inverse outer product with 'true' std's</span>
0270 <span class="comment">% hh_mat=inv(igg); % rescaled outer product hessian with 'true' std's</span>
0271 <span class="comment">% ihh=A'*igg*A; % inverted outer product hessian</span>
0272 <span class="comment">% hh_mat0=inv(A)'*hh_mat*inv(A); % outer product hessian with 'true' std's</span>
0273 <span class="keyword">end</span>
0274 <span class="keyword">if</span> hflag&lt;2,
0275 hessian_mat=hh_mat0(:);
0276 <span class="keyword">end</span>
0277
0278 <span class="keyword">if</span> isnan(hessian_mat),
0279 hh_mat0=eye(length(hh_mat0));
0280 ihh=hh_mat0;
0281 hessian_mat=hh_mat0(:);
0282 <span class="keyword">end</span>
0283 hh1=h1;
0284 htol1=htol;
0285 save hess
0286 <span class="comment">% 11/25/03 SA Created from Hessian_sparse (removed sparse)</span>
0287
0288</pre></div>
<hr><address>Generated on Fri 16-Jun-2006 09:09:06 by <strong><a href="http://www.artefact.tk/software/matlab/m2html/">m2html</a></strong> &copy; 2003</address>
</body>
</html>