Commit Graph

6898 Commits (2f13d6610d6842adc4a4b448eb45ebd4223967e6)

Author SHA1 Message Date
Sébastien Villemot d9b9f78392 Merge branch 'particle_check' into 'master'
Particle filters: provide error if trends or prefiltering is used

Closes #1690

See merge request Dynare/dynare!1695
2020-01-13 11:02:29 +00:00
Johannes Pfeifer 03a8759560 Particle filters: provide error if trends or prefiltering is used
Closes https://git.dynare.org/Dynare/dynare/issues/1690
2020-01-10 19:08:51 +01:00
Sébastien Villemot 7e770f69e7
Remove workaround for errors in MEX files
Because at some point throwing exceptions from MEX files (with mexErrMsgTxt())
was not working under Windows 64-bit, we had designed a workaround to avoid
using exceptions.

Most MEX files were returning an error code as their first (or sometimes last)
argument, and that code would have to be checked from the MATLAB code.

Since this workaround is no longer needed, this commit removes it. As a
consequence, the interface of many MEX files is modified.

For some background, see https://www.dynare.org/pipermail/dev/2010-September/000895.html
2020-01-10 18:33:11 +01:00
Stéphane Adjemian (Charybdis) 3e408ffd6b
Updated dseries submodule (flip method and geometric nanmean).
[skip ci]
2020-01-09 19:16:15 +01:00
Sébastien Villemot 4e314a529b
Bump minimal required Octave version to 4.4 2020-01-06 18:29:47 +01:00
Sébastien Villemot a95358accc Merge branch 'sim1' into 'master'
sim1.m: provide missing function input to nested function

See merge request Dynare/dynare!1691
2020-01-06 13:22:29 +00:00
Houtan Bastani bf102030cb
support saving exogenous variables in `dynasave`, `dynasave`; fix bugs in `dynasave`; add test
- `dynasave`: if a variable being saved was named `n` or `s`, the `eval` statements would break the code
- `dynasave`: use the `-struct` option to `save` to avoid `eval` statements
- `dynasave` and `dynatype`: do everything in 1 loop instead of 2
- `dynasave` and `dynatype`: use `strcmp` instead of `strfind`

- preprocessor update contains:
  - Partial reversion of global indentation of macro processor header files introduced in e2d5a83592634f0604d8c86409748cd2ec5906d2
  - Symbol List check pass: allow caller to specify the valid types of variables in a Symbol List
  - Allow `dynasave` and `dynatype` to support exogenous variables in their var_list

issue #1691
2020-01-06 12:45:44 +01:00
Johannes Pfeifer 632c0a3943 sim1.m: provide missing function input to nested function 2020-01-06 09:06:18 +01:00
Marco Ratto 2134f2616d
for parallel execution we need to initialize also prior_draw (used in slice sampler). 2020-01-02 17:48:29 +01:00
Sébastien Villemot 49dc997073
Global reindentation of MATLAB code (excluding submodules)
Also convert to Unix end-of-lines, and remove trailing whitespaces.
2019-12-20 16:30:27 +01:00
Willi Mutschler 45e9771eb8
Fixed bug regarding non-stationary variables in pruned moments 2019-12-20 12:30:53 +01:00
Willi Mutschler 8b9b49f8d7
Finished identification order=1|2|3
Note that I still need to do a code clean up (provide some licenses for functions from other people) and to double check order=3. There is also much room for speed and memory improvement, but the code works fine for now. I will also provide more information to the merge request soon about the detailed changes for future reference.
2019-12-20 12:28:55 +01:00
Sébastien Villemot c04c111d97
Merge branch 'rattoma/dynare-epilogue'
Ref. !1688
2019-12-20 11:51:41 +01:00
Stéphane Adjemian (Charybdis) b5d4b4059b Return an error if pruning is used with order>3 in estimation. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 031569fa96 Allow higher order approximations in nonlinear filters. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 227b2661cb Do not trap order>2 in estimation. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 4e0deb7987 Removed persistent variables. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 76e3c6ca68 Removed unnecessary parts of the code.
- Call resol instead of dynare_resolve.
 - Removed definition of constant and trend which are not used in nonlinear filters.
 - Cosmetic changes.
2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 6418e225bf Rewrote doc header. 2019-12-20 11:31:56 +01:00
Sébastien Villemot 996bdd6c64 New local_state_space_iteration_k MEX, for nonlinear filters at k-order
It applies the approximated policy function to a set of particles, using
Dynare++ routines.

There is support for parallelization, using Dynare++ multithreading
model (itself based on C++11 threads; we don’t use OpenMP because it is
incompatible with MKL). For the time being, default to a single thread. This
should be later refined through empirical testing.
2019-12-20 11:31:56 +01:00
Houtan Bastani f8fb8c0450
add missing comments/copyright to function 2019-12-20 10:37:06 +01:00
Marco Ratto 31c29d08c9 provisions for making shock decompositions for epilogue variables.
In case the epilogue formula is non-linear, the non additive non-linear term is distributed proportionally to the size of the individual shock contribution.
It is triggered by new option with_epilogue, applicable to commands:
1) shock_decomposition, realtime_shock_decomposition,
where preprocessor should trigger
options_.shock_decomp.with_epilogue=true;
2) initial_condition_decomposition
where preprocessor should trigger
options_.initial_condition_decomp.with_epilogue=true;
2019-12-19 22:19:39 +01:00
Marco Ratto aa50724379 Changed cumfix==0: splits non-additive components proportionally to size of shock contribution 2019-12-19 22:15:04 +01:00
Sébastien Villemot 1ac7344e42
Rollback introduction of +get and +set folders
Under Octave, having namespaces called “get” and “set” overshadows the builtin
functions with the same names, which are needed for graphics manipulation.

Therefore we go back to the initial function naming scheme, but moving all
those functions under an “accessors” subdirectory.

Among other things, this is a revert of
e4134ab59b and
c5e86fcb59.

Ref. !1655, !1686
2019-12-19 17:20:38 +01:00
Sébastien Villemot efa6c6c682
“datafile” option of “perfect_foresight_setup” (and “simul”) now equivalent to “initval_file”
Ref. #1663
2019-12-19 14:58:54 +01:00
Sébastien Villemot 0ba453dd0a
Fix file permissions 2019-12-19 10:47:55 +01:00
Sébastien Villemot 2c9ea629bd Merge branch 'utilsx' into 'master'
Moved get and set utilities to +get and +set

See merge request Dynare/dynare!1686
2019-12-19 09:42:19 +00:00
Sébastien Villemot 6ba10b88f2
Preprocessor: various provisions for improvements to shock decomposition
Accordingly update the MATLAB routines, the testsuite, and the manual.

In particular, “squeeze_shock_decomp” has been renamed to
“squeeze_shock_decomposition” for consistency with other commands.

Ref. #1687, !1655
2019-12-18 11:56:57 +01:00
Marco Ratto a6e3e7256a trap possible issues in slice iterations and save info file on progress 2019-12-17 23:24:48 +01:00
Sébastien Villemot e2f91abcaf Merge branch 'master' into 'master'
utilities + plot shock decompositions + init condition decompositions

See merge request Dynare/dynare!1655
2019-12-17 18:21:31 +00:00
Willi Mutschler 5a8c206760 Added parameter derivatives of perturbation solution up to 3 order
# Preliminary comments
I finished the identification toolbox at orders two and three using the pruned state space system, but before I merge request this, I decided to first merge the new functionality to compute parameter derivatives of perturbation solution matrices at higher orders. So after this is approved, I merge the identification toolbox.
I guess @rattoma, @sebastien, and @michel are best choices to review this.
I outline the main idea first and then provide some more detailed changes I made to the functions.

***

# Main idea
This merge request is concerned with the *analytical*computation of the parameter derivatives of first, second and third order perturbation solution matrices, i.e. using _closed-form_ expressions to efficiently compute the derivative of  $g_x$ , $g_u$, $g_{xx}$, $g_{xu}$, $g_{uu}$, $g_{\sigma\sigma}$, $g_{xxx}$, $g_{xxu}$, $g_{xuu}$, $g_{uuu}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ *with respect to model parameters*  $\theta$.  Note that $\theta$ contains model parameters, stderr and corr parameters of shocks. stderr and corr parameters of measurement errors are not yet supported, (they can easily be included as exogenous shocks). The availability of such derivatives is beneficial in terms of more reliable analysis of model sensitivity and parameter identifiability as well as more efficient estimation methods, in particular for models solved up to third order, as it is well-known that numerical derivatives are a tricky business, especially for large models.

References for my approach are:
* Iskrev (2008, 2010) and Schmitt-Grohé and Uribe (2012, Appendix)  who were the first to compute the parameter derivatives analytically at first order, however, using inefficient (sparse) Kronecker products.
* Mutschler (2015) who provides the expressions for a second-order, but again using inefficient (sparse) Kronecker products.
* Ratto and Iskrev (2012) who show how the first-order system can be solved accurately, fast and efficiently using existing numerical algorithms for generalized Sylvester equations by taking the parameter derivative with respect to each parameter separately.
* Julliard and Kamenik (2004) who provide the perturbation solution equation system in tensor notation at any order k.
* Levintal (2017) who introduces permutation matrices to express the perturbation solution equation system in matrix notation up to fifth order.
Note that @rattoma already implemented the parameter derivatives of $g_x$ and $g_u$ analytically (and numerically), and I rely heavily on his work in `get_first_order_solution_params_derivs.m` (previously `getH.m`). My additions are mainly to this function and thus it is renamed to `get_perturbation_params_derivs.m`.

The basic idea of this merge request is to take the second- and third-order perturbation solution systems in Julliard and Kamenik (2004), unfold these into an equivalent matrix representation using permutation matrices as in Levintal (2017). Then extending Ratto and Iskrev (2012) one takes the derivative with respect to each parameter separately and gets a computational problem that is linear, albeit large, as it involves either solving generalized Sylvester equations or taking inverses of highly sparse matrices. I will now briefly summarize the perturbation solution system at third order and the system that results when taking the derivative with respect to parameters.

## Perturbation Solution
The following systems arise at first, second, and third order:
$(ghx): f_{x} z_{x} = f_{y_{-}^*} + f_{y_0} g_{x} + f_{y_{+}^{**}} g^{**}_{x} g^{*}_{x}= A g_{x} + f_{y_{-}^*}=0$

$(ghu): f_{z} z_{u} = f_{y_0} g_{u} + f_{y_{+}^{**}} g^{**}_{x} g^{*}_{u} + f_{u}= A g_u + f_u = 0$

$(ghxx) : A g_{xx} + B g_{xx} \left(g^{*}_{x} \otimes g^{*}_{x}\right) + f_{zz} \left( z_{x} \otimes z_{x} \right) = 0$

$(ghxu) : A g_{xu} + B g_{xx} \left(g^{*}_{x} \otimes g^{*}_{u}\right) + f_{zz} \left( z_{x} \otimes z_{u} \right) = 0$

$(ghuu) : A g_{uu} + B g_{xx} \left(g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zz} \left( z_{u} \otimes z_{u} \right) = 0$

$(ghs2) : (A+B) g_{\sigma\sigma} +  \left( f_{y^{**}_{+}y^{**}_{+}} \left(g^{**}_{u} \otimes g^{**}_{u}\right) + f_{y^{**}_{+}} g^{**}_{uu}\right)vec(\Sigma) = 0$

$(ghxxx) : A g_{xxx} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{x} \otimes g^{*}_{x}\right) + f_{y_{+}}g^{**}_{xx} \left(g^{*}_x \otimes g^{*}_{xx}\right)P_{x\_xx} + f_{zz} \left( z_{x} \otimes z_{xx} \right)P_{x\_xx} + f_{zzz} \left( z_{x} \otimes z_{x} \otimes z_{x} \right) = 0$

$(ghxxu) : A g_{xxu} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{x} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{x} \otimes z_{x} \otimes z_{u} \right) + f_{zz} \left( \left( z_{x} \otimes z_{xu} \right)P_{x\_xu} + \left(z_{xx} \otimes z_{u}\right) \right) + f_{y_{+}}g^{**}_{xx} \left( \left(g^{*}_{x} \otimes g^{*}_{xu}\right)P_{x\_xu} + \left(g^{*}_{xx} \otimes g^{*}_{u}\right) \right) = 0$

$(ghxuu) : A g_{xuu} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{x} \otimes z_{u} \otimes z_{u} \right)+ f_{zz} \left( \left( z_{xu} \otimes z_{u} \right)P_{xu\_u} + \left(z_{x} \otimes z_{uu}\right) \right) + f_{y_{+}}g^{**}_{xx} \left( \left(g^{*}_{xu} \otimes g^{*}_{u}\right)P_{xu\_u} + \left(g^{*}_{x} \otimes g^{*}_{uu}\right) \right) = 0$

$(ghuuu) : A g_{uuu} + B g_{xxx} \left(g^{*}_{u} \otimes g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{u} \otimes z_{u} \otimes z_{u} \right)+ f_{zz} \left( z_{u} \otimes z_{uu} \right)P_{u\_uu} + f_{y_{+}}g^{**}_{xx} \left(g^{*}_{u} \otimes g^{*}_{uu}\right)P_{u\_uu}  = 0$

$(ghx\sigma\sigma) : A g_{x\sigma\sigma} + B g_{x\sigma\sigma} g^{*}_x + f_{y_{+}} g^{**}_{xx}\left(g^{*}_{x} \otimes g^{*}_{\sigma\sigma}\right) + f_{zz} \left(z_{x} \otimes z_{\sigma\sigma}\right) + F_{xu_{+}u_{+}}\left(I_{n_x} \otimes vec(\Sigma)\right) = 0$
$F_{xu_{+}u_{+}} = f_{y_{+}^{\ast\ast}} g_{xuu}^{\ast\ast} (g_x^{\ast} \otimes I_{n_u^2}) + f_{zz} \left( \left( z_{xu_{+}} \otimes z_{u_{+}} \right)P_{xu\_u} + \left(z_{x} \otimes z_{u_{+}u_{+}}\right) \right) + f_{zzz}\left(z_{x} \otimes z_{u_{+}} \otimes z_{u_{+}}\right)$

$(ghu\sigma\sigma) : A g_{u\sigma\sigma} + B g_{x\sigma\sigma} g^{*}_{u} + f_{y_{+}} g^{**}_{xx}\left(g^{*}_{u} \otimes g^{*}_{\sigma\sigma}\right) + f_{zz} \left(z_{u} \otimes z_{\sigma\sigma}\right) + F_{uu_{+}u_{+}}\left(I_{n_u} \otimes vec(\Sigma_u)\right) = 0$
$F_{uu_{+}u_{+}} = f_{y_{+}^{\ast\ast}} g_{xuu}^{\ast\ast} (g_u^{\ast} \otimes I_{n_u^2})  + f_{zz} \left( \left( z_{uu_{+}} \otimes z_{u_{+}} \right)P_{uu\_u} + \left(z_{u} \otimes z_{u_{+}u_{+}}\right) \right) + f_{zzz}\left(z_{u} \otimes z_{u_{+}} \otimes z_{u_{+}}\right)$

A and B are the common perturbation matrices:

$A = f_{y_0} + \begin{pmatrix} \underbrace{0}_{n\times n_{static}} &\vdots& \underbrace{f_{y^{**}_{+}} \cdot g^{**}_{x}}_{n \times n_{spred}} &\vdots& \underbrace{0}_{n\times n_{frwd}}  \end{pmatrix}$and $B = \begin{pmatrix} \underbrace{0}_{n \times n_{static}}&\vdots & \underbrace{0}_{n \times n_{pred}} & \vdots & \underbrace{f_{y^{**}_{+}}}_{n \times n_{sfwrd}} \end{pmatrix}$

and $z=(y_{-}^{\ast}; y; y_{+}^{\ast\ast}; u)$ denotes the dynamic model variables as in `M_.lead_lag_incidence`, $y^\ast$ denote state variables, $y^{\ast\ast}$ denote forward looking variables, $y_+$ denote the variables with a lead, $y_{-}$ denote variables with a lag, $y_0$ denote variables at period t, $f$ the model equations, and $f_z$ the first-order dynamic model derivatives, $f_{zz}$ the second-order dynamic derivatives, and $f_{zzz}$ the third-order dynamic model derivatives. Then:
$z_{x} = \begin{pmatrix}I\\g_{x}\\g^{**}_{x} g^{*}_{x}\\0\end{pmatrix}$, $z_{u} =\begin{pmatrix}0\\g_{u}\\g^{**}_{x} \cdot g^{*}_{u}\\I\end{pmatrix}$, $z_{u_{+}} =\begin{pmatrix}0\\0\\g^{**}_{u}\\0\end{pmatrix}$
$z_{xx} = \begin{pmatrix} 0\\g_{xx}\\g^{**}_{x} \left( g^{*}_x \otimes g^{*}_{x} \right) + g^{**}_{x} g^{*}_{x}\\0\end{pmatrix}$, $z_{xu} =\begin{pmatrix}0\\g_{xu}\\g^{**}_{xx} \left( g^{*}_x \otimes g^{*}_{u} \right) + g^{**}_{x} g^{*}_{xu}\\0\end{pmatrix}$, $z_{uu} =\begin{pmatrix}0\\g_{uu}\\g^{**}_{xx} \left( g^{*}_u \otimes g^{*}_{u} \right) + g^{**}_{x} g^{*}_{uu}\\0\end{pmatrix}$,
$z_{xu_{+}} =\begin{pmatrix}0\\0\\g^{**}_{xu} \left( g^{*}_x \otimes I \right)\\0\end{pmatrix}$, $z_{uu_{+}} =\begin{pmatrix}0\\0\\g^{**}_{xu} \left( g^{*}_{u} \otimes I \right)\\0\end{pmatrix}$, $z_{u_{+}u_{+}} =\begin{pmatrix}0\\0\\g^{\ast\ast}_{uu}\\0\end{pmatrix}$, $z_{\sigma\sigma} = \begin{pmatrix}0\\ g_{\sigma\sigma}\\ g^{\ast\ast}_{x}g^{\ast}_{\sigma\sigma} + g^{\ast\ast}_{\sigma\sigma}\\0 \end{pmatrix}$

$P$ are permutation matrices that can be computed using Matlab's `ipermute` function.

## Parameter derivatives of perturbation solutions
First, we need the parameter derivatives of first, second, third, and fourth derivatives of the dynamic model (i.e. g1,g2,g3,g4 in dynamic files), I make use of the implicit function theorem: Let $f_{z^k}$ denote the kth derivative (wrt all dynamic variables) of the dynamic model, then let $df_{z^k}$ denote the first-derivative (wrt all model parameters) of $f_{z^k}$ evaluated at the steady state. Note that $f_{z^k}$  is a function of both the model parameters $\theta$  and of the steady state of all dynamic variables $\bar{z}$, which also depend on the parameters. Hence, implicitly $f_{z^k}=f_{z^k}(\theta,\bar{z}(\theta))$  and $df_{z^k}$ consists of two parts:
1. direct derivative wrt to all model parameters given by the preprocessor in the `_params_derivs.m` files
2. contribution of derivative of steady state of dynamic variables (wrt all model parameters): $f_{z^{k+1}} \cdot d\bar{z}$
Note that we already have functionality to compute $d\bar{z}$ analytically.

Having this, the above perturbation systems are basically equations of the following types
$AX +BXC = RHS$ or $AX = RHS$
Now when taking the derivative (wrt to one single parameter $\theta_j$), we get
$A\mathrm{d}\{X\} + B\mathrm{d}\{X\}C = \mathrm{d}\{RHS\} - \mathrm{d}\{A\}X -  \mathrm{d}\{B\}XC - BX\mathrm{d}\{C\}$
or
$A\mathrm{d}\{X\}  = \mathrm{d}\{RHS\} - \mathrm{d}\{A\}X$
The first one is a Sylvester type equation, the second one can be solved by taking the inverse of $A$. The only diffculty and tedious work arrises in computing (the highly sparse) derivatives of $RHS$.

***

# New functions: `
## get_perturbation_params_derivs.m`and `get_perturbation_params_derivs_numerical_objective.m`
* The parameter derivatives up to third order are computed in the new function`get_perturbation_params_derivs.m` both analytically and numerically. For numerical derivatives `get_perturbation_params_derivs_numerical_objective.m` is the objective for `fjaco.m` or `hessian_sparse.m` or `hessian.m`.
* `get_perturbation_params_derivs.m` is basically an extended version of the previous `get_first_order_solution_params_derivs.m` function.
* * `get_perturbation_params_derivs_numerical_objective.m`builds upon `identification_numerical_objective.m`. It is used for numerical derivatives, whenever `analytic_derivation_mode=-1|-2`. It takes from `identification_numerical_objective.m` the parts that compute numerical parameter Jacobians of steady state, dynamic model equations, and perturbation solution matrices. Hence, these parts are removed in `identification_numerical_objective.m` and it only computes numerical parameter Jacobian of moments and spectrum which are needed for identification analysis in `get_identification_jacobians.m`, when `analytic_derivation_mode=-1` only.
* Detailed changes:
      * Most important: notation of this function is now in accordance to the k_order_solver, i.e. we do not compute derivatives of Kalman transition matrices A and B, but rather the solution matrices ghx,ghu,ghxx,ghxu,ghuu,ghs2,ghxxx,ghxxu,ghxuu,ghuuu,ghxss,ghuss in accordance with notation used in `oo_.dr`. As a byproduct at first-order, focusing on ghx and ghu instead of Kalman transition matrices A and B makes the computations slightly faster for large models (e.g. for Quest the computations were faster by a couple of seconds, not much, but okay).
      * Removed use of `kstate`, see also Dynare/dynare#1653 and Dynare/dynare!1656
      * Output arguments are stored in a structure `DERIVS`, there is also a flag `d2flag` that computes parameter hessians needed only in `dsge_likelihood.m`.
      * Removed `kronflag` as input. `options_.analytic_derivation_mode` is now used instead of `kronflag`.
      * Removed `indvar`, an index that was used to selected specific variables in the derivatives. This is not needed, as we always compute the parameter derivatives for all variables first and then select a subset of variables. The selection now takes place in other functions, like `dsge_likelihood.m`.
      * Introduced some checks: (i) deterministic exogenous variables are not supported, (ii) Kronecker method only compatible with first-order approximation so reset to sylvester method, (iii) for purely backward or forward models we need to be careful with the rows in `M_.lead_la	g_incidence`, (iv) if `_params_derivs.m` files are missing an error is thrown.
      * For numerical derivatives, if mod file does not contain an `estimated_params_block`, a temporary one with the most important parameter information is created.
## `unfold_g4.m`
* When evaluating g3 and g4 one needs to take into account that these do not contain symmetric elements, so one needs to use `unfold_g3.m` and the new function `unfold_g4.m`. This returns an unfolded version of the same matrix (i.e. with symmetric elements).

***

# New test models
`.gitignore` and `Makefile.am` are changed accordingly. Also now it is possible to run test suite on analytic_derivatives, i.e. run `make check m/analytic_derivatives`

## `analytic_derivatives/BrockMirman_PertParamsDerivs.mod`
* This is the Brock Mirman model, where we know the exact policy function $g$ for capital and consumption. As this does not imply a nonzero $g_{\sigma\sigma}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ I added some artificial equations to get nonzero solution matrices with respect to $\sigma$. The true perturbation solution matrices  $g_x$ , $g_u$, $g_{xx}$, $g_{xu}$, $g_{uu}$, $g_{\sigma\sigma}$, $g_{xxx}$, $g_{xxu}$, $g_{xuu}$, $g_{uuu}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ are then computed analytically with Matlab's symbolic toolbox and saved in `nBrockMirmanSYM.mat`. There is a preprocessor flag that recreates these analytical computations if changes are needed (and to check whether I made some errors here ;-) )
* Then solution matrices up to third order and their parameter Jacobians are then compared to the ones computed by Dynare's `k_order_solver` and by `get_perturbation_params_derivs` for all `analytic_derivation_mode`'s. There will be an error if the maximum absolute deviation is too large, i.e. for numerical derivatives (`analytic_derivation_mode=-1|-2`) the tolerance is choosen lower (around 1e-5); for analytical methods we are stricter: around 1e-13 for first-order,  1e-12 for second order, and 1e-11 for third-order.
* As a side note, this mod file also checks Dynare's `k_order_solver` algorithm and throws an error if something is wrong.
* This test model shows that the new functionality works well. And analytical derivatives perform way better and accurate than numerical ones, even for this small model.
## `analytic_derivatives/burnside_3_order_PertParamsDerivs.mod`
* This builds upon `tests/k_order_perturbation/burnside_k_order.mod` and computes the true parameter derivatives analytically by hand.
      * This test model also shows that the new functionality works well.

## `analytic_derivatives/LindeTrabandt2019.mod`
* Shows that the new functionality also works for medium-sized models, i.e. a SW type model solved at third order with 35 variables (11 states). 2 shocks and 20 parameters.
* This mod file can be used to tweak the speed of the computations in the future.
* Compares numerical versus analytical parameter derivatives (for first, second and third order). Note that this model clearly shows that numerical ones are quite different than analytical ones even at first order!
## `identification/LindeTrabandt2019_xfail.mod`
* This model is a check for issue Dynare/dynare#1595, see fjaco.m below, and will fail.
* Removed `analytic_derivatives/ls2003.mod` as this mod file is neither in the testsuite nor does it work.

***

# Detailed changes in other functions
## `get_first_order_solution_params_derivs.m`
* Deleted, or actually, renamed to `get_perturbation_params_derivs.m`, as this function now is able to compute the derivatives up to third order

## `identification_numerical_objective.m`
* `get_perturbation_params_derivs_numerical_objective.m`builds upon `identification_numerical_objective.m`. It takes from `identification_numerical_objective.m` the parts that compute numerical parameter Jacobians of steady state, dynamic model equations, and perturbation solution matrices. Hence, these parts are removed in `identification_numerical_objective.m` and it only computes numerical parameter Jacobian of moments and spectrum which are needed for identification analysis in `get_identification_jacobians.m`, when `analytic_derivation_mode=-1` only.

## `dsge_likelihood.m`
* As `get_first_order_solution_params_derivs.m`is renamed to `get_perturbation_params_derivs.m`, the call is adapted. That is,`get_perturbation_params_derivs` does not compute the derivatives of the Kalman transition `T`matrix anymore, but instead of the dynare solution matrix `ghx`. So we recreate `T` here as this amounts to adding some zeros and focusing on selected variables only.
* Added some checks to make sure the first-order approximation is selected.
* Removed `kron_flag` as input, as `get_perturbation_params_derivs` looks into `options_.analytic_derivation_mode` for `kron_flag`.

## `dynare_identification.m`
* make sure that setting `analytic_derivation_mode` is set both in `options_ident` and `options_`. Note that at the end of the function we restore the `options_` structure, so all changes are local. In a next merge request, I will remove the global variables to make all variables local.

## `get_identification_jacobians.m`
* As `get_first_order_solution_params_derivs.m`is renamed to `get_perturbation_params_derivs.m`, the call is adapted. That is,`get_perturbation_params_derivs` does not compute the derivatives of the Kalman transition `A` and `B` matrix anymore, but instead of the dynare solution matrix `ghx` and `ghu`. So we recreate these matrices here instead of in `get_perturbation_params_derivs.m`.
* Added `str2func` for better function handles in `fjaco.m`.

## `fjaco.m`
* make `tol`an option, which can be adjusted by changing `options_.dynatol.x`for identification and parameter derivatives purposes.
* include a check and an informative error message, if numerical derivatives (two-sided finite difference method) yield errors in `resol.m` for identification and parameter derivatives purposes. This closes issue  Dynare/dynare#1595.
* Changed year of copyright to 2010-2017,2019

***

# Further suggestions and questions
* Ones this is merged, I will merge request an improvement of the identification toolbox, which will work up to third order using the pruned state space. This will also remove some issues and bugs, and also I will remove global variables in this request.
* The third-order derivatives can be further improved by taking sparsity into account and use mex versions for kronecker products etc. I leave this for further testing (and if anybody actually uses this ;-) )
2019-12-17 18:17:09 +00:00
Marco Ratto c5e86fcb59 Moved
get_param_by_name --> get.param_by_name
set_param_value --> set.param_value
plus the additional set utility:
set.shock_stderr_value
\
2019-12-17 17:42:25 +01:00
Marco Ratto 9f721c5763 renamed utilities to +get format 2019-12-17 09:44:15 +01:00
Sébastien Villemot 279bb7bc16 Merge branch 'mh_recover' into 'master'
make mh_recover robust to crashed parallel jobs

See merge request Dynare/dynare!1684
2019-12-16 09:30:48 +00:00
Marco Ratto 9a07171a7c trap case where there is no list of variables to squeeze 2019-12-15 17:05:47 +01:00
Marco Ratto 65d72866c3 provisions for squeeze when oo_ is output argument of plot_shock_decomposition.
Also trap with error situation when new computations are triggered after having squeezed results in oo_.
2019-12-15 16:53:43 +01:00
Marco Ratto 17e87e2a4c added steady state info on xls file shock decomposition 2019-12-15 15:40:10 +01:00
Marco Ratto 4c6b803945 use optional variable list as fourth input argument, to complement automatic list based on options_.plot_shock_decomp.i_var. 2019-12-15 15:40:10 +01:00
Marco Ratto 44eae1300d trap plot_end_date larger than actual length of smoother 2019-12-15 15:40:10 +01:00
Marco Ratto eb73cf4273 implement provisions for options_.no_graph.plot_shock_decomposition 2019-12-15 15:40:10 +01:00
Marco Ratto 83f38c9533 check also whether shock_decomposition field exists (happens when skipinsample=0 and realtime=1) 2019-12-15 15:40:10 +01:00
Marco Ratto 71fa8cac9f fixed bug with steady state of annualized variables requiring auxiliary while doing expand 2019-12-15 15:40:10 +01:00
Marco Ratto ac511ca756 bug fix for computing shock decomp for auxiliary variable with groups 2019-12-15 15:40:10 +01:00
Marco Ratto c8c473cc60 provisions for changing the preprocessor call to plot_shock_decoposition with one output argument 2019-12-15 15:40:10 +01:00
Marco Ratto 6932ac4993 fixed name of new option to a more meaningful one: max_nrows. 2019-12-15 15:40:10 +01:00
Marco Ratto 019545e2b3 fixed bugs in interactive mode of plot shock decomp that led to crashed with aoa 2019-12-15 15:40:10 +01:00
Marco Ratto 2d1402b899 to reduce memory in oo_, only store results for periods specified in save_realtime 2019-12-15 15:40:10 +01:00
Marco Ratto cc11a054c4 when varlist=0, we only compute decomposition and return restuls in oo_ without plotting. useful when squeezed oo_ misses initval decomp for requested variables. 2019-12-15 15:40:10 +01:00
Marco Ratto 7d70a1917b fixed bug of aoa with aux variable. completed computation of annualized_realtime_forecast_shock_decomposition.pool which so far triggered an error (vintage=0 in realtime shock decomp) 2019-12-15 15:40:10 +01:00
Marco Ratto 8b9d7490e3 plot_shock_decomposition stores info of plotted variables. This can be optionally used by squeeze_shock_decomp to store in oo_ only variables plotted so far. Users can optionally define the list of vars to sotre in squeezed oo_ for shock decomps. Also allow aoa plots for lists of input variables. fixed error in wrong field name var_type of q2a. fixed aoa call, storing the wrong decomp data for interactive mode. 2019-12-15 15:40:10 +01:00
Marco Ratto f167c01eee trap case when option is cell 2019-12-15 15:40:10 +01:00
Marco Ratto 3cbb039fbf utilities to get steadystate smoothed updated of endogenous by name, irfs by shock and endo name, shock std err by shock name 2019-12-15 15:40:10 +01:00
Sébastien Villemot 416532b6c3
dseries: new “nanmean” and “backcast” methods 2019-12-14 11:11:31 +01:00
Sébastien Villemot 7170ac0423 Merge branch 'ramsey' into 'master'
New Ramsey syntax

See merge request Dynare/dynare!1685
2019-12-13 17:32:42 +00:00
Sébastien Villemot ce4aa9a8e1
Preprocessor update
— deprecation warning for “simul” (#1683)
— Ramsey: use information from transformed model for filling
  M_.nonzero_hessian_eqs. (Closes: #1681)
  Also, M_.hessian_eq_zero is not generated if order = 1
— New field M_.endo_trends (#1648)
2019-12-13 18:22:12 +01:00
Johannes Pfeifer c0c3a4c05c evaluate_planner_objective.m: add warning if order>1 is used
Also modernizes output format
2019-12-13 14:37:00 +01:00
Marco Ratto da4baa5d50 make mh_recover robust to crashed parallel jobs 2019-12-12 22:24:30 +01:00
Sébastien Villemot 6b308ccbd8 Merge branch 'dynare_solve' into 'master'
dynare_solve.m: return with valid solution even if Jacobian is ill-behaved

See merge request Dynare/dynare!1682
2019-12-12 20:06:38 +00:00
Johannes Pfeifer 1b56a56e78 dynare_solve.m: return with valid solution even if Jacobian is ill-behaved
In pathological cases, the Jacobian at the initial but true steady state value is ill-behaved. In this case we should return with the valid steady state instead of trying random starting values
2019-12-12 19:08:43 +01:00
Marco Ratto 35b63e87d6 properly handle labelling, grouping and expand under init2shocks option 2019-12-12 16:45:57 +01:00
Marco Ratto 06c2b49897 provisions for init2shocks: initial conditions are added to its correspondiong shock for shock decompositions, while they are removed in initial condition decompositions. 2019-12-12 15:21:47 +01:00
Johannes Pfeifer 9208bcf049 Modified harmonic mean: Transform hard-coded tolerance to parameter
Closes https://git.dynare.org/Dynare/dynare/issues/1666
2019-12-12 11:49:57 +01:00
Dóra Kocsis 716b18989d Factorize interpret_resol_info into print_info. Closes: Dynare/dynare#1308 2019-12-09 17:02:15 +01:00
Sébastien Villemot 3f3f49c465 Merge branch 'flip_diff' into 'master'
Flip diff

See merge request Dynare/dynare!1677
2019-12-06 15:29:07 +00:00
Houtan Bastani dec7d4fdb4
dynare.m: simplify code 2019-12-06 12:01:58 +01:00
Houtan Bastani 3a223e9c08
fix uncommon bug in parsing dynare command line options
previously, `nopathchange` and `nopreprocessoroutput` were set even if they were values instead of option names.

`nopathchange` would further remove all options that contained `'nopathchange'`

e.g. `dynare example1.mod savemacro=nopathchange` would erroneously set `nopathchange` to true and would delete the `savemacro` option altogether

In the fix, just check that the match starts in position 1 as, if the argument passed is longer than the matching pattern (e.g. nopathchangee), the preprocessor will stop processing with a usage error
2019-12-06 12:01:57 +01:00
Marco Ratto 5fe495852e Intregrate full list of options to initial_condition_decompositions, including defaults.
Adapt figure names when initial conditions are ploteed instead of shocks
2019-12-05 16:50:18 +01:00
Marco Ratto a14b174f93 provide info about flip diff options in the name of files and figures 2019-12-05 14:41:06 +01:00
Marco Ratto 79bdde79e0 implement new options flip and diff for plotting decompositions 2019-12-05 13:20:12 +01:00
Stéphane Adjemian (Charybdis) 2184195826 Updated dseries submodule.
[ci skip]
2019-12-05 12:10:36 +01:00
Dóra Kocsis b9af92eb8a Save conditional forecast output in oo_. Closes: Dynare/dynare#1672 2019-11-29 15:25:05 +01:00
Houtan Bastani f2205ed4a1
add nograph option to plot_shock_decomposition
In updating the preprocessor, also update call to plot_icforecast
2019-11-29 15:17:41 +01:00
Houtan Bastani 14384c233f
reporting: correctly handle `showDate` option of report 2019-11-29 12:02:00 +01:00
Houtan Bastani 3ef8564279
reporting: add ability to make title page 2019-11-29 11:36:00 +01:00
Sébastien Villemot c5537e23ae Merge branch 'tolerance' into 'master'
Nonlinear solver: tolerance for first Newton iteration is now smaller than for subsequent ones

See merge request Dynare/dynare!1674
2019-11-27 13:21:51 +00:00
Michel Juillard f70f7761db Nonlinear solver: tolerance for first Newton iteration is now
smaller than for subsequent ones. Closes: #1668
2019-11-27 14:19:01 +01:00
Sébastien Villemot 4a8dd1e8fa Merge branch 'steadystate_file' into 'master'
Remove globals from user-defined steadystate file

See merge request Dynare/dynare!1673
2019-11-26 17:26:19 +00:00
Johannes Pfeifer 0c01c31491 Remove global variables from user-defined steady state files
M_ and options_ are now passed as an input; params is an output as in the _steadystate2.m files
2019-11-26 17:05:14 +01:00
Sébastien Villemot 55d7397d40
Remove utilites for converting oo_ between Dynare versions
It’s not clear that such utilities are useful (people should rather update
their MATLAB). It’s also a lot of work, and it’s probably better to focus on
documenting the incompatibilities.

Closes: #915
2019-11-26 16:26:50 +01:00
Michel Juillard d0247277f9 after check, oo_.dr isn't empty but oo_.dr.ghx + friends don't exist 2019-11-25 21:54:53 +01:00
Michel Juillard f6daa359b7 fix tolf and tolx options for sim1_purely_forward 2019-11-23 22:47:17 +01:00
Michel Juillard 2652cb0dd7 fix tolf and tolx options for sim1_purely_backward 2019-11-23 22:44:10 +01:00
Sébastien Villemot 900fbe86e7 Merge branch 'plot_identification' into 'master'
plot_identification.m: fix placing of white dots for infinite values

See merge request Dynare/dynare!1668
2019-11-22 09:29:26 +00:00
Johannes Pfeifer 8a171b82a8 plot_identification.m: fix placing of white dots for infinite values 2019-11-21 16:25:19 +01:00
Sébastien Villemot a327e2c9b8 Merge branch 'tex_subfolder' into 'master'
Move remaining LaTeX-file to subfolder

Closes #1655

See merge request Dynare/dynare!1667
2019-11-21 14:15:54 +00:00
Johannes Pfeifer 8e5fd8dc4e Move creation of subfolder outside of TeX-check
Required if we only want the figure
2019-11-21 11:58:16 +01:00
Johannes Pfeifer 0075776716 Laplace approximation: filter out non-positive definite Hessians
Closes https://git.dynare.org/Dynare/dynare/issues/1659
2019-11-21 11:32:13 +01:00
Johannes Pfeifer a59534a284 Move remaining figures and related TeX output from main folder to graphs 2019-11-21 10:14:48 +01:00
Johannes Pfeifer c6b757646e Move remaining LaTeX-files to subfolder
Closes https://git.dynare.org/Dynare/dynare/issues/1655
2019-11-21 10:14:48 +01:00
Stéphane Adjemian (Charybdis) 9f5f4b6981 Cosmetic changes. 2019-11-18 21:48:52 +01:00
Sébastien Villemot 98bdf76581
MATLAB R2009b compatibility fixes
— use XLS instead of XLSX in testsuite datafiles
— use optimset instead of optimoptions
— use quadv/quadl/quadgk instead of integral
— fix race condition in load_m_file_data_legacy.m tests
— use fallback implementation for intersect(…, 'stable')
2019-11-15 18:27:37 +01:00
Sébastien Villemot caf0c8e1f8
Trust region solver: recompute Jacobian only when necessary
Previously, the solver would recompute the Jacobian at every iteration. But, if
an iteration fails, the current point is not moved (only the radius of the
trust region changes), hence it is not necessary to recompute the Jacobian in
that case. This commit implements this optimization.
2019-11-14 16:03:50 +01:00
Dóra Kocsis 4c49eddb97 Add fallback implementation for contains and splitlines functions. 2019-11-13 13:05:28 +01:00
Houtan Bastani 0f61c5de7d
add new features to reporting
* split up report output, creating new files for the preamble, the body of the report, and each individual page of the report.
* allow the user to set the page number
* allow for the removal of headers and footers from a given page
2019-11-06 13:50:21 -05:00
Houtan Bastani c978b86a72
reporting: make horizontal line extend the width of the page 2019-10-25 17:31:46 +02:00
Houtan Bastani 5f2d60f945
fix reporting bugs with addData 2019-10-24 12:03:33 +02:00
Houtan Bastani be3c48604b
change name of mex folder in macOS to conform with Dynare minimum 2019-10-23 18:59:36 +02:00
Houtan Bastani 908523dbb8
fix bug that caused reporting not to work in MATLAB < R2012a 2019-10-23 17:37:31 +02:00
Houtan Bastani ac689b83ee
fix bug in reporting on Windows where `filesep`t results in a tab character 2019-10-23 15:11:43 +02:00
Stéphane Adjemian (Charybdis) 1211d4c01e Updated dseries submodule (changed merge behaviour). 2019-10-16 20:58:49 +02:00