Commit Graph

1803 Commits (ec908386b67d1b27e0dc4174ef64796f74246318)

Author SHA1 Message Date
Johannes Pfeifer 7c8f486727 Allow setting initial state for Kalman filter
Supersedes https://git.dynare.org/Dynare/dynare/-/merge_requests/1522
2021-01-28 17:24:27 +01:00
Sébastien Villemot 6ac1af7035
Merge branch 'filtered_variables' of git.dynare.org:JohannesPfeifer/dynare 2021-01-28 16:53:30 +01:00
Sébastien Villemot af67b4f7c9
Testsuite: Octave 6 compatibility fix
In Octave ≤ 5, it was impossible to take a function handle for a function in a
package (“+” folder). This bug has been fixed in Octave 6, so we can use the
same codepath as MATLAB in that case. See
https://savannah.gnu.org/bugs/index.php?46659
2021-01-28 16:36:43 +01:00
Sébastien Villemot 25d19fef96
Testsuite: increase numerical tolerance for minimal_state_space_system/sw_minimal.mod
This is necessary for MATLAB R2014a on some runners.
2021-01-27 18:24:26 +01:00
Sébastien Villemot 6e6bc3f6c9
Testsuite: rename test files for Octave compatibility
Rename histval_initval_file/ramst_data.mod to ramst_data_generate.mod.
Otherwise, there is a conflict between the +ramst_data folder and the generated
ramst_data.m.

Similarily, rename shock_decomposition/shock_decomposition_backward.mod to
shock_decomposition/shock_decomp_backward.mod (otherwise it conflicts with
matlab/shock_decomposition_backward.m).
2021-01-27 18:24:22 +01:00
Johannes Pfeifer c596b8890e Trend_diagnostics_MCMC_common.inc: do not store smoother results on disk 2021-01-27 17:42:15 +01:00
Johannes Pfeifer 8ad8bf05b3 gsa/ls2003a.mod: reduce run-time 2021-01-26 10:39:38 +01:00
Johannes Pfeifer e9ea837b28 Reduce run-time of tune_mh_jscale/fs2000.mod 2021-01-26 10:10:29 +01:00
Johannes Pfeifer c507eefc23 testsuite: check values of analytic derivation likelihood to prevent regression 2021-01-24 17:41:50 +01:00
Johannes Pfeifer ec69174173
Improvements to particle tests 2021-01-22 21:19:07 +01:00
Sébastien Villemot c9a6637ca5
Testsuite: compatibility fix for MATLAB R2014a
Double quotes strings are not supported under older MATLAB versions.
2021-01-19 16:14:25 +01:00
Sébastien Villemot 783598bdeb
Testsuite / methods of moments: compatbility fix for MATLAB R2014a
MATLAB R2014a does not like some of the variable names that are passed to
table(). Disable that call, as under Octave.
2021-01-19 16:10:40 +01:00
Sébastien Villemot 68af1c80fe
Testsuite: cosmetic changes to TeX and debug options
– Whenever possible, pass the “tex” option to Dynare commands instead of
  fiddling with options_.TeX;
— For the remaining explicit options_.TeX and also for options_.debug,
  use a boolean value instead of an integer.
2021-01-19 15:50:14 +01:00
Sébastien Villemot 654da6ecb8 Merge branch 'octave_fixes' into 'master'
Don't do bgp/solow-1 in Octave

See merge request Dynare/dynare!1780
2021-01-18 11:28:36 +00:00
Stéphane Adjemian (Charybdis) 78a4bca383
Use nonlinear filter. 2021-01-16 17:43:30 +01:00
Stéphane Adjemian (Charybdis) 0f3dd003b7
Run mod files in tests/particle. 2021-01-16 17:42:17 +01:00
Johannes Pfeifer ad17b8ec33
Particle filtering: provide more informative error messages in typical problematic cases
Also adds test cases
2021-01-15 22:11:20 +01:00
Sébastien Villemot 91b0ba64c6
Testsuite: rename all test files that have the name “example.mod”
That filename conflicts with Octave’s builtin example() function.
2021-01-15 17:19:41 +01:00
Sébastien Villemot 23cb1879e9
Testsuite: fix PAC test filename 2021-01-15 17:19:41 +01:00
Willi Mutschler 0a880825a1
MoM: Testsuite fixes for Octave 2021-01-14 10:04:10 +01:00
Willi Mutschler a697002a4e
MoM: Gradient-based optimizers can use analytical Jacobian for GMM
New option analytic_jacobian for mode_compute 1, 3, and 13
2021-01-14 10:03:39 +01:00
Stéphane Adjemian (Odysseus) b87690a9f1 Revert "Add tests for matrix notation"
This reverts commit 8d4a80956c.
2021-01-12 16:47:56 +01:00
Stéphane Adjemian (Odysseus) 0e1c9312e2
Merge remote-tracking branch 'Dynare/master' into enterprise 2021-01-11 11:06:55 +01:00
Willi Mutschler dabe0ada33
MoM: Add test for optimizers
Note that mode_compute=11 and 12 need some fixing
2021-01-07 14:14:13 +01:00
Willi Mutschler 2b554e0511
MoM: cosmetical changes to description of options in testsuite 2021-01-07 13:39:45 +01:00
Willi Mutschler ba8d5d364d
MoM: Update copyright years 2021-01-07 10:32:35 +01:00
Willi Mutschler 562a9c737f
MoM: Improve testsuite
- add Andreasen, Fernandez-Villaverde, Rubio-Ramirez (2017) test models
- move models to dedicated folders
- add `make m/method_of_moments` and `make o/method_of_moments` commands to run testsuite only for method of moments
2021-01-06 14:19:17 +01:00
Johannes Pfeifer 9c52aec23d Adjust output of loglinear option to make clear when a logged variable is used
- use of function for getting labels will allow to accommodate https://git.dynare.org/Dynare/dynare/-/issues/349
- incidentally fixes a bug in the display of the simulated variance decomposition for observables with measurement error
2020-12-23 13:53:44 +01:00
Sébastien Villemot 5deaca993b
Merge branch 'discretion' of git.dynare.org:JohannesPfeifer/dynare 2020-12-22 12:29:38 +01:00
Sébastien Villemot 3d912401d5
Testsuite: Octave compatibility fixes in histval_initval_file_unit_tests.m 2020-12-22 12:27:19 +01:00
Johannes Pfeifer 4b793da2c3 discretionary_policy: allow for non-linear model to be used 2020-12-18 18:22:39 +01:00
Willi Mutschler eae5e2f029
MoM: Add analytical standard errors for GMM
Reset analytic_derivation_mode for steadystate file parameter changes


MoM: Fix GMM analytical standard errors wrong dimensions in autocovs
2020-12-18 16:16:10 +01:00
Johannes Pfeifer 4cf9f7d1a7 stoch_simul.m: reports second moments based on pruned state space at order=2 when pruning is specified
Related to https://git.dynare.org/Dynare/dynare/-/merge_requests/1744
2020-12-17 09:45:28 +01:00
Stéphane Adjemian (Charybdis) e066556d7a
Added integration test for d8ddd9ef6. 2020-12-07 22:25:54 +01:00
Sébastien Villemot 1c0d1409ea
Fix the use of [static]/[dynamic] equations with the “block” option 2020-12-07 16:24:13 +01:00
Sébastien Villemot fdf5bfbb58
Testsuite: cosmetic fix 2020-12-07 15:10:06 +01:00
Sébastien Villemot 4cc946842b
Add further Octave 6 test for intersect(…, 'stable')
Forgotten in c160d2407.

[skip ci]
2020-12-04 18:41:53 +01:00
Sébastien Villemot 8ba1a15262
Remove outdated Occbin files 2020-12-03 10:50:58 +01:00
Sébastien Villemot c160d24074
Octave 6 implements intersect(…, 'stable')
[skip ci]
2020-11-30 18:26:50 +01:00
Willi Mutschler fcef3ca09f
Don't do bgp/solow in Octave
[skip ci]
The levenberg-marquardt algorithm is not available in octave's implementation of fsolve, so we check the verbatim block only in matlab
2020-11-26 17:12:57 +01:00
Sébastien Villemot f27852d1ec
Add test for mshocks block 2020-11-26 15:48:57 +01:00
Willi Mutschler 3c1087a369
Fix for sylvester3a for purely forward-looking models
In purely forward-looking models ghx is empty and sylvester3a gives a "Operands to the || and && operators must be convertible to logical scalar values" error, as e becomes a "1×0 empty double row vector" and not a logical.
2020-11-21 09:46:10 +01:00
Johannes Pfeifer 904296738d bgp-tests: Octave compatibility fixes 2020-11-19 13:42:35 +01:00
Sébastien Villemot fcb52478e4
Preprocessor: replace exogenous with lead/lags by auxiliary variables in deterministic models
Several tests need to be adapted, because they were implicitly making the
assumption that there is no auxiliary variable.

Incidentally, this closes #1731. This commit therefore also removes the
workaround introduced in 0391dbbeb1.
2020-11-09 16:34:12 +01:00
Johannes Pfeifer d38a4428cc Move macro_processor/example1_macro.mod to subfolder 2020-11-02 16:00:03 +01:00
Johannes Pfeifer b4f3eb4a1c Add unit test for ifndef
See https://git.dynare.org/Dynare/dynare/-/issues/1747
2020-11-02 16:00:03 +01:00
Sébastien Villemot cf68570813
Testsuite: workaround for glibc bug #19329
This bug is causing random crashes in the testsuite on our runners.

See https://sourceware.org/bugzilla/show_bug.cgi?id=19329

(cherry picked from commit f46dfba961)
2020-10-23 15:32:07 +02:00
Sébastien Villemot 299769c51b
Testsuite/MATLAB: output stderr to log file
(cherry picked from commit 2a472a0a6b)
2020-10-23 15:32:06 +02:00
Sébastien Villemot f46dfba961
Testsuite: workaround for glibc bug #19329
This bug is causing random crashes in the testsuite on our runners.

See https://sourceware.org/bugzilla/show_bug.cgi?id=19329
2020-10-23 12:35:35 +02:00
Sébastien Villemot 2a472a0a6b
Testsuite/MATLAB: output stderr to log file 2020-10-22 18:25:42 +02:00
Sébastien Villemot 6f415dff61
Testsuite: add missing line continuation symbol in the list of .mod files
This was causing some tests to be skipped.

The bug had been introduced e7162a2a69.
2020-10-21 16:36:18 +02:00
Sébastien Villemot e21cb1ad6b
LMMCP: fix the purely backward and purely forward cases
Closes: #1720
2020-10-21 16:30:40 +02:00
Sébastien Villemot 33d8bc55f3
Redisable tests/pac/trend-component-31/example.mod, it is not yet ready (some fixes are needed in the M code) 2020-10-16 18:11:01 +02:00
Sébastien Villemot 8e3127d69e
Preprocessor rebase
– PAC model: fixes to M_.pac.MODEL.ar (information about autoregressive part)
  + Fix order of items in this structure. Previously, items were ordered
    according to the declaration order of parameters. Now, items are order
    according to lag order (first lag appears first)
  + Gracefully handle the case where there is no autoregressive part
    (Ref. Dynare/preprocessor#52)
– Block decomposition:
  + bugfix for solve one boundary blocks with mfs>0
  + fix evaluation of recursive variables in solved blocks
  (Ref. Dynare/dynare#1726)
— Fix various potential bugs with model local variable
— Ensure that negative constants are never marked as temporary terms

By the way, activate tests/pac/trend-component-31/example.mod, since it should
now work.
2020-10-16 16:53:26 +02:00
Sébastien Villemot 7270f722e4
Activate test for two PAC equations in the same file 2020-10-16 16:30:38 +02:00
Sébastien Villemot 71e3e0d49e
Fix extended_path with bytecode
By the way, document and enforce the fact that stochastic extended_path (i.e.
order > 0) is not compatible with either bytecode or block.

Closes: #1742
2020-10-14 17:25:58 +02:00
Sébastien Villemot 43311a230f
Testsuite: histval_initval_file_unit_tests.m is not a TLS file 2020-10-14 17:16:35 +02:00
Sébastien Villemot 8acdb9f7cc
Testsuite: MATLAB R2014a compatibility fixes 2020-10-08 17:26:17 +02:00
Sébastien Villemot e7162a2a69
Block decomposition: new tests for mfs>0
Ref. #1726
2020-10-07 19:01:09 +02:00
Stéphane Adjemian (Charybdis) 80bfbc0736
Merge non linear shock decomposition for backward models.
Closes #2.
2020-09-29 18:17:25 +02:00
Sébastien Villemot 8d4a80956c
Add tests for matrix notation
Closes: Enterprise/dynare#1
2020-09-29 16:52:20 +02:00
Sébastien Villemot 66361eb28d
Add test for shock decomposition of backward models
Ref: Enterprise/dynare#2
2020-09-28 18:26:29 +02:00
Stéphane Adjemian (Charybdis) e690c5de5a
Merge Enterprise/block_trust_region into enterprise. 2020-09-24 16:05:55 +02:00
Stéphane Adjemian (Charybdis) 17e6421ea3
Merged master into enterprise.
Fixed conflict in
 - doc/manual/source/running-dynare.rst
2020-09-23 15:28:19 +02:00
Stéphane Adjemian (Charybdis) a40f30cada
Closes #1761 (Fixes for histval_initval_file) 2020-09-23 12:19:18 +02:00
Sébastien Villemot d61f4166ad
Testsuite: add a test for a purely forward variable in block+bytecode mode
Closes: #1727
2020-09-22 16:46:58 +02:00
Michel Juillard c2e43c0093 fix histval_initval_file unit tests 2020-09-20 11:11:26 +02:00
Michel Juillard 9e94252d1a fix target values for periods 2020-09-19 12:18:24 +02:00
Michel Juillard 91b4cfd3a8 fixes handling of periods in histval_file/initval_file 2020-09-19 12:14:43 +02:00
Sébastien Villemot 865ab47fa9
Provide block_trust_region MEX under solve_algo 13 and 14
- block trust region solver now available under solve_algo=13
  It is essentially the same as solve_algo=4, except that Jacobian by finite
  difference is not handled. A test file is added for that case
- block trust region solver with shortcut for equations that can be evaluated
  is now available under solve_algo=14 (in replacement of the pure-MATLAB solver)

Closes: Enterprise/dynare#3
2020-09-18 18:01:56 +02:00
Sébastien Villemot fcc3a3cec2
Testsuite: fix repeated typo in equation name
[skip ci]
2020-09-18 17:40:44 +02:00
Stéphane Adjemian (Odysseus) 248ad18846 Merge remote-tracking branch 'Dynare/master' into enterprise
Fixed conflicts in:

 - .gitlab-ci.yml
 - matlab/dynare_config.m
 - matlab/dynare_solve.m
 - matlab/initvalf.m
 - matlab/perfect-foresight-models/make_ex_.m
 - matlab/perfect-foresight-models/perfect_foresight_setup.m
 - mex/build/matlab/Makefile.am
 - mex/build/matlab/configure.ac
 - mex/build/octave/Makefile.am
 - mex/build/octave/configure.ac
 - mex/sources/Makefile.am
 - preprocessor
 - tests/Makefile.am
2020-09-17 14:53:32 +02:00
Johannes Pfeifer 96be513ecf Remove matched_moments_ as input argument as it is now a field of M_ 2020-08-05 15:58:58 +02:00
Stéphane Adjemian (Odysseus) 579fb9e685 Fixed bug introduced in edede274f2. 2020-07-24 22:30:51 +02:00
Stéphane Adjemian (Charybdis) edede274f2
Make the detection of the target in PAC equation more robust.
In a PAC equation, the error correction term must be written as:

β×(yₜ₋₁-xₜ₋₁)

where x is the endogenous variable determined by the equation and y
the target for x. The ordering matters, if not respected the
preprocessor wll not identify the error correction term (hence the
target) and will throw an error. The target must be an endogenous
variable or the log of an endogenous variable. For a more general
target, ie an expression, one can create an auxiliary variable.

We impose the ordering, target comes first in the error correction
term, so that the stability condition is always β>0 (this constraint
can be enforced when estimating the PAC equation).
2020-07-24 17:36:31 +02:00
Stéphane Adjemian (Charybdis) 7f50b02910
Added test where the PAC equation doesn't have lags on the endogenous variable.
This example is currently failing.
2020-07-22 12:25:05 +02:00
Stéphane Adjemian (Charybdis) 897e2fc63b
Added test with two PAC equations in the same mod file. 2020-07-22 12:21:23 +02:00
Johannes Pfeifer 9fa24aa7be RBC_MoM: filter out additional pathological cases with try-catch 2020-07-17 20:35:53 +02:00
Johannes Pfeifer 791e5e9d97 RBC_MoM-test: filter out pathological cases
[skip CI]
2020-07-17 15:16:05 +02:00
Sébastien Villemot 1dbbef9f2e Merge branch 'moment_estimation' into 'master'
First implementation of moment estimation

See merge request Dynare/dynare!1750
2020-07-16 15:41:15 +00:00
Willi Mutschler 19b7d202d2 testsuite: Add test cases 2020-07-16 14:54:04 +02:00
Sébastien Villemot cda3253d2c Merge branch 'ME_traceplots' into 'master'
Fix trace plots with measurement errors and their correlations

See merge request Dynare/dynare!1752
2020-07-16 09:39:47 +00:00
Sébastien Villemot 499451d50a Merge branch 'pruned_state_space' into 'master'
Speed up pruned_state_space_system.m by e.g. using persistent variables

See merge request Dynare/dynare!1749
2020-07-16 09:39:01 +00:00
Johannes Pfeifer 540f0454d2 Code Review of GMM routines
- fix prefilter option
- Implement iterative GMM
2020-07-16 11:38:40 +02:00
Willi Mutschler 3615962a61 First draft of method of moments toolbox with GMM and SMM 2020-07-16 11:38:38 +02:00
Johannes Pfeifer 8a00ee3dff AnSchorfheide_pruned_state_space.mod: remove test part that only showcases that in Andreasen's code there is an error 2020-07-15 15:37:49 +02:00
Johannes Pfeifer f717712ed6 Fix trace plots with measurement errors and their correlations
Problem was introduced when transitioning to cell arrays
2020-07-15 14:34:04 +02:00
Sébastien Villemot 51278b2f90
Testsuite / External functions: also test coefficients of 2nd order policy function 2020-07-10 18:25:23 +02:00
Sébastien Villemot acdca03e43
Testsuite / External functions: add file missing in previous commit 2020-07-09 18:42:17 +02:00
Sébastien Villemot 5b88113668
Testsuite / External functions: add check for numerical results 2020-07-09 18:40:36 +02:00
Sébastien Villemot 4913cfb3f0 Merge branch 'third_order_moments' into 'master'
Add display of theoretical moments at order=3 with pruning

See merge request Dynare/dynare!1744
2020-06-30 14:38:22 +00:00
Johannes Pfeifer dfc7357636 Add display of theoretical moments at order=3 with pruning 2020-06-30 13:46:54 +02:00
Johannes Pfeifer ac728c0492 Add test cases for moments with measurement errors 2020-06-29 13:21:08 +02:00
Sébastien Villemot ee05ff6c6a
Block decomposition now compatible with 'use_dll' option 2020-06-23 15:49:27 +02:00
Willi Mutschler 42503e102e
✔️ Add unit tests for correct identification analysis 2020-06-20 22:38:47 +02:00
Willi Mutschler 97134cc362
🐛 Fix issue with endogenous params in steady state
Identification should switch to analytic_derivation_mode=-2 if steady state block/file changes parameter values. Dynare/dynare!1732 already adresses this when there is a prior specified. This fix also addresses this when there are no priors.

kim2.mod is now not only an integration test but also a unit test for this.
2020-06-20 22:38:47 +02:00
Sébastien Villemot d0fb9ce5b8
Preprocessor update
— Bug fix: allow external functions in model local variables (also add corresponding
  integration test)
— JSON output:
  + do not append two underscores after model local variable names in their
    definition
  + when writing the model after computing pass, use TEF terms for (not
    derived) external functions
  + write model local variables in parsed model (modfile.json) (Closes: #1723)
— In the driver file, provide mapping between model local variables and indices
  in the temporary terms vector (Closes: #1722)
2020-06-05 17:16:11 +02:00
Sébastien Villemot bd7eb2f88d
Fixes to solve_algo ≥ 9
— allow solve_algo=9 without block nor bytecode
— allow solve_algo≥9 with block without bytecode
— test solve_algo=9
2020-05-25 17:22:17 +02:00
Michel Juillard b70d99d1b4
Refactoring of initval_file and histval_file
initval_file and hisvfal_file are now more flexible and have functionalities
similar to option datafile in estimation.

Closes: #1671
2020-05-14 16:45:54 +02:00
Sébastien Villemot e5f45ef7b1
Testsuite: fix calls to fataltest.m 2020-05-13 14:58:09 +02:00
Sébastien Villemot ac1b7cc9e1
Testsuite: fix formula for maximum difference error test 2020-05-13 14:57:58 +02:00
Sébastien Villemot b2841ff98c
Testsuite: minor simplification 2020-05-13 12:57:14 +02:00
Sébastien Villemot 6e0f104d7e
Bump minimal MATLAB version to R2014a
Ref. #1713
2020-04-07 16:32:32 +02:00
Stéphane Adjemian (Charybdis) 046960c4a8
Merge remote-tracking branch community/master into enterprise 2020-04-03 11:33:46 +02:00
Johannes Pfeifer b8b05300c1 missing_DiffuseKalmanSmootherH1_Z.m: fix bug with variable dimension 2020-04-02 19:22:55 +02:00
Willi Mutschler 64a2a4388a
🐛 📖 Fixes TaRB with student t proposal distribution 2020-04-01 14:46:22 +02:00
Sébastien Villemot 43ed7b2b69
Merge remote-tracking branch 'community/master' into enterprise 2020-03-27 16:28:41 +01:00
Johannes Pfeifer 16cbfa7f29 noprint: add integration test 2020-03-08 09:01:00 +01:00
Sébastien Villemot 3ec1d15107
Fix interaction of “predetermined_variables” with “histval” (preprocessor#47)
Also add a corresponding regression test case.
2020-03-02 17:10:58 +01:00
Stéphane Adjemian c1541bebbf Merge branch 'Ramsey_estimation' into 'master'
testsuite: add integration test for estimation with Ramsey

See merge request Dynare/dynare!1719
2020-03-02 08:30:13 +00:00
Johannes Pfeifer 068a3e4bde testsuite: add integration test for estimation with Ramsey
[skip ci]
2020-03-02 09:17:36 +01:00
Stéphane Adjemian (Charybdis) 2083246a35
Add initial condition for mh_tune_jscale.
Closes #1717.
2020-02-29 13:25:24 +01:00
Johannes Pfeifer f89c1b58d8 Add integration test for steady state parameter updating 2020-02-25 14:48:29 +01:00
Johannes Pfeifer 3cf622cdc1 Output M_ in stoch_simul.m and discretionary_policy.m
Fixes https://git.dynare.org/Dynare/dynare/issues/1711 together with https://git.dynare.org/Dynare/preprocessor/-/merge_requests/15
2020-02-24 14:14:41 +01:00
Willi Mutschler f4dc2ee743
🐛 Fix wrong third-order computation in pruned state space system
Related to #1706

 Add unit test for pruned state space up to order 3


//


Changed tolerance in new test
2020-02-16 22:55:16 +01:00
Houtan Bastani 71def361fb
macro processor: add test of tuple index in @#for loop
[skip ci]
2020-02-14 17:29:41 +01:00
Sébastien Villemot 633f1ad85a
Testsuite: change the seed for some Octave test
Under Octave, the default seed in fs2000ns_uncorr_ME.mod leads to a generated
dataset that makes fs2000ns_corr_ME.mod and fs2000ns_corr_ME_missing.mod
fail (due to a large difference between univariate and multivariate diffuse
filters).
2020-02-13 18:12:10 +01:00
Sébastien Villemot 3b60e26e63
Testsuite: properly clean generated datafiles under kalman/likelihood_from_dynare/ 2020-02-12 18:02:08 +01:00
Johannes Pfeifer 30bfa16211 Fixes related to estimation under discretion 2020-02-06 13:30:00 +01:00
Sébastien Villemot a8e54d791a
Testsuite: set the random seed for unitary tests, for reproducibility 2020-02-04 18:19:35 +01:00
Sébastien Villemot 177c5c359f
Implement estimation under discretionary optimal policy
Also add a test case.

For the time being, estimating parameters that appear in the discount factor is
not supported.

Ref. #1173
2020-02-03 18:29:45 +01:00
Johannes Pfeifer 6cf5e6495a fs2000_simul.mod: make sure we use the estimated parameter vector
Makes the simulation converge without turning to homotopy
2020-01-31 13:22:47 +01:00
Stéphane Adjemian (Odysseus) d5e2b0f9f8 Updated integration test with new interface for aggregate routine. 2020-01-29 18:28:35 +01:00
Johannes Pfeifer 9e82432f87 Gali_discretion.mod: filter out NaN in planner_objective
Related to https://git.dynare.org/Dynare/dynare/issues/1699
2020-01-29 17:48:16 +01:00
Sébastien Villemot 7f02fe5a78
Testsuite: increase numerical tolerance for sw_minimal.mod test
Needed for some runners, e.g. lupi
2020-01-27 23:16:41 +01:00
Stéphane Adjemian (Charybdis) 82f642fbca
Added missing case in integration tests for particle filters.
There was not test for the conditional particle filter (CPF).
2020-01-27 22:57:41 +01:00
Sébastien Villemot fc27fad46d
Preprocessor: Do not simplify constant equations which have a tag attached
In particular, in the case of an 'mcp' tag, this could lead to a convergence
failure.

Also modify the lmmpc/rbcii.mod test so that it fails if convergence is not
obtained, in order to detect such regressions earlier.

Closes: #1697

This preprocessor update also includes the following changes:
— removal of license.txt, now merged in back in the main Dynare one
– various code simplifications and modernizations in the macro processor
— minor code simplification in parsing of the --+ options: … +-- line
2020-01-27 16:12:25 +01:00
Sébastien Villemot 01d5d734c6
CI: skip two further tests under MATLAB R2009b, that apparently trigger a MATLAB bug 2020-01-24 16:40:55 +01:00
Sébastien Villemot 3b9cc0e9f7
Testsuite: use whitespace-separated syntax for options in .mod file
Follows from d3e90a8dbf.
2020-01-24 16:01:17 +01:00
Willi Mutschler 3d51ee9e23
Added unit tests for minimal state space system 2020-01-24 14:20:05 +01:00
Willi Mutschler aa0f278edc
📃 Update license 2020-01-24 12:45:15 +01:00
Willi Mutschler 5525a7c515
🏇 Better minimal state space handling and unit tests 2020-01-24 12:45:08 +01:00
Willi Mutschler c4f7c416fa
🐛 Fix #1694 by robust rank tolerance and histc 2020-01-24 12:44:26 +01:00
Sébastien Villemot 66a5113500
Preprocessor: fix derivation order for “identification” without “stoch_simul”
Also add a regression test.

Closes: preprocessor#40
2020-01-20 17:28:05 +01:00
Sébastien Villemot e371b1a94b
New option “filtered_theoretical_moments_grid”, that supersedes “hp_ngrid”
The old option is left for backward-compatibility purposes, but it has the same
effect as the new one.

Closes: #1093
2020-01-20 16:23:10 +01:00
Sébastien Villemot a753c18d61
Testsuite: add various missing files to source tarball 2020-01-16 16:54:47 +01:00
Sébastien Villemot 3ee9ada8d3
Octave compatibility fix: skip unit test for load_m_file_data_legacy.m
The failure is impossible to reproduce outside the runners. Skip the test, for
lack of a better solution.
2020-01-14 14:58:38 +01:00
Sébastien Villemot 9bac6a0d32
Octave compatibility fix: skip some tests under Octave 5.1, due to a bug in ordeig 2020-01-14 11:49:41 +01:00
Sébastien Villemot c5f50a34e2
MATLAB compatibility fix: skip identification/kim/kim2.mod test under R2009b
MATLAB crashes, most likely due to an internal bug.
2020-01-14 11:45:03 +01:00
Sébastien Villemot 7c2338034f
Testsuite: drop useless version check (we require at least MATLAB R2009b/7.9) 2020-01-14 11:26:06 +01:00
Sébastien Villemot fd13063a5c
Octave compatibility fix: “fig” format is unsupported 2020-01-13 18:33:47 +01:00
Sébastien Villemot bde2f1977c
MATLAB compatibility fix: skip mode_compute=3 test under R2009b
In this version, fminunc fails due to Inf value.
2020-01-13 18:30:28 +01:00
Sébastien Villemot d9b9f78392 Merge branch 'particle_check' into 'master'
Particle filters: provide error if trends or prefiltering is used

Closes #1690

See merge request Dynare/dynare!1695
2020-01-13 11:02:29 +00:00
Johannes Pfeifer 23cac089f1 fs2000_ns_common.inc: set order of approximation explicitly to 1 2020-01-12 17:13:24 +01:00
Sébastien Villemot 7e770f69e7
Remove workaround for errors in MEX files
Because at some point throwing exceptions from MEX files (with mexErrMsgTxt())
was not working under Windows 64-bit, we had designed a workaround to avoid
using exceptions.

Most MEX files were returning an error code as their first (or sometimes last)
argument, and that code would have to be checked from the MATLAB code.

Since this workaround is no longer needed, this commit removes it. As a
consequence, the interface of many MEX files is modified.

For some background, see https://www.dynare.org/pipermail/dev/2010-September/000895.html
2020-01-10 18:33:11 +01:00
Houtan Bastani bf102030cb
support saving exogenous variables in `dynasave`, `dynasave`; fix bugs in `dynasave`; add test
- `dynasave`: if a variable being saved was named `n` or `s`, the `eval` statements would break the code
- `dynasave`: use the `-struct` option to `save` to avoid `eval` statements
- `dynasave` and `dynatype`: do everything in 1 loop instead of 2
- `dynasave` and `dynatype`: use `strcmp` instead of `strfind`

- preprocessor update contains:
  - Partial reversion of global indentation of macro processor header files introduced in e2d5a83592634f0604d8c86409748cd2ec5906d2
  - Symbol List check pass: allow caller to specify the valid types of variables in a Symbol List
  - Allow `dynasave` and `dynatype` to support exogenous variables in their var_list

issue #1691
2020-01-06 12:45:44 +01:00
Willi Mutschler 45e9771eb8
Fixed bug regarding non-stationary variables in pruned moments 2019-12-20 12:30:53 +01:00
Willi Mutschler 8b9b49f8d7
Finished identification order=1|2|3
Note that I still need to do a code clean up (provide some licenses for functions from other people) and to double check order=3. There is also much room for speed and memory improvement, but the code works fine for now. I will also provide more information to the merge request soon about the detailed changes for future reference.
2019-12-20 12:28:55 +01:00
Sébastien Villemot d40b775260
Preprocessor: new “with_epilogue” option and related fixes
Ref. !1688
2019-12-20 11:57:34 +01:00
Sébastien Villemot c04c111d97
Merge branch 'rattoma/dynare-epilogue'
Ref. !1688
2019-12-20 11:51:41 +01:00
Stéphane Adjemian (Charybdis) 78c36dd0b7 Fixed data files for nonlinear filter's integration test. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Charybdis) 53c9d9de69 Use steady_state_model in tests/particles/dsge_base2.mod. 2019-12-20 11:31:56 +01:00
Stéphane Adjemian (Odysseus) 58da5e7120 Added timing for comparing the mex iterating on the nonlinear reduced form model.
local_state_space_iteration_k is significantly slower than old local_state_space_iteration_2...
2019-12-20 11:31:56 +01:00
Sébastien Villemot 996bdd6c64 New local_state_space_iteration_k MEX, for nonlinear filters at k-order
It applies the approximated policy function to a set of particles, using
Dynare++ routines.

There is support for parallelization, using Dynare++ multithreading
model (itself based on C++11 threads; we don’t use OpenMP because it is
incompatible with MKL). For the time being, default to a single thread. This
should be later refined through empirical testing.
2019-12-20 11:31:56 +01:00
Marco Ratto 87963acb3a adapted test function for plot shock decompositions. still needs the new option with_epilogue. 2019-12-19 22:23:28 +01:00
Sébastien Villemot 1ac7344e42
Rollback introduction of +get and +set folders
Under Octave, having namespaces called “get” and “set” overshadows the builtin
functions with the same names, which are needed for graphics manipulation.

Therefore we go back to the initial function naming scheme, but moving all
those functions under an “accessors” subdirectory.

Among other things, this is a revert of
e4134ab59b and
c5e86fcb59.

Ref. !1655, !1686
2019-12-19 17:20:38 +01:00
Sébastien Villemot efa6c6c682
“datafile” option of “perfect_foresight_setup” (and “simul”) now equivalent to “initval_file”
Ref. #1663
2019-12-19 14:58:54 +01:00
Sébastien Villemot 7e07d3e0fc
Testsuite: add further check to verify that “initval_file” works as expected
Ref. #1663
2019-12-19 14:51:59 +01:00
Sébastien Villemot e81c837c51
Testsuite: fix engine for testing M scripts
I’m not sure it has ever worked.

As a consequence, remove the workaround that had apparently been implemented in
tests/initval_file/ramst_initval_file.mod.
2019-12-19 14:51:56 +01:00
Sébastien Villemot 2c9ea629bd Merge branch 'utilsx' into 'master'
Moved get and set utilities to +get and +set

See merge request Dynare/dynare!1686
2019-12-19 09:42:19 +00:00
Sébastien Villemot f720f470bf
Preprocessor update
— “ramsey_policy”: bugfix when no option is passed
— various improvements to “epilogue” (preprocessor#36)
— compatibility with Bison 3.5
— bugfix in search for constant equations
— new “planner_discount_latex_name” option of “ramsey_policy” (Closes: #1686)
2019-12-18 17:46:38 +01:00
Sébastien Villemot cc54fff571
Testsuite: fix typo in m/optimal_policy/Ramsey rule 2019-12-18 17:29:12 +01:00
Sébastien Villemot 6ba10b88f2
Preprocessor: various provisions for improvements to shock decomposition
Accordingly update the MATLAB routines, the testsuite, and the manual.

In particular, “squeeze_shock_decomp” has been renamed to
“squeeze_shock_decomposition” for consistency with other commands.

Ref. #1687, !1655
2019-12-18 11:56:57 +01:00
Marco Ratto e4134ab59b fixed calls to moved utilities 2019-12-17 22:26:38 +01:00
Stéphane Adjemian (Odysseus) 432faa3fae Fixed trailing whitespace warning.
[skip ci]
2019-12-17 21:59:13 +01:00
Sébastien Villemot e2f91abcaf Merge branch 'master' into 'master'
utilities + plot shock decompositions + init condition decompositions

See merge request Dynare/dynare!1655
2019-12-17 18:21:31 +00:00
Willi Mutschler 5a8c206760 Added parameter derivatives of perturbation solution up to 3 order
# Preliminary comments
I finished the identification toolbox at orders two and three using the pruned state space system, but before I merge request this, I decided to first merge the new functionality to compute parameter derivatives of perturbation solution matrices at higher orders. So after this is approved, I merge the identification toolbox.
I guess @rattoma, @sebastien, and @michel are best choices to review this.
I outline the main idea first and then provide some more detailed changes I made to the functions.

***

# Main idea
This merge request is concerned with the *analytical*computation of the parameter derivatives of first, second and third order perturbation solution matrices, i.e. using _closed-form_ expressions to efficiently compute the derivative of  $g_x$ , $g_u$, $g_{xx}$, $g_{xu}$, $g_{uu}$, $g_{\sigma\sigma}$, $g_{xxx}$, $g_{xxu}$, $g_{xuu}$, $g_{uuu}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ *with respect to model parameters*  $\theta$.  Note that $\theta$ contains model parameters, stderr and corr parameters of shocks. stderr and corr parameters of measurement errors are not yet supported, (they can easily be included as exogenous shocks). The availability of such derivatives is beneficial in terms of more reliable analysis of model sensitivity and parameter identifiability as well as more efficient estimation methods, in particular for models solved up to third order, as it is well-known that numerical derivatives are a tricky business, especially for large models.

References for my approach are:
* Iskrev (2008, 2010) and Schmitt-Grohé and Uribe (2012, Appendix)  who were the first to compute the parameter derivatives analytically at first order, however, using inefficient (sparse) Kronecker products.
* Mutschler (2015) who provides the expressions for a second-order, but again using inefficient (sparse) Kronecker products.
* Ratto and Iskrev (2012) who show how the first-order system can be solved accurately, fast and efficiently using existing numerical algorithms for generalized Sylvester equations by taking the parameter derivative with respect to each parameter separately.
* Julliard and Kamenik (2004) who provide the perturbation solution equation system in tensor notation at any order k.
* Levintal (2017) who introduces permutation matrices to express the perturbation solution equation system in matrix notation up to fifth order.
Note that @rattoma already implemented the parameter derivatives of $g_x$ and $g_u$ analytically (and numerically), and I rely heavily on his work in `get_first_order_solution_params_derivs.m` (previously `getH.m`). My additions are mainly to this function and thus it is renamed to `get_perturbation_params_derivs.m`.

The basic idea of this merge request is to take the second- and third-order perturbation solution systems in Julliard and Kamenik (2004), unfold these into an equivalent matrix representation using permutation matrices as in Levintal (2017). Then extending Ratto and Iskrev (2012) one takes the derivative with respect to each parameter separately and gets a computational problem that is linear, albeit large, as it involves either solving generalized Sylvester equations or taking inverses of highly sparse matrices. I will now briefly summarize the perturbation solution system at third order and the system that results when taking the derivative with respect to parameters.

## Perturbation Solution
The following systems arise at first, second, and third order:
$(ghx): f_{x} z_{x} = f_{y_{-}^*} + f_{y_0} g_{x} + f_{y_{+}^{**}} g^{**}_{x} g^{*}_{x}= A g_{x} + f_{y_{-}^*}=0$

$(ghu): f_{z} z_{u} = f_{y_0} g_{u} + f_{y_{+}^{**}} g^{**}_{x} g^{*}_{u} + f_{u}= A g_u + f_u = 0$

$(ghxx) : A g_{xx} + B g_{xx} \left(g^{*}_{x} \otimes g^{*}_{x}\right) + f_{zz} \left( z_{x} \otimes z_{x} \right) = 0$

$(ghxu) : A g_{xu} + B g_{xx} \left(g^{*}_{x} \otimes g^{*}_{u}\right) + f_{zz} \left( z_{x} \otimes z_{u} \right) = 0$

$(ghuu) : A g_{uu} + B g_{xx} \left(g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zz} \left( z_{u} \otimes z_{u} \right) = 0$

$(ghs2) : (A+B) g_{\sigma\sigma} +  \left( f_{y^{**}_{+}y^{**}_{+}} \left(g^{**}_{u} \otimes g^{**}_{u}\right) + f_{y^{**}_{+}} g^{**}_{uu}\right)vec(\Sigma) = 0$

$(ghxxx) : A g_{xxx} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{x} \otimes g^{*}_{x}\right) + f_{y_{+}}g^{**}_{xx} \left(g^{*}_x \otimes g^{*}_{xx}\right)P_{x\_xx} + f_{zz} \left( z_{x} \otimes z_{xx} \right)P_{x\_xx} + f_{zzz} \left( z_{x} \otimes z_{x} \otimes z_{x} \right) = 0$

$(ghxxu) : A g_{xxu} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{x} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{x} \otimes z_{x} \otimes z_{u} \right) + f_{zz} \left( \left( z_{x} \otimes z_{xu} \right)P_{x\_xu} + \left(z_{xx} \otimes z_{u}\right) \right) + f_{y_{+}}g^{**}_{xx} \left( \left(g^{*}_{x} \otimes g^{*}_{xu}\right)P_{x\_xu} + \left(g^{*}_{xx} \otimes g^{*}_{u}\right) \right) = 0$

$(ghxuu) : A g_{xuu} + B g_{xxx} \left(g^{*}_{x} \otimes g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{x} \otimes z_{u} \otimes z_{u} \right)+ f_{zz} \left( \left( z_{xu} \otimes z_{u} \right)P_{xu\_u} + \left(z_{x} \otimes z_{uu}\right) \right) + f_{y_{+}}g^{**}_{xx} \left( \left(g^{*}_{xu} \otimes g^{*}_{u}\right)P_{xu\_u} + \left(g^{*}_{x} \otimes g^{*}_{uu}\right) \right) = 0$

$(ghuuu) : A g_{uuu} + B g_{xxx} \left(g^{*}_{u} \otimes g^{*}_{u} \otimes g^{*}_{u}\right) + f_{zzz} \left( z_{u} \otimes z_{u} \otimes z_{u} \right)+ f_{zz} \left( z_{u} \otimes z_{uu} \right)P_{u\_uu} + f_{y_{+}}g^{**}_{xx} \left(g^{*}_{u} \otimes g^{*}_{uu}\right)P_{u\_uu}  = 0$

$(ghx\sigma\sigma) : A g_{x\sigma\sigma} + B g_{x\sigma\sigma} g^{*}_x + f_{y_{+}} g^{**}_{xx}\left(g^{*}_{x} \otimes g^{*}_{\sigma\sigma}\right) + f_{zz} \left(z_{x} \otimes z_{\sigma\sigma}\right) + F_{xu_{+}u_{+}}\left(I_{n_x} \otimes vec(\Sigma)\right) = 0$
$F_{xu_{+}u_{+}} = f_{y_{+}^{\ast\ast}} g_{xuu}^{\ast\ast} (g_x^{\ast} \otimes I_{n_u^2}) + f_{zz} \left( \left( z_{xu_{+}} \otimes z_{u_{+}} \right)P_{xu\_u} + \left(z_{x} \otimes z_{u_{+}u_{+}}\right) \right) + f_{zzz}\left(z_{x} \otimes z_{u_{+}} \otimes z_{u_{+}}\right)$

$(ghu\sigma\sigma) : A g_{u\sigma\sigma} + B g_{x\sigma\sigma} g^{*}_{u} + f_{y_{+}} g^{**}_{xx}\left(g^{*}_{u} \otimes g^{*}_{\sigma\sigma}\right) + f_{zz} \left(z_{u} \otimes z_{\sigma\sigma}\right) + F_{uu_{+}u_{+}}\left(I_{n_u} \otimes vec(\Sigma_u)\right) = 0$
$F_{uu_{+}u_{+}} = f_{y_{+}^{\ast\ast}} g_{xuu}^{\ast\ast} (g_u^{\ast} \otimes I_{n_u^2})  + f_{zz} \left( \left( z_{uu_{+}} \otimes z_{u_{+}} \right)P_{uu\_u} + \left(z_{u} \otimes z_{u_{+}u_{+}}\right) \right) + f_{zzz}\left(z_{u} \otimes z_{u_{+}} \otimes z_{u_{+}}\right)$

A and B are the common perturbation matrices:

$A = f_{y_0} + \begin{pmatrix} \underbrace{0}_{n\times n_{static}} &\vdots& \underbrace{f_{y^{**}_{+}} \cdot g^{**}_{x}}_{n \times n_{spred}} &\vdots& \underbrace{0}_{n\times n_{frwd}}  \end{pmatrix}$and $B = \begin{pmatrix} \underbrace{0}_{n \times n_{static}}&\vdots & \underbrace{0}_{n \times n_{pred}} & \vdots & \underbrace{f_{y^{**}_{+}}}_{n \times n_{sfwrd}} \end{pmatrix}$

and $z=(y_{-}^{\ast}; y; y_{+}^{\ast\ast}; u)$ denotes the dynamic model variables as in `M_.lead_lag_incidence`, $y^\ast$ denote state variables, $y^{\ast\ast}$ denote forward looking variables, $y_+$ denote the variables with a lead, $y_{-}$ denote variables with a lag, $y_0$ denote variables at period t, $f$ the model equations, and $f_z$ the first-order dynamic model derivatives, $f_{zz}$ the second-order dynamic derivatives, and $f_{zzz}$ the third-order dynamic model derivatives. Then:
$z_{x} = \begin{pmatrix}I\\g_{x}\\g^{**}_{x} g^{*}_{x}\\0\end{pmatrix}$, $z_{u} =\begin{pmatrix}0\\g_{u}\\g^{**}_{x} \cdot g^{*}_{u}\\I\end{pmatrix}$, $z_{u_{+}} =\begin{pmatrix}0\\0\\g^{**}_{u}\\0\end{pmatrix}$
$z_{xx} = \begin{pmatrix} 0\\g_{xx}\\g^{**}_{x} \left( g^{*}_x \otimes g^{*}_{x} \right) + g^{**}_{x} g^{*}_{x}\\0\end{pmatrix}$, $z_{xu} =\begin{pmatrix}0\\g_{xu}\\g^{**}_{xx} \left( g^{*}_x \otimes g^{*}_{u} \right) + g^{**}_{x} g^{*}_{xu}\\0\end{pmatrix}$, $z_{uu} =\begin{pmatrix}0\\g_{uu}\\g^{**}_{xx} \left( g^{*}_u \otimes g^{*}_{u} \right) + g^{**}_{x} g^{*}_{uu}\\0\end{pmatrix}$,
$z_{xu_{+}} =\begin{pmatrix}0\\0\\g^{**}_{xu} \left( g^{*}_x \otimes I \right)\\0\end{pmatrix}$, $z_{uu_{+}} =\begin{pmatrix}0\\0\\g^{**}_{xu} \left( g^{*}_{u} \otimes I \right)\\0\end{pmatrix}$, $z_{u_{+}u_{+}} =\begin{pmatrix}0\\0\\g^{\ast\ast}_{uu}\\0\end{pmatrix}$, $z_{\sigma\sigma} = \begin{pmatrix}0\\ g_{\sigma\sigma}\\ g^{\ast\ast}_{x}g^{\ast}_{\sigma\sigma} + g^{\ast\ast}_{\sigma\sigma}\\0 \end{pmatrix}$

$P$ are permutation matrices that can be computed using Matlab's `ipermute` function.

## Parameter derivatives of perturbation solutions
First, we need the parameter derivatives of first, second, third, and fourth derivatives of the dynamic model (i.e. g1,g2,g3,g4 in dynamic files), I make use of the implicit function theorem: Let $f_{z^k}$ denote the kth derivative (wrt all dynamic variables) of the dynamic model, then let $df_{z^k}$ denote the first-derivative (wrt all model parameters) of $f_{z^k}$ evaluated at the steady state. Note that $f_{z^k}$  is a function of both the model parameters $\theta$  and of the steady state of all dynamic variables $\bar{z}$, which also depend on the parameters. Hence, implicitly $f_{z^k}=f_{z^k}(\theta,\bar{z}(\theta))$  and $df_{z^k}$ consists of two parts:
1. direct derivative wrt to all model parameters given by the preprocessor in the `_params_derivs.m` files
2. contribution of derivative of steady state of dynamic variables (wrt all model parameters): $f_{z^{k+1}} \cdot d\bar{z}$
Note that we already have functionality to compute $d\bar{z}$ analytically.

Having this, the above perturbation systems are basically equations of the following types
$AX +BXC = RHS$ or $AX = RHS$
Now when taking the derivative (wrt to one single parameter $\theta_j$), we get
$A\mathrm{d}\{X\} + B\mathrm{d}\{X\}C = \mathrm{d}\{RHS\} - \mathrm{d}\{A\}X -  \mathrm{d}\{B\}XC - BX\mathrm{d}\{C\}$
or
$A\mathrm{d}\{X\}  = \mathrm{d}\{RHS\} - \mathrm{d}\{A\}X$
The first one is a Sylvester type equation, the second one can be solved by taking the inverse of $A$. The only diffculty and tedious work arrises in computing (the highly sparse) derivatives of $RHS$.

***

# New functions: `
## get_perturbation_params_derivs.m`and `get_perturbation_params_derivs_numerical_objective.m`
* The parameter derivatives up to third order are computed in the new function`get_perturbation_params_derivs.m` both analytically and numerically. For numerical derivatives `get_perturbation_params_derivs_numerical_objective.m` is the objective for `fjaco.m` or `hessian_sparse.m` or `hessian.m`.
* `get_perturbation_params_derivs.m` is basically an extended version of the previous `get_first_order_solution_params_derivs.m` function.
* * `get_perturbation_params_derivs_numerical_objective.m`builds upon `identification_numerical_objective.m`. It is used for numerical derivatives, whenever `analytic_derivation_mode=-1|-2`. It takes from `identification_numerical_objective.m` the parts that compute numerical parameter Jacobians of steady state, dynamic model equations, and perturbation solution matrices. Hence, these parts are removed in `identification_numerical_objective.m` and it only computes numerical parameter Jacobian of moments and spectrum which are needed for identification analysis in `get_identification_jacobians.m`, when `analytic_derivation_mode=-1` only.
* Detailed changes:
      * Most important: notation of this function is now in accordance to the k_order_solver, i.e. we do not compute derivatives of Kalman transition matrices A and B, but rather the solution matrices ghx,ghu,ghxx,ghxu,ghuu,ghs2,ghxxx,ghxxu,ghxuu,ghuuu,ghxss,ghuss in accordance with notation used in `oo_.dr`. As a byproduct at first-order, focusing on ghx and ghu instead of Kalman transition matrices A and B makes the computations slightly faster for large models (e.g. for Quest the computations were faster by a couple of seconds, not much, but okay).
      * Removed use of `kstate`, see also Dynare/dynare#1653 and Dynare/dynare!1656
      * Output arguments are stored in a structure `DERIVS`, there is also a flag `d2flag` that computes parameter hessians needed only in `dsge_likelihood.m`.
      * Removed `kronflag` as input. `options_.analytic_derivation_mode` is now used instead of `kronflag`.
      * Removed `indvar`, an index that was used to selected specific variables in the derivatives. This is not needed, as we always compute the parameter derivatives for all variables first and then select a subset of variables. The selection now takes place in other functions, like `dsge_likelihood.m`.
      * Introduced some checks: (i) deterministic exogenous variables are not supported, (ii) Kronecker method only compatible with first-order approximation so reset to sylvester method, (iii) for purely backward or forward models we need to be careful with the rows in `M_.lead_la	g_incidence`, (iv) if `_params_derivs.m` files are missing an error is thrown.
      * For numerical derivatives, if mod file does not contain an `estimated_params_block`, a temporary one with the most important parameter information is created.
## `unfold_g4.m`
* When evaluating g3 and g4 one needs to take into account that these do not contain symmetric elements, so one needs to use `unfold_g3.m` and the new function `unfold_g4.m`. This returns an unfolded version of the same matrix (i.e. with symmetric elements).

***

# New test models
`.gitignore` and `Makefile.am` are changed accordingly. Also now it is possible to run test suite on analytic_derivatives, i.e. run `make check m/analytic_derivatives`

## `analytic_derivatives/BrockMirman_PertParamsDerivs.mod`
* This is the Brock Mirman model, where we know the exact policy function $g$ for capital and consumption. As this does not imply a nonzero $g_{\sigma\sigma}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ I added some artificial equations to get nonzero solution matrices with respect to $\sigma$. The true perturbation solution matrices  $g_x$ , $g_u$, $g_{xx}$, $g_{xu}$, $g_{uu}$, $g_{\sigma\sigma}$, $g_{xxx}$, $g_{xxu}$, $g_{xuu}$, $g_{uuu}$, $g_{x\sigma\sigma}$, $g_{u\sigma\sigma}$ are then computed analytically with Matlab's symbolic toolbox and saved in `nBrockMirmanSYM.mat`. There is a preprocessor flag that recreates these analytical computations if changes are needed (and to check whether I made some errors here ;-) )
* Then solution matrices up to third order and their parameter Jacobians are then compared to the ones computed by Dynare's `k_order_solver` and by `get_perturbation_params_derivs` for all `analytic_derivation_mode`'s. There will be an error if the maximum absolute deviation is too large, i.e. for numerical derivatives (`analytic_derivation_mode=-1|-2`) the tolerance is choosen lower (around 1e-5); for analytical methods we are stricter: around 1e-13 for first-order,  1e-12 for second order, and 1e-11 for third-order.
* As a side note, this mod file also checks Dynare's `k_order_solver` algorithm and throws an error if something is wrong.
* This test model shows that the new functionality works well. And analytical derivatives perform way better and accurate than numerical ones, even for this small model.
## `analytic_derivatives/burnside_3_order_PertParamsDerivs.mod`
* This builds upon `tests/k_order_perturbation/burnside_k_order.mod` and computes the true parameter derivatives analytically by hand.
      * This test model also shows that the new functionality works well.

## `analytic_derivatives/LindeTrabandt2019.mod`
* Shows that the new functionality also works for medium-sized models, i.e. a SW type model solved at third order with 35 variables (11 states). 2 shocks and 20 parameters.
* This mod file can be used to tweak the speed of the computations in the future.
* Compares numerical versus analytical parameter derivatives (for first, second and third order). Note that this model clearly shows that numerical ones are quite different than analytical ones even at first order!
## `identification/LindeTrabandt2019_xfail.mod`
* This model is a check for issue Dynare/dynare#1595, see fjaco.m below, and will fail.
* Removed `analytic_derivatives/ls2003.mod` as this mod file is neither in the testsuite nor does it work.

***

# Detailed changes in other functions
## `get_first_order_solution_params_derivs.m`
* Deleted, or actually, renamed to `get_perturbation_params_derivs.m`, as this function now is able to compute the derivatives up to third order

## `identification_numerical_objective.m`
* `get_perturbation_params_derivs_numerical_objective.m`builds upon `identification_numerical_objective.m`. It takes from `identification_numerical_objective.m` the parts that compute numerical parameter Jacobians of steady state, dynamic model equations, and perturbation solution matrices. Hence, these parts are removed in `identification_numerical_objective.m` and it only computes numerical parameter Jacobian of moments and spectrum which are needed for identification analysis in `get_identification_jacobians.m`, when `analytic_derivation_mode=-1` only.

## `dsge_likelihood.m`
* As `get_first_order_solution_params_derivs.m`is renamed to `get_perturbation_params_derivs.m`, the call is adapted. That is,`get_perturbation_params_derivs` does not compute the derivatives of the Kalman transition `T`matrix anymore, but instead of the dynare solution matrix `ghx`. So we recreate `T` here as this amounts to adding some zeros and focusing on selected variables only.
* Added some checks to make sure the first-order approximation is selected.
* Removed `kron_flag` as input, as `get_perturbation_params_derivs` looks into `options_.analytic_derivation_mode` for `kron_flag`.

## `dynare_identification.m`
* make sure that setting `analytic_derivation_mode` is set both in `options_ident` and `options_`. Note that at the end of the function we restore the `options_` structure, so all changes are local. In a next merge request, I will remove the global variables to make all variables local.

## `get_identification_jacobians.m`
* As `get_first_order_solution_params_derivs.m`is renamed to `get_perturbation_params_derivs.m`, the call is adapted. That is,`get_perturbation_params_derivs` does not compute the derivatives of the Kalman transition `A` and `B` matrix anymore, but instead of the dynare solution matrix `ghx` and `ghu`. So we recreate these matrices here instead of in `get_perturbation_params_derivs.m`.
* Added `str2func` for better function handles in `fjaco.m`.

## `fjaco.m`
* make `tol`an option, which can be adjusted by changing `options_.dynatol.x`for identification and parameter derivatives purposes.
* include a check and an informative error message, if numerical derivatives (two-sided finite difference method) yield errors in `resol.m` for identification and parameter derivatives purposes. This closes issue  Dynare/dynare#1595.
* Changed year of copyright to 2010-2017,2019

***

# Further suggestions and questions
* Ones this is merged, I will merge request an improvement of the identification toolbox, which will work up to third order using the pruned state space. This will also remove some issues and bugs, and also I will remove global variables in this request.
* The third-order derivatives can be further improved by taking sparsity into account and use mex versions for kronecker products etc. I leave this for further testing (and if anybody actually uses this ;-) )
2019-12-17 18:17:09 +00:00
Marco Ratto 199b76c979 also test fo empty list of variables to squeeze 2019-12-15 17:17:10 +01:00
Marco Ratto 6783d51135 added squeeze call in test function 2019-12-15 16:55:02 +01:00
Johannes Pfeifer a4030c3676 testsuite: update ramsey_model to new command sequence using ramsey_model
Related to https://git.dynare.org/Dynare/dynare/issues/1678
2019-12-13 14:18:15 +01:00
Marco Ratto 2edf6d05a3 added testing for flip diff and init2shocks 2019-12-12 16:53:45 +01:00
Houtan Bastani 32872362b3
introduce changes to epilogue block
preprocessor#36
- detrend epilogue statements
- write names of epilogue variables in `M_.epilogue_names`
- output list of endogs used in epilogue equations in `M_.epilogue_var_list_`
- output static and dynamic epilogue files
2019-12-11 16:30:37 +01:00
Houtan Bastani 40e4317ef2
macro processor test file: remove utf-8 character from variable 2019-12-10 16:53:12 +01:00
Houtan Bastani 2017b32f31
fix bug in macro processor with short circuit operators || and &&
#1676
2019-12-10 16:53:12 +01:00
Sébastien Villemot 6a89783b65
Merge remote-tracking branch 'community/master' into enterprise 2019-12-03 15:39:59 +01:00
Dóra Kocsis b9af92eb8a Save conditional forecast output in oo_. Closes: Dynare/dynare#1672 2019-11-29 15:25:05 +01:00
Houtan Bastani 3ef8564279
reporting: add ability to make title page 2019-11-29 11:36:00 +01:00
Johannes Pfeifer 0c01c31491 Remove global variables from user-defined steady state files
M_ and options_ are now passed as an input; params is an output as in the _steadystate2.m files
2019-11-26 17:05:14 +01:00
Johannes Pfeifer fc6c78e5ea testsuite: transform old steady state files to steady_state_model-blocks
Allows for easier maintenance
2019-11-26 17:02:12 +01:00
Sébastien Villemot 8065e9d439
Build system: by default, error out if some dependency is missing
In particular, if either MATLAB or Octave is missing, one needs to pass either
--disable-matlab or --disable-octave.

Moreover, several new configure flags have been introduced for disabling some
components:
--disable-doc
--disable-dynare++
--disable-mex-dynare++
--disable-mex-ms-sbvar
--disable-mex-kalman-steady-state
2019-11-26 13:53:42 +01:00
Houtan Bastani 7a320d2944
fix macro processor bug that resulted in incorrect casts from string to bool 2019-11-26 11:19:15 +01:00
Stéphane Adjemian (Charybdis) 7e7f57d7d1 Encoding fixes. 2019-11-25 11:48:09 +01:00
Michel Juillard cc0f2865a6 use tolf option inside simul() 2019-11-24 09:56:35 +01:00
Dóra Kocsis 75a929051f Update sur and surgibbs function outputs. 2019-11-22 10:59:01 +01:00
Sébastien Villemot 628a8a6ff3
Testsuite: remove obsolete conditionals for testing whether Octave version > 3.4
We now support Octave ≥ 4.2 only.

By the way, in another test, use the isoctave function instead of manual test.
2019-11-19 15:25:59 +01:00
Sébastien Villemot 12d7d8cef2
Testsuite: use XLSX for Octave and MATLAB ≥ R2012a, and XLS for MATLAB < R2012a
In 98bdf76581, all XLSX files were removed in
favour of XLS, for compatibility with MATLAB R2009b.

But this broke the very same tests under Octave, since XLS support is not very
good there.

This commit implements a solution that works everywhere, by using XLSX whenever
possible, and XLS otherwise.
2019-11-19 12:49:01 +01:00
Sébastien Villemot e04ccd9b01
Fix construction of source tarball
Was broken by the previous commit, which renamed some datafile without updating
EXTRA_DIST in tests/Makefile.am.
2019-11-15 22:49:45 +01:00
Sébastien Villemot 98bdf76581
MATLAB R2009b compatibility fixes
— use XLS instead of XLSX in testsuite datafiles
— use optimset instead of optimoptions
— use quadv/quadl/quadgk instead of integral
— fix race condition in load_m_file_data_legacy.m tests
— use fallback implementation for intersect(…, 'stable')
2019-11-15 18:27:37 +01:00
Sébastien Villemot 42f172dec3
perfect_foresight_problem MEX: optimization for linear models
When the model is linear, there is no need to reevaluate the Jacobian for each
time period, since it is invariant.

Closes: #1662
2019-11-14 14:45:10 +01:00
Dóra Kocsis b4cd1c0f57 Add routine printing equations where a searched variable appears in. 2019-11-08 17:33:27 +01:00
Sébastien Villemot 56e7144b91
Merge remote-tracking branch 'community/master' into enterprise 2019-10-23 16:30:29 +02:00
Dóra Kocsis 3648ccb8ff Adjust result output for NLS and Iterative OLS. 2019-10-10 10:10:23 +02:00
Houtan Bastani 43c73a74c8
test for matlab namespace function in steady_state_model block. closes #1639 2019-10-09 16:39:33 +02:00
Houtan Bastani cd83a7790e
add .m files from 4a1b5eccb2 to EXTRA_DIST
[skip ci]
2019-10-09 16:35:32 +02:00
Houtan Bastani 4a1b5eccb2
Support MATLAB namespaces in function calls #1639
shorten test names because adding `_namespace` made some .mod file names too long
2019-10-09 16:22:32 +02:00
Sébastien Villemot 8875ac14d8
Merge remote-tracking branch 'community/master' into enterprise 2019-10-09 12:32:12 +02:00
DoraK 41c66583ac Add linear combination support for growth neutrality in Iiterative OLS. 2019-10-08 23:18:40 +02:00
Sébastien Villemot b3d1e8412b
Add support for mode_compute=1 under Octave
Since version 1.6, the optim Forge package has an implementation of fmincon.
Hence we can now use mode_compute=1 under Octave.

This commit also adds tests/optimizers/fs2000_1.mod to the testsuite. It will
be skipped under MATLAB if the optimization toolbox is not there, or under
Octave if optim ≥ 1.6 is not there.
2019-10-08 18:38:15 +02:00
Sébastien Villemot f91018c319
Fix test broken by 1668d50aaf 2019-10-02 15:06:13 +02:00
Stéphane Adjemian (Charybdis) fb3335f191 Added unitary tests for aggregate routine. 2019-10-02 10:41:32 +02:00
Sébastien Villemot 5d8babb15c
Merge remote-tracking branch 'community/master' into enterprise 2019-10-02 10:36:48 +02:00
Houtan Bastani 32932ce741
preprocessor: submodule update 2019-10-02 10:32:01 +02:00
Sébastien Villemot 9c9e2b443e
Merge remote-tracking branch 'community/master' into enterprise 2019-10-01 14:54:52 +02:00
Stéphane Adjemian (Charybdis) ad73aac0ad Fixed integration test.
Need to increase accuracy (set tolf 1e-9 instead of 1e-6) if some
endogenous variables appear under logs (which is the case for x3).
2019-09-30 14:29:19 +02:00
Houtan Bastani 715ae0ce39
preprocessor: submodule update 2019-09-30 12:28:29 +02:00
Sébastien Villemot e2c57dc5f5
Increase test tolerance 2019-09-26 17:35:24 +02:00
Sébastien Villemot 88c38c9ebc
Fixed some trend_component_model tests
Those tests broke because they were declaring an endogenous (var_allows_…) that
is not used in the model, and this is no longer allowed by the preprocessor.

Those variables were introduced by commit
dccabecee7 (on the old ecb-master branch), at the
same time as a second PAC equation was introduced.

The fix consists in making the second PAC equation different from the first
one, defining a different endogenous.
2019-09-26 17:35:15 +02:00
Sébastien Villemot cbb59fe6f8
Merge remote-tracking branch 'community/master' into enterprise 2019-09-26 16:54:27 +02:00
Sébastien Villemot e9da62eca9
Add missing exceptions to .gitignore 2019-09-26 15:19:26 +02:00
Sébastien Villemot e521dcf081
Drop useless LICENSE file 2019-09-26 15:19:14 +02:00
Sébastien Villemot f665117879
Remove spurious indentation changes
This commits reverts various spurious indentation changes that were on the
ecb-master but not on the master branch.
2019-09-26 15:17:54 +02:00
Sébastien Villemot 42392df3bc
Put binary test datafiles in the git repository
This makes the testsuite robust to network failures.
2019-09-19 14:20:00 +02:00
Houtan Bastani e043c60903
pass M_, options_, oo_ as arguments to stoch_simul, simult_, discretionary_policy. closes dynare#1197 2019-09-12 11:57:01 +02:00
Houtan Bastani 53ce6222c2
remove unused variable in dynare_config 2019-09-05 11:48:19 +02:00
Sébastien Villemot b73f5b7816
Compatibility fix for Octave 5
The sleep() function has been removed in favor of pause().
2019-08-22 17:03:19 +02:00
Houtan Bastani 5994321cd7
preprocessor: submodule update 2019-08-19 16:58:14 +02:00
Houtan Bastani ea06dfbcd5
add tests for casting and range in macro processor 2019-08-07 11:30:16 -04:00
Houtan Bastani f2b61b3e9e
submodule update: preprocessor; doc: introduce comprehensions 2019-06-26 15:09:55 +02:00
Houtan Bastani cb0e84a2c4
submodule update: preprocessor 2019-06-25 10:12:20 +02:00
Houtan Bastani 69229b6b2a
submodule update: preprocessor 2019-06-24 11:16:21 +02:00
Houtan Bastani 42e8b55274
add macro test to file 2019-06-24 11:16:21 +02:00
Houtan Bastani b4d2239da2
submodule update: preprocessor: rework macroprocessor 2019-06-21 17:11:14 +02:00
Stéphane Adjemian (Charybdis) efc174115e Use verbatim block. 2019-06-17 11:33:56 +02:00
Stéphane Adjemian (Charybdis) 75945a62ac Fixed plot_contribution routine. 2019-05-14 22:12:33 +02:00
Houtan Bastani 13547964dc
change graph in reporting test to show how to associate integers with time periods 2019-05-10 11:02:46 +02:00
Sébastien Villemot b0841b4136
Fix test files following change in the preprocessor when there is no parameter or exogenous 2019-04-30 16:42:53 +02:00
Stéphane Adjemian (Charybdis) ea68f849d5 Renamed routine. 2019-04-29 23:49:04 +02:00
Stéphane Adjemian (Charybdis) 92e7010b97 Added integration test. 2019-04-29 23:39:14 +02:00
Sébastien Villemot b2957127ad
k-order DLL: check that simulated time series at order 9 is correct
Ref #217
2019-04-29 15:48:03 +02:00
Sébastien Villemot 7ddf3046b7
Improve description of k-order tests 2019-04-29 15:46:47 +02:00
Stéphane Adjemian (Charybdis) b6a80229a9 Added integration test for cherrypick and agregate routines. 2019-04-27 18:12:53 +02:00
Sébastien Villemot f1b16ced4e
k-order DLL: check computed policy functions at order 9 against Burnside's model
Ref #217
2019-04-26 18:46:04 +02:00
Stéphane Adjemian (Charybdis) 64dc44740b Efficiency change. 2019-04-26 16:20:45 +02:00
Sébastien Villemot c5c1307725
Minor simplification 2019-04-26 14:44:36 +02:00
Stéphane Adjemian ee891948e0 Moved IntegrationAgorithm under options_.ep.stochastic 2019-04-18 18:03:57 +02:00
Stéphane Adjemian 62b0106dde Streamlined extended path examples. 2019-04-18 18:03:57 +02:00
Michel Juillard d6866b2e6b fixed saving of pseudo_data variables 2019-04-18 18:03:57 +02:00
Stéphane Adjemia (Scylla) 66e44dd830
Added missing routine.
Example for agregate routine.
2019-04-10 09:56:30 +02:00
Stéphane Adjemia (Scylla) 8740355407
Rewrote evaluate routine.
- Can handle more than one equation.
 - Can handle identities.
 - Forbids dynamic equations.
 - Can handle following LHS y, diff(y), diff(diff(y)), log(y), diff(log(y)) and
   diff(diff(log(y))), other transformations will result in an error.
 - Added integration tests.

Remark 1. In the integration tests I compare the values returned by the
          evaluate routine with the values computed with the simulation
          routines. Normally the discrepancies should be small, but this is not
          the case when the endogenous variable appear under a log on the
          LHS. My current conclusion is that this has more to do with the
          cumulation of the accuracy errors in the simulation routine (a
          sequence of Newton algorithms) rather than with the evaluate routine.

Remark 2. Currently the only allowed nonlinear transformation on the LHS
          endogenous variable is the log. It is not difficult to generalise, at
          some point I had all the matlab functions allowed by Dynare,
          but this would complicate the code for not much gain.
2019-04-08 11:01:34 +02:00
Stéphane Adjemia (Scylla) 94b20fe7e8
Activate two integration tests. 2019-04-02 22:57:23 +02:00
Sébastien Villemot 774e60f04c
options_.risky_steadystate is an (undocumented) boolean value
By the way, enable the option in example1_korder.mod (it does not work, but no
harm since it’s not in the testsuite).
2019-04-02 19:16:13 +02:00
Stéphane Adjemia (Scylla) aa499671ce
Added the possibility to rename an endogenous variable in equation. 2019-04-02 18:06:47 +02:00
Stéphane Adjemia (Scylla) f39f57b9b5
Added parameter values in cherrypick/agregate routines. 2019-04-02 07:11:32 +02:00
Stéphane Adjemian (Charybdis) bb0660d506
Added integration tests (trend component models). 2019-03-29 17:58:03 +01:00
Stéphane Adjemian (Charybdis) 7b7bca8bda
Generalised trend component model.
Added the possibility to associate more than one trend to an endogenous
variable. The number of error correction equations is longer required to be
equal to the number of trends.
2019-03-29 17:52:46 +01:00
Houtan Bastani d242ed88a2
dyn_ols: add date range option 2019-03-26 15:04:16 +01:00
Houtan Bastani 9ab4c6f80d
ols-style equations: handle additive elements separated by + and -
use same algorithm as in preprocessor to facilitate future changes
2019-03-26 11:06:30 +01:00
Stéphane Adjemian (Charybdis) a77de6ce75
Revert last patch on Makefile.am. 2019-03-25 20:09:14 +01:00
Sébastien Villemot d1df9317ab
Indentation fixes (tabs are meaningful in Makefiles) 2019-03-25 18:47:49 +01:00
Sébastien Villemot e7dfc6ab36
The MATLAB testsuite now works under Windows, so no need to special-case it 2019-03-25 18:47:49 +01:00
Stéphane Adjemian (Charybdis) 8a8ef89004
Added example for new agregate() routine. 2019-03-25 18:16:38 +01:00
Sébastien Villemot 0f28974c2c
Provisions for MATLAB R2019a
In particular, use the new "-batch" option for running the testsuite.
2019-03-22 16:43:40 +01:00
Sébastien Villemot 4a62fbb687
Remove spurious file in EXTRA_DIST, introduced in 666c9b8003 2019-03-21 18:30:44 +01:00
Sébastien Villemot 44b50d41ae
Fix EOL convention of some files add in 666c9b8003 2019-03-20 16:49:52 +01:00
Willi Mutschler 666c9b8003 Improvement of Identification Toolbox
# Improvements
  * heavily commented (also auxiliary functions) and changed notation to make all the functions (hopefully) more readable and understandable, and hence, easier to debug
  * added identification criteria of Komunjer and Ng (2011, Econometrica) and Qu and Tkachenko (2012, Quantitative Economics)
  * tests can be turned of, i.e. nostrength disables identification strenght, noreducedform disables reduced form criteria, nomoments disables moment criteria, nospectrum disables spectrum criteria, nominimal disables minimal system criteria
  * all kronflags (analytic_derivation_mode) actually work in all functions
  * added functionality when there is correlation in Sigma_e and when one wants to consider corr parameters of exogenous shocks. Previously, (1) corr parameters were not allowed when calling identification and (2) when Sigma_e was not diagonal then the toolbox relied on numerical derviatives only (kronflag=-1). Now it is possible to handle both identification of corr parameters as well as correct analytical derivatives when Sigma_e is not diagonal with all possible kronflag values (-1|-2|0|1)
  * all plots and results are stored in the same folder named identification (previously there was another one with a capital I (Identification))

# Needed changes to preprocessor
  * add as field to options_ident:

    - tex (same as in options_)
    - nostrength (to turn off identification strength)
    - noreducedform (to turn off reduced form criteria)
    - nomoments (to turn off Iskrev's moment criteria)
    - nominimal (to turn off Komunjer and Ng's minimal system criteria)
    - nospectrum (to turn off Qu and Tkachenko's spectrum criteria)

  * add to options_ident:
    - normalize_jacobians (whether to normalize Jacobians or not)
    - grid_nbr (integer used to discretize the interval [-pi;pi]
    - tol_rank (tolerance level to compute ranks)
    - tol_deriv (tolerance level to select nonzero columns in derivatives)
    - tol_sv (tolerance level to select nonzero singular values)
    - ChecksViaSubsets (for debugging purposes, uses different function to find problematic parameter sets)
    - max_dim_subsets_groups (for debugging purposes, used for ChecksViaSubsets)

# Further Suggestions
  * Rename getH.m into getParamsDerivReducedForm.m to make the purpose of this function evident
  * Rename getJJ.m into getIdentificationJacobians.m to make the purpose of this function evident
  * Rename thet2tau.m into IdentificationNumericalObjectiveFunction.m to make the purpose of this function evident
  * dYss, d2Yss, dg1 should also include derivatives wrt to stderr and corr parameters (even though these are just 0), as in other functions (getJJ, dynare_estimation) we always add these manually
  * I am pretty sure the current handling in getH.m of dYss and d2Yss is not correct in the case of nonstationary variables (if g2static is nonempty), I added a warning message, as I am not sure whether this is ever used
  * It would be straigthforward to also include stderr and corr parameters of measurement errors (these is not possible right now). Should I do this?
  * Computations of d2A and d2Om need to be checked, as the differences between computing these with analytically (kronflag=0|1) or numerically kronflag=-1|-2 is really large for the example model of AnSchorfheide.
  * I am not sure how to best normalize Qu and Tkachenko's G matrix. It looks (and in the Gaussian case actually is) very similar to the Ahess matrix. So I used the same normalization rule as for the Ahess matrix. See comments in identification_checks.m. Anyone has a better idea? Please also check the models in test/identification/cgg for differences.
  * parts that are unclear to me are marked by a [@wmutschl] tag
  * the run time of tests/identification/as2007.mod increases from 0h01m27s to 0h03m46s (as Qu and Tkachenko's G matrix takes a little while to compute). One could decrease prior_mc=250 to prior_mc=150.

# New functions
  * commutation: Returns Magnus and Neudecker's commutation matrix that solves k*vec(X)=vec(X')
  * DerivABCD: Derivative of X(p)=A(p)*B(p)*C(p)*D(p) w.r.t to p as in Magnus and Neudecker (1999), p. 175
  * DeriveMinimalState: Derives minimal state space system by checking observability and controllability of all possible combinations of variables
  * duplication: Duplication Matrix (and its Moore Penrose Inverse) as defined by Magnus and Neudecker (2002), p.49, Dp*vec(X) = X
  * identification_checks_via_subsets: finds problematic parameters in a bruteforce fashion: It computes the rank of the Jacobians for all possible parameter combinations, if the rank condition is not fullfilled, these parameter sets are flagged as non-identifiable. For debugging purposes only, as the current identification_checks.m (based on nullspace and multicorrelation coefficients) is much faster

# Detailed changes in getH.m
  * functionality improvements

    - heavily commented (also auxiliary functions) and changed notation of several variables to make this function (hopefully) more readable and understandable, and hence, easier to debug
    - added functionality when Sigma_e is not diagonal and/or when one wants to consider corr parameters of exogenous shocks independent of the value of kronflag
    - fixed function for all values of kronflag, i.e. kronflag=-2|-1|0|1. Previosuly, only kronflag=-2|0 were working, all other kronflags ran into errors (-1 was actually never called , but was dealt with in getJJ.m). I assume kronflag=-1|1 was used only for debugging issues, but still was not working. I fixed this now, the function now works out-of-the-box for all kronflag values.
    - I also outlined and documented what each kronflag does and point to the corresponding equations in Ratto and Iskrev (2012) or Iskrev (2010,Appendix A)
    - the function additionally outputs the Jacobians of B and Sig, which are needed for Qu and Tkachenko (2012) and Komunjer and Ng (2011)'s criteria
    - Moved computation of Jacobian of tau=[ys;vec(A);vech(B * M_.Sigma_e * B')] into getJJ.m to have all Jacobians which are needed for identification in one place. That is, getH.m computes first and second parameter derivatives of (1) reduced-form solution, (2) steady state and (3) Jacobian of dynamic model, whereas getJJ computes and sets up all Jacobians which are used for identification purposes. Therefore, getH might be useful more generally for other purposes than identification. For instance, when doing a GMM estimation, we could use this function to compute analytically the gradient of the moments and provide this to the optimizer used in a GMM context.

  * output arguments

    - renamed `H` (Jacobian wrt parameters of tau=[ys;vec(A);vech(B * M_.Sigma_e * B')] into dTAU, (as H is very confusing, e.g. in other functions it is a Hessian, or Hss and H2ss is also just the steady state. Morevoer, tau is used in Iskrev(2010) for the steady state and reduced-form solution)
    - renamed `Hss` (Jacobian of steady state wrt model parameters only) into `dYss` (as H is very confusing here, see above)
    - renamed `H2ss` (Hessian wrt model parameters only of ys) into d2Yss (as H is very confusing, see above)
    - renamed `gp` into `dg1`, where g1 corresponds to the same variable as in dynamic model files. Note that in params_deriv files gp lacks the contribution of Jacobian wrt steady state and dg1 includes this using the implicit function theorem as outlined in Ratto and Iskrev (2012). Hence, dg1 denotes Jacobian wrt to parameters. It is useful and important to distinguish gp and dg1.
    - added `dB` (Jacobian wrt parameters of solution matrix B) needed for Qu and Tkachenko (2012) as well as Komunjer and Ng (2011)
    - added `dSig` (Jacobian wrt parameters of M_.Sigma_e) needed for Qu and Tkachenko (2012) as well as Komunjer and Ng (2011)

  * input arguments

    - renamed `indx` (index of model parameters to be checked) into `indpmodel`, the p makes it more clear that this is a parameter index
    - renamed `indexo` (index of stderr parameters) into `indpstderr`, the p makes it more clear that this is a parameter index
    - renamed `iv` (index of variables to consider) into `indvar`
    - Renamed `M_` to `M`, `estim_params_` to `estim_params`, `options_` to `options` , `oo_` to `oo` to visualize that these are local and not global variables
    - included `indpcorr` a matrix of indices for corr parameters to be checked

  * misc

    - distinguished clearly between variables in DR or in declaration order without overwriting this in between
    - added which functions call getH.m
    - updated copyright to 2010-2019

# Detailed changes in getJJ.m

  * functionality improvements

    - heavily commented and changed notation of several variables to make this function (hopefully) more readable and understandable, and hence, easier to debug
    - added functionality when Sigma_e is not diagonal and/or when one wants to consider corr parameters of exogenous shocks independent of the value of kronflag
    - tidied the function up, such that it sets up all Jacobians which are needed for identification, i.e. Iskrev's J matrix, Qu and Tkachenko (2012)'s G matrix, Komunjer and Ng (2011)'s D matrix, reduced-form solution (dTAU), linear rational expectation (i.e. Jacobian of steady state and dynamic model equations dLRE).
    - dTAU is now constructed in getJJ instead of in getH (see comment above in getH.m)
    - works for all kronflags, i.e. for numerical derivatives (-1 and -2) as well as for analytical derivatives based on kronecker products (1) or Sylvester Equations (0)
    - added functionality for stderr and corr parameters independent of the value of kronflag (previously this was only possible with numerical derivatives, now it works for all kronflags)
    - finds minimal state vector needed for Komunjer and Ng (2011)'s criteria (function `DeriveMinimalState.m`)
    - moved computations from kronflag=-1 (which were used in case of corr in shock block) into getH.m, so that getJJ now only sets up the Jacobians for LRE, Iskrev's J, Qu and Tkachenko's G and Komunjer and Ng's D, whereas getH computes the Jacobians (wrt parameters) of A, B, Sigma_e, Om, Yss and g1. This should simplify debugging as everything is now in one place and not in two

  * output arguments

    - renamed `JJ` into `J`
    - renamed `H` into `dTAU` (as H is very confusing, e.g. in other functions it is a Hessian, or Hss and H2ss is also just the steady state. Morevoer, tau is used in Iskrev(2010) for the steady state and reduced-form solution)
    - renamed `gp` into `dLRE`, as this corresponds to Jacobian of LRE=[Yss;vec(g1)] where g1 is the Jacobian of the dynamic model equations.
    - renamed `gam` into `MOMENTS`
    - added `G` for Qu and Tkachenko's Jacobian matrix G
    - added `D` for Komunjer and Ng's Jacobian matrix D
    - reordered output arguments

  * input arguments

    - added `options_ident` as input argument; hence, `kronflag`, `nlags` and `useautocorr` are removed from input arguments as these are available in options_ident
    - Renamed `M_` to `M`, `estim_params_` to `estim_params`, `options_` to `options` , `oo_` to `oo` to visualize that these are local and not global variables
    - renamed `indx` (index of model parameters to be checked) into `indpmodel`, the p makes it more clear that this is a parameter index
    - renamed `indexo` (index of stderr parameters) into `indpstderr`, the p makes it more clear that this is a parameter index
    - added `indpcorr` (index of corr parameters)
    - renamed `mf` (index of VAROBS variables) into `indvobs`

  * misc

    - updated copyright to 2010-2019
    - provided some comments on several ways to compute the spectral density matrix
    - added which functions call getJJ.m

# Detailed changes in thet2tau.m

  * functionality improvements

    - heavily commented and changed notation of several variables to make this function (hopefully) more readable and understandable, and hence, easier to debug
    - Added output option to compute spectral density matrix
    - Reorded and added some output option.
    - Instead of Om, `outputflag=0` computes B and Sigma_e, which are needed for Qu and Tkachenko as well as Komunjer and Ng. The Jacobian of Om is then computed in getJJ or getH from Jacobian of B and Sigma_e. Due to some testing with An and Schorfheide model this seems to be more accurate when I compare these with the analytical derivatives. The old behavior (computing Om directly) can be restored by setting `outputflag=-2`.
    - In total this function can now be used to compute numerically Jacobians of Yss, A, B, Sigma_e, Om, g1, autocovariogram and spectral density
    - Clearly distinguished (and commented) on the different outputs of this function.
    - Works for all types of parameters, ie. model, stderr and corr.
    - This function can now also be used when there is no estimated_params block. Previously, there was an error when there was no estimated_params block when calling `set_all_parameters` as this requires some information in `estim_params`. I fixed this by providing a temporary local estim_parms structure with the necessary information on model, stderr and corr parameters. In this way, this can be easily extended to also include stderr and corr parameters of measurement errors.

  * output arguments

    - renamed `tau` into `out`, as this function computes *very* different things (and not only tau) depending on an input flag

  * input arguments

    - renamed `flagmoments` into `outputflag` as this function does not only compute moments but many other things (see above)
    - renamed `indx` (index of model parameters to be checked) into `indpmodel`, the p makes it more clear that this is a parameter index
    - renamed `indexo` (index of stderr parameters) into `indpstderr`, the p makes it more clear that this is a parameter index
    - added `indpcorr` (index of corr parameters)
    - merged `mf` (index of observable variables) and `iv` (index of variables to consider) into a single index `indvar` as there is no need to distinguish between these two indices (they were never used in combination)
    - added `grid_nbr` (number of grid points to compute spectral density)
    - reordered input arguments

  * misc

    - added which functions call thet2tau
    - updated copyright to 2010-2019

# Detailed changes in identification_analysis.m

  * functionality improvements

    - heavily commented and changed notation of several variables to make this function (hopefully) more readable and understandable, and hence, easier to debug
    - renamed `dg1` to `dLRE`, renamed `vecg1` to `lre`, renamed `H` to `dTAU` (see comments above)
    - added option `numzerotolderiv` with default `1.e-8` used for non-zero derivatives
    - added option `numzerotolrank` with default `1.e-10` used for rank computations
    - added theoretical identification analysis based on Komunjer and Ng (2011)'s method, i.e. steady state and observational equivalent spectral densities within a minimal system
    - added theoretical identification analysis based on Qu and Tkachenko (2012)'s method, i.e. steady state and spectral density
    - restructured the code slightly to combined chunks of code that belong together on the one hand, and on the other hand to differentiate between the different criteria
    - added call to new function `identification_checks_via_subsets.m` (see above for the definition of the functionality) to perform identification checks differently as find it more intuitive and (most likely) more precise.

  * input arguments

    - removed `bounds` and `dataset_` as input argument, because these are not needed
    - moved `name_tex` and `tittxt` into `options_ident` as these two inputs are only used in `ident_bruteforce.m` and already set in `dynare_identification.m`

  * output arguments

    - added `ide_spectrum` structure for Qu and Tkachenko's criteria based on the spectral density
    - added `ide_minimal` structure for Komunjer and Ng's criteria based on the minimal state space system
    - reordered output arguments

  * misc

    - added which functions call identification_analysis
    - updated copyright to 2010-2019

# Detailed changes in dynare_identification.m

  * functionality improvements

    - heavily commented and changed notation of several variables to make this function (hopefully) more readable and understandable, and hence, easier to debug
    - included more options (and default values) which can be set by the user, i.e. nostrength, nomoments, nominimal, nospectrum, tex, tol_rank, tol_deriv, tol_sv, grid_nbr, ChecksViaSubsets, max_dim_subsets_group
    - instead of turning warnings globally off, I specified the relevant warnings for matlab and octave, respectively, off
    - improved the warning messages slightly
    - restructured chunks of code with respect to different criteria

  * output arguments

    - renamed arguments: TAU to STO_TAU, GAM to STO_MOMENTS, LRE to STO_LRE, gp to STO_si_dLRE, H to STO_si_dTAU, JJ to STO_si_J
    - added arguments: STO_G and STO_D for the two new criteria

  * misc

    - added which functions call dynare_identification
    - updated copyright to 2010-2019

# Detailed changes in identification_checks.m

  * functionality improvements

    - added checks for Komunjer and Ng's D matrix. Note that the Jacobian D=[D_par D_rest], where D_par depends on the parameters and D_rest does not. So this is taken into account.
    - added checks for Qu and Tkachenko's G matrix. Note that the Jacobian G is a Gram matrix with dimension nparam x nparam, similar to Ahess. So this is taken into account. I am, however, not sure whether this is correct regarding the multicorrelation and pairwise correlation coefficients. Please double check.
    - the rank is now actually computed at the prespecified tolerance level (and not Matlab's default level), so this is in accordance to the further analysis of problematic parameter sets

  * output arguments

    - added the rank to output arguments which is later also displayed
    - replaced the J or JJ part in the variable names with X as this function is used for all sorts of Jacobians, not only Iskrev's J

  * input arguments

    - renamed hess_flag to output_flag (and clearly outlined what each value does)
    - added tol_rank and tol_sv as input arguments, such that the tolerance levels can be changed by the user and not preimplemented in this function
    - added param_nbr which is needed for Komunjer and Ng's D matrix

  * misc

    - updated copyright to 2010-2019

# Detailed changes in ident_bruteforce.m

  * functionality improvements

    - the output directory was set with a capital I, i.e. Identification, whereas in all other functions we rely on lower case i, i.e. identification. I changed this to lower-cases, so everything is now saved in the same folder.
    - changed displayed strings to be more precise with the corresponding papers and notation

  * input arguments

    - renamed `n` to `max_dim_cova_group` to name options the same across functions
    - renamed `pnames_TeX` to `name_tex` to name options the same across functions
    - added `tol_deriv` as tolerance level which can be changed by the user

  * misc

    - Added some comments
    - updated copyright to 2010-2019

# Detailed changes in disp_identification.m

  * functionality improvements

    - this function displays the same output for different Jacobians, hence I put the common code into a for loop. This should simplify changing the output that is printed to the console. Previously the code was simply repeated for the different criteria and only the strings changed.
    - some settings relevant for the computation are now printed as a summary to the console
    - the tolerance level, rank and required rank are always displayed on the command line to see how many problematic sets there are and which tolerance level was used
    - the function is also able to display problematic parameters computed by the new function `identification_checks_via_subsets.m` which is only used for debugging.

  * input arguments

    - added `idespectrum` structure for analysis based on Qu and Tkachenko
    - added `ideminimal` structure for analysis based on Komunjer and Ng
    - added `options_ident` to have all necessary settings in a structure

  * misc

    - Added some comments
    - Removed uncommented code that was not used as this was redundant and probably an artifact of the original programming?!
    - updated copyright to 2010-2019

# Detailed changes in dsge_likelihood.m

  * misc

    - adjusted call of getH due to changes of input and output arguments
    - updated copyright to 2010-2019

# Detailed changes in cosn.m

  * misc

    - commented functionality, input and output arguments of this function
    - updated copyright to 2010-2019
2019-03-20 15:44:54 +00:00
Stéphane Adjemian (Charybdis) 567ca19000
Added routine to evaluate the RHS of an equation. 2019-03-19 07:08:55 +01:00
Houtan Bastani 7858a88764
add dependency for test for parallelization of test suite 2019-03-15 17:01:25 +01:00
Houtan Bastani 543470b331
separate ECB tests so as to be able to test everything in one go 2019-03-15 16:44:59 +01:00
Houtan Bastani 4ac5ebbefc
add var AR test 2019-03-15 14:22:50 +01:00
Houtan Bastani dccabecee7
fix trend-component tests 2019-03-15 13:29:15 +01:00
Stéphane Adjemian (Charybdis) d3db092c81
Added a cherry-pick routine.
Extracts equations from a mod file and produce .inc files (equations, lists of
parameters, endogenous variables and exogenous variables) that can be included
in a mod file that will be used to simulate the model.

If an innovation has a tag `(used='estimationonly')` it will be excluded from
the generated files (ie list of shocks and equations).
2019-03-14 11:04:10 +01:00
Stéphane Adjemian (Charybdis) 52eed33c42
Fixed growth neutrality correction in presence of exogenous variables.
Also fixes growth neutrality correction in models with non optimizing
agents (correction was not taking into account the value of the share of non
optimizing agents).
2019-03-11 11:54:53 +01:00
Stéphane Adjemian (Charybdis) 6997e0a4a6
Added the possibility to have exogenous variables in the optimal part of PAC.
Works with iterative ols and nls.
2019-03-08 14:34:41 +01:00
Houtan Bastani ba416f12ad
makefile: use tabs instead of spaces 2019-03-08 11:16:26 +01:00
Houtan Bastani d7246b3489
ols-style routines: allow user to specify parameters to be estimated 2019-03-08 11:15:33 +01:00
Houtan Bastani 6b4b2bb8b3
fix path to test 2019-03-07 18:06:53 +01:00
Stéphane Adjemian (Charybdis) d4814c6b92
Removed pointer to missing file in Makefile.am. 2019-03-07 17:33:40 +01:00
Stéphane Adjemian (Charybdis) c8393a6a45
Updated list of non versioned files in under tests subfolder. 2019-03-07 17:09:36 +01:00
Stéphane Adjemian (Charybdis) dac08da6a4
Added the possibility to tag a varexo variable as an observed variable. 2019-03-07 15:31:34 +01:00
Houtan Bastani 92b1c25b9f
fix typo 2019-03-07 10:47:02 +01:00
Houtan Bastani f3a10d0238
add test for estimating certain parameters in sur 2019-03-07 10:47:02 +01:00
Stéphane Adjemian (Charybdis) 40918f4316
Fixed condition in integration tests for ec.istarget. 2019-03-07 09:39:14 +01:00
Stéphane Adjemian (Charybdis) 9eb64a0b64
Updated preprocessor submodule.
Fixes exogenous variables in PAC equation without non optimizing behaviour.
2019-03-06 15:39:00 +01:00
Houtan Bastani 59ab4f4b34
update tests 2019-03-04 15:08:19 +01:00
Stéphane Adjemian (Charybdis) cdadab34b4
Streamlined code in print_expectations and allow diff and unary op in growth option of pac_model. 2019-03-02 22:36:13 +01:00
Stéphane Adjemian (Charybdis) d00b57541e
Allow exogenous variables in pac.estimation.iterative_ols routine.
The parameters associated to these additional variables can be estimated or calibrated.
2019-03-01 23:32:47 +01:00
Stéphane Adjemian (Charybdis) fd98e83d78
Adapted integration tests for 8e60ded. 2019-02-28 16:23:04 +01:00
Stéphane Adjemian (Charybdis) f07b1e8028
Account for exogenous variables in PAC's RoT part.
Fixes the iterative_ols estimation of PAC equation when the Rule of Thumbs (non
optimizing) part of the equations contains endogenous and/or exogenous variables.
2019-02-27 15:53:25 +01:00
Stéphane Adjemian (Charybdis) 379431b05b
Added integration test.
Check that the content of pac.[pacmodel].equations.[eqtag].non_optimizing_behaviour.vars is correct.
2019-02-27 14:22:59 +01:00
Sébastien Villemot b6f0071501
Testsuite: activate k_order_perturbation/fs2000k4.mod 2019-02-26 18:59:42 +01:00
Houtan Bastani 043ecfc3b2
Revert "surgibbs: fix bug in param values output"
This reverts commit d6c5618a74.
2019-02-26 16:54:26 +01:00
Houtan Bastani d6c5618a74
surgibbs: fix bug in param values output 2019-02-26 15:42:51 +01:00
Houtan Bastani 0c8867cbad
fix typo 2019-02-26 12:52:38 +01:00
Stéphane Adjemian (Charybdis) 445f88df55
Fixed PAC/MCE with non optimizing behaviour.
Alse added integration tests for PAC/MCE.
2019-02-25 23:07:57 +01:00
Sébastien Villemot cbf09080f9
Octave compatibility fix 2019-02-25 17:46:40 +01:00
Stéphane Adjemian (Charybdis) 1aa7f4a93f
Added PAC with Model Consistent Expectations. 2019-02-25 17:31:58 +01:00
Sébastien Villemot 073f5dd6f7
Fix function header 2019-02-25 15:28:23 +01:00
Houtan Bastani 030737b9c0
sur: fix input checks 2019-02-25 14:23:48 +01:00
Stéphane Adjemian (Charybdis) 0af3068569
Handle situations where a PAC model is used in more than one equation. 2019-02-22 17:58:24 +01:00
Stéphane Adjemian (Charybdis) 4943bf5fd1
Updated integration tests fro trend component models. 2019-02-21 12:52:12 +01:00
Houtan Bastani a7165e2c12
fix tests 2019-02-19 12:34:30 +01:00
Sébastien Villemot 1f84bc051d
Under Octave, the "statistics" Forge package is now a requirement
The gamrnd fallback under matlab/missing/stats/ does not work under Octave
because the +gamrnd/ folder is not accessible (it has the same name as the
function, which does not work under Octave).

Instead of fixing this, rather make the statistics toolbox a requirement, since
anyways it is very easy to obtain under Octave.

Accordingly:
- do not try to run the unit tests of matlab/missing/stats/ under Octave
- merge the matlab/missing/stats-matlab/ into matlab/missing/stats/, since this
  directory is now MATLAB-only.

Also:
- move matlab/distributions/+gamrnd/ under matlab/missing/stats/ for
  consistency
- in the manual: remove obsolete link to Octave downloads on the Dynare
  website; update URL of Octave Forge

Closes #1638
2019-02-18 17:41:10 +01:00
Sébastien Villemot 548a6283ca
Testsuite: also move fataltests.m under tests/utils/ 2019-02-17 12:46:26 +01:00
Sébastien Villemot ce70e2a9ca
Testsuite: do not add top-level tests/ directory to the MATLAB/Octave path
This directory is cluttered with so many files that it creates problem (e.g.
dynare-command-options/ramst.mod was broken under Octave because of this).

Move files that have to be found through the path in a utils/ subdirectory.
2019-02-15 18:43:20 +01:00
Sébastien Villemot 9339869942
Octave compatibility fix in solow.mod test
- use optimoptions instead of optimset
- since one can't take the handle of a function in a package, use str2func to
  workaround it

It still does not work, the solver does not converge…
2019-02-15 18:43:19 +01:00
Stéphane Adjemia (Scylla) 3cb3b4aee5
Allow models with leads in bgp.write().
Also added tests to check that we are able to identify the Balanced Growth
Path, note that it does not work with tests/bgp/fs2000.mod.
2019-02-15 16:59:26 +01:00
Sébastien Villemot 9786ce3d07
Testsuite: display full stacktrace on Octave failures 2019-02-14 17:45:40 +01:00
Houtan Bastani eb52cc9953
preprocessor: submodule update 2019-02-14 17:41:07 +01:00
Sébastien Villemot 0b1b365ed3
Testsuite: invert colors (green vs red) for expected failures
I.e. a test that is expected to fail and that actually fails will be displayed
in green. If it actually succeeds, it will be red.
2019-02-14 17:03:35 +01:00
Sébastien Villemot c8f3b5b96c
Various updates to .gitignore files 2019-02-13 16:12:16 +01:00