Variable indices would be incorrect in the evaluated Jacobian if recursive
variables were present. This would lead to incorrect results and/or crashes in
bytecode MEX. This bug has been exposed by commit
f45a99fc68, which actually enabled mfs>0 for
static models.
Commit 23b0c12d8e introduced caching in chain
rule derivation (used by block decomposition), which increased speed for mfs >
0, but actually decreased it for mfs=0.
This patch introduces the pre-computation of derivatives which are known to be
zero using symbolic a priori (similarly to what is done in the non-chain rule
context). The algorithms are now identical between the two contexts (both
symbolic a priori + caching), the difference being that in the chain rule
context, the symbolic a priori and the cache are not stored within the ExprNode
class, since they depend on the list of recursive variables.
This patch brings a significant performant improvement for all values of the
“mfs” option (the improvement is greater for small values of “mfs”).
Input and output ranges should not overlap when calling std::set_union(),
otherwise the behaviour is undefined.
It seems that in this precise case the computation would still be
correct (though inefficient), because of the properties of std::set or because
of the specific implementation in libstdc++. But it’s better to be on the safe
side.
In a dynamic context, the only potentially non-null derivatives of
STEADY_STATE(…) are the parameters. We know that the derivatives w.r.t. other
variables are zero, so store that information in non_null_derivatives.
More precisely, incorrect equation normalization could occur in the presence of
cos, sin, tan, cosh and x^n (where n is an even integer).
Also add some comments explaining why some other rules are (hopefully) correct.
Note that DynamicModel::determineBlockDerivativesType(), it’s legitimate to
replace max_{lead,lag} by max_endo_{lead,lag}, because for exogenous
lag=lead=0, and we no longer compute derivatives w.r.t. to endogenous that do
not belong to the block (so-called “other” endogenous).
As a consequence, and as a temporary measure, always output the
non-block-decomposed legacy representation.
Also drop the block kalman filter output, and drop now useless variables in
M_.block_structure.
— No longer call std::exit() from threads when compilation fails, that function
is marked as not thread-safe under GNU/Linux; and it leads to deadlocks under
Windows. Rather store the list of failed objects, and exit with a message and
an error code from the main thread when that list is not empty at the end of
preprocessing.
– Fix the condition used for waiting until all compilation threads finish;
checking that the queue is empty is not enough, since a compilation may be
ongoing. So also track objects whose compilation is ongoing.
It would previously forget to write the indices for the highest derivation
order (e.g. if order=3, then it would only write sparce indices up to 2nd
order).
Ref. dynare#1859
Such a decomposition has to be simulated with periods as the outer loop and
blocks as the inner loop.
It is enabled by default for purely backward/forward/static models, as long as
the “block” option is not given. In that case, “mfs” is also set to 3 by
default (until that value becomes the global default).
M_.time_recursive_block_decomposition is set to “true” when that decomposition
has been performed, “false” otherwise for the traditional decomposition (the
latter has to be simulated with blocks as the outer loop and periods as the
inner loop).
It is no longer used in the new implementation of solve_algo={12,14}.
However, the M_.lhs field that was created by setup_solvers.m is used from
other places, so reimplement it at the preprocessor level.
The files are created under <basename>/+debug/dynamic_resid.m and
<basename>/+debug/static_resid.m.
Their purpose is to evaluate separately the LHS and RHS of each equation.
Within a block, if mfs>0, the recursive variables would not appear in recursive
order. This could lead to wrong results in case of dependency relationships
between recursive variables.
The new representation is only supported for MATLAB/Octave, C and Julia output
for the time being. Bytecode and JSON are unsupported.
This commit adds new fields in M_.
This is a preliminary step for dynare#1859.
– print the information only for the final matching (and not the intermediary
ones that may have failed);
– print the equation name next to its number.
In an “evaluate” block, the LHS of a renormalized equation (such as “log(x)=…”)
could be associated to a temporary term that would then be incorrectly
computed: that temporary term would be evaluated *before* (and not after) the
evaluation of the associated variable (“x” in the example).
Should have no impact though, since diff nodes are already substituted out at
that point. But it’s better to implement it properly, in case we change the
substitution rules later.
By the way, make the computeSubExprContainingVariable method protected.
The previous system would spawn as many threads as there are object files to be
compiled (which could lead to hundreds of threads for large block-decomposed
models). This could pose a memory usage problem (even when not just waiting,
threads require memory for their own stack).
– DataTree::packageDir() now takes a std::string_view, returns a
std::filesystem::path, and no longer creates that directory
– DataTree::writeToFileIfModified() now takes a std::filesystem::path as
argument
– Do not call DataTree::writeToFileIfModified() for generating MATLAB/Octave
files, since it does not work (the directory inside which the file is written
has been deleted by the preprocessor just before)
– Consistently use DataTree::packageDir() everywhere (for compatibility with
planner_objective)
Sébastien Villemot
Stéphane Adjemian (Guts)
Johannes Pfeifer
MichelJuillard