stepan
/
dynare
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add docker containers

kalman-mex
Willi Mutschler 2023-07-09 11:09:36 +02:00
parent d4a8d0fc50
commit b23c394972
No known key found for this signature in database
GPG Key ID: 91E724BF17A73F6D
4 changed files with 368 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -58,6 +58,7 @@ a 32-bit Octave.
1. [**Fedora, CentOS or RHEL**](#fedora-centos-or-rhel)
1. [**Windows**](#windows)
1. [**macOS**](#macos)
1. [**Docker**](#docker)
## General Instructions
@ -556,3 +557,8 @@ Alternatively, you can create a `startup.m` file or change the system default PA
Last tested on:
- macOS Ventura 13.3.1 (MacBook Air M1, MacBook Pro M2 MAX, M2 Virtual Machine using Parallels, Intel Virtual Machine using Quickemu)
## Docker
We offer a variety of pre-configured Docker containers for Dynare, pre-configured with Octave and MATLAB including all recommended toolboxes.
These are readily available for your convenience on [Docker Hub](https://hub.docker.com/r/dynare/dynare).
The docker folder contains [information and instructions](docker/README.md) to interact, built and customize the containers.

200
docker/Dockerfile Normal file
View File

@ -0,0 +1,200 @@
# Copyright 2023 Dynare Team
# This file is part of Dynare.
#
# Dynare is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Dynare is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Dynare. If not, see <https://www.gnu.org/licenses/>.
##############################################################################################################
# INFORMATION: #
# This Dockerfile installs Dynare (https://dynare.org) #
# - with MATLAB including the following toolboxes (https://github.com/mathworks-ref-arch/matlab-dockerfile) #
# MATLAB Symbolic_Math_Toolbox Statistics_and_Machine_Learning_Toolbox Optimization_Toolbox #
# Econometrics_Toolbox Parallel_Computing_Toolbox Control_System_Toolbox Global_Optimization_Toolbox #
# - with Octave including the following toolboxes (using debian packages): #
# octave-control octave-econometrics octave-io octave-statistics octave-struct octave-parallel #
# It also renames the GCC libraries of MATLAB (similar to the matlab-support package) to avoid conflicts #
# with the system libraries and adds the path of Dynare to the MATLAB and Octave startup scripts. #
# #
# MATLAB LICENSE: #
# The container is created using a network license, so no information on the license is inside the container #
# see https://git.dynare.org/dynare/dynare/docker/README.md#matlab-license for more information. #
##############################################################################################################
# Default values which MATLAB and Dynare release to install in the container
# The Dynare release must conform to a corresponding tag on https://git.dynare.org/dynare/dynare
# MATLAB release must conform to a corresponding tag on https://hub.docker.com/r/mathworks/matlab/tags
# Octave version the one shipped with the Ubuntu version used in the base container (in 20.04 it is 6.4.0)
ARG MATLAB_RELEASE=R2023a
ARG DYNARE_RELEASE=5.4
# Specify the list of products to install into MATLAB with mpm
ARG MATLAB_PRODUCT_LIST="Symbolic_Math_Toolbox Statistics_and_Machine_Learning_Toolbox Optimization_Toolbox Econometrics_Toolbox Parallel_Computing_Toolbox Control_System_Toolbox Global_Optimization_Toolbox"
# Specify MATLAB Install Location.
ARG MATLAB_INSTALL_LOCATION="/opt/matlab/${MATLAB_RELEASE}"
# Specify license server information using the format: port@hostname
ARG LICENSE_SERVER
# Specify the base image with MATLAB installed.
FROM mathworks/matlab:${MATLAB_RELEASE}
USER root
# Declare build arguments to use at the current build stage.
ARG MATLAB_RELEASE
ARG MATLAB_PRODUCT_LIST
ARG MATLAB_INSTALL_LOCATION
ARG LICENSE_SERVER
ARG DYNARE_RELEASE
# Install mpm dependencies.
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get update \
&& apt-get install --no-install-recommends --yes \
wget \
unzip \
ca-certificates \
&& apt-get clean \
&& apt-get autoremove \
&& rm -rf /var/lib/apt/lists/*
# Run mpm to install additional toolboxes for MATLAB in the target location and delete the mpm installation afterwards.
# If mpm fails to install successfully, then print the logfile in the terminal, otherwise clean up.
RUN wget -q https://www.mathworks.com/mpm/glnxa64/mpm \
&& chmod +x mpm \
&& ./mpm install \
--release=${MATLAB_RELEASE} \
--destination=${MATLAB_INSTALL_LOCATION} \
--products ${MATLAB_PRODUCT_LIST} \
|| (echo "MPM Installation Failure. See below for more information:" && cat /tmp/mathworks_root.log && false) \
&& rm -f mpm /tmp/mathworks_root.log
# Install dynare dependencies.
RUN export DEBIAN_FRONTEND=noninteractive \
&& apt-get update \
&& apt-get install --no-install-recommends --yes \
build-essential \
gfortran \
libboost-graph-dev \
libgsl-dev \
libmatio-dev \
libslicot-dev \
libslicot-pic \
libsuitesparse-dev \
flex \
libfl-dev \
bison \
autoconf \
automake \
texlive \
texlive-publishers \
texlive-latex-extra \
texlive-fonts-extra \
texlive-font-utils \
texlive-latex-recommended \
texlive-science \
texlive-plain-generic \
lmodern \
python3-sphinx \
tex-gyre \
latexmk \
libjs-mathjax \
doxygen \
x13as \
liboctave-dev \
octave-control \
octave-econometrics \
octave-io \
octave-statistics \
octave-struct \
octave-parallel \
gnuplot \
fonts-freefont-otf \
ghostscript \
epstool \
git \
&& apt-get clean \
&& apt-get autoremove \
&& rm -rf /var/lib/apt/lists/*
# Rename libraries (see matlab-support package: https://salsa.debian.org/debian/matlab-support/-/blob/master/debian/matlab-support.postinst)
RUN if [ -f "${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgcc_s.so.1" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgcc_s.so.1 ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgcc_s.so.1.bak; \
fi && \
if [ -f "${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libstdc++.so.6" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libstdc++.so.6 ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libstdc++.so.6.bak; \
fi && \
if [ -f "${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgfortran.so.5" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgfortran.so.5 ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libgfortran.so.5.bak; \
fi && \
if [ -f "${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libquadmath.so.0" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libquadmath.so.0 ${MATLAB_INSTALL_LOCATION}/sys/os/glnxa64/libquadmath.so.0.bak; \
fi && \
if [ -f "${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libfreetype.so.6" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libfreetype.so.6 ${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libfreetype.so.6.bak; \
fi
# Fix for epstopdf latex errors, i.e. LIBTIFF_4.0 not found
RUN if [ -f "${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libtiff.so.5" ]; then \
mv ${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libtiff.so.5 ${MATLAB_INSTALL_LOCATION}/bin/glnxa64/libtiff.so.5.bak ; \
fi
# Note: Uncomment one of the following two ways to configure the license server.
# WE DO NOT WANT OPTION 2!!!!
# Option 1. Specify the host and port of the machine that serves the network licenses
# if you want to store the license information in an environment variable. This
# is the preferred option. You can either use a build variable, like this:
# --build-arg LICENSE_SERVER=27000@MyServerName or you can specify the license server
# directly using: ENV MLM_LICENSE_FILE=27000@flexlm-server-name
ENV MLM_LICENSE_FILE=$LICENSE_SERVER
# Option 2. Alternatively, you can put a license file into the container (WE DON'T WANT THIS!!!)
# Enter the details of the license server in this file and uncomment the following line.
# COPY network.lic ${MATLAB_INSTALL_LOCATION}/licenses/
# Get Dynare sources as matlab user
USER matlab
WORKDIR /home/matlab
RUN git clone --depth 1 --branch ${DYNARE_RELEASE} --recurse-submodules https://git.dynare.org/dynare/dynare.git
# Compile Dynare
USER matlab
WORKDIR /home/matlab
RUN cd dynare \
&& autoreconf -si \
&& ./configure --with-matlab=${MATLAB_INSTALL_LOCATION} MATLAB_VERSION=${MATLAB_RELEASE} \
&& make -j$(($(nproc)+1))
# Add path of dynare to startup script for Octave.
RUN echo "addpath /home/matlab/dynare/matlab" >> /home/matlab/.octaverc
# Add path of dynare to startup script for MATLAB.
# Note that if startup.m file exists (in newer MATLAB containers), it is a MATLAB function
# and the last line is an "end", so we append the path to the second-to-last line
# For some reason we have to do this as root, otherwise the file is not writable
USER root
RUN filename="/home/matlab/Documents/MATLAB/startup.m" && \
if [ -f ${filename} ]; then \
tempfile=$(mktemp) && \
total_lines=$(wc -l < "$filename") && \
line_number=$((total_lines - 1)) && \
head -n "$line_number" "$filename" > "$tempfile" && \
echo "addpath('/home/matlab/dynare/matlab');" >> "$tempfile" && \
tail -n 1 "$filename" >> "$tempfile" && \
mv "$tempfile" "$filename" ; \
else \
echo "addpath('/home/matlab/dynare/matlab');" >> "$filename" ; \
fi && \
chown matlab:matlab "$filename"
# Set user and work directory.
USER matlab
WORKDIR /home/matlab

144
docker/README.md Normal file
View File

@ -0,0 +1,144 @@
# Dynare Docker Containers
We provide a range of pre-configured Docker containers for [Dynare](https://dynare.org), which include both Octave and MATLAB (pre-configured with Dynare already in the PATH) and all recommended toolboxes. These containers are ideal for using Dynare in CI/CD environments ([example Workflow for GitHub Actions](https://github.com/wmutschl/DSGE_mod/tree/master/.github/workflows)) or High Performance Computing clusters with either [Docker, ENROOT or Singularity](https://wiki.bwhpc.de/e/BwUniCluster2.0/Containers).
To minimize maintenance efforts while ensuring high levels of security, reliability, and performance, our Docker containers are built from the official [MATLAB container base image](https://hub.docker.com/r/mathworks/matlab) using a custom [Dockerfile](Dockerfile). For more information on building and customizing the containers, see the [built instructions and customization](#built-instructions-and-customization) section below. Additionally, we provide an example [docker-compose file](docker-compose.yml) for complete access to the Ubuntu Desktop via VNC.
## Supported tags
| Tags | Dynare Version | MATLAB® Version | Octave Version | Operating System | Base Image |
|--------|----------------|-----------------|----------------|------------------|-------------------------|
| latest | 5.4 | R2023a | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2023a |
| 5.4 | 5.4 | R2023a | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2023a |
| 5.3 | 5.3 | R2022b | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2022b |
| 5.2 | 5.2 | R2022a | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2022a |
| 5.1 | 5.1 | R2022a | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2022a |
| 5.0 | 5.0 | R2021b | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2021b |
| 4.6.4 | 4.6.4 | R2021a | 5.2.0 | Ubuntu 20.04 | mathworks/matlab:R2021a |
## How to interact with the container
To pull the latest image to your machine, execute:
```sh
docker pull dynare/dynare:latest
```
In the following we assume that you have access to a MATLAB license using e.g. a [network license server of a University](https://uni-tuebingen.de/de/3107#c4656) and show different workflows how to interact with the container.
Obviously, you would need to adjust the environmental variable `MLM_LICENSE_FILE` to your use-case, please refer to the [MATLAB license](#matlab-license) section on licensing information and how to pass a personal license.
Alternatively, if you don't have access to a license or the closed-source mentality of MATLAB is an irreconcilable issue for you, you can equally well use Dynare with the free and open-source alternative Octave.
### Run Dynare in an interactive MATLAB session in the browser
To launch the container with the `-browser` option, execute:
```sh
docker run -it --rm -p 8888:8888 -e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de --shm-size=512M dynare/dynare:latest -browser
```
You will receive a URL to access MATLAB in a web browser, for example: `http://localhost:8888` or another IP address that you can use to reach your server, such as through a VPN like [Tailscale](https://tailscale.com) if you are behind a firewall. Enter the URL provided into a web browser. If prompted, enter credentials for a MathWorks account associated with a MATLAB license. If you are using a network license manager, switch to the Network License Manager tab and enter the license server address instead. After providing your license information, a MATLAB session will start in the browser. This may take several minutes. To modify the behavior of MATLAB when launched with the `-browser` flag, pass environment variables to the `docker run` command. For more information, see [Advanced Usage](https://github.com/mathworks/matlab-proxy/blob/main/Advanced-Usage.md).
Note that the `-browser` flag is supported by base images starting from `mathworks/matlab:R2022a` using [noVNC](https://novnc.com). Some browsers, like Safari, may not support this workflow.
### Run Ubuntu desktop and interact with it via VNC
To start the Ubuntu desktop with activated VNC server, execute:
```sh
docker run -it --rm -p 5901:5901 -p 6080:6080 -e PASSWORD=dynare -e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de --shm-size=512M dynare/dynare:latest -vnc
```
To connect to the Ubuntu desktop, either:
- Point a browser to port 6080 of the docker host machine running this container (`http://hostname:6080`).
- Use a VNC client to connect to display 1 of the docker host machine (`hostname:1`). The VNC password is `matlab` by default, you can change that by adjusting the `PASSWORD` environment variable in the run command.
- If you are behind a firewall, we recommend to use a VPN such as [Tailscale](https://tailscale.com).
### Run Dynare with Octave in an interactive command prompt
To start the container and run Dynare with Octave in an interactive command prompt, execute:
```sh
docker run -it --rm --shm-size=512M dynare/dynare:latest octave
```
### Run Dynare with MATLAB non-interactively in batch mode
To start the container and run an example mod file using Dynare with MATLAB execute:
```sh
docker run --rm -e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de dynare/dynare:latest matlab -batch "cd dynare/examples; dynare example1"
```
### Run a bash shell inside the container
To start a bash shell inside the container, execute:
```sh
docker run -it --rm --shm-size=512M dynare/dynare:latest -shell
```
You can also non-interactively run a sequence of commands:
```sh
docker run --rm --shm-size=512M \
-e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de \
dynare/dynare:latest /bin/bash -c "\
cd /home/matlab/dynare/examples && \
matlab -batch 'dynare example1 console' && \
octave --eval 'dynare example1 console'"
```
### Run MATLAB Desktop using X11
To start the container and run MATLAB Desktop using X11, execute:
```sh
xhost +
docker run -it --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:ro -e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de --shm-size=512M dynare/dynare:latest
```
The Desktop window of MATLAB will open on your machine. Note that the command above works only on a Linux operating system with X11 and its dependencies installed.
## Additional information
### MATLAB license
To run this container, your license must be [configured for cloud use](https://mathworks.com/help/install/license/licensing-for-mathworks-products-running-on-the-cloud.html). Individual and Campus-Wide licenses are already configured for cloud use. If you have a different license type, please contact your license administrator to configure it for cloud use. You can identify your license type and administrator by viewing your MathWorks Account. Administrators can consult the "Administer Network Licenses" documentation. If you don't have a MATLAB license, you can obtain a trial license at [MATLAB Trial for Docker](https://de.mathworks.com/campaigns/products/trials/targeted/dkr.html).
#### Network license
If you're using a network license, you can pass the port and hostname via the `MLM_LICENSE_FILE` environmental variable in your `docker run` command or Docker Compose file. Here's an example `docker run` command that uses a network license:
```sh
docker run --init -it --rm -e MLM_LICENSE_FILE=27000@matlab-campus.uni-tuebingen.de --shm-size=512M dynare/dynare:latest matlab -batch "cd dynare/examples; dynare example1"
```
#### Personal License
To use a personal license, you must first create a license file via the MATHWORKS License Center, refer to [Option 2](https://de.mathworks.com/matlabcentral/answers/235126-how-do-i-generate-a-matlab-license-file#answer_190013) for detailed instructions.
For this process, you will need the `username` and a `host ID`. In the container, the username is predefined as 'matlab'.
The `host ID` corresponds to the MAC address of any network adapter in the container.
In Docker, you can supply a [randomly generated MAC address](https://miniwebtool.com/mac-address-generator/) (e.g., A6-7E-1A-F4-9A-92) during the docker run command.
Download the file from MATHWORKS License Center and ensure you provide the container with access to the license file by mounting it as a (read-only) volume.
Here is an example `docker run` command that utilizes a license file named `matlab-license.lic`, which is located in your home folder:
```sh
docker run --init -it --rm --mac-address A6-7E-1A-F4-9A-92 --shm-size=512M -v $HOME/matlab-license.lic:/licenses/license.lic:ro -e MLM_LICENSE_FILE=/licenses/license.lic dynare/dynare:latest matlab -batch "cd dynare/examples; dynare example1"
```
### Environment variables
When running the `docker run` command, you can specify environment variables using the `-e` option. The [base image](https://hub.docker.com/r/mathworks/matlab) documentation lists the available variables, such as `MLM_LICENSE_FILE`, `PASSWORD`, or `PROXY_SETTINGS`.
## Built instructions and customization
Here are the commands to create the Docker images available at [Docker Hub](https://hub.docker.com/r/dynare/dynare):
```sh
docker build --build-arg MATLAB_RELEASE=R2023a --build-arg DYNARE_RELEASE=5.4 -t dynare/dynare:latest .
docker build --build-arg MATLAB_RELEASE=R2023a --build-arg DYNARE_RELEASE=5.4 -t dynare/dynare:5.4 .
docker build --build-arg MATLAB_RELEASE=R2022b --build-arg DYNARE_RELEASE=5.3 -t dynare/dynare:5.3 .
docker build --build-arg MATLAB_RELEASE=R2022a --build-arg DYNARE_RELEASE=5.2 -t dynare/dynare:5.2 .
docker build --build-arg MATLAB_RELEASE=R2022a --build-arg DYNARE_RELEASE=5.1 -t dynare/dynare:5.1 .
docker build --build-arg MATLAB_RELEASE=R2021b --build-arg DYNARE_RELEASE=5.0 -t dynare/dynare:5.0 .
docker build --build-arg MATLAB_RELEASE=R2021a --build-arg DYNARE_RELEASE=4.6.4 -t dynare/dynare:4.6.4 .
```
If you need to customize the container, there are two ways to do so. You can either adjust the [Dockerfile](Dockerfile) and rebuild the container, or you can run the container interactively, make the necessary adjustments, and then commit the changes for later use. To commit changes to a container, use the `docker commit` command. This will create a new image with the changes you made. You can then use this image to start new containers with your customizations.
For more information on committing changes to a container, see the [Docker documentation](https://docs.docker.com/engine/reference/commandline/commit/) and how to [save changes in the containers](https://de.mathworks.com/help/cloudcenter/ug/save-changes-in-containers.html).
Note that if you plan to distribute your custom container, you should be aware of the licensing terms of any software included in the container.
The provided containers provide no inclusion or information about a MATLAB license file.
## License
This container includes commercial software products from The MathWorks, Inc. ("MathWorks Programs") and related materials. The MathWorks Programs are licensed under the MathWorks Software License Agreement, which is available in the MATLAB installation within this container.
The related materials in this container are licensed under separate licenses, which can be found in their respective folders.
Dynare is licensed under the GPL-3+.

18
docker/docker-compose.yml Normal file
View File

@ -0,0 +1,18 @@
version: '3'
services:
dynare:
image: dynare/dynare:latest
mac_address: A6-7E-1A-F4-9A-92 # randomly generated, used as host id in license file
environment:
PASSWORD: dynare
#MLM_LICENSE_FILE: 27000@matlab-campus.uni-tuebingen.de # use network license
MLM_LICENSE_FILE: /licenses/license.lic # use license file
volumes:
- $HOME/matlab-license.lic:/licenses/license.lic:ro # path to license file
- $HOME/scratch:/home/matlab/scratch # path to some shared folder from host system
ports:
- "5901:5901" # VNC server
- "6080:6080" # browser using NoVNC
shm_size: 512M # recommended as docker's default ist only 64M
command: -vnc # start Ubuntu desktop with VNC support