Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated the tests so they work again on CI! #909

Merged
merged 23 commits into from
Jan 31, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 16 additions & 39 deletions .github/workflows/run_tests_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,18 +59,17 @@ jobs:
if: matrix.shard == 1

- name: Run tests with code coverage
run: pytest -v --color=yes --cov-report term-missing --cov=bioptim --cov-report=xml:coverage.xml tests/shard${{ matrix.shard }}
run: pytest -v --color=yes --cov-report term-missing --cov=bioptim tests/shard${{ matrix.shard }}
if: matrix.os == 'ubuntu-latest'

- name: Archive coverage report
id: archive
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: coverage${{ matrix.shard }}
path: |
coverage.xml
.coverage

name: coverage-${{ matrix.shard }}
path: .coverage
if-no-files-found: error
include-hidden-files: true

merge-coverage:
needs: build
Expand All @@ -84,59 +83,37 @@ jobs:
run:
echo "PREFIX=${{ env.PREFIX_LINUX }}" >> $GITHUB_ENV


- name: Checkout code
uses: actions/checkout@v3

- name: Setup environment
uses: conda-incubator/setup-miniconda@v2
with:
miniforge-version: latest
use-mamba: true
activate-environment: bioptim
environment-file: environment.yml

- name: Print mamba info
run: |
conda info
# mamba list

- name: Install extra dependencies
run: |
sudo apt-get install -y python3-pip
sudo apt install -y python3-pip
pip3 install coverage

- name: Download all workflow run artifacts
id: download
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
pattern: coverage-*

- name: Rename coverage files
run: |
for shard in {1,2,3,4,5,6}; do
mv coverage${shard}/coverage.xml coverage${shard}.xml
mv coverage${shard}/.coverage* .coverage${shard}
mv coverage-${shard}/.coverage .coverage${shard}
done

- name: Show current dir content
run: pwd; ls -a -l

- name: Merge coverage reports
run: coverage combine .coverage1 .coverage2 .coverage3 .coverage4

- name: Show current dir content with new .coverage file
run: pwd; ls -a
run: coverage combine .coverage1 .coverage2 .coverage3 .coverage4 .coverage5 .coverage6

- name: Generate XML report
run: |
coverage xml
coverage report -m

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
- uses: codecov/codecov-action@v5
with:
#token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
#files: ./coverage1.xml, ./coverage2.xml, ./coverage3.xml, ./coverage4.xml
files: ./coverage.xml
flags: unittests
fail_ci_if_error: true
verbose: true
verbose: true
14 changes: 8 additions & 6 deletions bioptim/examples/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
import keyword
import os
import re
import sys
import subprocess
import sys

Check warning on line 7 in bioptim/examples/__main__.py

View check run for this annotation

Codecov / codecov/patch

bioptim/examples/__main__.py#L7

Added line #L7 was not covered by tests

import pyqtgraph as pg
from functools import lru_cache
from collections import OrderedDict
from PyQt5 import QtCore, QtGui, QtWidgets
from functools import lru_cache

Check warning on line 10 in bioptim/examples/__main__.py

View check run for this annotation

Codecov / codecov/patch

bioptim/examples/__main__.py#L10

Added line #L10 was not covered by tests

import pyqtgraph as pg
from pyqtgraph.Qt import QT_LIB, QtCore, QtGui, QtWidgets

Check warning on line 13 in bioptim/examples/__main__.py

View check run for this annotation

Codecov / codecov/patch

bioptim/examples/__main__.py#L12-L13

Added lines #L12 - L13 were not covered by tests

try:
import acados
Expand Down Expand Up @@ -137,6 +138,8 @@
]
)

app = pg.mkQApp()

Check warning on line 141 in bioptim/examples/__main__.py

View check run for this annotation

Codecov / codecov/patch

bioptim/examples/__main__.py#L141

Added line #L141 was not covered by tests


class Ui_Form(object):
def setupUi(self, Form):
Expand Down Expand Up @@ -206,7 +209,6 @@

path = os.path.abspath(os.path.dirname(__file__))
sys.path.insert(0, path)
app = pg.mkQApp()


QRegularExpression = QtCore.QRegularExpression
Expand Down Expand Up @@ -740,7 +742,7 @@
app = pg.mkQApp()
loader = ExampleLoader()
loader.ui.exampleTree.setCurrentIndex(loader.ui.exampleTree.model().index(0, 0))
app.exec()
pg.exec()

Check warning on line 745 in bioptim/examples/__main__.py

View check run for this annotation

Codecov / codecov/patch

bioptim/examples/__main__.py#L745

Added line #L745 was not covered by tests


if __name__ == "__main__":
Expand Down
4 changes: 2 additions & 2 deletions bioptim/interfaces/acados_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ def __acados_export_model(self, ocp):
x_sym = ocp.nlp[0].states.scaled.cx_start
u = ocp.nlp[0].controls.cx_start
u_sym = ocp.nlp[0].controls.scaled.cx_start
p = ocp.nlp[0].parameters.cx
p = ocp.nlp[0].parameters.scaled.cx
p_sym = ocp.nlp[0].parameters.scaled.cx
a = ocp.nlp[0].algebraic_states.cx_start
a_sym = ocp.nlp[0].algebraic_states.scaled.cx_start
Expand Down Expand Up @@ -294,7 +294,7 @@ def __set_constraints(self, ocp):
dt = nlp.dt
x = nlp.states.cx_start
u = nlp.controls.cx_start
p = nlp.parameters.cx
p = nlp.parameters.scaled.cx
a = nlp.algebraic_states.cx_start
d = nlp.numerical_timeseries.cx

Expand Down
16 changes: 8 additions & 8 deletions bioptim/interfaces/interface_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,19 +79,19 @@
interface.online_optim(interface.ocp, interface.opts.show_options)

# Thread here on (f and all_g) instead of individually for each function?
interface.sqp_nlp = {"x": v, "f": sum1(all_objectives), "g": all_g}
interface.nlp = {"x": v, "f": sum1(all_objectives), "g": all_g}
interface.c_compile = interface.opts.c_compile
options = interface.opts.as_dict(interface)

if interface.c_compile:
if not interface.ocp_solver or interface.ocp.program_changed:
nlpsol("nlpsol", interface.solver_name.lower(), interface.sqp_nlp, options).generate_dependencies("nlp.c")
nlpsol("nlpsol", interface.solver_name.lower(), interface.nlp, options).generate_dependencies("nlp.c")

Check warning on line 88 in bioptim/interfaces/interface_utils.py

View check run for this annotation

Codecov / codecov/patch

bioptim/interfaces/interface_utils.py#L88

Added line #L88 was not covered by tests
interface.ocp_solver = nlpsol("nlpsol", interface.solver_name, Importer("nlp.c", "shell"), options)
interface.ocp.program_changed = False
else:
interface.ocp_solver = nlpsol("solver", interface.solver_name.lower(), interface.sqp_nlp, options)
interface.ocp_solver = nlpsol("solver", interface.solver_name.lower(), interface.nlp, options)

interface.sqp_limits = {
interface.limits = {
"lbx": v_bounds[0],
"ubx": v_bounds[1],
"lbg": all_g_bounds.min,
Expand All @@ -100,13 +100,13 @@
}

if interface.lam_g is not None:
interface.sqp_limits["lam_g0"] = interface.lam_g
interface.limits["lam_g0"] = interface.lam_g
if interface.lam_x is not None:
interface.sqp_limits["lam_x0"] = interface.lam_x
interface.limits["lam_x0"] = interface.lam_x

# Solve the problem
tic = perf_counter()
interface.out = {"sol": interface.ocp_solver.call(interface.sqp_limits)}
interface.out = {"sol": interface.ocp_solver.call(interface.limits)}
interface.out["sol"]["solver_time_to_optimize"] = interface.ocp_solver.stats()["t_wall_total"]
interface.out["sol"]["real_time_to_optimize"] = perf_counter() - tic
interface.out["sol"]["iter"] = interface.ocp_solver.stats()["iter_count"]
Expand Down Expand Up @@ -172,7 +172,7 @@
try:
penalty = penalty.expand()
except RuntimeError:
# This happens mostly when there is a Newton decent in the penalty
# This happens mostly when, for instance, there is a Newton decent in the penalty
pass
return penalty(vertcat(*dt, v[len(dt) :]))

Expand Down
12 changes: 6 additions & 6 deletions bioptim/interfaces/ipopt_interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@
)
from .solver_interface import SolverInterface
from ..interfaces import Solver
from bioptim.optimization.solution.solution import Solution
from ..optimization.non_linear_program import NonLinearProgram
from ..misc.enums import (
SolverType,
)
from ..optimization.non_linear_program import NonLinearProgram
from ..optimization.solution.solution import Solution


class IpoptInterface(SolverInterface):
Expand All @@ -27,9 +27,9 @@ class IpoptInterface(SolverInterface):
Options irrelevant of a specific ocp
opts: IPOPT
Options of the current ocp
ipopt_nlp: dict
nlp: dict
The declaration of the variables Ipopt-friendly
ipopt_limits: dict
limits: dict
The declaration of the bound Ipopt-friendly
lam_g: np.ndarray
The lagrange multiplier of the constraints to initialize the solver
Expand Down Expand Up @@ -64,8 +64,8 @@ def __init__(self, ocp):
self.opts = Solver.IPOPT()
self.solver_name = SolverType.IPOPT.value

self.ipopt_nlp = {}
self.ipopt_limits = {}
self.nlp = {}
self.limits = {}
self.ocp_solver = None
self.c_compile = False

Expand Down
12 changes: 6 additions & 6 deletions bioptim/interfaces/sqp_interface.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import numpy as np

from bioptim.optimization.solution.solution import Solution
from .interface_utils import (
generic_online_optim,
generic_solve,
Expand All @@ -9,11 +10,10 @@
)
from .solver_interface import SolverInterface
from ..interfaces import Solver
from bioptim.optimization.solution.solution import Solution
from ..optimization.non_linear_program import NonLinearProgram
from ..misc.enums import (
SolverType,
)
from ..optimization.non_linear_program import NonLinearProgram


class SQPInterface(SolverInterface):
Expand All @@ -26,9 +26,9 @@ class SQPInterface(SolverInterface):
Options irrelevant of a specific ocp
opts: SQP
Options of the current ocp
sqp_nlp: dict
nlp: dict
The declaration of the variables SQP-friendly
sqp_limits: dict
limits: dict
The declaration of the bound SQP-friendly
lam_g: np.ndarray
The lagrange multiplier of the constraints to initialize the solver
Expand Down Expand Up @@ -63,8 +63,8 @@ def __init__(self, ocp):
self.opts = Solver.SQP_METHOD()
self.solver_name = SolverType.SQP.value

self.sqp_nlp = {}
self.sqp_limits = {}
self.nlp = {}
self.limits = {}
self.ocp_solver = None
self.c_compile = False

Expand Down
4 changes: 3 additions & 1 deletion bioptim/models/biorbd/biorbd_model.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import os
from pathlib import Path
from typing import Callable

import biorbd_casadi as biorbd
Expand Down Expand Up @@ -89,7 +91,7 @@ def name(self) -> str:

@property
def path(self) -> str:
return self.model.path().relativePath().to_string()
return self.model.path().absolutePath().to_string()

def copy(self):
return BiorbdModel(self.path)
Expand Down
5 changes: 3 additions & 2 deletions bioptim/models/biorbd/multi_biorbd_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,12 @@ def deep_copy(self, *args):
raise NotImplementedError("Deep copy is not implemented yet for MultiBiorbdModel class")

@property
def path(self) -> (list[str], list[str]):
def path(self) -> tuple[list[str], list[str]]:
return [model.path for model in self.models], [model.path for model in self.extra_models]

def copy(self):
return MultiBiorbdModel(tuple(self.path[0]), tuple(self.path[1]))
all_paths = self.path
return MultiBiorbdModel(tuple(all_paths[0]), tuple(all_paths[1]))

def serialize(self) -> tuple[Callable, dict]:
return MultiBiorbdModel, dict(bio_model=tuple(self.path[0]), extra_bio_models=tuple(self.path[1]))
Expand Down
8 changes: 8 additions & 0 deletions bioptim/models/protocols/stochastic_biomodel.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ def sensory_reference(self, time, states, controls, parameters, algebraic_states
def reshape_to_matrix(var, shape):
"""
Restore the matrix form of the variables

See Also
--------
reshape_to_vector
"""

if var.shape[0] != shape[0] * shape[1]:
Expand Down Expand Up @@ -83,6 +87,10 @@ def reshape_to_cholesky_matrix(var, shape):
def reshape_to_vector(matrix):
"""
Restore the vector form of the matrix

See Also
--------
reshape_to_matrix
"""
shape_0, shape_1 = matrix.shape[0], matrix.shape[1]
if isinstance(matrix, np.ndarray):
Expand Down
6 changes: 3 additions & 3 deletions resources/plotting_server.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""
This file is an example of how to run a bioptim Online plotting server. Apart on Macos, this is usually not the way to run a
bioptim server as it is easier to run it as an automatic multiprocess. This is achieved by setting
`Solver.IPOPT(online_optim=OnlineOptim.MULTIPROCESS_SERVER)`.
This file is an example of how to run a bioptim Online plotting server. Apart on Macos, this is usually not the way
to run a bioptim server as it is easier to run it as an automatic multiprocess. This is achieved by setting
`Solver.IPOPT(online_optim=OnlineOptim.MULTIPROCESS_SERVER)`.
If set to OnlineOptim.SERVER, then the plotting server is mandatory.

Since the server runs usings sockets, it is possible to run the server on a different machine than the one running the
Expand Down
Loading
Loading