Skip to content

Commit

Permalink
GH-44703: [CI][MATLAB][Packaging] Update MATLAB CI and crossbow pac…
Browse files Browse the repository at this point in the history
…kaging workflows to build against MATLAB `R2024b` (#44704)

### Rationale for this change

MATLAB [R2024b](https://www.mathworks.com/products/new_products/latest_features.html) is now available for use with the [matlab-actions/setup-matlab](https://github.com/matlab-actions/setup-matlab) GitHub Action.

We should update the [matlab.yml CI workflow](https://github.com/apache/arrow/blob/aab7d81aeec1e8f106bdda953cdeb00f7f78b355/.github/workflows/matlab.yml#L126), as well as the [crossbow packaging workflows for the MATLAB MLTBX files](https://github.com/apache/arrow/blob/aab7d81aeec1e8f106bdda953cdeb00f7f78b355/dev/tasks/matlab/github.yml#L34) to build against R2024b.

### What changes are included in this PR?

1. Updated the `.github/workflows/matlab.yml` CI workflow file to build the MATLAB Interface against MATLAB R2024b.
2. Updated the `dev/tasks/matlab/github.yml` `crossbow` packaging workflow to build the MATLAB MLTBX files against MATLAB R2024b.
3. Bumped `mathworks/libmexclass` version to commit [cac7c3630a086bd5ba41413af44c833cef189c09](mathworks/libmexclass@cac7c36) to work around mathworks/libmexclass#92

### Are these changes tested?

Yes.

1. CI workflow [successfully passed on all platforms in `mathworks/arrow`](https://github.com/mathworks/arrow/actions/runs/11805483560).
2. Crossbow job: https://github.com/ursacomputing/crossbow/actions/runs/11805816426.

### Are there any user-facing changes?

Yes.

1. All changes to the MATLAB interface will now be built against R2024b.
2. The MATLAB MLTBX release artifacts will now be built against R2024b.

### Notes

1. Thank you @ sgilmore10 for your help with this pull request!
* GitHub Issue: #44703

Authored-by: Kevin Gurney <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
  • Loading branch information
kevingurney authored Nov 16, 2024
1 parent df40f7a commit 133e114
Show file tree
Hide file tree
Showing 6 changed files with 31 additions and 21 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/matlab.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Install ccache
run: sudo apt-get install ccache
- name: Setup ccache
Expand Down Expand Up @@ -123,7 +123,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Install ccache
run: brew install ccache
- name: Setup ccache
Expand Down Expand Up @@ -162,7 +162,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Download Timezone Database
shell: bash
run: ci/scripts/download_tz_database.sh
Expand Down
14 changes: 14 additions & 0 deletions dev/archery/archery/crossbow/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,19 @@ def __init__(self, head, branch, remote, version, r_version, email=None):
self.r_version = r_version
self.no_rc_version = re.sub(r'-rc\d+\Z', '', version)
self.no_rc_r_version = re.sub(r'-rc\d+\Z', '', r_version)
# MAJOR.MINOR.PATCH Versioning
#
# Excludes:
#
# 1. Release Candidate (RC) string components (e.g. -rc123)
# 2. Dev string components (e.g. .dev123)
#
# Example:
#
# '19.0.0.dev66' ->
# '19.0.0'
self.no_rc_no_dev_version = \
re.sub(r'\.dev\d+\Z', '', self.no_rc_version)
# Semantic Versioning 1.0.0: https://semver.org/spec/v1.0.0.html
#
# > A pre-release version number MAY be denoted by appending an
Expand Down Expand Up @@ -1195,6 +1208,7 @@ def from_config(cls, config, target, tasks=None, groups=None, params=None):
versions = {
'version': target.version,
'no_rc_version': target.no_rc_version,
'no_rc_no_dev_version': target.no_rc_no_dev_version,
'no_rc_semver_version': target.no_rc_semver_version,
'no_rc_snapshot_version': target.no_rc_snapshot_version,
'r_version': target.r_version,
Expand Down
10 changes: 5 additions & 5 deletions dev/tasks/matlab/github.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Build MATLAB Interface
env:
{{ macros.github_set_sccache_envvars()|indent(8) }}
Expand Down Expand Up @@ -73,7 +73,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Build MATLAB Interface
env:
{{ macros.github_set_sccache_envvars()|indent(8) }}
Expand All @@ -99,7 +99,7 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Install sccache
shell: bash
run: arrow/ci/scripts/install_sccache.sh pc-windows-msvc $(pwd)/sccache
Expand Down Expand Up @@ -146,13 +146,13 @@ jobs:
- name: Install MATLAB
uses: matlab-actions/setup-matlab@v2
with:
release: R2024a
release: R2024b
- name: Run commands
env:
MATLABPATH: arrow/matlab/tools
ARROW_MATLAB_TOOLBOX_FOLDER: arrow/matlab/install/arrow_matlab
ARROW_MATLAB_TOOLBOX_OUTPUT_FOLDER: artifacts/matlab-dist
ARROW_MATLAB_TOOLBOX_VERSION: {{ arrow.no_rc_version }}
ARROW_MATLAB_TOOLBOX_VERSION: {{ arrow.no_rc_no_dev_version }}
uses: matlab-actions/run-command@v2
with:
command: packageMatlabInterface
Expand Down
2 changes: 1 addition & 1 deletion dev/tasks/tasks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ tasks:
ci: github
template: matlab/github.yml
artifacts:
- matlab-arrow-{no_rc_version}.mltbx
- matlab-arrow-{no_rc_no_dev_version}.mltbx

############################## Arrow JAR's ##################################

Expand Down
2 changes: 1 addition & 1 deletion matlab/tools/cmake/BuildMatlabArrowInterface.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ set(MATLAB_ARROW_LIBMEXCLASS_CLIENT_FETCH_CONTENT_NAME libmexclass)
# libmexclass is accessible for CI without permission issues.
set(MATLAB_ARROW_LIBMEXCLASS_CLIENT_FETCH_CONTENT_GIT_REPOSITORY "https://github.com/mathworks/libmexclass.git")
# Use a specific Git commit hash to avoid libmexclass version changing unexpectedly.
set(MATLAB_ARROW_LIBMEXCLASS_CLIENT_FETCH_CONTENT_GIT_TAG "ca3cea6")
set(MATLAB_ARROW_LIBMEXCLASS_CLIENT_FETCH_CONTENT_GIT_TAG "cac7c3630a086bd5ba41413af44c833cef189c09")
set(MATLAB_ARROW_LIBMEXCLASS_CLIENT_FETCH_CONTENT_SOURCE_SUBDIR "libmexclass/cpp")

# ------------------------------------------
Expand Down
18 changes: 7 additions & 11 deletions matlab/tools/packageMatlabInterface.m
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,11 @@

toolboxFolder = string(getenv("ARROW_MATLAB_TOOLBOX_FOLDER"));
outputFolder = string(getenv("ARROW_MATLAB_TOOLBOX_OUTPUT_FOLDER"));
toolboxVersionRaw = string(getenv("ARROW_MATLAB_TOOLBOX_VERSION"));
toolboxVersion = string(getenv("ARROW_MATLAB_TOOLBOX_VERSION"));
if isempty(toolboxVersion)
error("ARROW_MATLAB_TOOLBOX_VERSION environment variable value is empty." + ...
"ARROW_MATLAB_TOOLBOX_VERSION should follow the general form: ${MAJOR_VERSION}.${MINOR_VERSION}.${PATCH_VERSION}.");
end

appendLicenseText(fullfile(toolboxFolder, "LICENSE.txt"));
appendNoticeText(fullfile(toolboxFolder, "NOTICE.txt"));
Expand All @@ -27,15 +31,7 @@

disp("Toolbox Folder: " + toolboxFolder);
disp("Output Folder: " + outputFolder);
disp("Toolbox Version Raw: " + toolboxVersionRaw);

versionPattern = regexpPattern("^[0-9]+\.[0-9]+\.[0-9]+");
toolboxVersion = extract(toolboxVersionRaw, versionPattern);
if isempty(toolboxVersion)
error("Unable to extract MAJOR.MINOR.PATCH version string from " + toolboxVersionRaw);
end

disp("Toolbox Version:" + toolboxVersion);
disp("Toolbox Version: " + toolboxVersion);

identifier = "ad1d0fe6-22d1-4969-9e6f-0ab5d0f12ce3";
opts = matlab.addons.toolbox.ToolboxOptions(toolboxFolder, identifier);
Expand Down Expand Up @@ -85,4 +81,4 @@ function appendNoticeText(filename)
"This product includes software from The MathWorks, Inc. (Apache 2.0)"
" * Copyright (C) 2024 The MathWorks, Inc."];
writelines(noticeText, filename, WriteMode="append");
end
end

0 comments on commit 133e114

Please sign in to comment.