Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(build): Print out build metrics #12142

Closed
wants to merge 7 commits into from
Closed

feat(build): Print out build metrics #12142

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
a148b5e
Print out various metrics.
kgpai Jan 22, 2025
6be2180
Export extra cmake flags.
kgpai Jan 22, 2025
6a1ac40
unify build step, fix cmake flags and upload size file as artifact
assignUser Jan 22, 2025
13af77c
Merge branch 'main' into print_build_stats
assignUser Jan 22, 2025
b9927b6
fix find invocation
assignUser Jan 22, 2025
14090bd
use exec but with grouping
assignUser Jan 22, 2025
b723889
fix artifact name
assignUser Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 17 additions & 24 deletions .github/workflows/build-metrics.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ permissions:

jobs:
metrics:
name: Linux ${{ matrix.type }} with adapters
name: Linux ${{ matrix.link-type }} - ${{ matrix.type }} with adapters
if: ${{ github.repository == 'facebookincubator/velox' }}
runs-on: ${{ matrix.runner }}
container: ghcr.io/facebookincubator/velox-dev:adapters
Expand All @@ -59,7 +59,6 @@ jobs:
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}

- name: Make ${{ matrix.link-type }} - ${{ matrix.type }} Build
if: ${{ matrix.link-type == 'static' }}
env:
MAKEFLAGS: 'MAX_HIGH_MEM_JOBS=8 MAX_LINK_JOBS=4'
run: |
Expand All @@ -73,44 +72,38 @@ jobs:
"-DVELOX_ENABLE_ABFS=ON"
"-DVELOX_ENABLE_REMOTE_FUNCTIONS=ON"
"-DVELOX_MONO_LIBRARY=ON"
"-DVELOX_BUILD_SHARED=${{ matrix.link-type == 'shared' && 'ON' || 'OFF' }}"
)
make '${{ matrix.type }}'

- name: Make ${{ matrix.link-type }} - ${{ matrix.type }} Build
if: ${{ matrix.link-type == 'shared' }}
env:
MAKEFLAGS: 'MAX_HIGH_MEM_JOBS=8 MAX_LINK_JOBS=4'
run: |
EXTRA_CMAKE_FLAGS=(
"-DVELOX_ENABLE_BENCHMARKS=ON"
"-DVELOX_ENABLE_ARROW=ON"
"-DVELOX_ENABLE_PARQUET=ON"
"-DVELOX_ENABLE_HDFS=ON"
"-DVELOX_ENABLE_S3=ON"
"-DVELOX_ENABLE_GCS=ON"
"-DVELOX_ENABLE_ABFS=ON"
"-DVELOX_ENABLE_REMOTE_FUNCTIONS=ON"
"-DVELOX_MONO_LIBRARY=ON"
"-DVELOX_BUILD_SHARED=ON"
)
make '${{ matrix.type }}'
echo "CMake Flags: ${EXTRA_CMAKE_FLAGS[*]}"
make '${{ matrix.type }}' EXTRA_CMAKE_FLAGS="${EXTRA_CMAKE_FLAGS[*]}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat , did not know this trick.


- name: Log binary sizes
run: |
mkdir -p /tmp/metrics
sizes_file=/tmp/metrics/object_sizes
echo "sizes_file=$sizes_file" >> $GITHUB_ENV

pushd '_build/${{ matrix.type }}'

# . to also check in lib for mono build
find . -type f -name 'libvelox*.so' -o -name 'libvelox*.a' -exec ls -l -BB {} \; |
find . -type f -name 'libvelox*.so' -o -name 'libvelox*.a' | xargs ls -l -BB |
awk '{print $5, $9; total += $5} END {print total," total_lib_size"}' > $sizes_file

find velox -type f -name '*.o' -exec ls -l -BB {} \; |
find velox -type f -name '*.o' | ls -l -BB |
awk '{print $5, $9; total += $5} END {print total," total_obj_size"}' >> $sizes_file

find velox -type f -name 'velox_*' -exec ls -l -BB {} \; |
find velox -type f -name 'velox_*' | ls -l -BB |
awk '{print $5, $9; total += $5} END {print total," total_exec_size"}' >> $sizes_file

echo "::group::Size file for ${{ matrix.link-type }} - ${{ matrix.type }}"
cat $sizes_file
echo "::endgroup::"

- uses: actions/upload-artifact@v4
with:
path: ${{ env.sizes_file }}

- name: Copy ninja_log
run: cp _build/${{ matrix.type }}/.ninja_log /tmp/metrics/.ninja_log

Expand Down
Loading