Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Reduce feast-server container image size & fix dev image build #4781

Merged
merged 1 commit into from
Nov 22, 2024
Merged

fix: Reduce feast-server container image size & fix dev image build #4781

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 12 additions & 16 deletions sdk/python/feast/infra/feature_servers/multicloud/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
FROM debian:11-slim
RUN apt update && \
apt install -y \
jq \
python3 \
python3-pip \
python3-dev \
build-essential
FROM python:3.11-slim-bullseye

RUN pip install pip --upgrade
RUN pip install "feast[aws,gcp,snowflake,redis,go,mysql,postgres,opentelemetry,grpcio]"
RUN pip install --no-cache-dir pip --upgrade
RUN pip install --no-cache-dir "feast[aws,gcp,snowflake,redis,go,mysql,postgres,opentelemetry,grpcio]"


RUN apt update
RUN apt install -y -V ca-certificates lsb-release wget
RUN wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt update
RUN apt -y install libarrow-dev
RUN apt update && apt install -y -V ca-certificates lsb-release wget && \
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && apt update && \
apt install -y \
jq \
libarrow-dev && \
apt remove -y lsb-release wget && \
apt-get clean && rm -rf /var/cache/apt/lists

# modify permissions to support running with a random uid
RUN mkdir -m 775 /.cache
RUN chmod g+w $(python3 -c "import feast.ui as _; print(_.__path__)" | tr -d "[']")/build/projects-list.json
38 changes: 21 additions & 17 deletions sdk/python/feast/infra/feature_servers/multicloud/Dockerfile.dev
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,28 @@
FROM debian:11-slim
FROM python:3.11-slim-bullseye

RUN apt update && \
apt install -y \
jq \
python3 \
python3-pip \
python3-dev \
build-essential
RUN pip install --no-cache-dir pip --upgrade
RUN pip install --no-cache-dir pip-tools

RUN pip install pip --upgrade
COPY . .
RUN apt update && apt install -y -V ca-certificates lsb-release wget make git curl gcc && \
curl -sL https://deb.nodesource.com/setup_20.x | bash - && \
wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && apt update && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one of these days we should investigate on the reason why these arrow source code bundle for debian is installed.
AFAIK, installing the pyarrow python package is the only thing we need to connect an arrow system.
I will track a GH issue to investigate and in case clear the dockerfile.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#4785

apt install -y \
jq \
nodejs \
libarrow-dev && \
npm install -g yarn && \
apt remove -y lsb-release wget && \
apt-get clean && rm -rf /var/cache/apt/lists

RUN pip install "feast[aws,gcp,snowflake,redis,go,mysql,postgres,opentelemetry,grpcio]"
COPY . /feast
WORKDIR /feast
RUN make install-python-ci-dependencies && pip cache purge
ENV NPM_TOKEN '//registry.npmjs.org/:_authToken'
RUN make build-ui && yarn cache clean

WORKDIR /

RUN apt update
RUN apt install -y -V ca-certificates lsb-release wget
RUN wget https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
RUN apt update
RUN apt -y install libarrow-dev
# modify permissions to support running with a random uid
RUN mkdir -m 775 /.cache
RUN chmod g+w $(python3 -c "import feast.ui as _; print(_.__path__)" | tr -d "[']")/build/projects-list.json
Loading