Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare for article release #73

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,14 @@ floydhub/PROJECT_NAME:VERSION-ENV-gpu
```

Contains docker images for popular deep learning frameworks including: Tensorflow, PyTorch and Torch.

## Repository's structure

Add image from the blog & link

TODO: Floydker link

TODO: CI link?

TODO: base & dl explanation (copy from the article)

1 change: 1 addition & 0 deletions base/buildpack-deps/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM buildpack-deps:bionic
MAINTAINER FloydHub "[email protected]"

RUN apt-get update && apt-get install -y --no-install-recommends \
sudo \
Expand Down
4 changes: 3 additions & 1 deletion dl/dl-base/dl-base-2.x.x.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,9 @@ RUN pip --no-cache-dir install \


# Install and Enable jupyter-widgets - see https://github.com/jupyter-widgets/ipywidgets/tree/master/packages/jupyterlab-manager
RUN jupyter labextension install @jupyter-widgets/[email protected]
RUN jupyter labextension install \
@jupyter-widgets/[email protected] \
jupyterlab-plotly

{%- if arch == 'gpu' %}
# Update Cmake to the latest version to build xgboost on CUDA
Expand Down
58 changes: 3 additions & 55 deletions dl/tensorflow/tests/2.0/run.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#! /bin/bash
#############################
# TF 1.7 Testing Script #
# TF 2.0 Testing Script #
#############################

#TF Session
# TF Session
python tf_keras_mnist.py

# TF Eager
Expand All @@ -13,56 +13,4 @@ python mnist_eager.py
python keras_mnist_test.py


#TODO Move from tf.tf_cnn_benchmarks to perfzero https://github.com/tensorflow/benchmarks/tree/master/perfzero

# # Delete benchmarks folder if exists
# if test -d benchmarks; then
# rm -rf benchmarks
# fi

# Clone Benchmarks and go to a certain commit
# git clone https://github.com/tensorflow/benchmarks.git
# cd benchmarks/scripts/tf_cnn_benchmarks

# # Check if GPU is installed (quick hack)
# which nvidia-smi &> /dev/null
# ISGPU=$?

# if test $ISGPU -eq 1; then
# # CPU or CPU2?
# NCORE=`cat /proc/cpuinfo | grep processor | wc -l`

# # Quick Benchmark on AlexNet (syntethic images, no data transformation, channel last)
# if test $NCORE -eq 2; then
# # CPU test (defined initial lr otherwise Loss is NaN )
# python tf_cnn_benchmarks.py --device=cpu --kmp_blocktime=0 --nodistortions --model=alexnet --data_format=NHWC --batch_size=32 --num_inter_threads=1 --num_intra_threads=$NCORE --init_learning_rate=0.00001
# else
# # CPU2 test
# python tf_cnn_benchmarks.py --device=cpu --kmp_blocktime=0 --nodistortions --model=alexnet --data_format=NHWC --batch_size=64 --num_inter_threads=1 --num_intra_threads=$NCORE --init_learning_rate=0.00001
# fi
# else
# GPU_DEV_NAME=$(python -c 'from __future__ import print_function; import tensorflow as tf; print(tf.test.gpu_device_name())' 2> /dev/null)
# case ${GPU_DEV_NAME} in
# /device:GPU:*)
# echo "GPU load test passed. Device found: ${GPU_DEV_NAME}"
# ;;
# *)
# echo "ERROR: Tensorflow not able to find GPU device! Device found: ${GPU_DEV_NAME}"
# exit 1
# ;;
# esac

# # GPU or GPU2?
# nvidia-smi | grep -q V100
# ISV100=$?

# # Quick Benchmark on Resnet-50 (syntethic images, no data transformation, channel first)
# if test $ISV100 -eq 1; then
# # GPU test
# python tf_cnn_benchmarks.py --device=gpu --num_gpus=1 --batch_size=64 --model=resnet50 --nodistortions --data_format=NCHW
# else
# # GPU2 test (with MXP)
# python tf_cnn_benchmarks.py --device=gpu --num_gpus=1 --batch_size=256 --model=resnet50 --nodistortions --data_format=NCHW --use_fp16=true
# fi
# fi

# Consider to move from tf.tf_cnn_benchmarks to perfzero https://github.com/tensorflow/benchmarks/tree/master/perfzero
6 changes: 0 additions & 6 deletions test_gpu_imgs.sh

This file was deleted.

50 changes: 0 additions & 50 deletions testbase.sh

This file was deleted.