Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add wrappers for new tests approach with hspec #4

Merged
merged 17 commits into from
Feb 5, 2025
Merged

add wrappers for new tests approach with hspec #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
f95dc5a
wip: add wrappers for new tests approach with hspec
Intey Jan 30, 2025
9a92f33
test files for both approaches
Intey Jan 31, 2025
f8e2123
wrong names for test files
Intey Jan 31, 2025
9cd83c9
haskell: checks for both approach is green
Intey Jan 31, 2025
620cb92
haskell clean
Intey Feb 2, 2025
70e99f0
haskell: conditionally make it less verbose
Intey Feb 2, 2025
b03cca0
haskell: make silent by default
Intey Feb 3, 2025
2b6f596
haskell: remove 'Progess n/N' of test run for new approach
Intey Feb 3, 2025
69674c1
haskell: possibly fix issue with handling those marks
Intey Feb 3, 2025
f045f8f
haskell: now the error text is visible
Intey Feb 3, 2025
342f7aa
haskell: reduce compile output and color changes
Intey Feb 3, 2025
a82130f
haskell: supress build log on old task type
Intey Feb 3, 2025
71d9f26
haskell: review change request: optimization
Intey Feb 3, 2025
ca40bff
haskell: clean&color
Intey Feb 3, 2025
2ce511b
haskell: show runtime errors
Intey Feb 4, 2025
053c271
haskell: timeout was too long
Intey Feb 4, 2025
1d26aad
Increase test timeout
mesozoic-drones Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 17 additions & 15 deletions haskell/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,9 @@ FROM haskell:9.8.2-slim-buster
RUN apt-get -y update && apt-get -y upgrade \
&& apt-get -y install python3

WORKDIR /home/code_runner

# Directory for task launching
RUN mkdir /home/code_runner/task
COPY task/run-task.sh /home/code_runner/task
RUN stack install --install-ghc --resolver lts-23.0 hspec hspec-api hspec-core QuickCheck

COPY package.yaml /home/code_runner
COPY config.yaml /home/code_runner

# Directory for task launching
RUN mkdir /home/code_runner/playground
COPY playground/run-playground.sh /home/code_runner/playground
WORKDIR /home/code_runner

# Change access write for code_runner home directory to root recursively
RUN useradd -ms /bin/bash -u 1000 code_runner \
Expand All @@ -24,10 +15,21 @@ RUN useradd -ms /bin/bash -u 1000 code_runner \

USER code_runner

RUN stack new user-code \
&& cp /home/code_runner/config.yaml /home/code_runner/.stack/ \
&& cp /home/code_runner/package.yaml /home/code_runner/user-code/
RUN stack new user-code

RUN cd /home/code_runner/user-code/ && stack build
COPY config.yaml /home/code_runner/.stack


# cache warming for all dependencies including tests
COPY task/MachineFormatter.hs /home/code_runner/user-code/test/
COPY package.yaml /home/code_runner/user-code
RUN cd /home/code_runner/user-code/ && stack build --test --fast

# Directory for task launching
RUN mkdir /home/code_runner/task
COPY task/run-task.sh /home/code_runner/task
# Directory for task launching
RUN mkdir /home/code_runner/playground
COPY playground/run-playground.sh /home/code_runner/playground

ENTRYPOINT ["bash"]
7 changes: 6 additions & 1 deletion haskell/package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,11 @@ ghc-options:
- -Widentities
- -Wincomplete-record-updates
- -Wincomplete-uni-patterns
- -Wmissing-export-lists
- -Wmissing-home-modules
- -Wpartial-fields
- -Wredundant-constraints
- -Wno-missing-export-lists
- -Wno-type-defaults
Intey marked this conversation as resolved.
Show resolved Hide resolved

library:
source-dirs: src
Expand All @@ -59,3 +60,7 @@ tests:
- -with-rtsopts=-N
dependencies:
- user-code
- hspec
- hspec-api
- hspec-core
- QuickCheck
41 changes: 41 additions & 0 deletions haskell/task/MachineFormatter.hs
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Нужен, чтобы изменить стиль форматирования тестов.
Было

task 1
  user should define the @ToJSON@ for Int [✔️]
  user should define the @ToJSON@ for Point 5 3 [✘]
  user should define the @ToJSON@ for Point 7 3 [✘]
triangleArea
  should return correct area [✔️]
    +++ OK, passed 100 tests.

Failures:

  /home/intey/projects/senjun/hs-builder/example/Task1Spec.hs:21:45: 
  1) task 1 user should define the @ToJSON@ for Point 5 3
       expected: "{\"x\": 5, \"y\": 3}"
        but got: "{\"x\":5,\"y\":3}"

  To rerun use: --match "/task 1/user should define the @ToJSON@ for Point 5 3/" --seed 1857915877

  /home/intey/projects/senjun/hs-builder/example/Task1Spec.hs:23:45: 
  2) task 1 user should define the @ToJSON@ for Point 7 3
       expected: "{\"x\":7, \"y\":3}"
        but got: "{\"x\":7,\"y\":3}"

  To rerun use: --match "/task 1/user should define the @ToJSON@ for Point 7 3/" --seed 1857915877

Randomized with seed 1857915877

Finished in 0.0006 seconds
4 examples, 2 failures

Стало

task 1.user should define the @ToJSON@ for Point 5 3
? "{\"x\": 5, \"y\": 3}"
- "{\"x\":5,\"y\":3}"

task 1.user should define the @ToJSON@ for Point 7 3
? "{\"x\":7, \"y\":3}"
- "{\"x\":7,\"y\":3}"

Чтобы это работало, надо в файле тестов регистрировать этот форматтер и запускать stack test --test-arguments="-f machine".

Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{-
Simple wrapper, that make output more parsable
-}
module MachineFormatter (
formatter
) where

import Test.Hspec.Api.Format.V2

import Data.List (intercalate)

import Control.Monad (unless)

formatOnlyFalsy :: Format
formatOnlyFalsy event = case event of
ItemDone path item -> unless (isSuccess item) $ putStrLn (formatItem path item)
_ -> return ()

formatter :: (String, FormatConfig -> IO Format)
formatter = ("machine", \ _config -> return formatOnlyFalsy)

formatItem :: Path -> Item -> String
formatItem path item = joinPath path <> formatResult item

formatResult :: Item -> String
formatResult item = case itemResult item of
Success {} -> ""
Pending {} -> "-"
Failure _ reason -> case reason of
ExpectedButGot _ b c -> "\n? " ++ b ++ "\n- " ++ c ++ "\n"
Intey marked this conversation as resolved.
Show resolved Hide resolved
Reason r -> "\n" ++ r
ColorizedReason r -> "\n" ++ r
r -> "\n" ++ show r

joinPath :: Path -> String
joinPath (groups, requirement) = intercalate "." $ groups ++ [requirement]

isSuccess :: Item -> Bool
isSuccess item = case itemResult item of
Success -> True
_ -> False
75 changes: 50 additions & 25 deletions haskell/task/run-task.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

# parse flags - single letters prefixed with hyphen before each argument
# example: sh run.sh -c never -j 4

color="never"
file=""
task_type="code"

while getopts c:f:v: flag
do
case "${flag}" in
Expand All @@ -11,41 +16,61 @@ do
esac
done

stack_additional_opts=""
stack_additional_opts="--verbosity warn"
stack_test_additional_opts="-f machine"

if [ ! -z "$color" ];
if [ -n "${color}" ];
then
stack_additional_opts="${stack_additional_opts} --color ${color}"
# stack test colored by default
stack_test_additional_opts="${stack_test_additional_opts} --no-color"
fi

f="$(basename -- $file)"

f_capture="/tmp/capture.txt"
rm $f_capture > /dev/null 2>&1

# if grep not find `module Main` in the user code - than it is the new style of tests
new_task_type=0
grep "module Main" -q "${HOME}/task/$f" || new_task_type=1

# TODO it's suspicious that only "code" type of task
# and if it is not "code" user code is ok by default

# if exists file with user code
# run user code
if [ $task_type = "code" ]; then
cp /home/code_runner/task/$f /home/code_runner/user-code/app/Main.hs

# go to /home/code_runner/user_code for stack compiling and running
cd /home/code_runner/user-code

if ! ( timeout 10s stack build $stack_additional_opts --verbosity warn --ghc-options '-fno-warn-missing-export-lists -Wno-type-defaults' && stack exec user-code-exe | tee $f_capture ); then
echo user_solution_error_f936a25e
exit
project_dir="${HOME}/user-code"
build_command="stack build ${stack_additional_opts}"
# old test approach for tasks is all the Main module and it's output is
# parsed by python
if [ ${new_task_type} -eq 0 ]; then
cp "${HOME}/task/$f" "${project_dir}/app/Main.hs"
cd ${project_dir}
# we need to build explicitly to pass additional options, like color
if ! ( timeout 10s stack run ${stack_additional_opts} | tee $f_capture ); then
echo user_solution_error_f936a25e
exit
fi
echo user_code_ok_f936a25e
# run analyze of the output
timeout 5s python3 "${HOME}/task/${f}_tests" $f_capture
if [ "$?" -ne 0 ]; then
echo tests_cases_error_f936a25e
exit
fi
echo user_solution_ok_f936a25e
# for new approach we use usercode as library which is mouted to the project
else
cp "${HOME}/task/$f" "${project_dir}/src/UserCode.hs"
cp "${HOME}/task/${f}_tests" "${project_dir}/test/Spec.hs"
# go to /home/code_runner/user_code for stack compiling and running
cd ${project_dir}
if ! (timeout 10s ${build_command}); then
echo user_solution_error_f936a25e
exit
fi
echo user_code_ok_f936a25e
if ! (timeout 10s stack test ${stack_additional_opts} --progress-bar=none --test-arguments="${stack_test_additional_opts}"); then
echo tests_cases_error_f936a25e
exit
fi
echo user_solution_ok_f936a25e
fi
fi
echo user_code_ok_f936a25e

# run tests
timeout 10s python3 "/home/code_runner/task/${f}_tests" $f_capture
if [ "$?" -ne 0 ]; then
echo tests_cases_error_f936a25e
exit
fi

echo user_solution_ok_f936a25e