Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a docker-compose run --rm ci-admin command #44

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add a docker-compose run --rm ci-admin command #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM ubuntu:latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're going to a lot of trouble to install Python here. Perhaps you should use python:3.11 instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that would make it a lot simpler!
I ran into an issue using the python:3.11 image. It's related to the in-tree event_loop fixture.
The log looks like this:

==================================== ERRORS ====================================
____________ ERROR at setup of check_default_branches_for_git_repos ____________

fixturedef = <FixtureDef argname='event_loop' scope='session' baseid=''>

    @pytest.hookimpl(hookwrapper=True)
    def pytest_fixture_setup(
        fixturedef: FixtureDef,
    ) -> Generator[None, Any, None]:
        """Adjust the event loop policy when an event loop is produced."""
        if fixturedef.argname == "event_loop":
            # The use of a fixture finalizer is preferred over the
            # pytest_fixture_post_finalizer hook. The fixture finalizer is invoked once
            # for each fixture, whereas the hook may be invoked multiple times for
            # any specific fixture.
            # see https://github.com/pytest-dev/pytest/issues/5848
            _add_finalizers(
                fixturedef,
                _close_event_loop,
                _restore_event_loop_policy(asyncio.get_event_loop_policy()),
                _provide_clean_event_loop,
            )
            outcome = yield
            loop: asyncio.AbstractEventLoop = outcome.get_result()
            # Weird behavior was observed when checking for an attribute of FixtureDef.func
            # Instead, we now check for a special attribute of the returned event loop
            fixture_filename = inspect.getsourcefile(fixturedef.func)
            if not getattr(loop, "__original_fixture_loop", False):
>               _, fixture_line_number = inspect.getsourcelines(fixturedef.func)

/usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:756: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/lib/python3.11/inspect.py:1240: in getsourcelines
    lines, lnum = findsource(object)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

object = <function event_loop at 0xffff9511e980>

    def findsource(object):
        """Return the entire source file and starting line number for an object.
    
        The argument may be a module, class, method, function, traceback, frame,
        or code object.  The source code is returned as a list of all the lines
        in the file and the line number indexes a line in that list.  An OSError
        is raised if the source code cannot be retrieved."""
    
        file = getsourcefile(object)
        if file:
            # Invalidate cache if needed.
            linecache.checkcache(file)
        else:
            file = getfile(object)
            # Allow filenames in form of "<something>" to pass through.
            # doctest monkeypatches linecache module to enable
            # inspection, so let linecache.getlines to be called.
            if not (file.startswith('<') and file.endswith('>')):
                raise OSError('source code not available')
    
        module = getmodule(object, file)
        if module:
            lines = linecache.getlines(file, module.__dict__)
        else:
            lines = linecache.getlines(file)
        if not lines:
>           raise OSError('could not get source code')
E           OSError: could not get source code

/usr/local/lib/python3.11/inspect.py:1077: OSError
=============================== warnings summary ===============================
check_default_branches.py::check_default_branches_for_git_repos
  /usr/local/lib/python3.11/site-packages/_pytest/fixtures.py:1069: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
  Plugin: asyncio, Hook: pytest_fixture_setup
  OSError: could not get source code
  For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning
    result = ihook.pytest_fixture_setup(fixturedef=self, request=request)

check_worker_pools.py::check_gcp_ssds
  /usr/local/lib/python3.11/site-packages/pytest_asyncio/plugin.py:814: DeprecationWarning: pytest-asyncio detected an unclosed event loop when tearing down the event_loop
  fixture: <_UnixSelectorEventLoop running=False closed=False debug=False>
  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. The scopes of your custom "event_loop" fixtures do not overlap
      3. Your code does not modify the event loop in async fixtures or tests
  
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
ERROR src/ciadmin/check/check_default_branches.py::check_default_branches_for_git_repos

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* squints * maybe its this

  pytest-asyncio will close the event loop for you, but future versions of the
  library will no longer do so. In order to ensure compatibility with future
  versions, please make sure that:
      1. Any custom "event_loop" fixture properly closes the loop after yielding it
      2. The scopes of your custom "event_loop" fixtures do not overlap
      3. Your code does not modify the event loop in async fixtures or tests


ENV DEBIAN_FRONTEND=noninteractive
ENV TASKCLUSTER_ROOT_URL=https://firefox-ci-tc.services.mozilla.com/

RUN apt update && \
apt install -y software-properties-common curl git

RUN add-apt-repository ppa:deadsnakes/ppa && \
apt update && \
apt install -y python3.11 python3.11-venv python3.11-dev

RUN curl -sS https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
python3.11 get-pip.py && \
rm get-pip.py

WORKDIR /home/fxci-config

RUN mkdir -p /home/fxci-config/requirements

COPY requirements/*.txt /home/fxci-config/requirements/

RUN python3.11 -m pip install -r /home/fxci-config/requirements/test.txt

COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh

ENTRYPOINT ["entrypoint.sh"]
8 changes: 8 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: '3.8'

services:
ci-admin:
build: .
volumes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make it possible to forward TASKCLUSTER_ACCESS_TOKEN and TASKCLUSTER_CLIENT_ID here. Without them, you can't run apply (if it's ever needed), or even diff, which requires limited scopes.

- .:/home/fxci-config
working_dir: /home/fxci-config
5 changes: 5 additions & 0 deletions entrypoint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

python3.11 -m pip install --root-user-action=ignore -e .

ci-admin "$@"