[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

generate-ci'd emscripten pytest fails before reaching module code #2071

Open
1 of 2 tasks
chrysn opened this issue May 5, 2024 · 5 comments
Open
1 of 2 tasks

generate-ci'd emscripten pytest fails before reaching module code #2071

chrysn opened this issue May 5, 2024 · 5 comments
Labels
bug Something isn't working target/emscripten Emscripten build target

Comments

@chrysn
Copy link
chrysn commented May 5, 2024

Bug Description

In cbor-diag-py, which is an almost trivial1 project following the maturing scaffolding of maturin new and generate-ci, the emscripten tests fail, seemingly before they even touch code.

Excerpt of the latest CI run

 
[pytest]
Run set -e
xbuild environment already exists, skipping download                            
Installing xbuild environment                                                   
Creating Pyodide virtualenv at .venv                                            
... Configuring virtualenv                                                      
... Installing standard library                                                 
Successfully created Pyodide virtual environment!                               
Looking in indexes: https://pypi.org/simple, file:///home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index
Looking in links: dist
Collecting cbor-diag
  Downloading https://cdn.jsdelivr.net/pyodide/v0.25.1/full/cbor_diag-1.0.1-cp311-cp311-emscripten_3_1_46_wasm32.whl (362 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 362.9/362.9 kB 14.3 MB/s eta 0:00:00
Installing collected packages: cbor-diag
Successfully installed cbor-diag-1.0.1
WARNING: Location 'file:/home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index/pip/' is ignored: it is neither a file nor a directory.
Looking in indexes: https://pypi.org/simple, file:///home/runner/work/cbor-diag-py/cbor-diag-py/.pyodide-xbuildenv/xbuildenv/pyodide-root/pypa_index
Collecting pytest
  Downloading pytest-8.2.0-py3-none-any.whl.metadata (7.5 kB)
Collecting iniconfig (from pytest)
  Downloading https://cdn.jsdelivr.net/pyodide/v0.25.1/full/iniconfig-2.0.0-py3-none-any.whl (14 kB)
Requirement already satisfied: packaging in ./.venv/lib/python3.11/site-packages (from pytest) (23.1)
Collecting pluggy<2.0,>=1.5 (from pytest)
  Downloading pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB)
Downloading pytest-8.2.0-py3-none-any.whl (339 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 339.2/339.2 kB 8.8 MB/s eta 0:00:00
Downloading pluggy-1.5.0-py3-none-any.whl (20 kB)
Installing collected packages: pluggy, iniconfig, pytest
Successfully installed iniconfig-2.0.0 pluggy-1.5.0 pytest-8.2.0
============================= test session starts ==============================
platform emscripten -- Python 3.11.3, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/runner/work/cbor-diag-py/cbor-diag-py
configfile: pytest.ini
collected 1 item / 10 errors

==================================== ERRORS ====================================
_ ERROR collecting emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py _
ImportError while importing test module '/home/runner/work/cbor-diag-py/cbor-diag-py/emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/lib/python311.zip/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
emsdk-cache/emsdk-main/upstream/emscripten/test/test_benchmark.py:20: in <module>
    import clang_native
emsdk-cache/emsdk-main/upstream/emscripten/test/clang_native.py:10: in <module>
    from tools.shared import PIPE, run_process, CLANG_CC, CLANG_CXX
E   ModuleNotFoundError: No module named 'tools'
_ ERROR collecting emsdk-cache/emsdk-main/upstream/emscripten/test/test_browser.py _
ImportError while importing test module '/home/runner/work/cbor-diag-py/cbor-diag-py/emsdk-cache/emsdk-main/upstream/emscripten/test/test_browser.py'.
Hint: make sure your test modules/packages have valid Python names.

Given that nothing in the errors appear to be related to the cbor-diag module, it appears that this is an issue with maturin or the actions added in generate-ci. I'm a bit lost at why the emscripten tests are even called, do you have any pointers to track this down further?

Your maturin version (maturin --version)

1.5.1

Your Python version (python -V)

3.11.8

Your pip version (pip -V)

24.0

What bindings you're using

pyo3

Does cargo build work?

  • Yes, it works

If on windows, have you checked that you aren't accidentally using unix path (those with the forward slash /)?

  • Yes

Steps to Reproduce

The issue is with what happens on GitHub actions, I don't think this can be reproduced locally.

Footnotes

  1. The package uses altered paths and a manual init to avoid the public modulename.modulename API guarantee tracked in Defining a module foo results in foo.foo being available #1399 ↩
@chrysn chrysn added the bug Something isn't working label May 5, 2024
chrysn added a commit to chrysn/cbor-diag-py that referenced this issue May 5, 2024
@chrysn
CI: Restore emscripten test removal
6471f81 
Workaround-For: PyO3/maturin#2071
chrysn added a commit to chrysn/cbor-diag-py that referenced this issue May 5, 2024
@chrysn
CI: Restore emscripten test removal
920610f 
Workaround-For: PyO3/maturin#2071
@messense
Copy link
Member
messense commented May 6, 2024

I don't know either, @hoodmane any idea?

@messense messense added the target/emscripten Emscripten build target label May 6, 2024
@hoodmane
Copy link
Contributor
hoodmane commented May 6, 2024

cc @ryanking13.

I'm not sure why it's trying to run those tests but I think they were meant to run in the emscripten source tree and weren't meant to run on install. I wonder why they are running here.

@ryanking13
Copy link

It looks like emscripten is installed under emsdk-cache directory (the directory specified in actions-cache-folder option in the setup-emsdk action). And emscripten contains a bunch of Python test files in their repository, so pytest added them to the test suite.

So I guess you need to

  1. Install emscripten somewhere out of current working directory, or
  2. Specify the test file paths clearly when running pytest

@chrysn
Copy link
Author
chrysn commented May 7, 2024

Just to not drop the ball, is that something I can act on in my repository (doesn't seem like it, it doesn't touch emsdk installation), something where we'd need to change the generate output, or something where we'd need to pull in maintainers of the action repositories supplying the CI steps?

@ryanking13
Copy link
ryanking13 commented May 7, 2024 •
edited
Loading

is that something I can act on in my repository

Well, I think the easiest way, and what I would like to recommend, is specifying the files/directory to test by pytest clearly. By adding testpaths in pytest.ini, so that even if some files are unintentionally created in the local directory during CI, pytest will only put the specified directory in the test scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working target/emscripten Emscripten build target
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@chrysn @messense @hoodmane @ryanking13