Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in rule plot_contact_map #5

Open
OdysseyXD opened this issue Apr 26, 2021 · 1 comment
Open

Error in rule plot_contact_map #5

OdysseyXD opened this issue Apr 26, 2021 · 1 comment

Comments

@OdysseyXD
Copy link

Met the error below while running the pipeline

Error in rule plot_contact_map:
    jobid: 0
    output: /home/yyxu/data/hic/XK_1030/dlohic_test/pipeout/0a-qc/PAM2_R2.test.chroms_cmap, /home/yyxu/data/hic/XK_1030/dlohic_test/pipeout/0a-qc/PAM2_R2.test.global_cmap.svg

RuleException:
KeyError in line 434 of /home/yyxu/data/Snakefile:
'1'
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/snakemake-6.0.5-py3.6.egg/snakemake/executors/__init__.py", line 2347, in run_wrapper
  File "/home/yyxu/data/Snakefile", line 434, in __rule_plot_contact_map
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/dlo_hic-0.3.9-py3.6-linux-x86_64.egg/dlo_hic/tools/helper/plot_contact_map.py", line 84, in _main
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/dlo_hic-0.3.9-py3.6-linux-x86_64.egg/dlo_hic/utils/plot/contactmap.py", line 202, in plot_chrom
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/snakemake-6.0.5-py3.6.egg/snakemake/executors/__init__.py", line 568, in _callback
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/concurrent/futures/thread.py", line 56, in run
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/snakemake-6.0.5-py3.6.egg/snakemake/executors/__init__.py", line 554, in cached_or_run
  File "/home/yyxu/.conda/envs/dlohic/lib/python3.6/site-packages/snakemake-6.0.5-py3.6.egg/snakemake/executors/__init__.py", line 2359, in run_wrapper
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/yyxu/data/.snakemake/log/2021-04-26T231824.939013.snakemake.log

My conda env:

# packages in environment at /opt/anaconda/envs/dlohic:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
amply                     0.1.4                    pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
asciitree                 0.3.3                      py_2    conda-forge
atk-1.0                   2.36.0               h3371d22_4    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
biopython                 1.76             py36h516909a_0    conda-forge
bokeh                     2.3.1            py36h5fab9bb_0    conda-forge
bwa                       0.7.17               h5bf99c6_8    bioconda
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.17.1               h7f98852_1    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.16.0            h6cf1ce9_1008    conda-forge
certifi                   2020.12.5        py36h5fab9bb_1    conda-forge
chardet                   4.0.0                    pypi_0    pypi
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
contextvars               2.4                        py_0    conda-forge
cooler                    0.8.11             pyh3252c3a_0    bioconda
coreutils                 8.25                          1    bioconda
cycler                    0.10.0                     py_2    conda-forge
cython                    0.29.23          py36hc4f0c31_0    conda-forge
cytoolz                   0.10.1                   pypi_0    pypi
dask                      2021.3.0           pyhd8ed1ab_0    conda-forge
dask-core                 2021.3.0           pyhd8ed1ab_0    conda-forge
datrie                    0.8.2                    pypi_0    pypi
dbus                      1.13.6               h48d8840_2    conda-forge
dill                      0.3.3              pyhd8ed1ab_0    conda-forge
distributed               2021.3.0         py36h5fab9bb_0    conda-forge
dlo-hic                   0.3.9                    pypi_0    pypi
docutils                  0.17.1b1.dev0            pypi_0    pypi
expat                     2.3.0                h9c3ff4c_0    conda-forge
filelock                  3.0.12                   pypi_0    pypi
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      2.001                hab24e00_0    conda-forge
font-ttf-source-code-pro  2.030                hab24e00_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.13.1            hba837de_1005    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
fsspec                    0.9.0              pyhd8ed1ab_1    conda-forge
gdk-pixbuf                2.42.6               h04a7f16_0    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
giflib                    5.2.1                h36c2ea0_2    conda-forge
gitdb                     4.0.7                    pypi_0    pypi
gitpython                 3.1.14                   pypi_0    pypi
glib                      2.68.1               h9c3ff4c_0    conda-forge
glib-tools                2.68.1               h9c3ff4c_0    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  2.47.0               he056042_1    conda-forge
gst-plugins-base          1.18.4               h29181c9_0    conda-forge
gstreamer                 1.18.4               h76c114f_0    conda-forge
gtk2                      2.24.33              hab0c2f8_0    conda-forge
gts                       0.7.6                h64030ff_2    conda-forge
h5py                      3.1.0           nompi_py36hc1bc4f5_100    conda-forge
harfbuzz                  2.8.0                h83ec7ef_1    conda-forge
hdf5                      1.10.6          nompi_h6a2412b_1114    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
htslib                    1.12                 h9093b5e_1    bioconda
icu                       68.1                 h58526e2_0    conda-forge
idna                      2.10                     pypi_0    pypi
immutables                0.15             py36h8f6f2f9_0    conda-forge
importlib-metadata        3.10.1           py36h5fab9bb_0    conda-forge
ipython-genutils          0.2.0                    pypi_0    pypi
java-jdk                  8.0.112                       1    bioconda
jinja2                    2.11.3             pyh44b312d_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
jsonschema                3.2.0              pyhd8ed1ab_3    conda-forge
jupyter-core              4.7.1                    pypi_0    pypi
kiwisolver                1.3.1            py36h605e78d_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libclang                  11.1.0          default_ha53f305_0    conda-forge
libcurl                   7.76.1               hc4aaa36_0    conda-forge
libdeflate                1.7                  h7f98852_5    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc                    7.2.0                h69d50b8_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_19    conda-forge
libgd                     2.3.2                h78a0170_0    conda-forge
libgfortran-ng            9.3.0               hff62375_19    conda-forge
libgfortran5              9.3.0               hff62375_19    conda-forge
libglib                   2.68.1               h3e27bee_0    conda-forge
libgomp                   9.3.0               h2828fa1_19    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libllvm11                 11.1.0               hf817b99_2    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.1                 hfd2b0eb_2    conda-forge
librsvg                   2.50.3               hfa39831_1    conda-forge
libssh2                   1.9.0                ha56f1ee_6    conda-forge
libstdcxx-ng              9.3.0               h6de172a_19    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libtool                   2.4.6             h58526e2_1007    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp                   1.2.0                h3452ae3_0    conda-forge
libwebp-base              1.2.0                h7f98852_2    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.10               h72842e0_4    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
mafft                     7.475                h779adbc_1    bioconda
mako                      1.1.4                    pypi_0    pypi
markupsafe                1.1.1            py36h8f6f2f9_3    conda-forge
matplotlib                3.3.4            py36h5fab9bb_0    conda-forge
matplotlib-base           3.3.4            py36hd391965_0    conda-forge
msgpack-python            1.0.2            py36h605e78d_1    conda-forge
multiprocess              0.70.11.1        py36h8f6f2f9_1    conda-forge
mysql-common              8.0.23               ha770c72_1    conda-forge
mysql-libs                8.0.23               h935591d_1    conda-forge
nbformat                  5.1.3                    pypi_0    pypi
ncurses                   6.2                  h58526e2_4    conda-forge
nspr                      4.30                 h9c3ff4c_0    conda-forge
nss                       3.63                 hb5efdd6_0    conda-forge
numpy                     1.19.5           py36h2aa4a07_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openjdk                   8.0.112          zulu8.19.0.1_3    conda-forge
openjpeg                  2.4.0                hf7af979_0    conda-forge
openssl                   1.1.1k               h7f98852_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pairix                    0.3.7            py36h30a8e3e_3    bioconda
pandas                    1.1.5            py36h284efc9_0    conda-forge
pango                     1.42.4               h69149e4_5    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
perl                      5.32.0               h36c2ea0_0    conda-forge
pillow                    8.1.2            py36ha6010c0_1    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
psutil                    5.8.0            py36h8f6f2f9_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pulp                      2.4                      pypi_0    pypi
pyfaidx                   0.5.9.5            pyh3252c3a_0    bioconda
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyqt                      5.12.3           py36h5fab9bb_7    conda-forge
pyqt-impl                 5.12.3           py36h7ec31b9_7    conda-forge
pyqt5-sip                 4.19.18          py36hc4f0c31_7    conda-forge
pyqtchart                 5.12             py36h7ec31b9_7    conda-forge
pyqtwebengine             5.12.1           py36h7ec31b9_7    conda-forge
pyrsistent                0.17.3           py36h8f6f2f9_2    conda-forge
pysam                     0.16.0.1         py36h61e5637_3    bioconda
python                    3.6.13          hffdb5ce_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.6                     1_cp36m    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.4.1            py36h8f6f2f9_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
ratelimiter               1.2.0.post0              pypi_0    pypi
readline                  8.0                  he28a2e2_2    conda-forge
requests                  2.25.1                   pypi_0    pypi
samtools                  1.12                 h9aed4be_1    bioconda
scipy                     1.5.3            py36h9e8f40b_0    conda-forge
seaborn                   0.11.1                   pypi_0    pypi
setuptools                49.6.0           py36h5fab9bb_3    conda-forge
simplejson                3.8.1                    py36_0    bioconda
six                       1.15.0             pyh9f0ad1d_0    conda-forge
smart-open                5.0.0                    pypi_0    pypi
smmap                     4.0.0                    pypi_0    pypi
snakemake                 6.1.1                    pypi_0    pypi
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
sqlite                    3.35.4               h74cdb3f_0    conda-forge
tabix                     0.2.6                ha92aebf_0    bioconda
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.10               h21135ba_1    conda-forge
toolz                     0.11.1                     py_0    conda-forge
toposort                  1.6                      pypi_0    pypi
tornado                   6.1              py36h8f6f2f9_1    conda-forge
traitlets                 5.0.5                    pypi_0    pypi
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.4                   pypi_0    pypi
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
wrapt                     1.12.1                   pypi_0    pypi
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.0                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge

Info of my data:

  • genome file
    • merged with a virus genome as one chromosome of host genome in .fa file
  • chrom_sizes file
    • created by cut -f1,2 genome.fa.fai | head -n21 >>chrom.sizes
    • no 'chr' in both .fa file or .sizes file
  • output contain all files except plot and results report

Since the conda env was reinstall recently, those packages required were updated.
And this error did not show up before the reinstall action .
@OdysseyXD
Copy link
Author

OdysseyXD commented Apr 28, 2021 •
edited
Loading

Following the error, I found that only one file of my dataset have been converted into .hic file, but somehow sometimes all of my dataset can all be converted into .hic, .cool, .mcool file.
Then I try to rerun the pipeline and few more output file come out.

For example:

Sample: A1, A2; P1, P2

    1. First_run with output_format=['.hic']
Output: P2.hic
    1. RE_run with output_format=['.hic']
Output: A1.hic, P2.hic
    1. First_run with output_format=['.hic', '.cool']
Output: 
    P2.hic, P2.cool, P2.mcool, P2.cool.tmp.bed
    1. RE_run with output_format=['.hic', '.cool']
Output: 
   A1.hic,
   P2.hic, P2.cool, P2.mcool, P2.cool.tmp.bed
    1. Third_run with output_format=['.hic', '.cool']
Output: 
   A1.hic, A1.cool, A1.mcool, A1.cool.tmp.bed
   P2.hic, P2.cool, P2.mcool, P2.cool.tmp.bed
   A2.hic, A2.cool, A2.mcool, A2.cool.tmp.bed

跑流程第一次时只有一个文件有输出就报错，再跑一次再出来一些再报错，再跑一次再出来一些再报错。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@OdysseyXD