Coarse cool keeping bins() columns

[ddepierre]

Hi,
I have a cool file as such:

cooler.Cooler(cool_file).bins()[:1000]
chrom start end cov_cis_raw cov_tot_raw weight
0 chr1 0 10000 0 0 NaN
1 chr1 10000 20000 0 0 NaN
2 chr1 20000 30000 0 0 NaN
3 chr1 30000 40000 0 0 NaN
4 chr1 40000 50000 0 0 NaN
.. ... ... ... ... ... ...
995 chr1 9950000 9960000 188 192 0.175841
996 chr1 9960000 9970000 171 173 0.224719
997 chr1 9970000 9980000 199 202 0.206420
998 chr1 9980000 9990000 197 197 0.148978
999 chr1 9990000 10000000 256 259 0.127959

I want to coarse it while keeping "cov_cis_raw" and "cov_tot_raw" columns, but when I coarse it:

cooler.coarsen_cooler(cool_file, output_uri=coarsed_cool_file ,chunksize=1000000, factor=100)

cooler.Cooler(coarsed_cool_file).bins()[:10]
chrom start end weight
0 chr1 0 1000000 NaN
1 chr1 1000000 2000000 NaN
2 chr1 2000000 3000000 NaN
3 chr1 3000000 4000000 0.024616
4 chr1 4000000 5000000 0.028092
5 chr1 5000000 6000000 0.033317
6 chr1 6000000 7000000 0.029853
7 chr1 7000000 8000000 0.027963
8 chr1 8000000 9000000 0.025852
9 chr1 9000000 10000000 0.028878

I was wondering if there is a way to coarse keeping those columns.
"columns" param from coarse is for .pixels() columns names
maybe something in kwarg but I don't find what.

Best,
David

[sergpolly]

cov_cis_raw and cov_tot_raw - sounds familiar - aren't these output of the cooltools.coverage ?
If this is the case I'd recommend re-running cooltools.coverage on a coarsened cooler-file ...

I don't think cooler coarsen can do this for you. You can probably achieve something like that using plain pandas by assigning larger bins to smaller bins, and then aggregating cov_cis_raw values on larger bins - and then storing the result manually inside the cooler using plain h5py-way. But this is much more complicated and error-prone than simply recalculating coverage at lower resolution after coarsening .

[nvictus]

I don't think cooler coarsen can do this for you.

Correct. We don't currently provide a way to coarsen bin columns, since there are many ways one could do it and in many cases it's not a good idea (e.g. like balancing weights).

For coverage it is straightforward to do manually and you can append the new column using the lower level h5py interface as @sergpolly suggested:

with clr.open("r+") as f:  # f is an h5py.Group
    f["bins/cov_cis_raw"] = data

or

with clr.open("r+") as f:
    f["bins"].create_dataset(name="cov_cis_raw", data=data, compression="gzip", compression_opts=6)

to also apply compression to the new column.