Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 03 23:02
    alexpreynolds commented #296
  • Dec 03 03:07
    nvictus commented #296
  • Dec 03 02:35
    nvictus commented #296
  • Dec 02 21:03
    alexpreynolds opened #296
  • Nov 14 21:46
    KunFang93 commented #295
  • Nov 14 05:28
    KunFang93 edited #295
  • Nov 14 05:25
    KunFang93 opened #295
  • Oct 23 06:41
    TakatoG opened #294
  • Oct 17 15:16
    chenggang108 opened #293
  • Oct 12 20:17
    ericmalekos opened #292
  • Oct 12 08:45
    Phlya closed #291
  • Oct 12 08:45
    Phlya commented #291
  • Oct 11 17:58
    BeetleMilk commented #291
  • Oct 11 16:53
    Phlya commented #291
  • Oct 11 16:49
    BeetleMilk edited #291
  • Oct 11 16:46
    BeetleMilk opened #291
  • Sep 27 20:22
    Jelena4 closed #290
  • Sep 27 20:22
    Jelena4 commented #290
  • Sep 27 20:02
    agalitsyna commented #290
  • Sep 27 19:52
    Jelena4 commented #290
nandankita
@nandankita
bin1_id bin2_id count balanced
1 2059 1
1 8659 1
1 9335 1
5 9247 1
6 3015 1
6 4647 1
7 3307 1
8 1278 1
8 2774 1
Traceback (most recent call last):
File "/usr/bin/cooler", line 9, in <module>
load_entry_point('cooler==0.5.2', 'console_scripts', 'cooler')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 716, in call
return self.main(args, kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback,
ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(
args, **kwargs)
File "/home/ankita/4DN/cooler/cooler/cooler/cli/dump.py", line 162, in dump
f, sep='\t', index=False, header=False, float_format='%g')
File "/usr/lib64/python2.7/site-packages/pandas/core/frame.py", line 1344, in to_csv
formatter.save()
File "/usr/lib64/python2.7/site-packages/pandas/formats/format.py", line 1551, in save
self._save()
File "/usr/lib64/python2.7/site-packages/pandas/formats/format.py", line 1652, in _save
self._save_chunk(start_i, end_i)
File "/usr/lib64/python2.7/site-packages/pandas/formats/format.py", line 1678, in _save_chunk
lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
File "pandas/lib.pyx", line 1075, in pandas.lib.write_csv_rows (pandas/lib.c:19767)
IOError: [Errno 32] Broken pipe
Please let me know what am I doing wrong.
Nezar Abdennur
@nvictus
Hey Ankita! You're doing nothing wrong, actually. You can essentially ignore that message: it's printed to stderr, so if you redirect the output it won't carry over. It's caused by head closing the pipe, and cooler should be suppressing the error, but I think it's not getting caught in Python 2. Let me try patching it.
I pushed a patch. Try pulling changes from master and let me know if the traceback goes away.
nandankita
@nandankita
Ohh Thanks! I will just try that now.
nandankita
@nandankita
Yeah the patch fixed the error! To do this step:
mat = c.matrix(balance=True).fetch('chr5:10,000,000-15,000,000')
I need to have a .cool file which has bins/weight column,
So, first I thought if I just do balancing, it should be good, but that doesn't seem to be the case, should I calculate weight first and then write to the file?
Nezar Abdennur
@nvictus
Yes, if you use cooler balance it will write the weights to the file.
nandankita
@nandankita
Do you mean this is correct?
cooler dump --header --balanced HB_Cool.cool | head
bin1_id bin2_id count balanced
1 2059 1
1 8659 1
1 9335 1
5 9247 1
6 3015 1
6 4647 1
7 3307 1
8 1278 1
8 2774 1
of course I did this first
cooler balance HB_Cool.cool
variance is 115532.067933
variance is 1880.1202993
variance is 142.980174133
variance is 16.5192880813
variance is 1.92663361978
variance is 0.25196914841
variance is 0.0346924669294
variance is 0.00558787607212
variance is 0.00103577668573
variance is 0.000235077586937
variance is 6.00085555451e-05
variance is 1.68418189549e-05
variance is 4.89101936518e-06
Nezar Abdennur
@nvictus
yes... if a bin was filtered during balancing, it will have a weight of NaN. So the balanced value will be NaN, which prints as empty
you can check with cooler dump --annotate weight
nandankita
@nandankita

Nezar, I tried do cooler balance and then cooler dump, then I thought weights should be there in my cool file, but I still get this error through python API
matb = c.matrix(balance=True).fetch('chr5:10,000,000-15,000,000')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "cooler/core.py", line 360, in fetch
File "cooler/api.py", line 311, in _slice
File "cooler/api.py", line 560, in matrix
ValueError: No column 'bins/weight' found. Use cooler.ice to calculate balancing weights.

Is there any step I am missing?

Also another thing, I tried is cooler show, which again gave me error :(

cooler show --out test.png --dpi 200 HB_Cool.cool chr3:0-80,000,000
Traceback (most recent call last):
File "/usr/bin/cooler", line 9, in <module>
load_entry_point('cooler==0.5.2', 'console_scripts', 'cooler')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 716, in call
return self.main(args, kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback,
ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(
args, kwargs)
File "/home/ankita/4DN/cooler/cooler/cooler/cli/show.py", line 135, in show
'log10' : 'log 10 ( relative contact frequency '}[scale]))
File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 1976, in colorbar
ret = gcf().colorbar(mappable, cax = cax, ax=ax,
kw)
File "/usr/lib64/python2.7/site-packages/matplotlib/figure.py", line 1387, in colorbar
cb = cbar.colorbar_factory(cax, mappable, kw)
File "/usr/lib64/python2.7/site-packages/matplotlib/colorbar.py", line 1173, in colorbar_factory
cb = Colorbar(cax, mappable,
kwargs)
File "/usr/lib64/python2.7/site-packages/matplotlib/colorbar.py", line 854, in init
ColorbarBase.init(self, ax, **kw)
TypeError: init() got an unexpected keyword argument 'label'

Am I missing steps?
So sorry to bother you like this. I am just running into the errors!
Nezar Abdennur
@nvictus
No worries. These reports are helpful!
The first one is surprising
If you restart the python interpreter, is the weights column still missing?
You can also check using h5py.
f = h5py.File('myfile.cool', 'r')
'weight' in f['bins']
Anton will look into cooler show. I suspect it's related to the matplotlib version. Could you print the results of pip list?
nandankita
@nandankita
I am home and can't able to connect to my system from here. I'll let you know first thing tomorrow morning!
Anton Goloborodko
@golobor
I think I fix this issue in the master branch. Nezar, how long ago did you merge develop back into master? The master branch somehow didn't get the proper imports from matplotlib (i.e. using "Agg" when out is None)
Nezar Abdennur
@nvictus
yeah, that change is in develop... i'll cherry-pick it over to master
nandankita
@nandankita
[ankita@localhost cooler]$ pip list
-ipap11helper (0.1)
backports.ssl-match-hostname (3.4.0.2)
Beaker (1.5.4)
biopython (1.68)
blivet (0.61.15.37)
Brlapi (0.6.0)
cffi (0.8.6)
chardet (2.2.1)
click (6.6)
configobj (4.7.2)
configshell-fb (1.1.18)
cooler (0.5.2, /home/ankita/4DN/cooler/cooler)
coverage (3.6b3)
cryptography (0.8.2)
cupshelpers (1.0)
decorator (3.4.0)
di (0.3)
dnspython (1.12.0)
enum34 (1.0.4)
ethtool (0.8)
firstboot (19.5)
freeipa (2.0.0a0)
fros (1.0)
fuse-python (0.2.1)
h5py (2.6.0)
iniparse (0.4)
initial-setup (0.3.9.30)
ipaplatform (4.2.0)
ipapython (4.2.0)
IPy (0.75)
javapackages (1.0.0)
kerberos (1.1)
kitchen (1.1.1)
kmod (0.1)
langtable (0.0.31)
lxml (3.2.1)
Mako (0.8.1)
MarkupSafe (0.11)
matplotlib (1.2.0)
netaddr (0.7.5)
nose (1.3.0)
ntplib (0.3.2)
numpy (1.11.1)
pandas (0.18.1)
Paste (1.7.5.1)
perf (0.1)
pip (7.1.0)
ply (3.4)
policycoreutils-default-encoding (0.1)
pyasn1 (0.1.6)
pycparser (2.14)
pycups (1.9.63)
pycurl (7.19.0)
pyfaidx (0.4.6)
pygobject (3.14.0)
pygpgme (0.3)
pyinotify (0.9.4)
pykickstart (1.99.66.6)
pyliblzma (0.5.3)
pyOpenSSL (0.13.1)
pyparsing (1.5.6)
pyparted (3.9)
pysam (0.9.1.4)
pysmbc (1.0.13)
python-augeas (0.5.0)
python-dateutil (1.5)
python-default-encoding (0.1)
python-dmidecode (3.10.13)
python-ldap (2.4.15)
python-meh (0.25.2)
python-nss (0.16.0)
python-yubico (1.2.3)
pytz (2012d)
pyudev (0.15)
pyusb (1.0.0b1)
pyxattr (0.5.1)
qrcode (5.0.1)
rtslib-fb (2.1.57)
scipy (0.18.0)
seobject (0.1)
sepolicy (1.1)
setroubleshoot (1.1)
setuptools (0.9.8)
six (1.9.0)
slip (0.4.0)
slip.dbus (0.4.0)
SSSDConfig (1.13.0)
targetcli-fb (2.1.fb41)
Tempita (0.5.1)
urlgrabber (3.10)
urwid (1.1.1)
yum-langpacks (0.4.2)
yum-metadata-parser (1.1.4)
[ankita@localhost cooler]$
nandankita
@nandankita
After restarting python interpreter it worked for balance=True ! Also, now the 'weight' in f['bins'] is True as well.
nandankita
@nandankita
Can I try cooler show now?
nandankita
@nandankita
I am just wondering, if there is any way to create a cool file which is not binned?
Anton Goloborodko
@golobor
yes, you can try cooler show with the current version on master
btw, what do you mean by a cooler file that's not binned? Is it a file with individual read pairs? What kind of analyses do you have in mind?
nandankita
@nandankita

So, I am not sure if it is correct to ask but we do these steps
$ cooler makebins $CHROMSIZES_FILE 10000 > bins.10kb.bed
$ cooler cload bins.10kb.bed $CONTACTS_FILE out.cool

As I understood I am getting a cool file which is 10 kb binned. What if I don't want it to be binned and create a cool file ? Does that make sense?

nandankita
@nandankita
cooler show works now! Thanks a lot!
nandankita
@nandankita
Sorry but I tried cooler dump, it works mostly but not this one:
[ankita@localhost cooler]$ cooler dump -t pixels -r 10:10,000,000-20,000,000 -r2 10:30,000,000-80,000,000 --header --join HB_Cool.cool | head
Traceback (most recent call last):
File "/usr/bin/cooler", line 9, in <module>
load_entry_point('cooler==0.5.2', 'console_scripts', 'cooler')()
File "/usr/lib/python2.7/site-packages/click/core.py", line 716, in call
return self.main(args, kwargs)
File "/usr/lib/python2.7/site-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python2.7/site-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback,
ctx.params)
File "/usr/lib/python2.7/site-packages/click/core.py", line 534, in invoke
return callback(
args, kwargs)
File "/home/ankita/4DN/cooler/cooler/cooler/cli/dump.py", line 105, in dump
selector = c.matrix(as_pixels=True).fetch(range, range2)
File "/home/ankita/4DN/cooler/cooler/cooler/core.py", line 359, in fetch
i0, i1, j0, j1 = self._fetch(*args,
kwargs)
File "/home/ankita/4DN/cooler/cooler/cooler/api.py", line 317, in _fetch
region1 = parse_region(region, self._chromsizes)
File "/home/ankita/4DN/cooler/cooler/cooler/util.py", line 105, in parse_region
raise ValueError("Unknown sequence label: {}".format(chrom))
ValueError: Unknown sequence label: 10
Peter Kerpedjiev
@pkerpedjiev

AttributeError Traceback (most recent call last)

<ipython-input-1-8423f6653a05> in <module>()
10
11 where = np.flatnonzero
---> 12 chromsizes = cooler.read_chromsizes('http://s3.amazonaws.com/pkerp/data/mm9/chromInfo.txt') # defaults to reading chr#,X,Y,M
13 chromosomes = list(chromsizes.keys())
14 cumullengths = np.r[0, np.cumsum(chromsizes)]

AttributeError: module 'cooler' has no attribute 'read_chromsizes'

trying to run higlass_getter.py
Nezar Abdennur
@nvictus
@pkerpedjiev took offline
@nandankita Are you sure there's a chromosome called "10" in your data?
or is it chr10
nandankita
@nandankita
Yes! you are right! Thank you!
Nezar Abdennur
@nvictus
:)
Nezar Abdennur
@nvictus

As I understood I am getting a cool file which is 10 kb binned. What if I don't want it to be binned and create a cool file ? Does that make sense?

So what you are asking for is a binary version of the contact list? Cool files represent contact matrices so by definition they must be binned (i.e. aggregated) some way

You are not limited to fixed bin widths, however.
For example, you can use a bin table that segments the genome into restriction fragments. Then you will get a fragment-delimited matrix. Does that make sense?
cooler digest can make a fragment-delimited bin file
nandankita
@nandankita
Ohh okay! Now I understand. :D
Ilya Flyamer
@Phlya
Hi! I remember reading somewhere that cooler CLI can work directly with fragment files from hiclib, but couldn't find it anywhere now, is this the case? If so, is it the command cooler cload?
nandankita
@nandankita

Hi, I would like to import a bed file in Hi-Glass where it is overlapping intervals of chromosome as following:
id start end mid-point Value
chr1 1 100000 50000 53.696604901
chr1 5001 105000 55000 53.3929395224
chr1 10001 110000 60000 53.393254991
chr1 15001 115000 65000 53.3523366425 ..

as a hitile file. Any suggestion on how should I do this?

Nezar Abdennur
@nvictus
Hi Ankita. You should be able to generate a beddb file you can load into higlass using clodius aggregate bedfile
Also, we now have a community slack channel dedicated to HiGlass: http://bit.ly/higlass-slack
Lots more activity there
nandankita
@nandankita
Thank you Nezar! :D