These are chat archives for glue-viz/glue-vispy-viewers

3rd
May 2016
Thomas Robitaille
@astrofrog
May 03 2016 14:01
@PennyQ - I've made some fixes for the flood filling, see glue-viz/glue-3d-selection#8 and all the comments in there
Thomas Robitaille
@astrofrog
May 03 2016 18:52
It seems to fix the sharp switch

I got an error when make html as below

Exception occurred:
File "/Users/penny/anaconda/lib/python2.7/site-packages/astropy/logger.py", line 187, in _showwarning
if not isinstance(args[0], AstropyWarning):
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types
The full traceback has been saved in /var/folders/z0/ftx1hs8941s3b09whp8y82wm0000gn/T/sphinx-err-Ddqs6L.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at https://github.com/sphinx-doc/sphinx/issues. Thanks!
make: * [html] Error 1

Thomas Robitaille
@astrofrog
May 03 2016 18:55
Ok that's weird
I'll take a look to see what it could be
hmm actually can you try updating astropy?
conda install astropy
Penny Qian
@PennyQ
May 03 2016 18:56
ok
it works but actually there is no update on the astropy… condais a mystary…
Thomas Robitaille
@astrofrog
May 03 2016 18:58
so make html still fails, right?
or it works?
Penny Qian
@PennyQ
May 03 2016 18:58
it works!
Thomas Robitaille
@astrofrog
May 03 2016 18:58
ok :)
Penny Qian
@PennyQ
May 03 2016 19:03
Now I can see all the .html file from that doc/_build/html directory, so I will check the code to see if it’s included? How to previewe it through my terminal, or just use Chrome to open them?
Thomas Robitaille
@astrofrog
May 03 2016 19:04
just use chrome to open
you can do open doc/_build/html/index.html from terminal if you like
that opens it in the browser
Penny Qian
@PennyQ
May 03 2016 19:06
yes! it works, so I will check if any classes is missing here and add the cooresponding line into the .html file?
the glue.core.subset.Subset.html
Thomas Robitaille
@astrofrog
May 03 2016 19:07
no - as I mentioned in the issue you only need to add the missing classes in the __all__ line in the glue/core/subset.py file
the HTML files are generated from the code
so you should not edit the HTML directly
Penny Qian
@PennyQ
May 03 2016 19:08
oh, so the .html items are generated from the __all__ line
Thomas Robitaille
@astrofrog
May 03 2016 19:08
The only file you need to edit is subset.py
yes
Penny Qian
@PennyQ
May 03 2016 19:08
got it!
I will do it now, and create a new PR if there is update?
Thomas Robitaille
@astrofrog
May 03 2016 19:08
yep, thanks!
Penny Qian
@PennyQ
May 03 2016 19:16

should a function with decor be included in the __all__ or not? like

@contract(subsets='list(isinstance(Subset))', returns=Subset)
def _combine(subsets, operator):

Thomas Robitaille
@astrofrog
May 03 2016 19:24
Only public classes and functions should be included, so not functions that start with _
(the decorator part doesn't matter)
so _combine should not be included
Penny Qian
@PennyQ
May 03 2016 19:25
thanks, got it
Penny Qian
@PennyQ
May 03 2016 19:39
@astrofrog I update the subset.py and make html again, but when I checked the new doc/_build/html/api.html there is no change...
Thomas Robitaille
@astrofrog
May 03 2016 19:39
Ah yes, you need to make sure you re-install glue (from the version you have)
then run make html after
Penny Qian
@PennyQ
May 03 2016 19:40
can I use python setup.py develop and it could updates with my code change?
Thomas Robitaille
@astrofrog
May 03 2016 19:40
I think so
Thomas Robitaille
@astrofrog
May 03 2016 19:54
?
Does it work/
Penny Qian
@PennyQ
May 03 2016 19:54
yeah, I am creating pr for it
Thomas Robitaille
@astrofrog
May 03 2016 19:54
Perfect
Thanks!
Penny Qian
@PennyQ
May 03 2016 19:55
but I feel I can’t help with the instructions for those classes/func in that file :(
Thomas Robitaille
@astrofrog
May 03 2016 19:55
That's ok! As long as you've added missing ones
missing classes I mean
I can add the docs later
While I think of it, once you've finished that, make sure you take a look at glue-viz/glue-3d-selection#8
Penny Qian
@PennyQ
May 03 2016 19:57
cool, ok checking now, one quick question that does it mean that we’d better also put the comments like main glue so that later we could easily get our docs from it?
Thomas Robitaille
@astrofrog
May 03 2016 19:58
Sorry I'm not sure I understand, can you clarify what you mean?
Penny Qian
@PennyQ
May 03 2016 20:02
oh sorry, I mean later we will also generate such a website for the glue-3d-viewers, so we might also put comments like that way to make it easier for producing html files?
Thomas Robitaille
@astrofrog
May 03 2016 20:02
Ah yes, exactly, we can do that for the 3D plugin package in future yes
Unless we decide to merge it in to the main package of course
Penny Qian
@PennyQ
May 03 2016 20:06
Hmmm, I feel it’s good to leave clear comments, ha, I will try to add some into the 3d-veiwers code :)
Thomas Robitaille
@astrofrog
May 03 2016 20:08
Yes, commenting/docstrings is always good :)
Penny Qian
@PennyQ
May 03 2016 20:08
PyFITS should no longer be used (it's deprecated)
what happened to it?
Thomas Robitaille
@astrofrog
May 03 2016 20:09
It is now astropy.io.fits
Penny Qian
@PennyQ
May 03 2016 20:10
they are the same thing? or PyFits is an old name?
Thomas Robitaille
@astrofrog
May 03 2016 20:10
Yes they are the same thing, well astropy.io.fits has had more updates recently
PyFITS was merged into astropy
and PyFITS is no longer developed on its own
Penny Qian
@PennyQ
May 03 2016 20:16
Ha, Astropy is a bigger fish :)
I merged the glue-viz/glue-3d-selection#8
Thomas Robitaille
@astrofrog
May 03 2016 20:19
Ok - maybe you can now try it and see if it works properly for you?
Penny Qian
@PennyQ
May 03 2016 20:19
Thanks for this!
Thomas Robitaille
@astrofrog
May 03 2016 20:19
I haven't done a performance comparison of the scipy method vs cython yet
but if you want to do that, that would be great :)
Penny Qian
@PennyQ
May 03 2016 20:20
sure no problem, let me get a bigger data~
Thomas Robitaille
@astrofrog
May 03 2016 20:20
ok sounds good
Penny Qian
@PennyQ
May 03 2016 21:30

there is still a better performance of floodfill than scipy as

scipy takes 1.323516 sec and floodfill takes 0.293942

while another thing is the

get pos time is 163.842682 sec

lol...
Thomas Robitaille
@astrofrog
May 03 2016 21:32
ah ok, so the flood fill (cython or scipy) is not the limiting factor?
Penny Qian
@PennyQ
May 03 2016 21:32
coz sometimes the max_pos is not found and assigned as None, which might cause TypeError for the Cython code, so I add a loop to increase the region after each loop to make sure max_pos is assigned as a value...
Thomas Robitaille
@astrofrog
May 03 2016 21:33
ok
Penny Qian
@PennyQ
May 03 2016 21:33
no, it’s good actually, the get_max_pos is terribly slow...
my test data is 40MB FYI
Thomas Robitaille
@astrofrog
May 03 2016 21:34
ok
I'll try and think of how we can make get_pos more efficient
Penny Qian
@PennyQ
May 03 2016 21:36
-\ def get_max_pos(self):
    # TODO: search_region and max_value is critical for the performance!
    data = self.transform(self.pos_data)  # Map coordinates
    max_pos = None
    select_region = 4 
    # increase the search range after each loop to ensure the max_pos is found
    while max_pos is None:
        m1 = data > (self.selection_origin - select_region)
        m2 = data < (self.selection_origin + select_region)
        max_value = 0. # find a value larger than 0. 
        pick_selected = np.argwhere(m1[:,0] & m1[:,1] & m2[:,0] & m2[:,1])
        for item in pick_selected:
            index = tuple(self.pos_data[item].flatten())
            if self.vol_data[index] > max_value:
                max_value = self.vol_data[index]
                max_pos = index
        select_region *= 2
    return max_pos  # list argument for flood_fill_3d.cyfill()
wow, this looks good, what happened to the first line...
Thomas Robitaille
@astrofrog
May 03 2016 21:36
not sure what happened to the first line
Penny Qian
@PennyQ
May 03 2016 21:37
;) ok it’s fine~
Thomas Robitaille
@astrofrog
May 03 2016 21:46
Ok, so I have an idea
It would require completely changing get_max_pos, but it could make it waaaaaay faster
Penny Qian
@PennyQ
May 03 2016 21:47
wow!
Thomas Robitaille
@astrofrog
May 03 2016 21:47
I think
;)
Penny Qian
@PennyQ
May 03 2016 21:47
what’s that?
Thomas Robitaille
@astrofrog
May 03 2016 21:47
Ok so the slow part is that we have to transform ALL the voxels from visual to canvas space, right?
Penny Qian
@PennyQ
May 03 2016 21:47
yes
Thomas Robitaille
@astrofrog
May 03 2016 21:48
So a faster way is if we can transform the point where we click in canvas space into a ray in visual space
then we can just look at pixels along the ray in the visual
and I think I found a way to do that
Basically we can set up a transform like
tr = scatter.get_transform(map_from='canvas', map_to='visual')
Then we can transform two separate coordinates
Penny Qian
@PennyQ
May 03 2016 21:48
yes?
Thomas Robitaille
@astrofrog
May 03 2016 21:49
trpos1 = tr.map((200, 200, -1))
trpos2 = tr.map((200, 200, +1))
200,200 is the position where we click in the canvas
Penny Qian
@PennyQ
May 03 2016 21:49
ok
Thomas Robitaille
@astrofrog
May 03 2016 21:50
-1 and +1 are two positions along the line of sight (perpendicular to the canvas)
this then gives two sets of 3D coordinates
Well you have to do
trpos1 = trpos1[:3] / trpos1[3]
trpos2 = trpos2[:3] / trpos2[3]
Then trpos1 and trpos2 are 3D vectors in visual space
So then we have to just make sure we convert that to pixel space and then we can find a way to determine what the pixels along that ray are
Penny Qian
@PennyQ
May 03 2016 21:51
this is the homogeneous coordinates transfer?
Thomas Robitaille
@astrofrog
May 03 2016 21:51
I think so?
yes it is
Penny Qian
@PennyQ
May 03 2016 21:52
so we need a 4-dim array to save that ray pos + value?
Thomas Robitaille
@astrofrog
May 03 2016 21:52
(sorry was checking)
no, trpos1 is a 1D array with 3 elements
trpos1 and trpos2 are two points in 3D so you can find the equation of a line between them
then we can use the equation of that ray to find the intersection with each 'plane' of the cube
and in each plane we can find the closest pixel to the ray
and we just then find the peak one
Does that make sense?
Penny Qian
@PennyQ
May 03 2016 21:55
hmmm, so now we are not searching for the max value point along that ray, but instead find the cloese pixel along all ‘planes’ to the ray line?
Thomas Robitaille
@astrofrog
May 03 2016 21:56
Well we are still finding the max value along the ray
It's just that the easiest way to do that is to take the cube and check for each slice in the cube what the nearest pixel to the ray is
then for each slice you have one value
Penny Qian
@PennyQ
May 03 2016 21:56
yeah, it transfers the 3D problem to 2D images of muctiple channels
Thomas Robitaille
@astrofrog
May 03 2016 21:56
and you can then find the maximum of all those values
Does that make sense?
Maybe you can try and experiment with that to see if it is feasible?
Penny Qian
@PennyQ
May 03 2016 21:59
yeah, I am thinking if the ‘closest’ point in each slice is the ‘max’ one
Thomas Robitaille
@astrofrog
May 03 2016 22:00
Well for each slice you find the closest point, and then the max one is the max of all the closest points
Penny Qian
@PennyQ
May 03 2016 22:00
yeah but maybe the max value on that slice is not the closest one
Thomas Robitaille
@astrofrog
May 03 2016 22:00
right but we only care about voxels that the ray goes through right?
Penny Qian
@PennyQ
May 03 2016 22:01
yeah that’s true
Thomas Robitaille
@astrofrog
May 03 2016 22:01
we don't just want the maximum over the whole slice
Penny Qian
@PennyQ
May 03 2016 22:01
ok I will try it tonight~
Thomas Robitaille
@astrofrog
May 03 2016 22:01
ok :)
Penny Qian
@PennyQ
May 03 2016 22:01
ha, it looks quite promising ;)