Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 19:23
    sayandip18 commented #258
  • 19:22
    sayandip18 commented #258
  • 19:20
    sk1p commented #258
  • 19:17
    sayandip18 commented #258
  • 19:16
    sk1p commented #699
  • 19:10
    sk1p commented #258
  • 15:40
    codecov[bot] commented #699
  • 15:40
    codecov[bot] commented #699
  • 15:37
    codecov[bot] commented #699
  • 15:32
    codecov[bot] commented #699
  • 15:32
    codecov[bot] commented #699
  • 15:31
    codecov[bot] commented #699
  • 15:31
    codecov[bot] commented #699
  • 15:31
    codecov[bot] commented #699
  • 15:30
    codecov[bot] commented #699
  • 15:29
    codecov[bot] commented #699
  • 15:29
    Build #2338 passed
  • 15:14
    codecov[bot] commented #699
  • 15:14
    twentyse7en synchronize #699
  • 15:07
    sayandip18 commented #258
Shankhadeep Dey
@Iamshankhadeep
@sk1p I did come across it, but it does not have the code gen support for python and typescript.
So I skiped it and never mentioned.
Alexander Clausen
@sk1p
@Iamshankhadeep Ah, nice! It's always good to include things you considered, even if they don't fully match the requirements. For example it shows that you did comprehensive research, and it may be possible to "borrow" ideas from projects, even if they are not a 100% solution. For example, what do you think about implementing code generation yourself? See for example https://github.com/asyncapi/ts-nats-template/pull/1/files for a very specific code generator for the NATS framework - WIP from two days ago but implemented in under 1000 lines of template code
Shankhadeep Dey
@Iamshankhadeep
I guess it can be done but I need to write code generator for python and typescript both then.
Alexander Clausen
@sk1p
Indeed
Shankhadeep Dey
@Iamshankhadeep
I need to reasearch about asyncapi a lot then. To get to know about everything.
But then again we also need to maintain asyncapi for websocket and one for http. right?
Alexander Clausen
@sk1p
Could be possible to build a small prototype with it, to get to know it. With async API, I think it is possible to define "messages" that are transferred via HTTP
Shankhadeep Dey
@Iamshankhadeep
So what do you suggest about this?
Again we do need to play with tools to see if this can solve our requirement.
Alexander Clausen
@sk1p
I suggest to keep this in mind as a possible solution or part of a solution - for example, if the jsonschema-based approach has some downsides that we didn't anticipate, or if AsyncAPI turns out to have some nice properties, based on an initial prototype etc.
Shankhadeep Dey
@Iamshankhadeep
Yes I do understand. So I think I need to start too early to get this done in gsoc timeline. Did you have a look at the final draft? I shared it through gsoc website did you get it?
Alexander Clausen
@sk1p
Don't worry about getting the timeline 100% - once you start working, you may discover a completely different approach, or some details that make it hard to stick to the plan - it could happen that you only get a part of the plan finished, but still deliver a lot of value for our project - if you spend more time on finding the right approach. I saw the draft, yes. Don't have any comments right now, overall it looks good
Shankhadeep Dey
@Iamshankhadeep
Thanks I appreciate it. I will definitely try to add value to our project. So should I submit this as final proposal?
Alexander Clausen
@sk1p
@Iamshankhadeep sure, go ahead
@uellue I was distracted a bit this morning and built a jupyterlab integration for libertem-server: https://github.com/sk1p/libertem-jupyter-proxy - could a) be interesting for deployment in our institute and b) for tightening the integration, related to @twentyse7en 's project
Shankhadeep Dey
@Iamshankhadeep
Okay then. :thumbsup:
Anand Baburajan
@AnandBaburajan
@uellue and @sk1p, I'd like to become familiarized with UDFs. Would trying to focus on LiberTEM/LiberTEM#631 be helpful, or should I just try out UDFs? Thanks.
Alexander Clausen
@sk1p
@AnandBaburajan I'd say first trying to implement some simple UDFs would help most. For example, take one of the simple analyses that are available via the GUI (integration over a disk for example?) and try to reimplement both result images that are displayed. Then maybe play around with the roi parameter and see how that affects the results etc.
Anand Baburajan
@AnandBaburajan
:)
Sahan Wickramage
@SahanWickramage
@uellue @sk1p, I updated my proposal. Could you check it if you have some time?
Todd Chavez
@twentyse7en

@uellue I was distracted a bit this morning and built a jupyterlab integration for libertem-server

Cool! I'm not familiar with Jupyterlab , is it possible to directly run notebook from GUI using this?

Alexander Clausen
@sk1p
@SahanWickramage sorry, was in a conference call. I'll have a look in a bit
@twentyse7en It's a different way to use notebooks with a more integrated user interface
Alexander Clausen
@sk1p

Just a note that the GSoC proposal submission deadline is coming close, if you haven't submitted a final proposal, now would be a good time. ( @twentyse7en did you already submit? I couldn't find your proposal in the system last time I looked)

@SahanWickramage I'm sorry I can't give you more specific feedback. I can try to still add feedback to your proposal, but I guess it's cutting it too close to the deadline.

Todd Chavez
@twentyse7en
Yeah, I did the final submission yesterday @sk1p .
Alexander Clausen
@sk1p
@twentyse7en Ahh now I see - it just doesn't show up in the system when searching for "libertem". :+1:
Todd Chavez
@twentyse7en
I did setup the jupyter-proxy according to the instruction provided. But when I click the LiberTEM icon it opens a blank tab. How is this supposed to work?
Alexander Clausen
@sk1p
Are there any errors in the devtools console? It's supposed to start the LiberTEM GUI in that new tab. One thing to note is that it needs the current LiberTEM master version, with my latest changes in it.
Todd Chavez
@twentyse7en
Oh! I didn't update the LiberTEM to the current master.
Todd Chavez
@twentyse7en
now working perfectly :+1:
Alexander Clausen
@sk1p
:+1: :+1: :+1:
Todd Chavez
@twentyse7en
Can we have a button to make the server stop like in Jupyter notebook
quit.png
Alexander Clausen
@sk1p
@twentyse7en Hmm, could be useful, mostly when started from jupyter lab. One thing to take care with, however, is that jupyter lab may restart services on exit - so this may have to involve talking to some part of jupyter.
Anand Baburajan
@AnandBaburajan
drop.PNG

@sk1p In the 'happy' case, the dropdown is shown and if the timeout is hit, the textbox is shown (not as of now, as the RAW dialogue is shown in that case).
The dropdown is short, searchable and scrollable, so no need for n, right? Any suggestions? Thanks!

If the timeout is hit, it should still be possible to open the HDF5 file 'manually'

Can I do that in another PR?

Alexander Clausen
@sk1p

@AnandBaburajan nice!

Can I do that in another PR

Sure!

No suggestions right now, I think I have to try it myself once the PR is opened
Anand Baburajan
@AnandBaburajan
There's a small issue with passing the ds_paths, I'll open the PR once it's solved :)
Alexander Clausen
@sk1p
:+1:
Sahan Wickramage
@SahanWickramage
@sk1p feedback you gave is plenty. Thank you. I searched for some state of art compression surveys on google scholar. If I found something useful, I will put them on chat. Do you have any other suggestions to work on? @uellue @sk1p
Shankhadeep Dey
@Iamshankhadeep
Sorry everyone I have been busy for quite a few days regarding my college project. I will be active from tomorrow onwards.
Sayandip Halder
@sayandip18
In the radialfourier notebook, shouldn't there be a ds_path parameter in ctx.load function in cell no. 5?
Alexander Clausen
@sk1p

@SahanWickramage

Do you have any other suggestions to work on?

The last you worked on was the "empty state" issue, right? Do you want to continue with that? Do you need any more feedback for that? Otherwise, have a look at the issue tracker and see what looks possible to work on for you, and just ask any questions in the issue itself. There should still be plenty of issues marked as "good first issue", too.

@Iamshankhadeep No worries! Thanks for letting us know
@sayandip18 in this case, it is not required - ds_path is specific to the HDF5 file format, where one file can contain multiple data sets. In this case, the data is stored in the MIB format, which doesn't require this parameter. See also: https://libertem.github.io/LiberTEM/reference/dataset.html#mib https://libertem.github.io/LiberTEM/formats.html
Sayandip Halder
@sayandip18

This is what I did:
ds = ctx.load( 'hdf5', path='C:/Users/Sandip/Desktop/calibrationData_circularProbe.h5' ) cx,cy = 134,139

And this is the error I got:
`TypeError Traceback (most recent call last)

<ipython-input-4-6aefe2e8ebe7> in <module>
1 ds = ctx.load(
2 'hdf5',
----> 3 path='C:/Users/Sandip/Desktop/calibrationData_circularProbe.h5'
4 )
5 cx,cy = 134,139

c:\users\sandip.virtualenvs\libertem258\lib\site-packages\libertem\api.py in load(self, filetype, args, **kwargs)
88 """
89 # delegate to libertem.io.dataset.load:
---> 90 ds = self.executor.run_function(load, filetype,
args, **kwargs)
91 ds = ds.initialize(self.executor)
92 ds.set_num_cores(len(self.executor.get_available_workers()))

c:\users\sandip.virtualenvs\libertem258\lib\site-packages\libertem\executor\dask.py in run_function(self, fn, args, **kwargs)
135 fn_with_args = functools.partial(fn,
args, **kwargs)
136 future = self.client.submit(fn_with_args, priority=1)
--> 137 return future.result()
138
139 def map(self, fn, iterable):

c:\users\sandip.virtualenvs\libertem258\lib\site-packages\distributed\client.py in result(self, timeout)
216 if self.status == "error":
217 typ, exc, tb = result
--> 218 raise exc.with_traceback(tb)
219 elif self.status == "cancelled":
220 raise result

c:\users\sandip.virtualenvs\libertem258\lib\site-packages\libertem\io\dataset__init__.py in load()
29 """
30 cls = get_dataset_cls(filetype)
---> 31 return cls(args, *kwargs)
32
33

TypeError: init() missing 1 required positional argument: 'ds_path'`

Alexander Clausen
@sk1p
@sayandip18 indeed - if you are trying to open HDF5 data, you need to add the ds_path parameter
Sayandip Halder
@sayandip18
Oh I get it now. Thanks!