Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Muhammad Huzaifa
    @mhuzai
    Welcome!
    Jeffrey Zhang
    @JeffreyZh4ng
    Hi
    2 replies
    Georgios Zacharopoulos
    @GiorgioZacharo
    @mhuzai Hi Huzaifa. I have generated some graphs regarding data dependencies among functions at audio_pipeline. Do you have one that I could compare with for one of the functions?
    _ZN12ILLIXR_AUDIO7ABAudio11readNEncodeER8CBFormat.pdf
    Muhammad Huzaifa
    @mhuzai
    Hi @GiorgioZacharo , I do have DFGs for both recording and playback but they are at an algorithmic/logical level, not at a function level like yours are. I can convert them to a function level and then let you know.
    Georgios Zacharopoulos
    @GiorgioZacharo
    @mhuzai Great. Thanks for that!
    Sam Grayson
    @charmoniumQ

    Those of us at UIUC who work on ILLIXR decided we should do more of our deliberation in public in addition to our research meetings. There are a couple of venues for this:

    • Gitter is best for realtime communication. We can deliberate over design decisions and go back-and-forth on specific bugs here. However, Gitter is less discoverable, so we have other options too.

    • Once we have consensus around a plan for some bug or feature (deliberated on Gitter), whatever we decided can go into a GitHub issue. More specifics and details can be worked out in the issue's comment thread. Contributors can log progress through checklists and our GitHub Project board.

    • If these details are relevant in the future (e.g. if it is the rationale for an important decision), then can be put either in the high-level documentation (docs/), comments in the code, or in the root of the repo (in special cases) in their pull request. This way it is permanent and discoverable.

    In following my own suggestion, I am asking for feedback on the plan I have laid out above. Then I will roll that into a ILLIXR/ILLIXR#60, which will record our policies in the root of the repo.

    Rishi Desai
    @therishidesai
    Any thoughts on moving to Discord? A lot of other open source communities have moved to that (e.g: Rust) and it has been successful for them because the channel features are much nicer than Gitter
    1 reply
    Mattermost is another nice option
    Rishi Desai
    @therishidesai
    For mattermost I believe it is similar to Slack where it is an account per workspace
    Rishi Desai
    @therishidesai
    We would also have to self host mattermost unless we want the enterprise edition
    Jakob Bornecrantz
    @Wallbraker
    For MM you need an account per instance, but a instance can have multiple teams.
    Ryan A. Pavlik
    @rpavlik
    howdy! Thanks for choosing a chat system I already had running :-P (Not hard, I have mattermost, discord, gitter, slack, and even IRC open at all times...)
    Sam Grayson
    @charmoniumQ
    Hello all. We fixed a lot of usability problems related to installing ILLIXR, building ILLIXR, and connecting it to Monado. If you are working on master, skim the documentation for the new build process. If you are working on the latest release (v2-latest), no change yet, we're going to push out a release soon.
    Shulin Zhao
    @suz53
    Hi everyone, I was trying to run ILLIXR on a Ubuntu virtual machine but failed with GLX error. Is there a snapshot which can run as a virtual machine? Thanks.
    Muhammad Huzaifa
    @mhuzai
    Hi Shulin, thank you for trying out ILLIXR. We do not have an image at the moment but are certainly working towards building one. In the meantime, can you try out this branch of ILLIXR: https://github.com/ILLIXR/ILLIXR/tree/issue-57-headless It has some fixes for headless mode but those fixes might help run in a VM as well.
    Shulin Zhao
    @suz53
    Hi Muhammad, thank you very much for your help. I will 1) try the branch in a VM; and 2) at same time build the ILLIXR on a Ubuntu desktop. I will let you know any updates. Thanks again for your reply, and the great XR project!
    Shulin Zhao
    @suz53
    @mhuzai Hi Muhammad, I've tried the branch you shared with me, however, still same error. Error details: make: Leaving directory '/home/zsl/Desktop/ILLIXR/open_vins'
    Register N6ILLIXR13record_loggerE
    Register N6ILLIXR8gen_guidE
    Register N6ILLIXR11switchboardE
    thread,140464022648576,switchboard worker,0
    X Error of failed request: GLXBadFBConfig
    Major opcode of failed request: 152 (GLX)
    Minor opcode of failed request: 34 ()
    Serial number of failed request: 37
    Current serial number in output stream: 36
    Traceback (most recent call last):
    File "runner/main.py", line 301, in <module>
    main()
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/site-packages/click/core.py", line 829, in call
    return self.main(args, kwargs)
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback,
    ctx.params)
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(
    args, **kwargs)
    File "runner/main.py", line 299, in main
    asyncio.run(run_config(Path(config_path)))
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
    File "/home/zsl/miniconda3/envs/illixr-runner/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
    File "runner/main.py", line 291, in run_config
    await loadersloader
    File "runner/main.py", line 144, in load_native
    await subprocess_run(
    File "runner/main.py", line 91, in subprocess_run
    raise subprocess.CalledProcessError(return_code, cmd=shlex.join(args))
    subprocess.CalledProcessError: Command '../runtime/main.opt.exe ../ground_truth_slam/plugin.opt.so ../offline_imu_cam/plugin.opt.so ../open_vins/plugin.opt.so ../pose_prediction/plugin.opt.so ../gldemo/plugin.opt.so ../timewarp_gl/plugin.opt.so ../debugview/plugin.opt.so ../audio_pipeline/plugin.opt.so' returned non-zero exit status 1.
    Please let me know any suggestions for deploying ILLIXR on a VM. And I will try it on Nvidia Jetson board next. And after that, I will try to build on a Ubuntu desktop, and let you know the updates. Thanks again!
    5 replies
    Sam Grayson
    @charmoniumQ
    Are you running ILLIXR over SSH with X forwarding?
    DhairyaBahl
    @DhairyaBahl
    Hello....I am new here...Can anyone help me a bit in resolving issue#132 on how to set default resolution to 2K from VGA sized ...
    4 replies
    Matt Horsnell
    @matt2909
    Hello :wave: having spoken to some of you about ILLIXR I'm taking my first hesitant steps at building/running. As you might expect I'm trying to get this running on Arm. Question - do I need to pull a specific branch? setup specific environment files etc... Happy to be pointed at a doc :smile: thanks
    Muhammad Huzaifa
    @mhuzai
    Great to see you here, Matt! If you're running Ubuntu 18.04, then just starting with install_deps.sh would be the right thing. There is no Conda on Arm, which our build and running script needs, so that is something that we are working on right now. If you are not running Ubuntu 18.04, please let us know your OS and we can provide instructions accordingly.
    Matt Horsnell
    @matt2909
    I am running Ubuntu 20.04 on an AArch64 platform.
    There is no urgency to any of my questions by the way, but I have a few clear hours this week and wanted to dive in.
    Muhammad Huzaifa
    @mhuzai
    20.04 is great. We've tested fairly thoroughly on it as well. I would still recommend starting with the getting started page. One minor change would be to pull from master and not from v2-latest, as we've pushed quite a few usability changes since the last release.
    Matt Horsnell
    @matt2909
    Can you still run in Standalone mode? I tried following the steps here but for me at least I don't end up with any benchmark folders
    Ah my confusion, v1-latest structure and v2-latest/master seem quite different. Apologies, I'll keep playing...
    Matt Horsnell
    @matt2909
    Do you think you will be able to support an equivalent standalone mode in v2?
    3 replies
    Shulin Zhao
    @suz53

    Are you running ILLIXR over SSH with X forwarding?

    Yes.

    Muhammad Huzaifa
    @mhuzai
    That won't work due to certain OpenGL requirements. You'll have to try directly on a machine.
    3 replies
    Shulin Zhao
    @suz53
    Hello! the install_apt_deps.sh error: E: Unable to locate package librealsense2-dkms Is there a new apt source?
    30 replies
    nblei
    @nblei
    In the audio_pipeline there are quite a few objects which are 'free'd rather than 'delete'd, which is causing memory leaks to be reported by valgrind.
    Specifically, look at audio.cpp:28-36, the ABAudio destructor. These objects are allocated using new and then deallocated using free.
    2 replies
    nblei
    @nblei
    Replacing free with delete fixes the memory leaks.
    Muhammad Huzaifa
    @mhuzai
    Thanks for the feedback, @nblei. Would you like to open a PR with the fixes (so you get attribution)?
    Armand Behroozi
    @armandb_gitlab

    Hello,

    I am trying to run the open-vins component by itself using the “./run_illixr_msckf” script. I just want to double check that my usage is correct b/c I ran it overnight and no text or GUI popped up and it was still going.

    I wanted to test whether everything was installed and working properly so I went to The EuRoC MAV Dataset and downloaded the ASL Dataset format for Machine Hall 01 as a test (I used the ASL and not the ROS bag b/c the instructions say this is running w/o roscore). This is the command I used: “./run_illixr_msckf ../../mav0/cam0 ../../mav0/cam1 ../../mav0/imu0 ../../mav0/cam0/data ../../mav0/cam1/data.”

    I cloned the ILLIXR/open_vins repo, not the whole thing. Ubuntu 16.04 on my x86 desktop

    Is my usage correct or is there something I’m missing?

    Muhammad Huzaifa
    @mhuzai

    Hi @armandb_gitlab , thanks for checking out ILLIXR! Your usage is just slightly off. The first three parameters need to point to the data files inside the folders; e.g., /mav0/cam0/data.csv for the first camera. The final two parameters look good. Once you make the fix, you should see a lot of prints regarding tracking, initialization, and so on.

    And thank you for bringing this up. We should fix this issue on our end by checking the arguments and failing instead of silently hanging.

    1 reply
    nblei
    @nblei
    Is anyone aware of a fixed-point implementation and/or DSP implementation of the ambisonic audio algorithms?
    Muhammad Huzaifa
    @mhuzai
    I am not (although haven't looked that hard). We did run the audio pipeline through Hexagan DSP Simulator, but the results were abysmal with the code as-is. And we have no immediate plans to invest in handwritten DSP code at the moment.
    Armand Behroozi
    @armandb_gitlab

    image.png
    image.png
    When I'm running openVINS standalone on my Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz, I find that most of the time is spent in the tracking instead of the filter as reported in the paper.

    Could I get a similar screenshot from a run on the Intel Xeon E-2236 CPU? I want to try and work backwards from the specs try to figure out what's causing the difference in runtime on each of the different kernels.

    Also, on a slightly unrelated note, I noticed that KimeraVIO is the default VIO algorithm over openVINS. What was the reasoning behind this? I noticed that openVINS only takes opencv 3 and Kimera works w/ opencv 4 and kernels like the lktracker seem to look much different (and are probably better) on opencv4. Is this part of the reason?

    4 replies
    kaiming-uw
    @kaiming-uw
    Hello👋 My name is Kaiming Cheng and I'm working with Franzi Roesner & Yoshi Kohno on AR/VR security at UW CSE. I'm super excited about ILLIXR and would love to try some of our security ideas on this testbed. I wonder if ILLIXR can support multi-application currently?
    Muhammad Huzaifa
    @mhuzai

    Hey @kaiming-uw! Thanks for checking out ILLIXR. ILLIXR does indeed support running multiple applications together, courtesy of Monado. You can choose ILLIXR as the device, start the Monado service, and then launch multiple OpenXR applications together. We've tested that this works. However, I would like to mention two caveats:

    1. Our timewarp and spacewarp code has not been integrated into the Monado compositor yet. As a result, the display may not always be smooth; for instance, due to dropped frames. We are working hard to get this integration done soon.
    2. Multi-app/multi-layer compositing rules are somewhat tricky at the moment. Take a look at Section 10.4.2 of the OpenXR spec: https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#compositing
      So, if you would like the output from multiple applications to show up, you would either have to use different layers (e.g., projection vs. quad) or make the unused parts of a layer transparent such that the bottom later shows. If all the applications render to the projection layer and draw over the entire layer, then in the final image only one application will show up. (all the apps will be running though, and you'll still be able to measure their performance, power, scheduling, etc. effects)
      There is extension in the works that allows compositing based on depth, but it hasn't been finalized yet: https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html#XR_EXTX_overlay

    Let us know if you have questions!

    Armand Behroozi
    @armandb_gitlab
    image.png

    Hello,

    I was trying to profile openVINS with perf to find function hotspots and was running into some issues b/c there weren’t enough symbols. That led me to modify the CMakeLists of ov_msckf w/ -fno-omit-frame-pointer and configure and compile opencv w/ the same compiler flag. Now I have more symbols, but I’m getting nonsensical results like cv::detail::LKTrackerInvoker::operator() was using ~40% of execution time (perf data attached). I’ve tried creating the perf call graph with lbr and dwarf as well with similar results.
    In the paper, you all say you used perf and vtune to find hotspots. Did you mostly use perf to get the hotspots and did you run into similar issues? Was the solution to compile all the related libraries (opencv, boost, eigen) w/ frame pointers? Does vtune help avoid some of these challenges?

    Any help/advice would be appreciated.

    9 replies
    Armand Behroozi
    @armandb_gitlab

    Two quick questions:

    1. Just to double check, there isn't a standalone version for the IMU integrator and camera components right?
    2. In "Hologram", NUM_SPOTS is set to 2 b/c humans have 2 eyes correct? It has nothing to do with the number of focal planes or hologram pixels right?

    Also, for anyone with their GPUs connected to a server looking to run the openGL programs w/ GUIs, I had trouble getting stuff to work with "ssh -X" and ended up using virtual gl https://virtualgl.org/Main/HomePage. Would recommend.

    4 replies
    Armand Behroozi
    @armandb_gitlab

    Do you guys know if there are any reputable datasets with visual-inertial data for head-mounted displays?

    From what I understand, most of the datasets like EuRoC are for micro-aerial vehicles and self-driving cars. I was able to find others that use phones (NEAR: The NetEase AR...) or emulate head movements by moving a camera by hand. A lot of the 360 video datasets just have the head pose (not the IMU and camera readings).

    4 replies
    yjaycha
    @yjaycha
    Hello, This is Jay from Arm Applied Computer Vision team. I'm trying to try ILLXR on my macbook but getting started page seems to be broken, https://github.com/ILLIXR/ILLIXR/blob/master/getting_started.md. Do you have a documentation for install and using a demo data to try?
    4 replies
    yjaycha
    @yjaycha
    Hi, is there a eye-tracking in version 2? I see RITnet has been removed in version 2.
    1 reply
    ltstein
    @ltstein
    I am interested in combining XR and ROS, with the ROS platform able to access data produced the headsets or other XR devices. Do you think this would be better to integrate as an ILLIXR plugin or as a driver within Monado?
    6 replies
    Giordano Salvador
    @e3m3
    This community has moved to Discord (https://discord.gg/upkvy7x3W4). See you there!