Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jason Ren
    @jason718
    Hi guys, Hi Weichao, This is Jason
    I don't really get this , "making Unreal Editor the active window"
    Does it mean "play in selected viewport"
    Weichao Qiu
    @qiuwch
    I mean, use ctrl-tab to switch to the Unreal Editor after the script starts running.
    Jason Ren
    @jason718
    I see... I will test this after dinner and tell you the results.
    Jason Ren
    @jason718
    active editor window + high quality setting, I got ~2 fps. It's still not as fast as Weichao described (10 fps). Are there any other tricks that can speed up this? Or this is caused by hardware difference?
    Weichao Qiu
    @qiuwch
    @jason718 What map are you using? Let me try it on my side.
    Jason Ren
    @jason718
    @qiuwch I am trying on some maps (like urban city from PolyPixel) from the market.
    Weichao Qiu
    @qiuwch
    @jason718 I don't have this map, but I will try a map with similar scale and let you know later.
    Jason Ren
    @jason718
    @qiuwch Awesome! thanks! Acutally what I did is ‘vset’ the camera first and ‘vget’ the ‘lit’ picture. Every second I can got 1.5~2 pics
    Weichao Qiu
    @qiuwch
    @jason718 Did you also use ground truth?
    Jason Ren
    @jason718
    @qiuwch I am not sure what does “ground truth” mean here. But before extraction, the program read trajectories from *.json file
    Weichao Qiu
    @qiuwch
    I mean do you use depth, object_mask, etc. Or do you just use RGB image?
    Jason Ren
    @jason718
    Yep, I just use RGB pics
    Weichao Qiu
    @qiuwch
    Ok, l see.
    Jason Ren
    @jason718
    @qiuwch On windows, it's a little bit faster. the average speed is 0.4 second/ img ( 2.5 fps). the peak perforamance can be ~5 fps.
    Weichao Qiu
    @qiuwch
    Hi, @jason718 , sorry for late reply, I was busy with a project due yesterday. I got 8-9 fps in SunTemple during my testing. This is the script I used for testing. https://gist.github.com/qiuwch/10eb67e42ec2540e9e828ae827f3b64b
    My setting is project: SunTemple, UE4 version: 4.12, Windows 10, 500G SSD, Titan X * 1.
    I will spend my weekend optimizing speed, but not sure how soon I can get some progress. I can show you how to change resolution.
    Jason Ren
    @jason718
    @qiuwch Interesting. It seems like the fps is highly depend on map and hardware. BTW, how to change resolution and recompile on linux?
    Weichao Qiu
    @qiuwch
    @jason718 The resolution is currently hard coded in https://github.com/unrealcv/unrealcv/blob/master/Source/UnrealCV/Private/GTCaptureComponent.cpp. You can use the Makefile in the root folder to make the plugin (no document yet). I am traveling today and can not write in much details, sorry.
    ayush-agrawal
    @ayush-agrawal
    Is there a similar tutorial/file written for MATLAB as this one: https://github.com/unrealcv/unrealcv/blob/master/client/examples/generate-images.ipynb ? In particular, the dynamics of my agent (a humanoid robot) is written in MATLAB and I'm trying to generate synthetic dataset of images of the environment from a (virtual) camera on the robot.
    revilokeb
    @revilokeb
    @ayush-agrawal I am pretty sure there isnt yet, at least I havent come across so far. But it shouldnt be too hard to convert to Matlab from Python?
    ayush-agrawal
    @ayush-agrawal
    Has anybody gotten around with this issue unrealcv/unrealcv#40 ?
    ayush-agrawal
    @ayush-agrawal
    My bad, I was using an older version of the plugin. Works fine with v0.3.0
    Jason Ren
    @jason718
    @qiuwch Hi Weichao, any idea about how to automatically place shapenet models( or generally *.obj) into a existing game map?
    Weichao Qiu
    @qiuwch
    @jason718 I did not try to do it automatically. I manually imported a few ShapeNet models (*.obj) before. Many models of shapenet v1 have incorrect surface normal, but v2 is right. Are you interested in importing a lot of shapenet models to UE4?
    @ayush-agrawal Sorry I did not have time to maintain the MATLAB version, all my friends around are using python now.
    Jason Ren
    @jason718
    @qiuwch Yes I am interested in that. As you known in software like blender/maya, we can write script to import and place object. However, these engines don't have realistic game maps. I am thinking if we can use maps from UE4 and objects from ShapeNet, that would generate some great dataset.
    Weichao Qiu
    @qiuwch
    @jason718 I agree. Did you try batch import ShapeNet models into UE4? I think importing a few hundreds models is not a problem. But since the ShapeNet is so large, we might need to dynamically load models. I can not imagine a UE4 project with all shapnet models imported into it.
    Jason Ren
    @jason718
    @qiuwch I haven't tried batch import. Since there is no command for putting an object at specific location, I have decided not to use UE4 this time.
    Shuzhi Yu
    @shuzhiyu
    Hello everyone, I am thinking whether we can generate synthetic dataset for optical flow based on the current commands. Thank you!
    Máté
    @mkisantal

    Hello everyone, I'm also new here. :-)

    @shuzhiyu I'm also interested in getting optical flow data. I don't think it can be generated by unrealcv at this point, but hopefully others will correct me if I'm wrong.

    In the MPI Sintel dataset the authors "modified
    Blender’s internal motion blur pipeline to give us accurate motion vectors at
    each pixel" [http://files.is.tue.mpg.de/black/papers/ButlerECCV2012-corrected.pdf]

    Doing something similar might be possible in UE4 as well.

    Shuzhi Yu
    @shuzhiyu
    @mkisantal Thank you, Mate for the information.
    Weichao Qiu
    @qiuwch
    Hi, @shuzhiyu @mkisantal. For the optical flow, 1. I am thinking about how to design the API. It seems not intuitive to use the request/response cycle for optical flow. The request/response will affect the timing of UE4. There are some dynamical scene in UE4, such as this one (https://www.youtube.com/watch?v=EO0k92iVMjE). For this, the idea can be similar to Sintel, which is recording frame-by-frame difference. For static scenes, the motion comes from the camera motion, so we need a way that won't affect the camera motion speed. 2. The motion field is available in UE4 for the computation of motion blur effect, but it is not clear whether the time interval can be controlled. I did not spend much time on optical flow because there are a lot in the todo list.
    Shuzhi Yu
    @shuzhiyu
    @qiuwch Thank you, Weichao. I will try to see whether it is doable.
    Máté
    @mkisantal

    @qiuwch Thanks for your answer. Indeed, it's not trivial to combine it with the static scene renderings and the way the camera is moved now. I was thinking about implementing a vehicle model, which is responsible for moving the camera instead of the jumps with location/moveto/rotation commands. In a bit similar way how AirSim does, but with a lot simpler model. However, then view synchronization might get tricky (it is not solved in AirSim at this point)... So not necessarily a good direction.

    On top of this, optical flow is not necessarily the same as motion blur, if you decide to capture frames with a lower rate than the frame rate of your camera.

    Anyway, I'll keep thinking on this. I'm trying to use UnrealCV for an RL project, so it would be nice to have both motion blur on the captured images and optical flow accessible. But for now I'm just getting started with UE4 and UnrealCV. :-)

    Edgar Riba
    @edgarriba
    hi! is there any way to mute the sound in the example binaries by console command ?
    Gonçalo Marques
    @goncasmage1
    Hey guys, how's it going? :)
    I've been trying to use the UnrealCV plugin in these past few days, but can't get the object mask view to work. I understood from the documentation and source code that there is a server/client connection involved, but couldn't figure out how to set it up.
    Can you guys help me set it up, or even get it to run locally if possible?
    Thanks in advance!
    Raverss
    @Raverss
    @goncasmage1 Hey, had the same problem, it's an open issue on UnrealCV Github. It's bug in version 4.16-4.18. I downgraded to VXGI 4.15 branch and object masking is working just fine. What is bothering me tho is different resolution of object_mask from other views, any ideas why is that?
    Weichao Qiu
    @qiuwch
    @Raverss The image resolution is configured by the unrealcv config file. The object mask resolution is controlled by the game window. You need to adjust either of them to make them the same. This issue has been solved in the newest version which has not been released yet.
    Raverss
    @Raverss
    @qiuwch I should have read issues on your repo first before asking question :D I found your answer about running game in separate window with resolution specified in editor shortly after I wrote here.
    ajatOdin
    @ajatOdin
    Does anyone have advice with regards to engine compatibility and the plugin build tool? If I try to build for 4.18 or 4.2 I get an odd permissions issue and a failed build.
    atifkarim
    @atifkarim
    Hello, Anyone here who has used the UnrealCV plugin for 4.19 or 4.20 UE4 version? I have tried but failed. I am working in Windows OS.
    Michał Pełka
    @michalpelka
    Hello how to debug plugin code ?
    Victor Le
    @Coac
    Hello everyone, I am not too familiar with Unreal engine. Do you know if it exist something similar for Unity3D?