by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Andrew Bell
@abellgithub
@WouterDX : Run your pipeline in streaming mode to a single output. Then re-run THAT output as input in non-streaming mode to separate your points into the various areas.
WouterDX
@WouterDX
@abellgithub : Thx. For the second pipeline, if this should scale to larger extracts (1000s small polygons for instance), I should still solve that by doing the second pipeline sequentially (eg. loop with different values of filters.range) to deal with RAM I guess?
cojeffj
@cojeffj

Hello! new to PDAL - and trying to get rid of some noise. my JSON for pipeline:

{
    "pipeline": [
        "input.las",
        {
            "type": "filters.outlier",
            "method": "statistical",
            "multiplier": 3,
            "mean_k": 8
        },
        {
            "type": "filters.range",
            "limits": "Classification![7:7],Z[-100:3000]"
        },
        {
            "type": "writers.las",
            "compression": "false",
            "minor_version": "4",
            "dataformat_id": "0",
            "filename":"output.las"
        }
    ]
}

My command is:
pdal pipeline "c:\users\lucas\desktop\spanish_diggings\pipeline.json" --readers.las.filename="c:\users\lucas\desktop\spanish_diggings\raw\Spanish_723_L001_nnnn1_M01_r.las" --writers.las.filename="c:\users\lucas\desktop\spanish_diggings\denoise\test.las"

everything runs, but my output.las is an empty 2kb file. Any ideas?

Apologies in advance if this isn't the right place to ask.

Andrew Bell
@abellgithub
I don't see anything wrong with your pipeline. You should check the results coming out of the outlier filter without the range filter.
cojeffj
@cojeffj
@abellgithub that worked, removing the range filter from the pipeline. I'm testing again with the range filter, but with only the classified noise removal and not the Z limits. I'll report back.
cojeffj
@cojeffj
@abellgithub Also worked with just noise exclusion. It's the Z range that is causing the issues. Any idea why?
Andrew Bell
@abellgithub
Have you checked your Z values? If you've verified that they should be passed, you should submit a bug report and include the relevant data.
TJian
@tjtanaa
Could I know which function in Potree could be used to visualize the Entwine Point Tile that I have generated?
If there is a demo script that would be great.
Rickmc3280
@Rickmc3280
I was doing some searches online and found some sort of email thread that discussed a C# version of this? Does anybody know more? Intending to use a few of the functions in a C# app. (A few of the functions pull from other headers/cpp files and is pretty fully embedded so not sure how much I would need to rewrite, if I were to do that).
millionpoundhat
@millionpoundhat

When installing conda into a fresh environment with --

conda create -n entwine -c conda-forge entwine

When I try and run entwine I get --

entwine: error while loading shared libraries: libpdal_base.so.10: cannot open shared object file: No such file or directory

millionpoundhat
@millionpoundhat
(I activate the environment of course)
Howard Butler
@hobu
@millionpoundhat Entwine feedstock needs to be updated. I will do so
millionpoundhat
@millionpoundhat
@hobu Thank you :-)
lowbrows
@lowbrows
Pretty new to PDAL. I'm writing a python script to create some calibration QC tools and I need to ensure that the filters.info stage is in streaming mode. How can I set it or check it in python? With the command line there are lots of options and they are set the usual way like "--schema", see https://pdal.io/apps/info.html#info-command. Would I put the options in the pipeline.execute() function or somewhere in the pipeline stage? Thanks!
Andrew Bell
@abellgithub
@lowbrows : Streaming mode will be used if possible when using the pipeline interface. See the stage documentation for information on which stages are non-streaming. Generally speaking, the non-streaming stages are ones that need to sort, rearrange or find nearest neighbors. filters.info supports streaming: https://pdal.io/stages/filters.info.html#filters-info
lowbrows
@lowbrows
Thanks Andrew, I'm only using the readers.las and the filters.info in the pipeline. I thought if the streaming option was available I could process any size of file. However, when the script got to a file of about 15GB the Memory reached 100% and python quit. That why I was hoping to find a way to explicitly set it to streaming mode or at least check if the pipeline is running in streaming mode. Is there a way to do this or any way around this?
Andrew Bell
@abellgithub
@lowbrows Sorry, I didn't look closely enough. When using python, PDAL always runs in standard mode so that data can be set to PDAL from python or returned from PDAL to python. You certainly can get the same information by running pdal info from a shell and extracting the metadata. In this case the command will run in stream mode.
lowbrows
@lowbrows
That makes sense and is a good thing to know moving forward. Maybe I can figure out a workaround to stay in python and use the stdout of the shell. Thanks for the help!
Howard Butler
@hobu
@lowbrows we have chatted about implementing streaming in the Python bindings, but it hasn't been done yet. Happy to take a patch that does so if you've got the gumption
lowbrows
@lowbrows
Hello Howard, still have a lot of work to do for my project but I'll check back with you if/when I get it running and see if there is a way to make it work in a more general way for python pdal. Thanks
lowbrows
@lowbrows
Hi - trying to use the "where" option of filters.hexbin and keep getting errors. I've tried enclosing and not enclosing in quotes, brackets, parenthesis and everything else I can think of but I still get something like:
Unexpected argument 'where'
Could someone provide an example of how the 'where' option and the accompanying expressions are used. Also, would this invalidate streaming mode?
Andrew Bell
@abellgithub
Are you using version 2.2?
lowbrows
@lowbrows
Version 2.1.0 - Installed with conda
Andrew Bell
@abellgithub
Not supported until version 2.2.
lowbrows
@lowbrows
Ahh.. ok. Hypothetically then, if I were to use the 'where' option with the filters.hex would it invalidate streaming mode?
Andrew Bell
@abellgithub
No.
lowbrows
@lowbrows
Awesome! - Guess I'll be upgrading then. That is a very powerful and much needed feature. Thanks again Andrew.
Adam Steer
@adamsteer

hiya all. I was just contacted by a person going through a workshop exercise I made with filters.csf (https://github.com/adamsteer/f4g-oceania-pdal/blob/master/workshop/3-pipelines.md - running rpas-ground.jsonon a 68.6mb LAZ file), reporting that it consumed all the resources in a machine with 128GB RAM. I built the workshop on a 16GB Macbook, so it seemed odd. Just testing now, I’m consuming 20GB+ (RAM + swap) for the same exercise, with memory usage ramping up as soon as filters.csf starts work. PDAL version is 2.2 (using conda).

Using PDAL v2.1 (from conda), that same pipeline / dataset barely rates a memory usage blip from normal operations. I don’t know how to describe whats happening aside from 'PDAL 2.2 from conda uses a lot of memory for filters.csf'...

Andrew Bell
@abellgithub
@adamsteer : Have you isolated this to filters.csf?
Adam Steer
@adamsteer
In this case I think so. Memory usage jumps as soon as csf starts operating:
pdal pipeline rpas-ground.json --readers.las.filename=data/APPF-farm-sample.laz -v debug
(PDAL Debug) Debugging...
(pdal pipeline Debug) Executing pipeline in standard mode.
(pdal pipeline filters.elm Info) Classified 3 points as noise by Extended Local Minimum (ELM).
(pdal pipeline filters.csf Warning) Both NumberOfReturns and ReturnNumber are filled with 0's. Proceeding without any further return filtering.
from here RAM usage climbs with no further stdout spew

the data are here:
https://s3.amazonaws.com/f4g-pdal-workshop-sampledata/APPF-farm-sample.laz

pipeline is:

[
    {
      "type": "readers.las",
      "filename": "APPF-farm-sample.laz"
    },
    {
      "type": "filters.assign",
      "assignment": "Classification[:]=0"
    },
    {
      "type": "filters.elm"
    },
    {
      "type": "filters.csf",
      "ignore": "Classification[7:7]"
    },
    {
      "type": "filters.range",
      "limits": "Classification[2:2]"
    },
    {
      "type": "writers.las",
      "filename": "APPF-ground-default.laz"
    }
]
Andrew Bell
@abellgithub
I see a bug that was introduced. I'll fix.
Adam Steer
@adamsteer
fantastic, thanks!!
Guilhem
@gui2dev
Hi all,
I was wondering if making azure blob arbiter readable has ever been considered
Andrew Bell
@abellgithub
@adamsteer : A fix has been applied to master and the 2.2-maintenance branch for the issue you found.
eyou102
@eyou102
Hello, trying out PDAL/Entwine for the first time. Trying to build ept from csv
eyou102
@eyou102

Dataset = https://drive.google.com/file/d/11-qXqz18U2LdaoH2iVtu2PQnHnEe12Ud/view?usp=sharing

text file looks like:
X Y Z Uncertainty
403725.031 4122783.500 15.733 0.104
403725.125 4122783.500 15.745 0.085

at the command line:
'entwine build -i "~\csv_export\test.txt" -o "C:~\entwine" --srs "EPSG:26918"'

output is:
Scanning input
Encountered an error: No points found!
Exiting.

I figure I'm making a mistake with the switches or the file formatting?

Adam Steer
@adamsteer
@abellgithub thanks! credit goes to CRREL, who found the bug and told me. conda-forge hasn’t appeared to update yet, I will go build a pdal and test it out..
Hi @eyou102 could you post a smaller version of your test file? maybe just the first few thousand lines?
eyou102
@eyou102

To provide an update to this issue:

  • file paths must use forward slashes (UNIX), not backslashes (Windows)
    • if I use pdal translate to convert to LAS, I can successfully build the ept

I have not been able to get the readers.text workflow to work yet

Guilhem
@gui2dev
@eyou102, usually, I put all my inputs in a folder and use -i input/folder as argument
Adam Steer
@adamsteer
Hi @eyou102 I did not manage to find a way to step straight from text -> ept.. sorry! I think i am also missing something...
Howard Butler
@hobu
@adamsteer conda forge won't be updated for maintenance branch items unless there's an actual release
Adam Steer
@adamsteer
Gotcha - should have time to make a build and test CSF this weekend.