Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Partha Nath
    @ParthaNath28
    but when I fire a query like return jpaStreamer.stream(Entity.class).toList().stream().count(); it is taking almost 5 minutes to bring back a count of 200 thousand records.
    Is there an efficient way of getting the count without waiting for such an enormous time? Thanks
    Per Minborg
    @minborg
    Hi. return jpaStreamer.stream(Entity.class).count() is the way to go. If you enable logging of SQL querries, what do you see then?
    Per Minborg
    @minborg
    I think that looks as expected. If you cut and paste that code into an SQL tool, what result do you get?
    WHsat database type are you using?
    Partha Nath
    @ParthaNath28
    oracle
    the query works fine
    Per Minborg
    @minborg
    But you always get “1” as a result?
    When using the stream?
    Partha Nath
    @ParthaNath28
    By using the stream I am getting the correct number but it takes an enormous amount of time.
    3 minutes
    Per Minborg
    @minborg
    I mean, if you are using jpaStreamer.stream(Entity.class).count();
    Was the SQL dump from when you used the command jpaStreamer.stream(Entity.class).count();?
    Partha Nath
    @ParthaNath28
    jpaStreamer.stream(Entity.class).count()
    image.png
    I have pasted the screenshot
    from my intellij debugger window.
    Per Minborg
    @minborg
    ok. And the SQL dump when you run that stream is?
    Partha Nath
    @ParthaNath28
    yes
    The one I pasted above
    Per Minborg
    @minborg
    ok
    Strange. I will have to file an issue to verify this.
    Partha Nath
    @ParthaNath28
    ok
    thanks.
    Per Minborg
    @minborg
    Please add any extra information you can think of.
    (the issue is public)
    As a work-around, you could add a predicate that is always true in the stream pipeline using .filter() and see if that solves the problem (another renderer will be used in this case).
    .filter($Person.name.notEquals(“71287618717187hdvahashv”))
    Partha Nath
    @ParthaNath28
    ok
    thanks
    Partha Nath
    @ParthaNath28
    sorry this did not work either
    the sql is generated fine and giving correct record count
    but from java its still 1
    return jpaStreamer.stream(Entity.class)
    .filter(Entity$.primaryKeyIdentifier.isNotNull())
    .count();
    I tried this one
    Per Minborg
    @minborg
    ok. Let’s wait for the issue to get examined.
    Thanks
    vinit patel
    @vinit_r_patel_twitter
    Hi Per-Ake, I had reported an issue last week regarding my Eclipse based Spring STS IDE could not detect the generated class even though they got generated and compiled successfully. Finally, I figured out that Eclipse based IDE does not like a file name ending with "$". If I rename the file name by replacing "$" with "", the IDE does recognize it. Is there any way to generate the source file with "" suffix instead of "$"?
    Per Minborg
    @minborg
    Ooops. Accoding to the Java Language Specification, a class can end with “_” and “$”. This must be a bug in Spring STS IDE/
    There is no way to replace the “$” (easilly). JPA is using “_” for its typed fields.
    (metadata)
    Ghost
    @ghost~62da6df56da03739849a62de
    Hey, is it possible to efficiently stream blobs or huge (byte) arrays instead of loading them completely into memory with this?
    Per Minborg
    @minborg
    Do you mean stream blobs but not load the actual contents until you actually reference the BLOB?
    Ghost
    @ghost~62da6df56da03739849a62de
    Something like that. Let's say you store huge images with some meta-data and actual content. Is it possible to stream that 30mb blob from the DB to potentially a user, without loading the whole 30mb all at once?
    You have a table Image with id, and data (byte[] or something else) columns. You want that one particular image, but it's 30mb huge. Is it possible to omit loading it completely into memory?
    jpa-streamer is surely great if you have a big list of (small) results. You can stream, filter, etc. What if I want one huge result? Can I stream that too?
    I hope I could explain my problem with those questions
    Per Minborg
    @minborg
    You could define an entity with just the small thumbnail for example and then only retriev the larger entity version once you actually need the large data. Also, there might be methods in JPA itself that supports this lazy loading.
    Ghost
    @ghost~62da6df56da03739849a62de

    I don't think you understood my problem. I want to retrieve the larger entity efficiently.

    public class HugeImage{

    // some annotations like @Id
    private long Id;

    // some annotations like @Lob and Lazy_loading,
    private byte[] content;

    // getters, setters
    }

    How do I use streams to retrieve the content, without loading the complete content all at once?