Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jun 26 2017 20:07
    @remmeier banned @hibaymj
Roy
@depzor
Anyone familiar with the jpa n+1 issue? And how to prevent this with crnk?
Remo
@remmeier
@depzor question is, do you have one? Because crnk actually does quite a number of things to prevent them in the first place.
@rehab-reda0 Not the first time the topic has come up. As of yet there is no flag to ignore missing relationships. It will result in an error. Maybe @JsonApiRelation should allow to relax it. But generally not too fond of using it. An alternative approach could be if the parent repository returns a place holder with a status "DELETED" or something like that. Maybe could also be a visable option.
Remo
@remmeier
@SarangKanabargi there is an architecture chapter in the docs: http://www.crnk.io/releases/stable/documentation/#_architecture. Short answer is to model the business as a resource model. It requires a bit a different kind of thinking, but the end result can be a cleaner API.
Roy
@depzor
@remmeier yes I am experiencing multiple n+1 issues. Sometimes even more than > 100 queries are executed.
I know this can be solved by using JPA Entity graphs, but no clue how to integrate this with CRNK
Remo
@remmeier
you need to check where it is happening. Maybe there is an EAGER relation or something like that. crnk on its own should not have n+1 issues (graph loading and stuff is all taken care of).
Roy
@depzor
All marked as LAZY. I am using an older version of crnk: 2.11. Does that matter?
Roy
@depzor
I do use the "include" query parameter
Remo
@remmeier
Just now not aware of an issue. But 2.11 is really old and it should be a more or less smoth upgrade path. The one thing that did change in the meantime is a larger relationship handling refactoring. e.g. the JpaRelationshipRepository (or something like that) does not exist anymore. So things got (much) simpler. But a profiler or something should tell you where the loading happens. Generally crnk will be the one processing the "include", not JPA. So you would get a query for every inclusion. But not 1+n, rather 1+1
Roy
@depzor
Found the problem. we are using mapstruct to map entity to dto's, seems that during this mapping operation it triggers new queries to fetch some data
Remo
@remmeier
there is the possibility to customize the JpaEntityRepisitory to do Jpa graph loading. at least for newer version. but it is a bit undsr the hood
per default it will not load the entity graph in jpa, as it this can be a shource of trouble in many more complex scenarios.
Roy
@depzor
@Override
protected boolean fetchRelations() {
return true;
}
:)
this seems to activate it
duncanportelli
@duncanportelli
I am building a custom HttpStatusBehavior to return a custom Http Status code based on the type of the resource which I am reading from HttpStatusContext in getStatus(). When sending a DELETE request the ResponseDocument in HttpStatusContext is null since no data is returned in the response. How do you suggest getting the type of the resource then please?
Remo
@remmeier
maybe would be useful for the passed context to give access to QueryContext.
would be a little PR
Roy
@depzor
Is there any limitation for including 3rd level resources when fetchRelations is enabled? Getting a JPA error: "Unable to locate Attribute with the the given name ..."
Roy
@depzor
Ah issue seems to be fixed since version 3 in EntityGraphBuilderImpl :)
Roy
@depzor
One other question, when I enable entity graph within crnk by setting fetchRelations to true and I pass a OneToMany relation through the include query parameter
this makes the collection type also part of the entity graph which is really bad for performance (HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!)
Is there anyway to trigger a subselect query for collection types? So kind of a hybrid solution using entity graphs for one-to-one and many-to-one relationships and for collections a new sub select query?
Remo
@remmeier
are those nested objects crnk resources as well?
Roy
@depzor
yes all crnk resources
but all fetchPaths are passed through the EntityGraphBuilderImpl
Steve Brush
@brushs
@remmeier could you outline one of the use cases adding a TransactionRunner such as SpringTransactionRunner to a CRNK server instance solves? I understand the benefit if multiple actions are being performed behind a single call, however my understanding is only a single CRUD operation should be occurring. Is this for supporting multiple operation scenarios? Or are the single operations not as simple as they seem?
Remo
@remmeier
all crnk resources sounds perfect, because then you may not need any of it. dont populate relationships in mapstruct. at least not-loaded ones. crnk will then construct the graph.
Remo
@remmeier
I dont like the answer, but general graph loading in JPA/Hibernate is sh*t. not sure about most recent versions. Usually things explode in doing too much in a single query and limited options to prevent it or giving weird behavior. in crnk the graph loading is more robust and also accounts for things like security. for that reason never tried to go much further in jpa. one crnk limtation as of yet, it will assemble the graph in the response,.not the mapstruct.objects. sometimes the later would also be desirable.
transaction runner has different use cases. if the create/update/delete is more complex spanning multiple queries. operationsmodule supporting bulk mutation. and if desired database isolation when doing multiple reads/graph loading. for the later read committed is a frequent default, so not much impacf here.
Roy
@depzor
Thanks for pointing me into the right direction. I found a major problem in my code... well due Lombok..
using @data notation on my entity's

this creates a hashCode method for all properties and this triggers many lazy load queries when it creates a TupleElement at a certain point:

TupleElement(Object[] data) {
this.data = data;
this.hashCode = Arrays.hashCode(data);
}

Steve Brush
@brushs
thanks @remmeier -- is there an example available of a scenario with create/update/delete with multiple queries?
Roy
@depzor
Remo, I understand I dont need to populate relationships to the DTO to prevent new queries. However when I try to PATCH a relation it fails, because the relation isn't mapped into the dto.. why isnt the entity used here
Roy
@depzor
ah the entity is fetched through the JPA mapper. There I shouldn't override the dto relationships to the entity right because the dto relationships are null?
Roy
@depzor
hence the same properties (relationships) I set to ignore with mapstruct like @Mapping(target = "someRelation", ignore = true)
applies to both methods: map (entity to dto) and unmap (dto to entity)? Seems to work for a patch, not sure if I am missing any other cases..
Roy
@depzor
mmm you can immediately update relationships as well, that wont work with this solution
Remo
@remmeier
cannot quite follow.
sometimes it is simpler to use entities and db views rather than DTOs
to simplify relationshift handling in JPA I often make use of JsonApiRelationId to avoid all the relationship loading stuff and be closer to the relational model. so each relationship has a primitive and a entity-based atteibute mapped to the same column.
the docs/demo should have an example
updates will then use the primitve field
Remo
@remmeier
if a PATCH does not transmit a field, it will be ignored
Roy
@depzor
In my implementation of JpaMapper
E unmap(D dto)
I fetch the entity with entity manager and map all dto properties to the entity and for relations I only map it from dto if it's there otherwise I rely on existing entity data
that is my strategy at the moment
Roy
@depzor
Hi goodmorning, earlier you told me I shouldn't map relationships with mapstruct to the corresponding dto.
This is working very efficient when performing a GET. No wasted queries.
However in my JsonApiResource dto I have a json api relation property marked with @NotNull hence required when creating a new entity.
This is working fine for a POST. However when I try to do a PATCH and I omit the specific relation data from my request body
I receive a 422 error because the relationship is null. I can fix this by mapping this relationship in my JpaMapper.map(Tuple tuple) implementation.
But this solution has a side effect. When I perform another GET to fetch a list of my entity this triggers extra unwanted queries for this relationship, because I had to map this relationship in MapStruct.
Any better ideas for this particular problem?
Remo
@remmeier
I almost always make use of @JsonApiRelationId to complement @JsonApiRelation. It simplifies the handling. In particular for JPA with those lazy proxies that are prone to lazy loading issues. Or DB Views rather than DTOs as other simplification depending on the scenario (not always desired though).
Steve Brush
@brushs
@remmeier in the JPA Module docs (11.2) the docs describe one feature as "automatic transaction handling spanning requests" -- could you elaborate on that one a bit, in particular what is meant by "request" in this context?
Remo
@remmeier
that should rather be "spanning across a request" to be precise. No multi-request transaction things, except when batching them with the operations module.
Steve Brush
@brushs
I have a ResourceRepository for a DTO, and presumably would like to inject a reference to repository for an actual Entity tied to my DB to perform the actual work for creating records. I can autowire the Entity's repository if I actually create a class for that repository. However, I would prefer not to -- currently spring boot auto-configures all the repositories. Is there to inject or get a reference into my DTO's repository without creating the Entity repository manually? I tried through the ResourceRegistry but that doesn't seem to work.