by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 11:59
    jeenbroekstra commented #2431
  • 11:33
    hmottestad commented #2431
  • 11:31
    hmottestad commented #2431
  • 11:31
    hmottestad commented #2431
  • 11:30
    hmottestad commented #2431
  • 11:29
    hmottestad commented #2431
  • 05:54
    jeenbroekstra review_requested #2433
  • 05:54
    jeenbroekstra review_requested #2433
  • 05:54
    jeenbroekstra opened #2433
  • 05:40

    jeenbroekstra on GH-2431-jmh-maven-assembly

    GH-2431 all benchmarks now have… (compare)

  • 04:36
    jeenbroekstra commented #2431
  • 04:04
    jeenbroekstra commented #2431
  • 03:35
    jeenbroekstra commented #2431
  • 03:30
    jeenbroekstra commented #2431
  • 03:08
    github-actions[bot] opened #2432
  • 03:07

    jeenbroekstra on GH-1917-release-script-updates

    (compare)

  • 03:07

    jeenbroekstra on master

    GH-1917 switch to official gith… (compare)

  • 03:07
    jeenbroekstra closed #2428
  • 03:07
    jeenbroekstra closed #1917
  • 02:15
    jeenbroekstra commented #2431
Jeen Broekstra
@jeenbroekstra
Thanks for doing this Bart, I appreciate the tangle (and frustration). If at any point you have thoughts on how to make this easier I'd love to hear them.
Bart Hanssens
@barthanssens
in terms of frustration level, it's very low compared to e.g. having to fill out a 20+ survey in word, which could be replaced with some data exchange in RDF if it wouldn't be wrapped in red tape :-)
Bart Hanssens
@barthanssens
After some initial digging, looks like Sol tests are ok when upgrading from the current v7.x to 8.3, but some tests start failing somewhere between v8.3 and v8.5.1 (Lucene tests seem fine though, but I'll have to double check)
Bart Hanssens
@barthanssens
yup, it's solr 8.5 (and unfortunately also 8.6, so further upgrading won't help): 8.4 still all green, 8.5 has issues with more complex queries
Jeen Broekstra
@jeenbroekstra
Does that block your planned guava bump?
Bart Hanssens
@barthanssens
I think so, another option might be to upgrade to ES 7.6 (based on Solr/Lucene 8.4) instead of 7.7 or 7.8
(meanwhile it looks like the issue has something to do with change in highlighting / scores: SolrIndex.query seems to return the correct number of Solr Documents when querying, but does no highlighting in 8.5, while it does in 8.4)
Bart Hanssens
@barthanssens
Or perhaps test if ES 7.8 + Lucene 8.5 works, while keeping Solr at 8.4 (seems possible)
Jeen Broekstra
@jeenbroekstra
If that is workable I'm fine with it. I don't think our Solr support is such a widely used core feature that it warrants blocking upgrades in other areas of the platform.
Bart Hanssens
@barthanssens
Solr 8.4 / Lucene 8.5 seems to work :-)
Bart Hanssens
@barthanssens
another issue: elasticsearchrunner does not have options to set the java environment / path, which is used in the elasticsearchstore/benchmark java classes (setting it to a specific adoptopenjdk 8 location... I assume to have a consistent benchmark base, @hmottestad ?), maybe this could be fixed in a benchmark profile in pom.xml
Jeen Broekstra
@jeenbroekstra
hm? how can you set a specific adoptopenjdk location in code? Isn't that dependent on the machine on which it runs?
Bart Hanssens
@barthanssens
that's what I'm wondering as well
https://github.com/eclipse/rdf4j/blob/c22a0c5964de8caa1379376b2c18bc6828152993/core/sail/elasticsearch-store/src/test/java/org/eclipse/rdf4j/sail/elasticsearchstore/benchmark/AddBenchmark.java
in the PR I'm working on, I just ignore this (at least for now, to get the upgrade working)
Jeen Broekstra
@jeenbroekstra
Ah. That should not have made it into the repo, looks like some local testing setup by Havard. Fine with just ignoring it for now.
Bart Hanssens
@barthanssens
did we have some kind of upgrade note / path when going from ES 5.x to ES 6.x ?
Jeen Broekstra
@jeenbroekstra
I think so. Do you remember what RDF4J release that was?
Bart Hanssens
@barthanssens
ah, I knew there was a reason for release notes :-) don't know it by heart...
Jeen Broekstra
@jeenbroekstra
browsing back through it I can't immediately find it either...
Anyway, yes, the release notes are an ideal place for this I'd say.
Bart Hanssens
@barthanssens
first release with ES was 3.1, so... 3.3 ?
I'm probably mixing things up with Lucene upgrades, since RDF4J 3.1 started with ES 6.5
so this would be the first major upgrade for ES
Jeen Broekstra
@jeenbroekstra
We had elasticsearch full-text search before that I think. The elasticsearchstore is new. It's a bit confusing but we use ES for two different purposes now :)
Bart Hanssens
@barthanssens
Lucene for sure, which would boil down to the same thing I guess, since ES uses Lucene and every major change in Lucene says "re-index" in the release notes :-)
Jeen Broekstra
@jeenbroekstra
I've been meaning to ask: are you taking care of these version bumps because you yourself use the ES functionality? Or more out of a general "keeping things up to date" sense?
Bart Hanssens
@barthanssens
the latter actually, I occasionally use Lucene for the text searches, but not ES (store nor search)
Jeen Broekstra
@jeenbroekstra
fair enough, all the happier that you're willing to pick this up!
Bart Hanssens
@barthanssens
but sometimes it's related, like the guava dependency (which is not really an issue, but some people may feel uncomfortable when using older libraries with a CVE)
Jeen Broekstra
@jeenbroekstra
no i quite agree.
Bart Hanssens
@barthanssens
to be honest most of my projects just use the nativestore and Rio, nothing big :-)
and SHACL
Jeen Broekstra
@jeenbroekstra
I've been thinking about ways we can perhaps "isolate" the lucene/solr/elasticsearch modules, so that their dependencies don't interfere with the rest of the project. If we can somehow organize that, it might make these kinds of version bumps a bit less painful.
Bart Hanssens
@barthanssens
hmmz, perhaps, but that might bring us back to separate repos for stores / modules.... ?
Jeen Broekstra
@jeenbroekstra
well I was hoping to avoid that and instead do some maven pom magic (overriding managed dep versions, exclusions, etc). But I haven't got a fully formed plan yet.
Multiple versions on the same build path cause obvious problems so it would mean taking them out of the default multimodule build, and treating them as separate projects. We can do that even while still keeping them in the same repo.
but I grudge spending the time on this - I don't really use any of it myself :)
Bart Hanssens
@barthanssens
I'm sure someone would say "OSGi" :-)
Jeen Broekstra
@jeenbroekstra
I keep meaning to educate myself on that, but that spec requires a stiff drink before reading....
Bart Hanssens
@barthanssens
never used it
Jeen Broekstra
@jeenbroekstra
anyway, time for me to go for a walk. They only let us out for one hour each day, and the weather is going to be dreadful, but it's dry now.
Bart Hanssens
@barthanssens
as soon as I hear "enterprise", I have the tendency to run for the exit
Jeen Broekstra
@jeenbroekstra
not a trekkie then?
Bart Hanssens
@barthanssens
depends, original series, next generation and voyager ...
the rest of the franchise, mwa ... :-)
anyway, beaming myself to bed
Jeen Broekstra
@jeenbroekstra
alright, engage! Thanks Bart, and ttyl
Jeen Broekstra
@jeenbroekstra
@hmottestad just an FYI that I will take over and do the 3.3.1 patch release in the next hour or so - I'd like to have a patch before the weekend so I can bump our internal dependency.
Jeen Broekstra
@jeenbroekstra
RDF4J 3.3.1 now released - see https://rdf4j.org/news/2020/08/07/rdf4j-3.3.1-released/