Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Emilio
@elahrvivaz
i think the 1.9 versions are basically bug fix versions on top of 1.8
but some obscure API changed so they bumped minor versions
hopefully you aren't using that obscure api :)
James Srinivasan
@jrs53
I don't understand the libthrift issues - if they use semver, 0.9.3 & 0.9.1 should be compatible!¬
Emilio
@elahrvivaz
IF, haha
James Srinivasan
@jrs53
they don't? only place I found complaints about version mismatch was here
Emilio
@elahrvivaz
not sure
James Srinivasan
@jrs53
getting GeoCQEngineIndexChoiseTest.checkFilter:72 » UnsupportedOperation GeoCQEngineDe... with tests building 2.4 having upped accumulo version. Looking into it some more...
Emilio
@elahrvivaz
someone else was saying that test failed for them...
i'd try just using -DskipTests
James Hughes
@jnh5y
@urbanit nice to hear that you all got the C* Spark support working!
As I mentioned, I have been mentoring university students on the same project. They are (hopefully) putting up a partial solution this week. Perhaps you'll be in a position to review their week and compare it to your own to see if they caught anything your implementation ought to have.
@elahrvivaz Hmm... the GeoCQEngineIndexChoiseTest failures are annoying. @jrs53 do you happen to have more info about the build failure you saw?
Sotiris Roussos
@urbanit
@jnh5y that's great... At l(e)ast, we found a solution... Both :-)
James Hughes
@jnh5y
Indeed! Is your implementation working well for you or do you suspect there's further work to improve it?
(I'm happy either way since I don't use C* on a daily basis:). I'm hopeful what you have is great!)
Sotiris Roussos
@urbanit
Code was based on HBase... Except InputFormat that we copied Bigtable... Anyway, we arefinishing some details and I will send the code... Or a PR if tests are there...
James Srinivasan
@jrs53
-------------------------------------------------------------------------------
Test set: org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest
-------------------------------------------------------------------------------
Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 4.352 s <<< FAILURE! - in org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest
GeoCQEngine should::return correct number of results for basic spatial predicates filter [ Where intersects POLYGON [[-180 90, 0 90, 0 0, -180 0, -180 90]] ] [geo-only index](org.locationtech.geomesa.memory.cqen
gine.utils.GeoCQEngineIndexChoiseTest)  Time elapsed: 0.062 s  <<< ERROR!
java.lang.UnsupportedOperationException: GeoCQEngineDebugEnabled = false, debug mode disabled
        at org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest.checkFilter(GeoCQEngineIndexChoiseTest.scala:72)
@jnh5y first error, others similar
Emilio
@elahrvivaz
@jrs53 what OS are you building on?
James Srinivasan
@jrs53
CentOS 7 with Java 1.8.something
Emilio
@elahrvivaz
:+1:
James Srinivasan
@jrs53
using build/mvn
had hoped -am -pl :geomesa-accumulo-dist_2.11 would not build that bit
Emilio
@elahrvivaz
i wonder if it's some kind of timing issue, maybe the vals in that class need to be lazy
James Srinivasan
@jrs53
am using multithreaded build
does CI pass?
Emilio
@elahrvivaz
yeah
and i haven't seen it fail building locally either
James Hughes
@jnh5y
@elahrvivaz threading! That's a nice catch
maybe we should slam a 'sequential' at the top of GeoCQEngineIndexChoiseTest?
James Srinivasan
@jrs53
build/mvn clean install -T8 -pl :geomesa-accumulo-dist_2.11 -am is my command line
@jnh5y that's my usual solution to nasty test execution order things
Emilio
@elahrvivaz
@jnh5y it should be thread-safe for reads right?
@jrs53 I usually split the build and test phases, since tests fail sometimes it can save some time
mvn clean install -T8 -DskipTests followed by mvn surefire:test -T3
i've seen that the 'initialization' part of a test body can sometimes be run more than once, or out of order, or different things like that... but making things lazy or defs should fix that
we'd have to move the add features lines inside the lazy init block for each val
Damon Stone
@nomadgis
Is it possible to connect to a GM datastore from a Java only app and cql query and read from it?
Emilio
@elahrvivaz
@nomadgis yes, you just use the GeoTools API
for example, this page shows you how to create an HBase data store: https://www.geomesa.org/documentation/user/hbase/usage.html#creating-a-data-store
the other stores have similar sections
you can also take a look at the example code in our tutorials repo: https://github.com/geomesa/geomesa-tutorials
James Srinivasan
@jrs53
Is there a way of passing accumulo.connector to the Spark df reader? Seems to expect only string->string
Emilio
@elahrvivaz
it has to be serializable... for simplicity we use string->string
passing in the connector directly is mainly for test support
James Srinivasan
@jrs53
I'm trying to use it to test a Kerberos workaround
but org.apache.spark.sql.DataFrameReader expects options to be [String,String]
not [String,Serializable]