Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
James Hughes
@jnh5y
@elahrvivaz Hmm... the GeoCQEngineIndexChoiseTest failures are annoying. @jrs53 do you happen to have more info about the build failure you saw?
Sotiris Roussos
@urbanit
@jnh5y that's great... At l(e)ast, we found a solution... Both :-)
James Hughes
@jnh5y
Indeed! Is your implementation working well for you or do you suspect there's further work to improve it?
(I'm happy either way since I don't use C* on a daily basis:). I'm hopeful what you have is great!)
Sotiris Roussos
@urbanit
Code was based on HBase... Except InputFormat that we copied Bigtable... Anyway, we arefinishing some details and I will send the code... Or a PR if tests are there...
James Srinivasan
@jrs53
-------------------------------------------------------------------------------
Test set: org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest
-------------------------------------------------------------------------------
Tests run: 10, Failures: 0, Errors: 10, Skipped: 0, Time elapsed: 4.352 s <<< FAILURE! - in org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest
GeoCQEngine should::return correct number of results for basic spatial predicates filter [ Where intersects POLYGON [[-180 90, 0 90, 0 0, -180 0, -180 90]] ] [geo-only index](org.locationtech.geomesa.memory.cqen
gine.utils.GeoCQEngineIndexChoiseTest)  Time elapsed: 0.062 s  <<< ERROR!
java.lang.UnsupportedOperationException: GeoCQEngineDebugEnabled = false, debug mode disabled
        at org.locationtech.geomesa.memory.cqengine.utils.GeoCQEngineIndexChoiseTest.checkFilter(GeoCQEngineIndexChoiseTest.scala:72)
@jnh5y first error, others similar
Emilio
@elahrvivaz
@jrs53 what OS are you building on?
James Srinivasan
@jrs53
CentOS 7 with Java 1.8.something
Emilio
@elahrvivaz
:+1:
James Srinivasan
@jrs53
using build/mvn
had hoped -am -pl :geomesa-accumulo-dist_2.11 would not build that bit
Emilio
@elahrvivaz
i wonder if it's some kind of timing issue, maybe the vals in that class need to be lazy
James Srinivasan
@jrs53
am using multithreaded build
does CI pass?
Emilio
@elahrvivaz
yeah
and i haven't seen it fail building locally either
James Hughes
@jnh5y
@elahrvivaz threading! That's a nice catch
maybe we should slam a 'sequential' at the top of GeoCQEngineIndexChoiseTest?
James Srinivasan
@jrs53
build/mvn clean install -T8 -pl :geomesa-accumulo-dist_2.11 -am is my command line
@jnh5y that's my usual solution to nasty test execution order things
Emilio
@elahrvivaz
@jnh5y it should be thread-safe for reads right?
@jrs53 I usually split the build and test phases, since tests fail sometimes it can save some time
mvn clean install -T8 -DskipTests followed by mvn surefire:test -T3
i've seen that the 'initialization' part of a test body can sometimes be run more than once, or out of order, or different things like that... but making things lazy or defs should fix that
we'd have to move the add features lines inside the lazy init block for each val
Damon Stone
@nomadgis
Is it possible to connect to a GM datastore from a Java only app and cql query and read from it?
Emilio
@elahrvivaz
@nomadgis yes, you just use the GeoTools API
for example, this page shows you how to create an HBase data store: https://www.geomesa.org/documentation/user/hbase/usage.html#creating-a-data-store
the other stores have similar sections
you can also take a look at the example code in our tutorials repo: https://github.com/geomesa/geomesa-tutorials
James Srinivasan
@jrs53
Is there a way of passing accumulo.connector to the Spark df reader? Seems to expect only string->string
Emilio
@elahrvivaz
it has to be serializable... for simplicity we use string->string
passing in the connector directly is mainly for test support
James Srinivasan
@jrs53
I'm trying to use it to test a Kerberos workaround
but org.apache.spark.sql.DataFrameReader expects options to be [String,String]
not [String,Serializable]
Emilio
@elahrvivaz
Connector isn't even serializable though
James Srinivasan
@jrs53
how does that work then?
Emilio
@elahrvivaz
type erasure haha
it's kind of a hack, in other words
in spark, we actually do serialize the param map though
string is just easier to deal with since all our 'real' params are primitives anyway
James Srinivasan
@jrs53
so is it possible to pass in a pre-built Connector when using Spark?
Emilio
@elahrvivaz
no, b/c we can't serialize it
James Srinivasan
@jrs53
oh well, will need to make some other changes :-(
Emilio
@elahrvivaz
is the issue that accumulo isn't supporting the proxy auth right?
i saw your email to the accumulo list
you could add some new params to specify the proxy through string keys, we could use that to construct the connector appropriately