skip_on_arrow()function is not working as expected. I'm trying to fix that issue now. So, let's wait for all checks on sparklyr/sparklyr#2426 to complete, merge it, and then try rebase and merge your change soon.
So I'm back working on a few things in the sparknlp package. One thing I've not been able to get working has me stumped. There is a method called
annotateJava() on the
LightPipeline object. This method is supposed to return
Map[String, List[String]]. Where the List[String] is a
java.util.List. But when I get the object back into R it's looking like this:
$document <jobj> scala.collection.convert.Wrappers$SeqWrapper [French author who helped pioneer the science-fiction genre. Verne wrate about space, air, and underwater travel before navigable aircrast and practical submarines were invented, and before any means of space travel had been devised.]
it looks like the
List objects are somehow coming back into R as
scala.collection.convert.Wrappers$SeqWrapper and I haven't found any way to turn them into a list or array in R. Any ideas?
map(e => <someconversion)in scala
sparklyrrepo with a steps to reproduce this issue locally. Thanks Dave!
@JakeRuss I'm trying to connect to remote cassandra using host, port, user name and password
conf <- spark_config()
conf[["spark.cassandra.connection.ssl.enabled"]] = TRUE
conf[["spark.cassandra.connection.host"]] = cassandra_host
conf[["spark.cassandra.connection.port"]] = cassandra_port
conf[["spark.cassandra.auth.username"]] = cassandra_username
conf[["spark.cassandra.auth.password"]] = cassandra_password
config[["sparklyr.defaultPackages"]] <- c("org.apache.hadoop:hadoop-aws:2.7.3", "datastax:spark-cassandra-connector:2.0.0-RC1-s_2.11")
sc <- spark_connect(master = "local", version = "2.2.0", spark_home = spark_path, config = conf)
df <- spark_read_source(
name = "emp",
source = "org.apache.spark.sql.cassandra",
options = list(keyspace = "temp", table = "category_distribution"),
memory = FALSE)
but this is not working. please suggest a solution
Error in force(code) :
Failed while connecting to sparklyr to port (8880) for sessionid (52016): Gateway in localhost:8880 did not respond.
Parameters: --class, sparklyr.Shell, "C:\Users\TarunGupta2\Documents\R\win-library\3.6\sparklyr\java\sparklyr-2.4-2.11.jar", 8880, 52016
---- Output Log ----
/Java/jdk1.8.0_251\bin\java was unexpected at this time.
---- Error Log ----