spark-jobserver/spark-jobserver

REST job server for Apache Spark

itsmesrds

@itsmesrds

No @valan4ik, context is already created with cached rdd in it. I'm sending two rest calls to get result's out of cached data. while sending the second the request , when first request is executing . It throws that exception and context is getting killed.

itsmesrds

@itsmesrds

Hi @valan4ik , @bsikander ,

Is there any way to set the number of executors and executor memory for every job in a pre-created sparkContext ?
As far as i know we can set those parmeters for creating the context. But Is the Same will work for every job request ?
Basically spark parameter for every session in a context

Behroz Sikander

@bsikander

well not really. As soon as the context is created the executors are spawned and they allocate resources.

Your only option is to use Dynamic Resource Allocation from Spark and it should work out of the box with jobserver (I hope).

pgouda89

@pgouda89

Hi @bsikander @valan4ik , We need to set spray.can.server.keystorePW="<keystore password>" to enabled SSL on the spark job server. It is not secure as the env.conf will have the string form of the secure key. Do we have better approach to pass the keystore password? I am using Spark jobserver 0.9.

Rajendra

@rajnitsrinagar1_twitter

I submitted a Spark job on EMR cluster and specified 3 executors with 5 GB memory for each. But it is in accepted state for last 15-20 mins and when I checked available resources then I found that core nodes do not have enough memory available but task nodes have more than 50 GB memory available but still my job is in accepted state

So is it necessary to have required memory available on core nodes to start the spark application instead having resource available on task nodes

Narasimhaporeddy

@Narasimhaporeddy

Hi Can some one please help me in setting up ssl , I am using CDH 5.15.1 with Spark job server version 0.8.0 using spark 2.2 . I have configured environment.conf spray.can.server { keystore = /opt/cloudera/security/pki/server.jks and trustore as /opt/cloudera/security/pki/ca-certs.jks } with the respective passwords. All my cloudera services work fine with ssl/tls enabled with the same settings

can some tell me if there is something wrong which i am doing

@pgouda89 can you please let me know how you have set up ssl

@bsikander can you please help me if there is something else which we need to configure to enable bothe client and server auth

ssl

Narasimhaporeddy

@Narasimhaporeddy

Has anyone enabled ssl both server and client with job-server and CDH ?

Narasimhaporeddy

@Narasimhaporeddy

ERROR erver.HttpServerConnection [] [akka.tcp://JobServer@xxxxxxxxxx:42392/user/IO-HTTP/listener-0/8] - Aborting encrypted connection to /XXXXXXXXX:xxxx due to [SSLHandshakeException:null cert chain] -> [SSLHandshakeException:null cert chain] is the error i face

Narasimhaporeddy

@Narasimhaporeddy

all

i am using combinedDao with hdfs+postgres

and could not make it work. Can some one please help me with the syntax

jobdao = spark.jobserver.io.CombinedDAO

combineddao {
  rootdir = "/tmp/combineddao"
  binarydao {
    class = spark.jobserver.io.HdfsBinaryDAO

dit = "hdfs path"

}

metadatadao {
class = spark.jobserver.io.MetaDataSqlDAO
}

sqldao {

  # Slick database driver, full classpath
  slick-driver = slick.driver.PostgresDriver

  # JDBC driver, full classpath
  jdbc-driver = org.postgresql.Driver

url =

user=

password =

flyway.locations="db/postgresql/migration ---> in the documentation it is given as db/combineddao/postgresql/migration

i see this error from spark job server Caused by: org.postgresql.util.PSQLException: ERROR: relation "BINARIES_CONTENTS" does not exist

and is unable to start

ERROR internal.command.DbMigrate [] [] - Migration of schema "public" to version 0.7.6 failed! Changes successfully rolled back.

Caused by: org.flywaydb.core.internal.dbsupport.FlywaySqlScriptException:

Migration V0_7_6__add_bin_hash_column.sql failed

SQL State : 42P01
Error Code : 0
Message : ERROR: relation "BINARIES_CONTENTS" does not exist
Location : db/postgresql/migration/V0_7_6/V0_7_6add_bin_hash_column.sql (/data01/spark/spark-jobserver/job-server/file:/data01/spark/spark-jobserver/job-server/spark-job-server.jar!/db/postgresql/migration/V0_7_6/V0_7_6add_bin_hash_column.sql)
Line : 4

Narasimhaporeddy

@Narasimhaporeddy

Statement : ALTER TABLE "BINARIES_CONTENTS" ADD COLUMN "BIN_HASH" BYTEA

@bsikander @noorul can you guys please help me with this

0.8.0 sjs version using with spark 2.2 on cdh 5.15.1

Valentina

@valan4ik

@Narasimhaporeddy Hi, I think you may have some documentation/jobserver versions mismatch:
if you use jobserver 0.8.0, please check documentation for this version: https://github.com/spark-jobserver/spark-jobserver/tree/0.8.0

db/combineddao/postgresql/migration was introduced only recently and is not part of 0.8.0 release
I think hdfs+postgres DAO would work only if you use master branch
You may use version 0.10.0 and use hdfs+h2 DAO (https://github.com/spark-jobserver/spark-jobserver/tree/e3c3d3ce9ba81b63608130d3904161c8246fe064)

Narasimhaporeddy

@Narasimhaporeddy

@valan4ik thanks for pointing that out to me. Indeed i was referring to latest documentation. One quick Question does the latest version support spark 2.2.0 ?

Valentina

@valan4ik

@Narasimhaporeddy Some dependencies and queries were updated, I am not sure if it is fully backward compatible. You could give it a try by setting SPARK_VERSION variable in your configuration file :)
I think there is no other way to try new DAOs

Narasimhaporeddy

@Narasimhaporeddy

aww makes sense. @valan4ik thanks for the quick help :)

Narasimhaporeddy

@Narasimhaporeddy

Hey did any one enable kerberos in sjs 0.8.0 , can you please let me know how you configured it

i did try to set up use-as-proxy-user=on in shiro.ini and try to export the keytab as mentioned in some of the knime documentation but it did not work for me.

@valan4ik @bsikander i did not see any specific doc on enabling kerberos in specific for spark job server can you please help me with this

sudhakarkunchala

@sudhakarkunchala

Hello All,Can anybody help me how to use the spark-jobserver in windows and possible pls share any link.

Evan Chan

@velvia

@sudhakarkunchala what issue do you have on Windows? It is JVM based, in theory should work just fine....

Lomesh Agrawal

@Lomesh941

Does SJS compile with Scala 2.12?

Krishna sanjay

@cris7M

how to build spark-job-server without using sbt

calvinzhan

@calvinzhan

I am using Scala 2.12. When I built the program, I couldn't download io.spray jars. Doesn't it mean Scala 2.12 isn't supported yet?

Krishna sanjay

@cris7M

spark job server https enable?

@Lomesh941 have u compiled and got the jar?

calvinzhan

@calvinzhan

I got an error msg, when calling a job: "result": "classPath not found"
}),404,Not Found,Vector((Server,spray-can/1.3.4), (Access-Control-Allow-Origin,*), (Content-Length,59), (Date,Thu, 30 Jul 2020 02:34:10 GMT), (Content-Type,application/json; charset=UTF-8)),List())

Open an issue: spark-jobserver/spark-jobserver#1317
Could anyboy help?

pgouda89

@pgouda89

Hi @valan4ik , Do we support utf-8 spark context name? for example: curl -i -d "" 'http://<host>:8090/contexts/Sparkconext漢字?num-cpu-cores=2&memory-per-node=512M&context-factory=spark.jobserver.context.SessionContextFactory' . I got followign error upon executing the above curl command: HTTP/1.1 400 Bad Request
Server: spray-can/1.3.4
Date: Fri, 31 Jul 2020 00:00:36 GMT
Content-Type: text/plain; charset=UTF-8
Connection: close
Content-Length: 65

Illegal request-target, unexpected character '￦' at position 23-bash-4.2$

kz3r

@kz3r

Hey guys, quick question about the "max-jobs-per-context" setting. I'm running with context-per-jvm but it would be great if I could set different max-jobs limit for each context individually. Checking the usage of this setting in the source files, it only seems related to the "akka.threads" setting in the Spark Application, but I thought those was already deprecated and therefore useless (using Spark 2.3.2). Any ideas on how could I achieve this?

Where communities thrive

People

Repo info

Activity

Migration V0_7_6__add_bin_hash_column.sql failed