scala/scala

Discord is now Scala’s main chat platform. Please join us at https://discord.com/invite/scala

Sorixelle

@Sorixelle

What type is films?

RoelofWobben

@RoelofWobben

so a is the acc that holds the answer ?

Sorixelle

@Sorixelle

Oh wait, hang on

Martijn Hoekstra

@martijnhoekstra

try to annotate a and b with the types that you think they have

Sorixelle

@Sorixelle

a isn't the type you think it is

RoelofWobben

@RoelofWobben

Films is a list

Sorixelle

@Sorixelle

Remember, a is your accumulator, and b is the current object

RoelofWobben

@RoelofWobben

aha, that was I already suspected

vijendra singh

@viju0731_twitter

@viju0731_twitter
Batch entry was aborted. Call getNextException to see the cause
how do i call this in scala
any help

getting ths error

how to call getNextException here

Batch Entry 0 insert into PGSQL-Call getNextException to see the cause

this is exact error I am getting

i want to see the real cause why it is failing to update the batch using scala

Sorixelle

@Sorixelle

Hard to help without any context - where are you seeing this error? What are you doing to cause it?

vijendra singh

@viju0731_twitter

i am trying to make call to postgres from scala file

when i am trying to write the data in postgres getting this error

RoelofWobben

@RoelofWobben

Thanks, I think it's time to find out how I can see the type of something in intelij

vijendra singh

@viju0731_twitter

java.sql.BatchUpdateException: Batch entry 0 INSERT INTO

Dominic Egger

@GrafBlutwurst

@RoelofWobben scala is so nice as to put some convinience methods on specific lists. List[Int] has a sum method e.g. List(1,2,3).sum. Though technically I think a foldLeft where you sum and count at the same time is more efficient

Sorixelle

@Sorixelle

@RoelofWobben If you hover over something with Ctrl (or Cmd on macOS), it shows you the type signature

Dominic Egger

@GrafBlutwurst

i think the keyboard shortcut is ctrl+p for showing the signature of a parameter list under the cursor. I'd also recommend rebinding "type info" action in intellij to something better reachable

RoelofWobben

@RoelofWobben

@GrafBlutwurst so I can do something like (films.map(_.idbmRating).sum ?

to count all ratings

Sorixelle

@Sorixelle

That would give you the total of all ratings, yes

sum is equivalent to reduce(_ + _), or fold(0)(_ + _)

Dominic Egger

@GrafBlutwurst

I think .sum etc exits for all T : Numeric

Sorixelle

@Sorixelle

where fold can be either foldLeft or foldRight

Dominic Egger

@GrafBlutwurst

with the ususal caveat on reduce of empty maps :D

ethicalbeginner

@nitishram

@AmirSarvestani i loaded the xml how to parse and get root and chile elements

RoelofWobben

@RoelofWobben

oke, so it looks like to me reduce does the same as folds for non-empty lists

Thanks all

Sorixelle

@Sorixelle

reduce is a fold with the head as the accumulator, and the tail as the list to fold over

ethicalbeginner

@nitishram

which is best method to parse large xml of size more than 2gb in scala/spark

Otto Chrons

@ochrons

a SAX parser?

Dominic Egger

@GrafBlutwurst

@RoelofWobben one small thing I'd prefer list.foldLeft(Option.empty[Int]) over Int.MaxValue because I find it cleaner in the case of an empty list. but that's just opinion

Sorixelle

@Sorixelle

While I'm here, quick question:

def apply(xs: String*) = // bla bla bla
implicit class StringOps(val s: String) extends AnyVal {
  // bla bla bla
}

Is there a way to make the StringOps apply in the definition of the apply method?

Dominic Egger

@GrafBlutwurst

doesn't it in eg xs.map( s => s.blah) ?

Sorixelle

@Sorixelle

I mean in val whatever = Foo(/* here */)

given apply is defined in Foo

ethicalbeginner

@nitishram

@ochrons yes

Sorixelle

@Sorixelle

I could import the implicit class, but I'd rather not have to if possible

because everything in that StringOps is only meant to be used in apply

Dominic Egger

@GrafBlutwurst

I don't think so but I'm not sure

vijendra singh

@viju0731_twitter

Hello everyone getting this error


Caused by: java.sql.BatchUpdateException: Batch entry 0 INSERT INTO foo(C1,C2,C3,C4,C5) VALUES ('bar','2','1','09-13-2018','1747774463','300008') was aborted.  Call getNextException to see the cause.
    at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2762)
    at org.postgresql.core.v3.QueryExecutorImpl$1.handleError(QueryExecutorImpl.java:478)
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1999)
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:421)
    at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2929)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:215)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:277)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:276)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$33.apply(RDD.scala:920)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

Any help

Otto Chrons

@ochrons

as the error messages says, call getNextException to get the cause

vijendra singh

@viju0731_twitter

@ochrons Yes i am trying to catch the getNextException

in scala way

Where communities thrive

People

Repo info

Activity