These are chat archives for sbt/sbt

3rd
May 2017
Adelbert Chang
@adelbertc
May 03 07:31
(i have confirmed -target will override it and i assume it defaults to the JVM version used to launch SBT)
Alessandro Vermeulen
@spockz
May 03 08:46

When I run sbt on our project it is stuck on

[info] Resolving key references (10593 settings)

Any ideas on how to fix this? I removed project/target already

Jorge
@jvican
May 03 09:33
Wow, that's weird.
Did that happen when adding a plugin or modifying the build?
Alessandro Vermeulen
@spockz
May 03 09:36
@jvican I can’t reasonably pin point what triggered it. I only created a new val, but reverting that causes the same issue again
Jorge
@jvican
May 03 09:36
Is that an open source project?
Alessandro Vermeulen
@spockz
May 03 09:36
No unfortunately not :(
I cleaned my ivy directory and all target directories and now it is stuck at loading project definition...
Jorge
@jvican
May 03 09:37
@spockz Give me a way to reproduce the error (minimal example) and I will fix it.
Which sbt version are you using?
Alessandro Vermeulen
@spockz
May 03 09:37
@jvican 0.13.15
Jorge
@jvican
May 03 09:38
Use 0.13.13 instead
And tell me if it works
Alessandro Vermeulen
@spockz
May 03 09:40
@jvican still stuck at loading project again
[info] Loading project definition from /Users/alessandrovermeulen/sources/*redacted*/project
[debug] Running task... Cancel: Null, check cycles: false, forcegc: true
Jorge
@jvican
May 03 11:01
I've noticed using -J-server as a sbt flag makes a difference. Are we already recommending its use @dwijnand @eed3si9n ?
Running sbt -J-server clean compile runs about 15% faster than the version without the server mode in my computer. This is only for startup times, but perhaps we should consider recommending its use.
Dale Wijnand
@dwijnand
May 03 11:21
That's interesting. No I don't think we've ever recommended it. More generally, in the Scala community, can you find references in projects recommending it?
Jorge
@jvican
May 03 11:21
No, though it's true that I don't usually see recommendations to the users either.
Dale Wijnand
@dwijnand
May 03 11:22
Right, I'm thinking: should scalac be run with -server?
Jorge
@jvican
May 03 11:22
It makes sense that it has faster startup for long-running processes.
It depends how big your project is, I believe. But in big projects you may see a boost in performance.
I'll experiment by compiling scalac with -server and see what happens. Will do the same with sbt, hehe.
Dale Wijnand
@dwijnand
May 03 11:23
what's the default? There's both a -server and -client - what's the default behaviour?
Jorge
@jvican
May 03 11:24
Oh, now that you mention I think JDK8 has -server by default.
But the rest of the JVMs don't. Let me check it.
Dale Wijnand
@dwijnand
May 03 11:24
so why are you seeing a difference?
Jorge
@jvican
May 03 11:26
My bad, -server is not enabled by default. ;) I got confused with another flag.

From the official docs:

The JDK includes two flavors of the VM -- a client-side offering, and a VM tuned for server applications. These two solutions share the Java HotSpot runtime environment code base, but use different compilers that are suited to the distinctly unique performance characteristics of clients and servers. These differences include the compilation inlining policy and heap defaults.

Although the Server and the Client VMs are similar, the Server VM has been specially tuned to maximize peak operating speed. It is intended for executing long-running server applications, which need the fastest possible operating speed more than a fast start-up time or smaller runtime memory footprint.

The Server VM contains an advanced adaptive compiler that supports many of the same types of optimizations performed by optimizing C++ compilers, as well as some optimizations that cannot be done by traditional compilers, such as aggressive inlining across virtual method invocations. This is a competitive and performance advantage over static compilers. Adaptive optimization technology is very flexible in its approach, and typically outperforms even advanced static analysis and compilation techniques.

Dale Wijnand
@dwijnand
May 03 11:31
I wish it included detail about what the default behaviour is
Jorge
@jvican
May 03 11:31
Default behaviour is client (C1).
Then there's C1 + heuristics/optimizations.
And then C2 (server).
In theory, the startup with -server should be slower, but I haven't noticed that. The 15% speedup appears when I'm cleaning and compiling in the same sbt process.
I see speedup even if I don't clean and compile, and this time is ~2x.
jvican in /data/rw/code/scala/sbt                                     [13:34:31] 
> $ time sbt exit                                     [±parallel-scripted-tests]
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/sbt/project
[info] Set current project to sbtRoot (in build file:/data/rw/code/scala/sbt/)
/home/jvican/bin/sbt exit  40.11s user 0.58s system 304% cpu 13.347 total
jvican in /data/rw/code/scala/sbt                                    [13:34:08] 
> $ time sbt -J-server exit                          [±parallel-scripted-tests]
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/sbt/project
[info] Set current project to sbtRoot (in build file:/data/rw/code/scala/sbt/)
/home/jvican/bin/sbt -J-server exit  20.66s user 0.36s system 274% cpu 7.650 total
Jorge
@jvican
May 03 11:37
It would be interesting if you can reproduce this @dwijnand
Dale Wijnand
@dwijnand
May 03 11:38
no I can't
12:37:18 ! time sbt exit
[info] Loading global plugins from /Users/dnw/.dotfiles/.sbt/0.13/plugins
[info] Loading project definition from /d/sbt/project
[info] Resolving key references (10019 settings) ...
[info] Set current project to sbtRoot (in build file:/d/sbt/)

real    0m13.687s
user    0m24.727s
sys    0m1.412s

12:37:38 $ time sbt -J-server exit
[info] Loading global plugins from /Users/dnw/.dotfiles/.sbt/0.13/plugins
[info] Loading project definition from /d/sbt/project
[info] Resolving key references (10019 settings) ...
[info] Set current project to sbtRoot (in build file:/d/sbt/)

real    0m13.238s
user    0m24.501s
sys    0m1.301s
$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
my java -help says:
    -server      to select the "server" VM
                  The default VM is server,
                  because you are running on a server-class machine.
so that's why it makes 0 difference
on java 1.6 that's not true:
    -client      to select the "client" VM
    -server      to select the "server" VM
    -jvm      is a synonym for the "client" VM  [deprecated]
    -hotspot      is a synonym for the "client" VM  [deprecated]
                  The default VM is client.
Jorge
@jvican
May 03 11:40
> $ java -version
openjdk version "1.8.0_92"
OpenJDK Runtime Environment (build 1.8.0_92-b14)
OpenJDK 64-Bit Server VM (build 25.92-b14, mixed mode)
So that may be a difference between hotspot and OpenJDK.
Dale Wijnand
@dwijnand
May 03 11:41
anything interesting in java -help?
Jorge
@jvican
May 03 11:42
Wait, I have the same as you:
    -server      to select the "server" VM
                  The default VM is server,
                  because you are running on a server-class machine.
Martijn Hoekstra
@martijnhoekstra
May 03 11:42
I thought everybody(tm)(r) was running server-class machines ATM - that is: all 64bit machines, and all 32 bit machines with at least 2 GB memory
Jorge
@jvican
May 03 11:42
This makes no sense :(
Dale Wijnand
@dwijnand
May 03 11:42
maybe openjdk's -help is bullshit?
clearly something is a lot different under -server so that seems like the easiest explanation
why don't you try oracle's jdk see if it's better for you
Jorge
@jvican
May 03 11:44
Okay, I found the issue.
I thought I was comparing sbt faithfully, but I wasn't. For some reason, the first time sbt exit was doing something under the hood. Running it again was much faster because that computation was no longer performed.
jvican in /data/rw/code/scala/sbt                                     [13:34:51] 
> $ time sbt exit                                     [±parallel-scripted-tests]
[info] Loading global plugins from /home/jvican/.sbt/0.13/plugins
[info] Loading project definition from /data/rw/code/scala/sbt/project
[info] Set current project to sbtRoot (in build file:/data/rw/code/scala/sbt/)
/home/jvican/bin/sbt exit  20.67s user 0.49s system 276% cpu 7.661 total
That's the second run.
What I wonder is, what was sbt doing under the hood? There was no compilation going on, all the class files of the project are already compiled...
Dale Wijnand
@dwijnand
May 03 11:45
./shrug
G. Richard Bellamy
@rbellamy
May 03 15:23
If anyone has an example where the flyway SBT plugin tasks are wired up to run before a slick codegen task, which in turn are wired up to run before tests (either test or it), and compile, I would love to see it...
that's an example build.sbt for what I'm trying to do...
two database connections - one for normal compilation (sbt run), and one for test (sbt test) -- I and my team have spent a ridiculous amount of time on something that should just work, and it's still not even half baked
What I want to happen is:
  1. sbt test -> flywayClean in Test + flywayMigrate in Test + genTables in Test + test
  2. sbt compile -> flywayMigrate + genTables + compile
  3. sbt run -> sbt compile + run mainClass
G. Richard Bellamy
@rbellamy
May 03 15:38
What currently happens is:
  1. sbt test -> genTables in Compile + genTables in Test + test
seriously, how can I run slick codegen targeting two different data sources, one for Compile and one for Test
I know I'm sounding petulant, I'm just so so so frustrated
Alessandro Vermeulen
@spockz
May 03 19:27
dbConf in Test := {…}
and dbConf in Compile :=
Oh that looks like what you have, sorry
@rbellamy what are you trying to do? Have it generate two different codebases from two different configs when running a single command sbt test?
G. Richard Bellamy
@rbellamy
May 03 21:45
take a look at the three requirements I listed
@spockz ---^
basically, when a developer is actually developing the application, they'll be mostly targeting the testing database (e.g. test.conf). Other developers would need to have the migrations and codegen occur targeting their local database
so, test.conf points to localhost:5434 and application.conf points to localhost:5432 (default)
the language I'm using to describe it is: a test DB and a runtime DB
G. Richard Bellamy
@rbellamy
May 03 21:50
one thing in particular - the test DB runs PGDATA on top of a tmpfs mount - this gives me almost the same performance as H2 in-memory, but I actually get PostgreSQL syntax (H2 basically lies about it's compatibility with PostgreSQL)
anyway, so, this whole things is so ridiculously difficult
in the last six months I've come to love SBT - but this kind of thing makes me almost cry with frustration
@eed3si9n @dwijnand I've resisted @mentioning you folks since I know you're super busy - but I could really really use your guidance...
Just ping me if you have a moment...
@spockz thanks for looking!
Guillaume Martres
@smarter
May 03 22:19
I'm super confused by how sbt source watching is supposed to work
I have a subproject with ts files, I put includeFilter in unmanagedSources := NothingFilter | "*.ts" | "**.json", in it and checked that subproject/watchSources contains the files I want it to contain, but ~foo does not rerun the command when I change anything in that project
I had to add watchSources in Global ++= (unmanagedSources in Compile).value to my subproject
and then it started working
Dan Di Spaltro
@dispalt
May 03 23:10
if I have a task that's called twice in a single compilation unit, how would I figure out what's causing it?
or how would that even happen?
ritschwumm
@ritschwumm
May 03 23:28
inspect, maybe?
Dan Di Spaltro
@dispalt
May 03 23:53
yeah I've tried that, I think it's play + sbt-web-scalajs doing something weird