These are chat archives for sbt/sbt

13th
Apr 2015
Ali Salim Rashid
@arashi01
Apr 13 2015 04:11
Feared that might be the case! Will have a look at Slick's build and try to find a workaround.
Mhhh, just heard that new versions of SBT intend to remove build.scala? :-/
Dale Wijnand
@dwijnand
Apr 13 2015 10:36
Perhaps it's a bit premature but will sbt-doge be folded into sbt given it fixes underlying issues?
given it's necessary for sbt to build itself, might be good to have out the box, even as an opt-in bundled plugin like maven resolver
Josh Suereth
@jsuereth
Apr 13 2015 11:39
@dwijnand It's not actually needed for sbt to build itself
BUT, yes, the intention is to flush out issues in it, then refold back into core
Is there a term for being your own guinea pig?
sbt's build is currently set up so that @eed3si9n had to go through contortions to get sbt-doge working on the sbt build, and even then the incrmental compiler hackery is a bit odd, still
the basic gist there is: sbt was too clever in its own build for its own good
sorry that was in response to half-way in your answer, then I continued reading
Daniel Capo Sobral
@dcsobral
Apr 13 2015 18:48
@soc Yes and no. They really intend to remove “extends Build”, so all your stuff would be either .sbt, POSO, and plugins.
Sam Halliday
@fommil
Apr 13 2015 21:41
Hi all. I'm looking through with an eye to run my integration tests in several JVMs at the same time. http://www.scala-sbt.org/release/docs/Testing.html is there an easy way to specify how many JVMs to use without having to create a test grouping?
btw @eed3si9n I updated a few tickets about parallel test failures, which will hopefully shed some light on the problem.
eugene yokota
@eed3si9n
Apr 13 2015 21:43
I was just going to post some comments on #1890 here that I need a desktop laminate card to remind me what all the parallel semantics mean when you combine sbt's options and individual test frameworks parallelism
Sam Halliday
@fommil
Apr 13 2015 21:44
heh, yeah I know what you mean
quick question: can ForkedTestGroup be set on its own? I just want to say "use 4 JVMs for the tests" with no special rules on what goes where
eugene yokota
@eed3si9n
Apr 13 2015 21:45
if I understand correctly, there's a magical meaning in parallelExecution in Test that says map my (Junit/specs2/etc) tests to (sbt) tasks
I am not sure what you exactly mean, but parallelism by sbt is controlled using this notion. ForkedTestGroup is one of that tags right? - http://www.scala-sbt.org/0.13/docs/Parallel-Execution.html
Sam Halliday
@fommil
Apr 13 2015 21:49
aah, yes! thank you. The docs for this are confusing in the Testing.html page
concurrentRestrictions in Global := Seq(
  Tags.limit(Tags.ForkedTestGroup, 4)
),
still isn't running more than one JVM
eugene yokota
@eed3si9n
Apr 13 2015 21:52
fork in Test is true?
Sam Halliday
@fommil
Apr 13 2015 21:52
fork in IntegrationTest is true
eugene yokota
@eed3si9n
Apr 13 2015 21:54
there's a suspicious setting called testForkedParallel that's set to false in Defaults.cala
you might have to track down Defaults.scala's logic to see if ForkedTestGroup is in effect
anyway, I have to go to bed. That branch should be a good testground
(I've commented the above because I'm wanting to merge this change and I don't want to forget what I tried)
eugene yokota
@eed3si9n
Apr 13 2015 21:57
thanks for the updates
Sam Halliday
@fommil
Apr 13 2015 21:57
if you have any thoughts on what I'm doing wrong, I'd really appreciate a ping on it.
and also keen to hear about the parallel test failures/false successes
eugene yokota
@eed3si9n
Apr 13 2015 21:58
i might try to reproduce it locally and see what I find
Sam Halliday
@fommil
Apr 13 2015 22:02
thanks! If you want to see the successful failures, uncomment "with ParallelTestExecution" in EnsimeConfigFixture.scala and change the version of ForecastIOLib to 1.5.2. That causes a real integration test failure "it:test" but it is reported as a success by sbt.
(there are probably less convoluted ways of doing this, I admit :smile:)
Josh Suereth
@jsuereth
Apr 13 2015 22:28
@eed3si9n That setting is somethign to remove later, but it's compatibility. Pre-sbt 0.13.2-ish, fork in Test always meant sequential testing. After we implemented parallel tests, we added that setting so you could keep the same semantics
eugene yokota
@eed3si9n
Apr 13 2015 22:30
the ecosystem of parallelism surrounding testing is confusing tho, since test frameworks like specs2 seem to recommend we set it to false
Eric Torreborre
@etorreborre
Apr 13 2015 22:31
where did I recommend anything like that?
eugene yokota
@eed3si9n
Apr 13 2015 22:32

https://etorreborre.github.io/specs2/guide/SPECS2-3.4/org.specs2.guide.Runners.html#arguments

don’t execute the specifications in parallel parallelExecution in Test := false

Josh Suereth
@jsuereth
Apr 13 2015 22:33
yeah, well the reality is that area of our codebase may need more attention
eugene yokota
@eed3si9n
Apr 13 2015 22:33
at some point we decided to use sbt parallelism as a brute force way of running tests in parallel
and then as test frameworks got more sophisticated, we didn't need that on by default
Eric Torreborre
@etorreborre
Apr 13 2015 22:34
Maybe the formulation is not correct then. This is not a recommendation, just something that users might want to define
Josh Suereth
@jsuereth
Apr 13 2015 22:34
We need some work on test framework API, methinks
Eric Torreborre
@etorreborre
Apr 13 2015 22:35
I think that test frameworks can/should do lots of things as long as it stays under the same JVM
eugene yokota
@eed3si9n
Apr 13 2015 22:35
I routinely search for ways to reduce parallelism of unit or acceptance tests during development. we need some easy way to configuring this, and some good docs to turn up the nob
Josh Suereth
@jsuereth
Apr 13 2015 22:36
yeah... right now we only really control paralleism at the task level
test groupings is the mechanism in sbt, and it's not really intuitive
eugene yokota
@eed3si9n
Apr 13 2015 22:37
also #1890 says we always succeed (not sure if this is parallel with one JVM, or forking on this one)
@dcsobral Do you have some information on the POSO bit? My main issue is that I don't want to have files a) in my root directory b) spread over multiple places
that's why I'm going the build.scala + build.sbt + build.properties route
Daniel Capo Sobral
@dcsobral
Apr 13 2015 23:26
@soc POSO means any scala file you put in build/ will be compiled and available. You can use it to define constants such as library dependencies, etc.
But there still has to be a file somewhere which actually uses these constants and drives the build, right?
Daniel Capo Sobral
@dcsobral
Apr 13 2015 23:37
Well, sure. So we have this:
*.sbt: build definitions
project/*.scala: one of:
  1. POSO: basically, Scala libraries used by your build files
  1. Plugins (single use plugins, one off plugins, something like that), which adds stuff requiring plugin-like functionality, such as setting settings.
You can go back on scroll to March 25, and read discussions that day. And, if you do that, let me apologize in advance for my rantings you’ll have to go through.
So, if your goal is to have no *.sbt file, you are screwed. If you want a single sbt file, not per-module, that has been easy to do for a while now.
I prefer not having build files in my root directory ... what are my options?
Daniel Capo Sobral
@dcsobral
Apr 13 2015 23:42
If you want one sbt file per module and common stuff, that’s harder.
I pretty much don't care as long as it's not in my root dir
Daniel Capo Sobral
@dcsobral
Apr 13 2015 23:43
You are pretty much screwed.
argh. will be interesting to see how long I can remain on 0.13. then.
will suck for beginners though.
Josh Suereth
@jsuereth
Apr 13 2015 23:46
@soc probably a long time
Not sure how long the 1.0 transition will take. It's been 20 months since we had a breaking release
and why will this suck for beginners?
those statements need justification
in my experience, the hardest things for beginners is not understanding the build scripts themselves, but understanding which files actually contribute to the build
dumping stuff in the root is the easiest way to make beginners lives much harder
Josh Suereth
@jsuereth
Apr 13 2015 23:48
I disagree
before, I could say "anything that ever matters to your build is in project/"
Josh Suereth
@jsuereth
Apr 13 2015 23:48
build.sbt, pom.xml, build.xml, Makefile
Actually, you couldn't
it worked perfectly fine for me for plenty of projects
Josh Suereth
@jsuereth
Apr 13 2015 23:48
And, from the training courses i've given on sbt specifically, the .sbt is way easier for new users to cope with than .scala
What I'm saying is, someone could have violated that rule easily with a .sbt file somewhere
and having build separate from root is actually not a common thing in build tools, so you're actualyl violating assumptions of most build users (especially newcomers)
unless newcomers have no pre-existing build bias
again, it's not about understanding .sbt vs. .scala
it's about understanding which files are involved at all
scala/scala is a great example for that
Josh Suereth
@jsuereth
Apr 13 2015 23:50
Right, well we hope to simplify that by having 1 rule instead of open-ended confusion
I'm not sure that arguing about what's a common approach regarding build tools is making a good point ...
I think it's hard to find a stronger agreement for "all X are terrible" than for X := build tools
Josh Suereth
@jsuereth
Apr 13 2015 23:54
Actually, it's highly relevant when talking about beginners
Beginners come with bias and expectations
every time you violate it should be for a good reason, and is likely a source of frustration
the broken defaults are one reaso why everything sucks so much
Josh Suereth
@jsuereth
Apr 13 2015 23:54
no project, more than sbt, has taught me this
Right. But only violate if the default is truly broken
The issue with sbt now is the convention of mutliple directories is already in place
and dumping build definitions into the root is one of these things, which make everything harder than necessary
Josh Suereth
@jsuereth
Apr 13 2015 23:55
ANd the meta-meta-meta tunnel is a thing
not sure you saw, but we have a project which starts in project/project/plugins.sbt and tunnels all the way through the meta sbt builds
for instance in scala/scala about half of the files are in subtle ways related to the build. I'm not sure how a beginner is supposed to figure this out
Josh Suereth
@jsuereth
Apr 13 2015 23:56
I don't think I agree with you. I'd agree that scala/scala is subtle and bad
but I don't think that means you throw out having build defintiion in the root directory
for example, see google's "bazel"
there, there's ONE palce for a build file and it's root and it makes a hell of a lot of sense
g2g
don't use scala/scala to draw inference for all builds :)
what I'm saying is that there will always be plenty of stuff which wants to be in the root dir. no amount of "don't do that" will change it.
imho, the best approach is to not fight with this, and have an easy way to tell what's build-related nd what not
how long am I now involved in Scala ? half a decade ... I still have to do trial-and-error to figure out the effects various files have in the root dir.