These are chat archives for sbt/sbt

8th
Feb 2017
Dale Wijnand
@dwijnand
Feb 08 00:00
@pfn feel free to flesh out a proposal in a ticket
Perry
@pfn
Feb 08 00:01
probably sometime when I get fed up with := getting rid of my beloved <<=
might have to come up with some magic to get rid of .value if I got in there
hmm, probably not possible
since it basically converts Key[T] to T
(signature-wise)
pfn @pfn ponders evil implicits
Dale Wijnand
@dwijnand
Feb 08 00:05
it might be possible
Perry
@pfn
Feb 08 00:06
basically @compileTimeOnly implicit def keyTasTT: T = ???
Stephen Judkins
@stephenjudkins
Feb 08 00:18
I have a bit of a mystery here. my SBT session is totally stalling (all pool threads are dead, only main thread is running and waiting on a monitor in sbt.execute.processAll)
whether this happens depends on what I do in a task i've written, but all printlndebugging inside that task appears to be swallowed up
any tips on how to figure out what's going on?
Perry
@pfn
Feb 08 00:19
attach a debugger
and breakpoint on your sbt file
should be able to breakpoint from intellij on the build project
Stephen Judkins
@stephenjudkins
Feb 08 00:20
haha, I am doing that right now. i think it's choking on the macros or something, because the breakpoints are crazy and make no sense whatsoever
Perry
@pfn
Feb 08 00:20
just launch sbt with the jvm remote debug flags and attach to remote process from intellij
Stephen Judkins
@stephenjudkins
Feb 08 00:20
yes. currently attached.
Perry
@pfn
Feb 08 00:20
macros shouldn't mess up line numbers...
Stephen Judkins
@stephenjudkins
Feb 08 00:21
they usually don't... but something is!
Perry
@pfn
Feb 08 00:21
move the implementation of your task out of the x := { ... } block
and instead do x := myTaskDef.taskValue
Stephen Judkins
@stephenjudkins
Feb 08 00:21
already doing that
Perry
@pfn
Feb 08 00:22
and breakpoint inside of def myTaskDef = Def.task { ... } hmm, dunno what's up with the line numbers otherwise
oh, move the def myTaskDef into a build.scala file
perhaps .sbt doesn't generate correct line numbers when compiling
pfn @pfn shrugs
Perry
@pfn
Feb 08 00:23
(I'd imagine it should, still
)
Stephen Judkins
@stephenjudkins
Feb 08 00:23
it's not in an .sbt file, it's in a .scala file in project
Perry
@pfn
Feb 08 00:24
so if you replace your task with a no-op it doesn't cause this hang?
Stephen Judkins
@stephenjudkins
Feb 08 00:24
it appears so
OK. when I set a breakpoint inside the Def.task, the breakpoint is triggered when the task is first evaluated and loaded by SBT... not when it's actually ran
if you understand what I'm saying?
Perry
@pfn
Feb 08 00:26
yeah, I do, in that case, call a method from within the Def.task...
unraveling the onion
and breakpoint on that method
Stephen Judkins
@stephenjudkins
Feb 08 00:29
dear god this is frustrating
OK. if I go and make it (basically) a no-op, it runs...but the println debugging statements are still swallowed. what on earth?
for what it's worth, this task is ran by resourceGenerators in Compile <+= mungeTask and i'm triggering it with compile. I think I'm going to back out and give it its own task key and see if it's easier to figure out what's going on in that context
Perry
@pfn
Feb 08 00:39
resourceGenerators in Compile += mungeTask iirc
since it's supposed to be a List[Task[Seq[File]]
Stephen Judkins
@stephenjudkins
Feb 08 00:42
huh. it does have something to do with that
Perry
@pfn
Feb 08 00:43
the problem is your resourceGenerator is trying to run at build load time, and not at build time
or something like that
the types for re/sourceGenerators confuses me
Stephen Judkins
@stephenjudkins
Feb 08 00:43
our other example that worked similarly uses <+=. I suspect it was only working by accident...
Perry
@pfn
Feb 08 00:44
yeah, I had sourceGenerators working similarly by accident before as well
it still confuses me...
but yeah, it's one of those "follow the types, fool" kinda problems
Stephen Judkins
@stephenjudkins
Feb 08 00:50
right. well, the types didn't save me this time.
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:46
Is there anyway a plugin can internally store state between SBT compilations? For example, let's say I'm using sbt-assembly, and I want to keep state about a specific property and see if it was modified in the current build.sbt. Is that possible?
Dale Wijnand
@dwijnand
Feb 08 13:47
You mean between reloads?
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:48
Not exactly. Lets say I have a field in SBT assembly which is used for shading. I want to be able to see if the shading rules changed between invocations of the assembly command
Now I don't know if SBT creates a new instance for each run of the Assembly object, or if it has a single instance and only creates a fresh instance when it reloads
I think it's a more general question. If I write a plugin for SBT, can I keep state between compilations of the users build.sbt?
Dale Wijnand
@dwijnand
Feb 08 13:51
user build.sbt's are only recompile if you restart sbt or if you reload sbt
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:52
And when SBT is reloaded, all plugins are loaded with it I assume?
Dale Wijnand
@dwijnand
Feb 08 13:52
so, there is only a single instance of Assembly in memory between invocations of the assembly command
yep
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:52
Is there any general way of keeping state between reloads?
Say I want to know things about the last compilation
Dale Wijnand
@dwijnand
Feb 08 13:53
I believe so
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:53
Any "standard" way I mean
Dale Wijnand
@dwijnand
Feb 08 13:54
@pfn was showing me a few weeks ago, I can't remember exactly, but I think he was saying that setting values in the attribute map persist reloads
It's not a common use case, so it's not "standard", but still..
Yuval Itzchakov
@YuvalItzchakov
Feb 08 13:55
@dwijnand I see.
Dale Wijnand
@dwijnand
Feb 08 13:56
I had started a general purpose plugin, to diff the state of the settings between reloads
but I ran into some uninteresting problems so I abandoned it
Ivan Danov
@idanov
Feb 08 16:06
Hi guys, does anyone know how to call sbt new with a private repo for the template? I've tried adding a global ssh resolver, but it seems that it doesn't work... I've also tried browsing through the sbt and sbt-template-resolver source code, but I couldn't really find any settings for using my ssh key when accessing git repos...
OlegYch
@OlegYch
Feb 08 16:08
sbt.SshBasedRepository#as ?
Ivan Danov
@idanov
Feb 08 16:16

@OlegYch Do you mean something like this:

cat ~/.sbt/0.13/global.sbt                                                            1 ↵
val keyFile: File = new File("~/.ssh/id_rsa")
resolvers += Resolver.ssh("Bitbucket", "bitbucket.org") as("username", keyFile)

I've tried adding it, but it doesn't work for some reason....

OlegYch
@OlegYch
Feb 08 16:16
~ is bash feature
or other command processors..
Ivan Danov
@idanov
Feb 08 16:18
@OlegYch I've changed it to full path now, but still got the same error:
sbt new git@bitbucket.org:repo/startup-kit-scala.g8.git
[info] Set current project to workspace (in build file:/Users/username/workspace/)
Template not found for: git@bitbucket.org:repo/startup-kit-scala.g8.git
OlegYch
@OlegYch
Feb 08 16:22
dunno
Ivan Danov
@idanov
Feb 08 16:25
@OlegYch thanks for trying to help!
Maybe I should ask @eed3si9n , since it seems that he is the main contributor for sbt new .... I can only hope that he is on gitter....
Perry
@pfn
Feb 08 16:37
@dwijnand, it's been forever since I showed that to you, months at least :)
@YuvalItzchakov, you can use something like FileFunction.cached to check state between runs
write the state to file, and load it back up
also, you can save the result of an operation into a memory cache or disk state (xxxTask saveAs/storeAs storageKey)
that can subsequently be loaded by ... I forget what
;-)
but anything that has access to state can read it
there's Def(?).loadFromContext(storageKey, resolvedScope, state)
which returns Option[PreviousTaskResult]
so there's several ways to approach the problem
I forget which one saves to disk, probably storeAs
eugene yokota
@eed3si9n
Feb 08 16:52
my name was mentioned
Dale Wijnand
@dwijnand
Feb 08 16:52
I thought storeAs stores in the State
eugene yokota
@eed3si9n
Feb 08 16:53
@idanov private repo support in Giter8 was fixed recently, and I think it should be available in 0.13.14
foundweekends/giter8#248 (The issue was expedited as part of Lightbend subscription)
Perry
@pfn
Feb 08 16:55
@dwijnand, then I probably have it backwards
Ivan Danov
@idanov
Feb 08 17:01
@eed3si9n thanks! So one can only use it only with 0.13.14 and there's no workaround for 0.13.13?
eugene yokota
@eed3si9n
Feb 08 17:02
if you can reverse engineer how the whole thing is wired up you can try the new stuff
but better bet is to wait till 0.13.14-RC1
hopefully coming up soonish
Ivan Danov
@idanov
Feb 08 17:05
@eed3si9n Yes, I believe 0.13.14-RC1 would be my best bet... Thank you very much for your help!
eugene yokota
@eed3si9n
Feb 08 17:06
np
Edmondo Porcu
@edmondo1984
Feb 08 18:03
Hello ,I have a terribly stupid problem which I can't solve. I am trying to synchronize applying migration to my db at the beginning of my tests and I am using a synchronized method on an object.
trait SingletonInitializationDBFactory[A <: SlickFacade] {

  def dbName: String

  private[this] val logger = getLogger

  private val driver = new MySQLDriver {
    override def defaultSqlTypeName(tmd: JdbcType[_],
                                    sym: Option[FieldSymbol]): String = tmd.sqlType match {
      case java.sql.Types.DECIMAL => "DECIMAL(20,7)"
      case _                      => super.defaultSqlTypeName(tmd, sym)
    }
  }

  private var serviceInstance: A = _

  protected def build(profile: slick.driver.JdbcProfile, db: Database): A

  @volatile
  private var alreadyInitialized:Boolean = false

  def getService(config: Config): A = {
    SingletonInitializationDBFactory.synchronized{
      if(!alreadyInitialized){
        val driverParam = config.getString("driver")
        val urlParam = config.getString("url")
        val userParam = config.getString("user")
        val passwordParam = config.getString("password")
        val schemaNameParam = config.getString("schemaName")
        val jdbcOptsParam = if (config.hasPath("opts")) config.getString("opts") else ""

        val jdbcUrl: String = s"${urlParam}${schemaNameParam}$jdbcOptsParam"
        logger.info(s"Connecting to db at $jdbcUrl with username $userParam and password $passwordParam with driver $driverParam")
        val db = Database.forURL(jdbcUrl, userParam, passwordParam, driver = driverParam)
        val session = db.createSession()
        try session.force() finally session.close() // Creating and closing a session to force initializing
        serviceInstance = build(driver, db)

        // Create the Flyway instance
        val flyway = new Flyway
        flyway.setLocations("classpath:db/migrations")
        // Point it to the database
        flyway.setDataSource(jdbcUrl, userParam, passwordParam)
        // Start the migration
        flyway.migrate()

        Thread.sleep(1000)
        alreadyInitialized = true
        serviceInstance

      }
      else{
        serviceInstance
      }
    }

  }

}

object SingletonInitializationDBFactory {
  private val lock = new AnyRef
}
the problem is that although this is synchronized, I see multiple threads going in the critical section in parallel. This happens inside a multi module SBT build so I was wondering if this has to do with SBT 0.13.7 which maybe forks the JVM or with multiple classloader breaks the unicity of the lock
OlegYch
@OlegYch
Feb 08 18:13
flyway locks versions table you know...
so there is no need to synchronize further
Edmondo Porcu
@edmondo1984
Feb 08 18:14
@OlegYch well my logs shows two threads try to perform the migration
and boom
eugene yokota
@eed3si9n
Feb 08 18:14
if you're not specifying to fork the test, tests aren't forked to another JVM
Edmondo Porcu
@edmondo1984
Feb 08 18:14
unit_test_1 | [error] -------------
unit_test_1 | [error] SQL State : 42S01
unit_test_1 | [error] Error Code : 1050
unit_test_1 | [error] Message : Table 'schema_version' already exists
unit_test_1 | [error] Line : 17
unit_test_1 | [error] Statement : CREATE TABLE cr.schema_version (
@eed3si9n seems that sbt could be messing up with singleton and classloaders?
OlegYch
@OlegYch
Feb 08 18:15
oh ok you drop the schema afterwards..
eugene yokota
@eed3si9n
Feb 08 18:15
we do mess with classloaders
sbt 0.13.13 caches them IIRC, so you might want to try that
Edmondo Porcu
@edmondo1984
Feb 08 18:17
thanks
where should I place the sbt.version ?
eugene yokota
@eed3si9n
Feb 08 18:18
project/build.properties
Edmondo Porcu
@edmondo1984
Feb 08 18:18
@OlegYch I don't want to do migration / drop for each of my test, I want the first test to do that and the other wait
project/build.properties and what should I put inside? sbt.version = 0.13.13 ?
eugene yokota
@eed3si9n
Feb 08 18:18
yea
i'm not sure if that's a use case we do support tho. basically two running tasks communicating with each other
it might be better to reduce the parallelism of test to 1
Edmondo Porcu
@edmondo1984
Feb 08 18:39
it doesn't look like I have much choice Eugene
Perry
@pfn
Feb 08 18:45
are the 2 tasks in the same build?
if not, use a common object from a plugin to synchronize on
or any arbitrary object in sbt
if you have the same root-level build, then share the lock object that way
Derek Wickern
@dwickern
Feb 08 19:06
can i configure a task to not cache its value?
Perry
@pfn
Feb 08 19:07
tasks don't cache their value
Derek Wickern
@dwickern
Feb 08 19:09
really? hmm
Perry
@pfn
Feb 08 19:09
unless they're explicitly written to do so, they do not
Derek Wickern
@dwickern
Feb 08 19:09
calling someTask := someTask.value prevents the old definition's value from being garbage collected
Perry
@pfn
Feb 08 19:10
that doesn't "assign the value"
that means someTask is implemented by running someTask
Derek Wickern
@dwickern
Feb 08 19:10
replacing it with someTask := <implementation of old someTask> allows that value to be GC'd
Perry
@pfn
Feb 08 19:10
what is "that value"
someTask defines a Task
of course it can't be "garbage collected"
Derek Wickern
@dwickern
Feb 08 19:11
it's test:testLoader which evalutes to a ClassLoader
Perry
@pfn
Feb 08 19:11
someTask.value is just syntax sugar for unwrapping a Task[T] to T
it doesn't remove the task
Derek Wickern
@dwickern
Feb 08 19:15
i assumed that
testLoader := wrap(testLoader.value)
would be equivalent to
testLoader := wrap(<testLoader's default implementation>)
Perry
@pfn
Feb 08 19:17
wut
I suppose, sure
Derek Wickern
@dwickern
Feb 08 19:19
the first way keeps the classloader gc rooted
maybe a detail of how the macro is implemented
Perry
@pfn
Feb 08 19:19
there's no detail
all ways keeps the "classloader gc rooted"
no idea what you're trying to accomplish
a.value means return the result of evaluating a
a := a.value thus means, reimplement a in terms of evaluating the previous value of a
and a := wrap(a.value) means reimplement a in terms of evaluating the previous value of a and call wrap on that result
no matter what, the previous a is always present
Lars Hupel
@larsrh
Feb 08 19:33
Does anybody still care about sbt/sbt-core-next? There's this really annoying bug in sbt/sbt-core-next#8 which at least affects sbt-pgp
I'd be happy to file one or two PRs with some additional improvements but I'd like to know whether it's worth it :smile:
Dale Wijnand
@dwijnand
Feb 08 19:34
In what context are you hitting that?
Lars Hupel
@larsrh
Feb 08 19:34
see sbt/sbt-pgp#97
this happens because the latest version of sbt-pgp pulls in sbt-core-next
Dale Wijnand
@dwijnand
Feb 08 19:36
OK, let's fix that
Lars Hupel
@larsrh
Feb 08 19:36
see sbt/sbt-pgp#76 for the PR that introduced it
Dale Wijnand
@dwijnand
Feb 08 19:36
If you PR,
Lars Hupel
@larsrh
Feb 08 19:36
so I take it that sbt-core-next is not intended to be used?
Dale Wijnand
@dwijnand
Feb 08 19:37
Not directly. We're scavenging some bits for sbt-server, but I doubt it'll be used as is.
That's a tiny PR.
I'll try and have a look at the reverse PR
Lars Hupel
@larsrh
Feb 08 19:39
fair enough
Do you know about the fate of the background execution stuff?
Otherwise I'm going to pull that out and package it as a plugin
eek, wait a second, I can't do that
there's no license on sbt-core-next
Dale Wijnand
@dwijnand
Feb 08 19:40
Background execution will be in sbt 1
Lars Hupel
@larsrh
Feb 08 19:42
ah, cool
it's in git already, with a license on it ... that means I'm free to steal it :smile:
Dale Wijnand
@dwijnand
Feb 08 19:49
Does pushing to GitHub count as distribution? :)
Lars Hupel
@larsrh
Feb 08 19:51
only if it's FLOSS :stuck_out_tongue_winking_eye:
Perry
@pfn
Feb 08 20:49
when was runsBefore added? it's not in 0.13.8
oh, runBefore
and doesn't seem to do what I want
pfn @pfn sticks to using dependsOn
Yuval Itzchakov
@YuvalItzchakov
Feb 08 21:02
@pfn Is there anywhere I can read on storeAs and the likes?
Or see any sample code that uses them?
Perry
@pfn
Feb 08 21:02
@YuvalItzchakov, it's used extensively through sbt Defaults.scala
mostly for use with inputtask parser generation
but you can adapt it to do whatever you need
Yuval Itzchakov
@YuvalItzchakov
Feb 08 21:03
@pfn Ok, I'll look, thanks.
Dale Wijnand
@dwijnand
Feb 08 22:07
here's how runBefore works, according to my experiments with it: https://github.com/sbt/sbt/pull/2908/files#diff-5203fb1295fd0a937570a18050ea699aR28
Perry
@pfn
Feb 08 22:26
right, there's no guarantee of ordering of dependencies in the graph
when it will run
seems like it only runs immediately before the task, and not its dependencies
Adelbert Chang
@adelbertc
Feb 08 22:39
does anyone know if nested SBT multiprojects work as expected? e.g. one of the subprojects in a multiproject is itself a multiproject
OlegYch
@OlegYch
Feb 08 22:40
do you mean multiple builds?
Edmund Noble
@edmundnoble
Feb 08 22:40
Do you mean project/build.sbt file declaring other projects with project/build.sbt files?
but nested
Perry
@pfn
Feb 08 22:43
not sure how that would work...
Adelbert Chang
@adelbertc
Feb 08 22:43
so each project can declare its own build.sbt file right
Perry
@pfn
Feb 08 22:43
seems weird to projectA/projectB/projectC/task
dunno if that's supported
Adelbert Chang
@adelbertc
Feb 08 22:44
what if that project's build.sbt itself defined others
Perry
@pfn
Feb 08 22:44
no idea, try a quick sample and see
Adelbert Chang
@adelbertc
Feb 08 22:44
hm
uncharted territory
alright
Perry
@pfn
Feb 08 22:44
should be pretty quick to test
Adelbert Chang
@adelbertc
Feb 08 22:44
hold my beer
Edmund Noble
@edmundnoble
Feb 08 22:44
:hand: :beer:
:running:
Perry
@pfn
Feb 08 22:45
[perryn@perryn-mbr sandbox] $ cat build.sbt
lazy val a = project
lazy val b = project
lazy val c = project
[perryn@perryn-mbr sandbox] $ cat b/build.sbt
lazy val z = project
lazy val y = project
lazy val x = project
[perryn@perryn-mbr sandbox] $ sbt projects
[info] Set current project to sandbox (in build file:/Users/perryn/src/sandbox/)
[info] In file:/Users/perryn/src/sandbox/
[info]     a
[info]     b
[info]     c
[info]   * sandbox
[info]     x
[info]     y
[info]     z
seems to work fine, dunno what happens if the collide
looks like b/b is ignored if it collides
[perryn@perryn-mbr sandbox] $ sbt b/baseDirectory x/baseDirectory
[info] Set current project to sandbox (in build file:/Users/perryn/src/sandbox/)
[info] /Users/perryn/src/sandbox/b
[info] /Users/perryn/src/sandbox/b/x
Adelbert Chang
@adelbertc
Feb 08 22:47
youre fast
so it does work
Perry
@pfn
Feb 08 22:48
just don't have nested projects with the same id and it will be fine, it seems
Adelbert Chang
@adelbertc
Feb 08 22:49
going to try this out
thank you kind sir
Dale Wijnand
@dwijnand
Feb 08 22:55
the thing to check is if the root project still aggregates over x/y/z
Edmondo Porcu
@edmondo1984
Feb 08 23:39
what's the difference between the name and the artifact in Exclusion rules in sbt?
Perry
@pfn
Feb 08 23:39
name is the name, artifact is the actual file
people can put whatever they want under the name, by convention, it's name + extension
for the artifact
same as anywhere there's a coordinate
Edmondo Porcu
@edmondo1984
Feb 08 23:40
can I use a wildcard?
Perry
@pfn
Feb 08 23:41
why would you specify an artifact in the exclusion rule
unless you only want to exclude a specific artifact
Edmondo Porcu
@edmondo1984
Feb 08 23:41
(services excludeAll (ExclusionRule("com.mycompang",name = "some-facade-client*"))
Perry
@pfn
Feb 08 23:42
try it and see
Edmondo Porcu
@edmondo1984
Feb 08 23:42
it doesn't work
if I use a whatDependsOn I can't still find the artifact
Perry
@pfn
Feb 08 23:44
no idea what that is
Edmondo Porcu
@edmondo1984
Feb 08 23:46
is there a way to investigate transitive dependencies without show fullClasspath which actually runs a compile?
Perry
@pfn
Feb 08 23:46
dependencyClasspath
you can also render the ivy resolution xml files
Edmondo Porcu
@edmondo1984
Feb 08 23:48
how ?
Perry
@pfn
Feb 08 23:49
by using a browser
and rendering it
or using xslt-proc
xsltproc
depending on what platform you're on
Adelbert Chang
@adelbertc
Feb 08 23:59

im trying to override the compile command on my SBT project to do something after, prior to completion. for update i did something like this:

update := {
  val r = update.value
  /* do stuff here */
  r
}

when I try to do the same with compile

compile := {
  val r = compile.value
  /* do stuff here */
  r
}

I get...

Reference to undefined setting: root/*:compile from root/*:compile .. Did you mean root/compile:compile ?
replacing compile with compile:compile does not work