lihaoyi/mill

A place to chat about mill and ask questions. You can also use our new forum at Github Discussions (https://github.com/lihaoyi/mill/discussions)

Tobias Roeser

@tobiasroeser:matrix.org

[m]

But I'm not familiar with JMH, so I can't tell details

Ichoran

@Ichoran

Yeah. I'll have to judge my level of motivation here. Running it as a second test suite (not JMH, but in-JVM) is so easy that it's tempting to just do that.

Thanks! I'll probably need help with the integration if I decide it's necessary, but I'll run some manually and see whether I think it's critical.

Tobias Roeser

@tobiasroeser:matrix.org

[m]

Sure. I'll try to help. If you can show me how to run it manually, I probably can sketch a prototype

Lorenzo Gabriele

@lolgab

@Ichoran sjsonnet uses jmh and mill for its benchmarks: https://github.com/databricks/sjsonnet/blob/master/bench/src/main/scala/sjsonnet/MainBenchmark.scala
You can take some inspiration there, maybe

Ichoran

@Ichoran

@lolgab - Great, thanks for the pointer!

Hm, the documentation says to run the benchmarks with sbt :/

Lorenzo Gabriele

@lolgab

Oh no :-/ I guess we need mill-jmh

discobaba

@uncleweirdo_twitter

@tobiasroeser:matrix.org I started a compile last night and it was still compiling the same thing over and over when I got back this morning. I guess that problem hasn't found a cure, yet. It was just looping over the top project, no sub-projects, whatever that implies. Seeing as getting you something to build was a bust, I was wondering if your system is such that you could just add pile of logging and put an unadvertised/unsupported version somewhere where a change to .mill-version would pick it up and allow users to help you debug. Just a random thought ...

discobaba

@uncleweirdo_twitter

@tobiasroeser:matrix.org A clue is that as it compiles over and over, it doesn't agree on the number of sources. One time it is 2 Scala sources and 24 Java sources then 4 Scala sources and 180 Java sources and so on. Number of java files should be more like 600. It was working recently. Maybe I can figure out what broke it ...

Tobias Roeser

@tobiasroeser:matrix.org

[m]

You could override the target and print all files out to get a first clue.

discobaba

@uncleweirdo_twitter

Not that I overrode the target, but > mill show reds.allSourceFiles | grep ref | wc -l
[1/1] show
[1/1] show > [2/5] reds.sources
622

David Francoeur

@daddykotex

hi, thanks for everything
we're using mill a lot here and sometimes we write or own plugins (libraries pull in other builds)
we started looking into writing test for those and so far we've had to grab a copy of https://github.com/com-lihaoyi/mill/blob/main/main/test/src/util/TestEvaluator.scala to run our test

would you be interested in a PR that does the following:

extract this class (and part of the requirements (TestUtils, for example)) into a module to be published
remove utest explicit dependency so that the user leveraging TestEvaluator can use it with another test framework

(cross posting from scala#com-lihaoyi)

Tobias Roeser

@tobiasroeser:matrix.org

[m]

Hi @daddykotex ! Great that you enjoy Mill. This sounds interesting.

Out of interest: Have you tried (or are you aware of) mill-integrationtest, a plugin specifically for testing mill plugins? It is more loosely coupled to Mill than TestEvaluator, but on the other side, it allows more integrationtest-like setups and also supports tests against different Mill versions.

Maybe, you can share some test, so we can get a better idea how a test setup and a test case might look like.

David Francoeur

@daddykotex

ahh, let me check mill-integrationtest out
our test is very simple, we have a model that has a task writing files to a path and returning a pathref
our test is just running the test (using the TestEvaluator) and ensuring files were written to disk

Tobias Roeser

@tobiasroeser:matrix.org

[m]

Here is my disclaimer, I wrote mill-integrationtest. ;-)

https://github.com/lefou/mill-integrationtest

David Francoeur

@daddykotex

Here is my disclaimer, I wrote mill-integrationtest. ;-)

I've seen that, that does not bother me :)

Brian Maso

@bmaso

Is there a way to set the repositories used by the $ivy magic import on the command-line, or use environment variables? I would prefer not to use interp.repostiroies() ++= ... or import $repo if at all possible.

3 replies

Brian Maso

@bmaso

There seems to be what I would consider to be an error in mill 0.10.4 -- possibly in earlier version (I haven't checked). Here's what I see:

I've defined the following in a JAR archive that I build locally and add to the local maven repo (local maven coordinate "bmaso::mylib:0.0.1", package bmaso):

package bmaso

import upickle.default.{ReadWriter => RW, macroRW}

case class DatasetFile(filename: String, timestamp: Long, size: Long)

object DatasetFile {
  //...upickle RW so I can use this as the value type of a mill target...
  implicit val rw: RW[DatasetFile] = macroRW
}

I have this very simple build.sc script:

import $repo.`file://<my home dir>/.m2/repository`
import mill._, scalalib._
import $ivy.`bmaso::mylib:0.0.1`, bmaso._

def test = T {
  DatasetFile("test", 1L, 1L)
}

I would expect mill show test to just spit out the JSON representation of that DatasetFile object. Instead I get the following stack trace:

Exception in thread "MillServerActionRunner" java.lang.NoClassDefFoundError: scala/Serializable
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        ...(elided for readability)...
Caused by: java.lang.ClassNotFoundException: scala.Serializable
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at ammonite.runtime.SpecialClassLoader.findClass(ClassLoaders.scala:241)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 80 more

Could it be that using a custom value type loaded from a dynamically defined repo is simply not supported?

Brian Maso

@bmaso

It all works fine if you define the custom case class and its ReadWriter in a jar loaded from the startup set of repos, or defined in the script itself, or in a script imported from a static location (using $file). You just can't load from a JAR loaded from a dynamically-defined repo location

Lorenzo Gabriele

@lolgab

Interesting! Ammonite has special classloader code and it might be causing what you say. Can you open a bug with a minimal reproduction? It might be a solvable problem :)

David Francoeur

@daddykotex

Out of interest: Have you tried (or are you aware of) mill-integrationtest, a plugin specifically for testing mill plugins? It is more loosely coupled to Mill than TestEvaluator, but on the other side, it allows more integrationtest-like setups and also supports tests against different Mill versions.

I gave it a shot, but I got blocked. Our plugin pulls dependencies from an internal artifactory. When the IT test runs, it tries to pull dependencies maven central instead. it ignores the configured repositoriesTask. I tried configuring it in multiple places but I did not succeed (our plugin configures it, I added the configuration to the ittest module I created, I also added the configuration in the test itself)

7 replies

Tobias Roeser

@tobiasroeser:matrix.org

[m]

You can add additional content via perTestResources (see documentation: https://github.com/lefou/mill-integrationtest#configuration-and-targets), which could be a shared.sc which you generate and include in each project under test

3 replies

An application example can be found here: https://github.com/lefou/mill-integrationtest/blob/9aa12663ff67458d890c652813eb75a95e7c8805/build.sc#L144

cc @daddykotex

and how it is used: https://github.com/lefou/mill-integrationtest/blob/9aa12663ff67458d890c652813eb75a95e7c8805/itest/src/01-simple/build.sc#L3

Simon Parten

@Quafadas

I have the feeling that something silly is happening, but I can't see it. Can anyone?

import $ivy.`com.disneystreaming.smithy4s::smithy4s-codegen:0.12.16`

import coursier.maven.MavenRepository
import mill._
import mill.api.PathRef
import mill.scalalib._
import os.Path
import smithy4s.codegen.{Codegen => Smithy4s}
import smithy4s.codegen.CodegenArgs

import scala.util._

trait Smithy4sModule extends ScalaModule {
}

And I get this;

Compiling /Users/simon/Code/mill-full-stack/mill-full-stack/project/smithy4s.sc
/Users/simon/Code/mill-full-stack/mill-full-stack/project/smithy4s.sc:8: value codegen is not a member of object ammonite.$file.project.smithy4s
import smithy4s.codegen.{Codegen => Smithy4s}

IT's successfully downloaded the artefacts, but is claiming that smithy4s.codegen isn't part of them.

Which as far as I can tell, it is!

Would anyone have a hint on where to start?

Lorenzo Gabriele

@lolgab

@Quafadas I think the problem is that the file is called smithy4s which Ammonite uses to create the object smithy4s which wraps your code. You need to either change the imports to import _root_.smithy4s.codegen.{Codegen => Smithy4s} etc. or rename the file to some other name.

Simon Parten

@Quafadas

@lolgab Thankyou... it had the feeling of something silly, and it was exactly this.

Tobias Roeser

@tobiasroeser:matrix.org

[m]

haha, that's probably the most easy to create and hard to debug issue with Mill. Once a co-worker of me placed some scripts into a mill dir, and everthing went crazy.

Simon Parten

@Quafadas

:-)

JulGvoz

@JulGvoz

So I just installed mill, but it's pretty much invisible on my shell

JulGvoz

@JulGvoz

I know that I could just run every command with --color false, but that's very bothersome. Is there some option I can set somewhere?

1 reply

discobaba

@uncleweirdo_twitter

@tobiasroeser:matrix.org A clue on the compiling forever problem: the 9.x versions fail, too, after 0.9.4.

1 reply

Sakib Hadžiavdić

@sake92

What is the easiest way to have cross jvm/js project?
Similar to https://github.com/portable-scala/sbt-crossproject CrossType.Full

3 replies

Should I just override sources to add shared folder too?

jilen

@jilen

Hi, how can I avoid fully rebuild while only updating a single dependency version in build file ?

Tobias Roeser

@tobiasroeser:matrix.org

[m]

As long as you need to change the build.sc, you can't avoid it.

Li Haoyi

@lihaoyi

the Zinc incremental compiler helps sometimes, but yeah Mill isn't smart enough to do fine grained invalidation

jilen

@jilen

Does mill support remote cache ? I want to run a CI job to update dependency in the midnight, and others can reuse that build cache the next day.

Tobias Roeser

@tobiasroeser:matrix.org

[m]

Mill uses coursier to cache Maven/Ivy dependencies, so yes, you can cache the coursier specific cache/storage directory and restore before each CI-run, to avoid unnecessary downloads.

But compiler output and everything under ./out does not support distribution

Where communities thrive

People

Repo info

Activity