by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    legopiraat
    @legopiraat
    Currently i have the use case to copy over files to a newly created tmp folder. But i don't want to copy the files on a 1 to 1 bases. I would like to ignore some directories but i can't find a good way to do this.
    Like match something based on a regex and exclude those. Is there a way to do this?
    legopiraat
    @legopiraat
    for example:
      val tmpDir: File = File.newTemporaryDirectory("tmp-")
      val path: File = config.baseDir
      path.copyTo(tmpDir, exclude = "\\bgit\\b")
    Pathikrit Bhowmick
    @pathikrit
    @legopiraat : that is too much of a specific use case I think. What would probably help is to have copyTo have a filter File => Boolean option to accept or reject. But, then you can see that so many other APIs would deserve that filter and not just copyTo and now we made the interface of better-files crazy because of your one use case.
    legopiraat
    @legopiraat
    The reason i asked for this is because am unable to delete some files after i copy them to the tmp folder
    Pathikrit Bhowmick
    @pathikrit
    what you can do is path.listRecursively.filterNot(_.name.matches(regex)).foreach(f => f.copyTo(tmpDir / (path.relativize(f).path)))
    legopiraat
    @legopiraat
    oow! that's an excellent idea! Didn't even think of that :)
    Jason Shin
    @JasonShin
    Hey guys
    package services
    import javax.inject._
    import sun.misc.{ BASE64Encoder, BASE64Decoder }
    import better.files.{File => ScalaFile}
    import java.io.{ FileInputStream, FileOutputStream }
    import org.apache.commons.codec.binary.Base64
    import com.google.common.io.CharStreams
    import java.io.{ InputStream, InputStreamReader }
    import com.google.common.base.Charsets
    
    /**
      * FileEncoder.scala
      *
      * Context:
      * The service aim to provide easy-to-use file R/W operations throughout Stormwind.io code base.
      * It will have utilities such as getActionAsBase64
      */
    class FileSystem () {
      /**
        * Map ScalaFile to File for test mocking
        */
      val File = ScalaFile
      /**
        * Read file as a string
        *
        * @example
        * val fs = new FileSystem
        * fs.readFileAsString("~/tmp/test.txt")
        * // returns a base64 representation of the file
        *
        * @param path
        * @return
        * @sideEffect
        */
      def readFileAsString(path: String): String = {
        // Reading the file as a FileInputStream
        val fileInputStream = new FileInputStream(File(path).toJava)
        // TODO: Is there a way to read as Bytes directly?
        val str = CharStreams.toString(new InputStreamReader(fileInputStream, Charsets.UTF_8))
        return str
      }
    
      /**
        * Zips an action if a zip file under the action directory doesn't already exist
        * Also you can forcefully zip the task, which deletes already existing ones
        *
        * @example
        * fs.zipDirIfNotExist("github", "triggers", "list_webhooks")
        * // creates a zip file under tasks/github/list_webhooks/list_webhooks.zip
        *
        * @param appName
        * @param taskType
        * @param taskName
        * @param force - Forcefully delete the existing zip file
        */
      def zipTaskIfNotExist(
        appName: String,
        taskType: String,
        taskName: String,
        force: Boolean = true
      ): Boolean = {
        val pwd = System.getProperty("user.dir")
        val dirPath = s"${pwd}/tasks/${appName}/${taskType}/${taskName}"
        val zipPath = s"${dirPath}/${taskName}.zip"
        val dir = File(dirPath)
        val zipFile = File(zipPath)
        if (force && zipFile.exists()) {
          zipFile.delete()
        }
    
        if (!zipFile.exists()) {
          dir.zipTo(zipFile.path)
        }
        true
      }
    
      /**
        * Get the zip folder of action as Base64.
        * This is typically need to interact with the OpenWhisk API for creating actions
        * by passing in the base64 representation of the ZIP file.
        *
        * @example
        * val fs = new FileSystem
        * // Execution of the code below return a base64 representation of the task
        * val result = fs.getActionAsBase64("github", "triggers", "list_webhooks")
        *
        * @param appName - application name such as Github
        * @param taskType - taskType can be either actions or triggers
        * @param taskName - name of the task such as create_issue
        * @return
        */
      def getActionAsBase64(
        appName: String,
        taskType: String,
        taskName: String,
        // Use the default method if possible
        readFileAsString: (String) => String = this.readFileAsString
      ): String = {
        val pwd = System.getProperty("user.dir")
        val dirPath = s"${pwd}/tasks/${appName}/${taskType}/${taskName}"
        val zipPath = s"${dirPath}/${taskName}.zip"
    
        val content: String = readFileAsString(zipPath)
        // Encoding the file using Base64encoder
        val encoded = new BASE64Encoder().encode(content.getBytes(Charsets.UTF_8))
        return encoded.toString
      }
    }
    I have above class in my source code
    and I am trying to write unit test for zipTaskIfNotExist; how would you mock ScalaFile or how would you write a unit test for it?
    I'm currently apporaching it like
    
    class FakeFile(val path: String) {
      def exists(): Boolean = {
        if (path.endsWith(".zip")) {
          return true
        } else {
          return true
        }
      }
      def delete(): Unit = {}
    }
    
    /**
      * Unit tests can run without a full Play application.
      */
    class FileSystemSpec extends PlaySpec {
    
        "zipTaskIfNotExist" should {
          "return true without errors" in {
    
            val fs = new FileSystem {
              override val File = FakeFile
            }
    
            val result = fs.zipTaskIfNotExist("github", "hello", "zz")
            result must equal(true)
          }
        }
    
      }
    }
    But the compiler is complaining
    [error] /home/jason/Desktop/stormwind.io/test/unit/services/FileSystemSpec.scala:53:31: not found: value FakeFile
    [error]           override val File = FakeFile
    [error]                               ^
    [error] one error found
    Pathikrit Bhowmick
    @pathikrit
    i don't understand what is FakeFile either
    Pathikrit Bhowmick
    @pathikrit
    /all better-files is now released for Scala 2.13.0-M5
    moritz bust
    @busti
    Is there an operator that creates a file and then writes to it?
    moritz bust
    @busti
    Also, I would like to clear the contents of a folder and then Immediately write to it. However, this causes an AccessDeniedError to be thrown when writing contents to the, now empty, folder.
    Pathikrit Bhowmick
    @pathikrit

    Is there an operator that creates a file and then writes to it?

    Yes, file.createIfNotExits().write("hello")

    AccessDeniedError means you don't have permission to write to that folder. This is not coming from better-file but from the JDK std lib itself (in the case of I/O from the OS itself)
    moritz bust
    @busti
    Yeah, the thing is, that the first .delete call is successful in deleting the folder but I cannot create any new folder / files in that location until I have restarted the jvm.
    moritz bust
    @busti
    When I recursively delete the files using java.nio it works perfectly fine.
    def deleteAll(file: java.io.File) {
      val contents = file.listFiles()
      if (contents != null) {
        for (f <- contents) {
          FileUtil.deleteAll(f)
        }
      }
     file.delete()
    }
    Pathikrit Bhowmick
    @pathikrit
    Can you paste me your better-files code that does not work and the accompanying NIO code that works? They should do the same thing
    Kunal Kapoor
    @kunalkapoor
    Hey guys I'm new here. I've been looking for a good File IO library for Scala for a long time and I came across this recently. It looks very attractive.
    I also stumbled across https://contributors.scala-lang.org/t/adding-better-files-to-the-scala-platform/340/31 which mentions that better-files could be included in the Scala library itself. However, there is no response since Feb 2017 so I was curious as to the status of that. We all know Scala desperately needs a good IO library, and I'm wondering whether better-files is still on the road to becoming the standard Scala IO utility.
    Pathikrit Bhowmick
    @pathikrit
    There is no plan for better-files becoming part of Scala standard library. Honestly it is better this way since libraries can release improvements/features faster than Scala standard release cadence
    Kunal Kapoor
    @kunalkapoor
    Pathikrit Bhowmick
    @pathikrit
    Yes
    /rgb
    @RalfBarkow_twitter
    Have to add a mimetpye file as the first entry in a ZIP file and it must be stored uncompressed (“Stored” mode). Do I have to specify the compressionLevel and if yes, which one? compressionLevel = 0 ?
    /rgb
    @RalfBarkow_twitter
    Does better-files supports the "compression method", STORED/NO_COMPRESSION=0 ?
    Pathikrit Bhowmick
    @pathikrit
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit. 7-Zip shows "Deflate" though with this code:
    val zipFile: File = file.zipTo("test.zip".toFile, compressionLevel = 0)
    Pathikrit Bhowmick
    @pathikrit
    yeah but don't use Int (0), you can use Deflater.DEFAULT_COMPRESSION for clarity
    /rgb
    @RalfBarkow_twitter
    Yes, thank you very much, it's really much easier to read and understand. I use 'compressionLevel = Deflater.NO_COMPRESSION' now but 7-Zip still shows "Deflate"!?
    /rgb
    @RalfBarkow_twitter
    The issue seems to be comparable to dotnet/corefx#29735 : >Currently when a ZipArchive is created and an entry is added with the CompressionLevel set to "no-compression" the file inside the archive is still wrapped in a deflate stream. The deflate stream correctly does not to compress the file.
    Pathikrit Bhowmick
    @pathikrit
    But is this a bug in better-file? It seems its a bug in the unzipper ... unzip works just fine from command line..
    /rgb
    @RalfBarkow_twitter
    unzip -lv test.zip shows the method as Defl:N and not as Stored, so it might be a bug in better-file. (Implicits.scala, line 258: if (level == Deflater.NO_COMPRESSION) out.setMethod(ZipOutputStream.DEFLATED) seems to wrap the file inside the archive in a deflate stream.
    Pathikrit Bhowmick
    @pathikrit
    @RalfBarkow_twitter : That code was added here for some reason - https://github.com/pathikrit/better-files/pull/38/files
    Can you send a pull request perhaps or atleast file an issue with failing test case?
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit, will try to write a failing test case and file an issue then but unfortunately I won't be able to do it in the near future (next week is carnival).
    Pathikrit Bhowmick
    @pathikrit
    A failing test or same is enough and I can fix it
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit, my problem seems to be that I don't know how to specify the compression method for an individual ZIP file entry, and set it to STORED. (Btw I'm working on that "failing test or same".)
    Pathikrit Bhowmick
    @pathikrit
    Thanks!
    Just put what your "ideal API" would look like and I can work backwords from there
    /rgb
    @RalfBarkow_twitter
    How about val zipFile: File = file.zipTo("test.iirds").toFile.stored ?
    Pathikrit Bhowmick
    @pathikrit
    What does .stored mean?
    /rgb
    @RalfBarkow_twitter
    It should set the compression method to STORED.
    /rgb
    @RalfBarkow_twitter
    val zipFile: File = file.zipTo("test.iirds".toFile).setCompressionMethod(STORED)
    /rgb
    @RalfBarkow_twitter
    According to https://en.wikipedia.org/wiki/Zip_(file_format)#Compression_methods , the .ZIP File Format Specification documents the following compression methods: Store (no compression), ...
    I would like to use .store to specify the compression method for an individual ZIP file entry, and set it to STORED.
    /rgb
    @RalfBarkow_twitter