by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jason Shin
    @JasonShin
    Hey guys
    package services
    import javax.inject._
    import sun.misc.{ BASE64Encoder, BASE64Decoder }
    import better.files.{File => ScalaFile}
    import java.io.{ FileInputStream, FileOutputStream }
    import org.apache.commons.codec.binary.Base64
    import com.google.common.io.CharStreams
    import java.io.{ InputStream, InputStreamReader }
    import com.google.common.base.Charsets
    
    /**
      * FileEncoder.scala
      *
      * Context:
      * The service aim to provide easy-to-use file R/W operations throughout Stormwind.io code base.
      * It will have utilities such as getActionAsBase64
      */
    class FileSystem () {
      /**
        * Map ScalaFile to File for test mocking
        */
      val File = ScalaFile
      /**
        * Read file as a string
        *
        * @example
        * val fs = new FileSystem
        * fs.readFileAsString("~/tmp/test.txt")
        * // returns a base64 representation of the file
        *
        * @param path
        * @return
        * @sideEffect
        */
      def readFileAsString(path: String): String = {
        // Reading the file as a FileInputStream
        val fileInputStream = new FileInputStream(File(path).toJava)
        // TODO: Is there a way to read as Bytes directly?
        val str = CharStreams.toString(new InputStreamReader(fileInputStream, Charsets.UTF_8))
        return str
      }
    
      /**
        * Zips an action if a zip file under the action directory doesn't already exist
        * Also you can forcefully zip the task, which deletes already existing ones
        *
        * @example
        * fs.zipDirIfNotExist("github", "triggers", "list_webhooks")
        * // creates a zip file under tasks/github/list_webhooks/list_webhooks.zip
        *
        * @param appName
        * @param taskType
        * @param taskName
        * @param force - Forcefully delete the existing zip file
        */
      def zipTaskIfNotExist(
        appName: String,
        taskType: String,
        taskName: String,
        force: Boolean = true
      ): Boolean = {
        val pwd = System.getProperty("user.dir")
        val dirPath = s"${pwd}/tasks/${appName}/${taskType}/${taskName}"
        val zipPath = s"${dirPath}/${taskName}.zip"
        val dir = File(dirPath)
        val zipFile = File(zipPath)
        if (force && zipFile.exists()) {
          zipFile.delete()
        }
    
        if (!zipFile.exists()) {
          dir.zipTo(zipFile.path)
        }
        true
      }
    
      /**
        * Get the zip folder of action as Base64.
        * This is typically need to interact with the OpenWhisk API for creating actions
        * by passing in the base64 representation of the ZIP file.
        *
        * @example
        * val fs = new FileSystem
        * // Execution of the code below return a base64 representation of the task
        * val result = fs.getActionAsBase64("github", "triggers", "list_webhooks")
        *
        * @param appName - application name such as Github
        * @param taskType - taskType can be either actions or triggers
        * @param taskName - name of the task such as create_issue
        * @return
        */
      def getActionAsBase64(
        appName: String,
        taskType: String,
        taskName: String,
        // Use the default method if possible
        readFileAsString: (String) => String = this.readFileAsString
      ): String = {
        val pwd = System.getProperty("user.dir")
        val dirPath = s"${pwd}/tasks/${appName}/${taskType}/${taskName}"
        val zipPath = s"${dirPath}/${taskName}.zip"
    
        val content: String = readFileAsString(zipPath)
        // Encoding the file using Base64encoder
        val encoded = new BASE64Encoder().encode(content.getBytes(Charsets.UTF_8))
        return encoded.toString
      }
    }
    I have above class in my source code
    and I am trying to write unit test for zipTaskIfNotExist; how would you mock ScalaFile or how would you write a unit test for it?
    I'm currently apporaching it like
    
    class FakeFile(val path: String) {
      def exists(): Boolean = {
        if (path.endsWith(".zip")) {
          return true
        } else {
          return true
        }
      }
      def delete(): Unit = {}
    }
    
    /**
      * Unit tests can run without a full Play application.
      */
    class FileSystemSpec extends PlaySpec {
    
        "zipTaskIfNotExist" should {
          "return true without errors" in {
    
            val fs = new FileSystem {
              override val File = FakeFile
            }
    
            val result = fs.zipTaskIfNotExist("github", "hello", "zz")
            result must equal(true)
          }
        }
    
      }
    }
    But the compiler is complaining
    [error] /home/jason/Desktop/stormwind.io/test/unit/services/FileSystemSpec.scala:53:31: not found: value FakeFile
    [error]           override val File = FakeFile
    [error]                               ^
    [error] one error found
    Pathikrit Bhowmick
    @pathikrit
    i don't understand what is FakeFile either
    Pathikrit Bhowmick
    @pathikrit
    /all better-files is now released for Scala 2.13.0-M5
    moritz bust
    @busti
    Is there an operator that creates a file and then writes to it?
    moritz bust
    @busti
    Also, I would like to clear the contents of a folder and then Immediately write to it. However, this causes an AccessDeniedError to be thrown when writing contents to the, now empty, folder.
    Pathikrit Bhowmick
    @pathikrit

    Is there an operator that creates a file and then writes to it?

    Yes, file.createIfNotExits().write("hello")

    AccessDeniedError means you don't have permission to write to that folder. This is not coming from better-file but from the JDK std lib itself (in the case of I/O from the OS itself)
    moritz bust
    @busti
    Yeah, the thing is, that the first .delete call is successful in deleting the folder but I cannot create any new folder / files in that location until I have restarted the jvm.
    moritz bust
    @busti
    When I recursively delete the files using java.nio it works perfectly fine.
    def deleteAll(file: java.io.File) {
      val contents = file.listFiles()
      if (contents != null) {
        for (f <- contents) {
          FileUtil.deleteAll(f)
        }
      }
     file.delete()
    }
    Pathikrit Bhowmick
    @pathikrit
    Can you paste me your better-files code that does not work and the accompanying NIO code that works? They should do the same thing
    Kunal Kapoor
    @kunalkapoor
    Hey guys I'm new here. I've been looking for a good File IO library for Scala for a long time and I came across this recently. It looks very attractive.
    I also stumbled across https://contributors.scala-lang.org/t/adding-better-files-to-the-scala-platform/340/31 which mentions that better-files could be included in the Scala library itself. However, there is no response since Feb 2017 so I was curious as to the status of that. We all know Scala desperately needs a good IO library, and I'm wondering whether better-files is still on the road to becoming the standard Scala IO utility.
    Pathikrit Bhowmick
    @pathikrit
    There is no plan for better-files becoming part of Scala standard library. Honestly it is better this way since libraries can release improvements/features faster than Scala standard release cadence
    Kunal Kapoor
    @kunalkapoor
    Pathikrit Bhowmick
    @pathikrit
    Yes
    /rgb
    @RalfBarkow_twitter
    Have to add a mimetpye file as the first entry in a ZIP file and it must be stored uncompressed (“Stored” mode). Do I have to specify the compressionLevel and if yes, which one? compressionLevel = 0 ?
    /rgb
    @RalfBarkow_twitter
    Does better-files supports the "compression method", STORED/NO_COMPRESSION=0 ?
    Pathikrit Bhowmick
    @pathikrit
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit. 7-Zip shows "Deflate" though with this code:
    val zipFile: File = file.zipTo("test.zip".toFile, compressionLevel = 0)
    Pathikrit Bhowmick
    @pathikrit
    yeah but don't use Int (0), you can use Deflater.DEFAULT_COMPRESSION for clarity
    /rgb
    @RalfBarkow_twitter
    Yes, thank you very much, it's really much easier to read and understand. I use 'compressionLevel = Deflater.NO_COMPRESSION' now but 7-Zip still shows "Deflate"!?
    /rgb
    @RalfBarkow_twitter
    The issue seems to be comparable to dotnet/corefx#29735 : >Currently when a ZipArchive is created and an entry is added with the CompressionLevel set to "no-compression" the file inside the archive is still wrapped in a deflate stream. The deflate stream correctly does not to compress the file.
    Pathikrit Bhowmick
    @pathikrit
    But is this a bug in better-file? It seems its a bug in the unzipper ... unzip works just fine from command line..
    /rgb
    @RalfBarkow_twitter
    unzip -lv test.zip shows the method as Defl:N and not as Stored, so it might be a bug in better-file. (Implicits.scala, line 258: if (level == Deflater.NO_COMPRESSION) out.setMethod(ZipOutputStream.DEFLATED) seems to wrap the file inside the archive in a deflate stream.
    Pathikrit Bhowmick
    @pathikrit
    @RalfBarkow_twitter : That code was added here for some reason - https://github.com/pathikrit/better-files/pull/38/files
    Can you send a pull request perhaps or atleast file an issue with failing test case?
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit, will try to write a failing test case and file an issue then but unfortunately I won't be able to do it in the near future (next week is carnival).
    Pathikrit Bhowmick
    @pathikrit
    A failing test or same is enough and I can fix it
    /rgb
    @RalfBarkow_twitter
    Thanks Pathikrit, my problem seems to be that I don't know how to specify the compression method for an individual ZIP file entry, and set it to STORED. (Btw I'm working on that "failing test or same".)
    Pathikrit Bhowmick
    @pathikrit
    Thanks!
    Just put what your "ideal API" would look like and I can work backwords from there
    /rgb
    @RalfBarkow_twitter
    How about val zipFile: File = file.zipTo("test.iirds").toFile.stored ?
    Pathikrit Bhowmick
    @pathikrit
    What does .stored mean?
    /rgb
    @RalfBarkow_twitter
    It should set the compression method to STORED.
    /rgb
    @RalfBarkow_twitter
    val zipFile: File = file.zipTo("test.iirds".toFile).setCompressionMethod(STORED)
    /rgb
    @RalfBarkow_twitter
    According to https://en.wikipedia.org/wiki/Zip_(file_format)#Compression_methods , the .ZIP File Format Specification documents the following compression methods: Store (no compression), ...
    I would like to use .store to specify the compression method for an individual ZIP file entry, and set it to STORED.
    /rgb
    @RalfBarkow_twitter
    Pathikrit Bhowmick
    @pathikrit
    Ok thanks for the pointer. Will take a look
    Philippe Derome
    @phderome
    I am curious to see whether creating tar balls or extracting them is within mandate or roadmap of this library or whether people have pointers for this (other than Process API or haoyi's os API which work well).
    Is there any plan or consideration to include setfacl and getfacl interfaces to handle more complex permission schemes than what chmod allows?
    Pathikrit Bhowmick
    @pathikrit
    @phderome : Is tar balls is definately in scope of this project. Can you file an issue? PRs welcome too :)
    Philippe Derome
    @phderome
    I don't really feel up to the task to contribute to be honest. I'll submit issue. I heard that tar technically is GNU and not completely UNIX portable. I assume that it does not bother you or won't stop you.
    Philippe Derome
    @phderome
    so I see you noticed today it was a duplicate...
    Bloznelis
    @Bloznelis
      def canonicalPath: String =
        toJava.getAbsolutePath