by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev Any thoughts ?
    Alexander Myltsev
    @alexander-myltsev

    @tampler try this one:

    rule { ";" ~ oneOrMore(CharPredicate.Printable) ~ push(FNIL) }

    you don’t need to optional test. the rule I give you would fail if it fails to consume “;” at the start
    also, I think CharPredicate.Printable should do the job to consume everything before “\n”
    otherwise, please, check the appropriate at https://github.com/sirthias/parboiled2/blob/release-2.1/parboiled-core/src/main/scala/org/parboiled2/CharPredicate.scala. and let me know if you don’t find any
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev Thanks, worked like a charm. I was even able to simplify my AST by removing FNIL with the following:
    def skip: Rule0  = rule { ';' ~ oneOrMore(CharPredicate.Printable) ~ ws }
    ....
    def start = rule { optional(skip) ~ oneOrMore(circ | mod  | bundle) ~ EOI }
    vonchav
    @voonchav_gitlab
    Hi all, apologies if similar questions have already been asked (most likely). What is the recommended testing strategy to test against the parser you've written? I wrote a parser. It's quite nice and I make the grammar quite flexible, especially when dealing with whitespaces. However, that creates a challenge for writing tests to cover all/most cases. I find the tasks of having a decent coverage quite daunting. I thought about writing property tests and Gen's for ScalaCheck. But writing Gen's based off my own grammar is still quite manual - basically translating my syntax to Gen's bit by bit.... Just wondering any advice from the community. Thank you so much.
    Boris V.Kuznetsov
    @tampler
    @voonchav_gitlab Your testing strategy is to write tests for everything you wanna parse. With Zio Test, you can do the following:
    import zio.test._
    import zio.test.Assertion._
    
    import org.parboiled2.ParseError
    
    object ParserTests {
      val str0 =
        """node Regbuffer : 
      module Shit :""".stripMargin
    
      val stringSuite = suite("Simple string suite")(
        test("Node Module parsing") {
          val par = new MyParser(str0)
          val res = par.start.run()
          val out = res.fold(ex => par.formatError(ex.asInstanceOf[ParseError]), v => v)
          assert(out, equalTo(Vector(NodeAST("Regbuffer"), ModuleAST("Shit"))))
        })
    And easily write small tests in a suite for every aspect you wanna cover in your parser. Than combine them all to parse more complex grammars. Parboiled2 docs have enough explanations and examples on that. Highly recommended to read
    vonchav
    @voonchav_gitlab
    Thanks @tampler. It's interesting that you gave a zio-test as an example. I do use zio and wrote zio-test. But I didn't think I'd use zio-test for parboiled tests :) Yeah, I suppose I'd have to still manually write tests.... I'm interested in what you described, specifically the "combine them all to parse more complex grammars"... did you mean composing small tests into a bigger test? That is intriguing. Would you mind posting an example here? Also, can you post a link to the Parboiled2 docs that show testing? I read through the entire Parboiled2 readme markdown multiple times and some of its example source code. But not so much in regards to writing unit tests. Please point me to the right direction. Thanks again.
    Boris V.Kuznetsov
    @tampler
    Hello. I need to parse a key-value grammar with optional tags. Here's example: tag key0 : value0, key1 : value1. However, the output AST Node depends on whether or not I had a tag for that key. Here's my impl:
    def flip: Rule1[Boolean] = rule {
        capture(Word) ~ ws ~> (
          (s: String) => test(s == "tag0") ~ push(true)
        )
      }
      def record: Rule1[BusAST] = rule {
        flip ~ capture(Word) ~ Col ~ bus ~ ws ~> (
          (
            check: Boolean,
            name: String,
            tpe: String,
            len: Int
          ) => BusAST(if (check) Out else In, name, tpe, len)
        )
      }
    This impl fails on the second part key1 : value1. I also tried this:
    def record: Rule1[BusAST] = rule {
        optional(atomic("tag0")) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> (
          (
            _,
            name: String,
            tpe: String,
            len: Int
          ) => BusAST(if (true) Out else In, name, tpe, len)
        )
      }
    The later parses my input fine, but I can't instantiate the correct AST type since I'm missing the knowledge of whether or not I had "tag0" there. @alexander-myltsev any advice? Thanks
    Alexander Myltsev
    @alexander-myltsev
    let me know if that works for you:
    def record: Rule1[BusAST] = rule {
        optional(atomic(capture("tag0”))) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> (
          (
            tagOpt: Option[String],
            name: String,
            tpe: String,
            len: Int
          ) => BusAST(if (tagOpt.isDefined) Out else In, name, tpe, len)
        )
      }
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev Perfecto ! Worked like a charm, thank you!
    Alexander Myltsev
    @alexander-myltsev
    also, you could try to cut the corner:
    def record: Rule1[BusAST] = rule {
      (test("tag0") ~ push(Out) | push(In))  ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST 
    }
    but I’m not sure if (test("tag0") ~ push(Out) | push(In)) passes the typer and derives the correct least supertype of In and Out. you should check on your side
    Boris V.Kuznetsov
    @tampler
    The prev solution looks simpler and delivers a better logical sense. Thanks for your support, Parboiled2 is awesome !
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev For the sake of science, I tried the second option as :
     def record: Rule1[BusAST] = rule {
        test(true) ~ push(Out) | push(In) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST
      }
    2 issues with this option:
    1. I had to put test(true) to make this compile, since test requires Boolean on input, not String
    2. Still getting a compile error:
      {
      found   : org.parboiled2.Rule[shapeless.HNil,Product with Serializable :: shapeless.HNil]
      required: org.parboiled2.Rule[shapeless.HNil,chipa.parser.ast.BusAST :: shapeless.HNil]"
      }
      Any ideas how to fix ?
    Alexander Myltsev
    @alexander-myltsev
    ah, sorry. test there is wrong. try this one:
    def record: Rule1[BusAST] = rule {
      ("tag0" ~ push(Out) | push(In))  ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST 
    }
    the wrong thing in your solution is that you don’t take into account the precedance of | and ~ operators. ~ has higher one. that means that you’re making two alternatives:
    test(true) ~ push(Out) and push(In) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST. and apparently the least common supertype of Out and BusAST is Product with Serializable
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev Thank you for this fix! It indeed looks simpler and more Parboiled-style. I used that pattern to scrap my boilerplate a lot. Now parsing a custom language with a 120 lines-of-code parser :D . Thanks to you and Parboiled2 !
    Tieru
    @Tieru
    Hello. Is there any way to split a big class with multiple rules to smaller classes?
    Could you provide me an example with such case? I couldn't find any by myself
    Alexander Myltsev
    @alexander-myltsev
    Tieru
    @Tieru
    So the solution is using traits? Or I miss something else?
    Alexander Myltsev
    @alexander-myltsev
    Wow, missed your question, sorry
    Yes, trait is an answer
    Tieru
    @Tieru
    Thanks. It worked :)
    Travis Brown
    @travisbrown
    Has anyone tried porting parboiled2 to Dotty? I took a stab at it this weekend and have support for ~80% of its functionality, with a focus on what http4s needs, but before I put more time into it I want to make sure I'm not duplicating effort.
    vonchav
    @voonchav_gitlab
    OMG, Travis, you're a saint. I know the project is in good hands when you get involved. I use Parboiled2. I'd love that it's Dotty ready.
    Boris V.Kuznetsov
    @tampler
    @travisbrown do you need asap? I also want to work this with Dotty, but the later doesn't work well with Shapeless, which is a transient dep here
    Shapeless 3.0.0 is under dev. I think We'll need to port Parboiled2 to Parboiled3 first :D
    Travis Brown
    @travisbrown
    @tampler There's no rush on this from my perspective, since I have it migrated enough for what I needed this week. The subset of Shapeless that Parboiled uses is small enough that it's not much of an issue.
    Didac
    @umbreak

    I have a parsing condition to satisfy which is

    it matches if the sequence starts with '@' and it follows with one or more characters (as long as some are NOT alpha numeric), or if the sequence does not start with '@'

    So basically,
    matched: ‘one’, ’@one.two
    not matched: ‘@one'

    Didac
    @umbreak

    So I have this:

      def term: Rule0 = rule { termMatch1 | termMatch2 }
      private def termMatch1: Rule0 = rule { '@' ~ oneOrMore(All) }
      private def termMatch2: Rule0 = rule { oneOrMore(AlphaNum) }

    But I have to exclude terms that match

    private def termExclude: Rule0 = rule { '@' ~ oneOrMore(AlphaNum) }
    Mikhail Belikov
    @mbelikov_twitter
    hi guys! where can I find a list of prominent projects using the parboiled2?
    just need some production ready examples of parboiled2 usage
    Alexander Myltsev
    @alexander-myltsev
    @umbreak what’s the question?
    @mbelikov_twitter a good point to start is to look at usages at MVN Repository:
    https://mvnrepository.com/artifact/org.parboiled/parboiled/usages
    Boris V.Kuznetsov
    @tampler
    @mbelikov_twitter I have several prominent projects, which use Parboiled2. But they are all commercial, so I can't share. You are welcome to ask concrete questions
    Boris V.Kuznetsov
    @tampler
    Hello! I implemented some rule def Node: Rule[String :: String :: HNil, NodeAST :: HNil], which compiles fine. However, when I try to add it to the root rule, I'm getting an error message: Theoptional,zeroOrMore,oneOrMoreandtimesmodifiers can only be used on rules of typeRule0,Rule1[T]andRule[I, O <: I]!
    How do I fix that ? Thanks
    I'm adding it as oneOfMore(Node)
    Boris V.Kuznetsov
    @tampler
    Okay, fixed by changing the rule to def Node: Rule[HNil, NodeAST :: HNil]. Pls disregard this question. Thank you for a great library !
    Boris V.Kuznetsov
    @tampler

    Hello ! I'm thinking about the best approach to attack the following parsing problem. Suppose I have the following input grammar:

        when _T : 
          tmp <= UInt<1>("h00") 
          skip 
        else : 
          node _T_1 = add(tmp, UInt<1>("h01")) 
          node _T_2 = tail(_T_1, 1) 
          skip

    That's essentially a logic block, which spans multiple lines. It starts with a when statement and has two branches : when and else. As far as the number of lines in each branch is variable (zero to many) and right-hand-side deps may be implemented aside the when block, I can not put this statement into a fixed case class like

    final case class MyWhen( cond:String, node0: Node, node1:Node)

    How would you parse this input? Any advice and example is appreciated . Thank you!

    Boris V.Kuznetsov
    @tampler
    Is there any way to selectively parse the input? Say, I'd love to have a single pass, which parses only lines, which start with input or output like this:
    figure Area :
      topic Essential:
        input vision : Vision
        output sun : Sun
    Boris V.Kuznetsov
    @tampler
    @alexander-myltsev any thoughts ?
    Alexander Myltsev
    @alexander-myltsev

    @tampler

    How would you parse this input? Any advice and example is appreciated

    I’m not following what the problem is. derive Expr(statement: Seq[?]) from the Node.

    final case class MyWhen( cond:String, thenExpr: Node, elseExpr:Node)

    @tampler

    Is there any way to selectively parse the input? Say, I'd love to have a single pass, which parses only lines, which start with input or output like this:

    I suggest you to extract those lines with RegEx. and then apply a parser