sirthias/parboiled2

A macro-based PEG parser generator for Scala 2.10+

Boris V.Kuznetsov

@tampler

The later parses my input fine, but I can't instantiate the correct AST type since I'm missing the knowledge of whether or not I had "tag0" there. @alexander-myltsev any advice? Thanks

Alexander Myltsev

@alexander-myltsev

let me know if that works for you:

def record: Rule1[BusAST] = rule {
    optional(atomic(capture("tag0”))) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> (
      (
        tagOpt: Option[String],
        name: String,
        tpe: String,
        len: Int
      ) => BusAST(if (tagOpt.isDefined) Out else In, name, tpe, len)
    )
  }

Boris V.Kuznetsov

@tampler

@alexander-myltsev Perfecto ! Worked like a charm, thank you!

Alexander Myltsev

@alexander-myltsev

also, you could try to cut the corner:

def record: Rule1[BusAST] = rule {
  (test("tag0") ~ push(Out) | push(In))  ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST 
}

but I’m not sure if (test("tag0") ~ push(Out) | push(In)) passes the typer and derives the correct least supertype of In and Out. you should check on your side

Boris V.Kuznetsov

@tampler

The prev solution looks simpler and delivers a better logical sense. Thanks for your support, Parboiled2 is awesome !

Boris V.Kuznetsov

@tampler

@alexander-myltsev For the sake of science, I tried the second option as :

 def record: Rule1[BusAST] = rule {
    test(true) ~ push(Out) | push(In) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST
  }

2 issues with this option:

I had to put test(true) to make this compile, since test requires Boolean on input, not String

Still getting a compile error:

{
found   : org.parboiled2.Rule[shapeless.HNil,Product with Serializable :: shapeless.HNil]
required: org.parboiled2.Rule[shapeless.HNil,chipa.parser.ast.BusAST :: shapeless.HNil]"
}

Any ideas how to fix ?

Alexander Myltsev

@alexander-myltsev

ah, sorry. test there is wrong. try this one:

def record: Rule1[BusAST] = rule {
  ("tag0" ~ push(Out) | push(In))  ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST 
}

the wrong thing in your solution is that you don’t take into account the precedance of | and ~ operators. ~ has higher one. that means that you’re making two alternatives:
test(true) ~ push(Out) and push(In) ~ ws ~ capture(Word) ~ Col ~ bus ~ ws ~> BusAST. and apparently the least common supertype of Out and BusAST is Product with Serializable

Boris V.Kuznetsov

@tampler

@alexander-myltsev Thank you for this fix! It indeed looks simpler and more Parboiled-style. I used that pattern to scrap my boilerplate a lot. Now parsing a custom language with a 120 lines-of-code parser :D . Thanks to you and Parboiled2 !

Tieru

@Tieru

Hello. Is there any way to split a big class with multiple rules to smaller classes?

Could you provide me an example with such case? I couldn't find any by myself

Alexander Myltsev

@alexander-myltsev

hi @Tieru

take a look at that project parser https://github.com/sangria-graphql/sangria/blob/master/src/main/scala/sangria/parser/QueryParser.scala

Tieru

@Tieru

So the solution is using traits? Or I miss something else?

Alexander Myltsev

@alexander-myltsev

Wow, missed your question, sorry

Yes, trait is an answer

Tieru

@Tieru

Thanks. It worked :)

Travis Brown

@travisbrown

Has anyone tried porting parboiled2 to Dotty? I took a stab at it this weekend and have support for ~80% of its functionality, with a focus on what http4s needs, but before I put more time into it I want to make sure I'm not duplicating effort.

vonchav

@voonchav_gitlab

OMG, Travis, you're a saint. I know the project is in good hands when you get involved. I use Parboiled2. I'd love that it's Dotty ready.

Boris V.Kuznetsov

@tampler

@travisbrown do you need asap? I also want to work this with Dotty, but the later doesn't work well with Shapeless, which is a transient dep here

Shapeless 3.0.0 is under dev. I think We'll need to port Parboiled2 to Parboiled3 first :D

Travis Brown

@travisbrown

@tampler Current status is here: https://github.com/sirthias/parboiled2/issues/210#issuecomment-597505806

@tampler There's no rush on this from my perspective, since I have it migrated enough for what I needed this week. The subset of Shapeless that Parboiled uses is small enough that it's not much of an issue.

Didac

@umbreak

I have a parsing condition to satisfy which is

it matches if the sequence starts with '@' and it follows with one or more characters (as long as some are NOT alpha numeric), or if the sequence does not start with '@'

So basically,
matched: ‘one’, ’@one.two’
not matched: ‘@one'

Didac

@umbreak

So I have this:

  def term: Rule0 = rule { termMatch1 | termMatch2 }
  private def termMatch1: Rule0 = rule { '@' ~ oneOrMore(All) }
  private def termMatch2: Rule0 = rule { oneOrMore(AlphaNum) }

But I have to exclude terms that match

private def termExclude: Rule0 = rule { '@' ~ oneOrMore(AlphaNum) }

Mikhail Belikov

@mbelikov_twitter

hi guys! where can I find a list of prominent projects using the parboiled2?

just need some production ready examples of parboiled2 usage

Alexander Myltsev

@alexander-myltsev

@umbreak what’s the question?

@mbelikov_twitter a good point to start is to look at usages at MVN Repository:
https://mvnrepository.com/artifact/org.parboiled/parboiled/usages

Boris V.Kuznetsov

@tampler

@mbelikov_twitter I have several prominent projects, which use Parboiled2. But they are all commercial, so I can't share. You are welcome to ask concrete questions

Boris V.Kuznetsov

@tampler

Hello! I implemented some rule def Node: Rule[String :: String :: HNil, NodeAST :: HNil], which compiles fine. However, when I try to add it to the root rule, I'm getting an error message: Theoptional,zeroOrMore,oneOrMoreandtimesmodifiers can only be used on rules of typeRule0,Rule1[T]andRule[I, O <: I]!
How do I fix that ? Thanks

I'm adding it as oneOfMore(Node)

Boris V.Kuznetsov

@tampler

Okay, fixed by changing the rule to def Node: Rule[HNil, NodeAST :: HNil]. Pls disregard this question. Thank you for a great library !

Boris V.Kuznetsov

@tampler

Hello ! I'm thinking about the best approach to attack the following parsing problem. Suppose I have the following input grammar:

    when _T : 
      tmp <= UInt<1>("h00") 
      skip 
    else : 
      node _T_1 = add(tmp, UInt<1>("h01")) 
      node _T_2 = tail(_T_1, 1) 
      skip

That's essentially a logic block, which spans multiple lines. It starts with a when statement and has two branches : when and else. As far as the number of lines in each branch is variable (zero to many) and right-hand-side deps may be implemented aside the when block, I can not put this statement into a fixed case class like

final case class MyWhen( cond:String, node0: Node, node1:Node)

How would you parse this input? Any advice and example is appreciated . Thank you!

Boris V.Kuznetsov

@tampler

Is there any way to selectively parse the input? Say, I'd love to have a single pass, which parses only lines, which start with input or output like this:

figure Area :
  topic Essential:
    input vision : Vision
    output sun : Sun

Boris V.Kuznetsov

@tampler

@alexander-myltsev any thoughts ?

Alexander Myltsev

@alexander-myltsev

@tampler

How would you parse this input? Any advice and example is appreciated

I’m not following what the problem is. derive Expr(statement: Seq[?]) from the Node.

final case class MyWhen( cond:String, thenExpr: Node, elseExpr:Node)

@tampler

Is there any way to selectively parse the input? Say, I'd love to have a single pass, which parses only lines, which start with input or output like this:

I suggest you to extract those lines with RegEx. and then apply a parser

Behrang

@behrangsa

Hi all,

When defining a grammar in Parboiled 1, is there an option to specify that a specific term/token should only appear {n,m} times?

Also is it possible to create a Parboiled parser for a data format language like this:

SIZE CHAR{SIZE}

where CHAR appears SIZE times AND SIZE is a two digit zero padded number, e.g:

05HELLO
11PROGRAMMING

1 reply

Alexander Myltsev

@alexander-myltsev

@behrangsa This is a parboiled2 group. I'm afraid there are not many people who know parboiled1.

Behrang

@behrangsa

Unfortunately there's no one in my body who knows Scala :(

Boris V.Kuznetsov

@tampler

I suggest you to switch to Parboiled2 then. No reasons to use an old barely alive Parb1

Tobias Haf

@tobias.haf_gitlab

Hi, I used Parboiled1 (Java) in the past and I'm currently evaluating Parboiled2 due to its performance promises :-) I'm wondering how to provide proper location information (startOffset, endOffset) to the AST nodes pushed to the value stack.
In Parboiled1 I had two Actions within a rule; first ACTION to push the node with the current offset (startOffset) to the value stack and second ACTION to set the endOffset to this node:

Sequence(
    ACTION(factory.createNode()),
    matchingRules(),
    ACTION(factory.finalizeNode())
)

How should this be implemented in Parboiled2?

2 replies

Edward A. Maxedon, Sr.

@edadma

how do i add position information to an AST case class action so that a nice post-parsing error message can be displayed if needed?

Edward A. Maxedon, Sr.

@edadma

ok, i can write push(cursor) to get the cursor position into the AST element, but then how to i generate a nice error message based on that later showing where in the code in error occured?

Luke Hutchison

@lukehutch

Hi, I have been trying to build a parse tree out of every rule that matches the input. I tried wrapping each rule with capture, but this does not seem to compile:

original:

Z = rule : Rule0 { X | Y }

modified version:

Z = rule : Rule1[String] { capture( X | Y) ~> (_) }

What is the right way to build the parse tree with a node for each match?

Luke Hutchison

@lukehutch

Let me explain a bit better what I am trying to achieve...

I want to produce a parse tree consisting of either one node per matching rule, or one node per matching clause within each rule.

As far as I can tell, each rule acts as an HList. But I can't figure out how to collect the HList nodes together into a tree.

I see some clues here:

https://github.com/sirthias/parboiled2/blob/master/examples/src/main/scala/org/parboiled2/examples/JsonParser.scala

def JsonArray = rule(ws('[') ~ zeroOrMore(Value).separatedBy(ws(',')) ~ ws(']') ~> (JsArray(_: _*)))

However I can't seem to get the _: _* pattern working with my own tree node class.

Also different PEG clause types have different HList types (most PEG nodes have a single subclause, but terminals have none, so need to use capture and produce a String for the terminal match, and sequence nodes (X ~ Y ~ Z) have as many type parameters as they have terms).

I don't have much knowledge of Scala unfortunately... any help would be greatly appreciated.

Jevgeni Tarasov

@devjev

Hi guys! I'm going to resurrect an old topic, but what I've found here and on Google groups is from 2015 and some of the example links are dead now. What is the best current way to parse indentation-sensitive syntax with parboiled2? Think Python, Markdown lists, YAML, etc. Is it with runSubParser? Is it something else? Are there any examples?

Dmitry Katsubo

@dmak

Dear all, I am a newbie with Parboiled and Scala. I was given a task to translate Parboiled1 grammar (written in Java) to Parboiled2 / Scala v2.13.8.

The first issue I've encountered is how to translate the following rule:
```
Rule OperatorSlop() {
 Var<Boolean> negate = new Var<>(false);

 return Sequence(
             Sequence('-', negate.set(true)),
             Sequence(IgnoreCase("WS"), push(Builder.build(negate.get())))
 );
}
```
It actually captures the minus sign into a variable and re-uses it later. Note that the actual rule is more complicated than this, and several variables are involved. What would be the proper solution in Scala?

The following issue I've encountered that this code compiles

def Operator = rule {
"=" ~ push(1) |
 ">=" ~ push(2)
}

but when I move | to the next line, the code does not compile:

def Operator = rule {
"=" ~ push(1)
| ">=" ~ push(2)
}

What would be Parboiled2 alternative of function org.parboiled.BaseActions.match() (Returns the input text matched by the rule immediately preceding the action expression that is currently being evaluated. This call can only be used in actions that are part of a Sequence rule and are not at first position in this Sequence.)
Thanks!

Where communities thrive

People

Repo info

Activity