COMMENT = _{ "#" ~ (!NEWLINE ~ ANY)* ~ NEWLINE? } but I'm having trouble coming up with a way to get the comment rule to skip # within quotes. Seems like this has got to be a common problem for comments. Is there some trick to it?
I understand that I can (and should) use SOI and EOI around in my start rule, but I would also like to parse strings using non-start rules, but make sure that the parse matches the entire string.
Currently, I have a nasty approach in which I'm going through the pairs and finding the lowest start position and the highest end position. But I feel that there must be some nicer, and more general way to do this.
Is there an idiomatic way to have variants of the same grammar. Suppose (asking for a friend) one were trying to develop parser that followed the specification for email addresses from RFC5322.
I (I mean my friend) would like to have a way to
- Optionally disallow expansion of any of the obsolete non-terminals
- Optionally disallow of comment-folding-white-space.
On a related note, is there a way to expand to null. I found that I get an error with a rule like
cfws = { }
Hi there, I am trying to ignore some characters when matching an atomic rule. I want to match hex numbers as such 0xFF_aa_b and so on, and my current rule works fine:
hexnum = @{
"0x"
~ ASCII_HEX_DIGIT
~ (ASCII_HEX_DIGIT | ("_" ~ &ASCII_HEX_DIGIT))*
}Except it always matches hexnum: "0xFF_aa_b", and I would like it to match as hexnum: "FFaab" to alleviate the cleanup toil afterwards.
This answer on Reddit has a broken link, but it seems to be roughly what I want. I already tried rearranging several times, but no success.
Seems like DROP might be useful in this case too, but theres not much documentation on it, and what I tried so far hasn't worked.
Can anyone help? Thanks in advance
@luizberti_gitlab, I am very new to pest, so I might be completely wrong, but I believe that you shouldn't try to achieve what you are after in the grammar itself. Instead you do that in how you process the resulting pairs.
I'm facing something similar, in that I have a pest grammar for RFC5322 email addresses in which
(first comment) jsmith@ (second comment)\nexample.com (third comment)
is a valid address. But I want to prune what the specs call "comments and folding whitespace (CFWS)", but I assume that I do that not in the pest specification itself, but in my handling of non-terminal nodes to eliminate.
That is, I plan to walk through the pairs created by a parse and drop any pair corresponding to the the rule for cwfs. I just haven't gotten around to trying that yet.
So I think you will need to create a rule specifically for 0x and another for _ so that you can walk through your pairs and filter those. This will mean that you need to change your hexnum rule to be a compound atomic.
Again, I am just figuring this stuff out for myself now. It is possible that my thinking about all of this is entirely wrong.
I am just starting to use pest_consume and finding it super helpful in many cases, but I am wondering if there is a more concise way to handle optionals. I have quite a few rules with multiple optional clauses, this is a simpler one..
module_body = { export_list? ~ import_list? ~ assignment_list }So when dealing with this using pest_consume, it seems I have to cover each possible permutation, something vaguely like:
fn module_body(node: Node) -> Result<...> {
Ok(match_nodes!(node.into_children();
[assignment_list(a)] => ...,
[export_list(e), assignment_list(a)] => ...,
[import_list(i), assignment_list(a)] => ...,
[export_list(e), import_list(i), assignment_list(a)] => ...,
))
}I'm not looking forward to handling cases like the one below, which if I'm right ,will need 32 match cases , I'll probably just treat is as a list of nodes to avoid that
snmp_object_type_macro_type = { "OBJECT-TYPE"
~ snmp_syntax_part
~ snmp_units_part?
~ snmp_access_part
~ snmp_status_part
~ snmp_descr_part?
~ snmp_refer_part?
~ snmp_index_part?
~ snmp_def_val_part? }
(] {) i'd like it to report both the missing ) and the missing (
Hi! I'm trying out Pest today. I modified the CSV grammar example from the book to include my grammar. However, when I get a parse error, it looks really ugly:
thread 'main' panicked at 'unsuccessful parse: Error { variant: ParsingError { positives: [var_id], negatives: [] }, location: Pos(12), line_col: Pos((2, 12)), path: None, line: " occurs(", continued_line: None }', src\main.rs:15:16
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\csv-tool.exe` (exit code: 101)How can I get it to look pretty like on pest.rs main page?
unwrap_or_else(|e| panic!("{}", e))
Now it looks much better:
thread 'main' panicked at ' --> 2:12
|
2 | occurs(
| ^---
|
= expected var_id', src\main.rs:16:25But what if my users don't know what a var_id is? Is there a pattern for including more useful info?
I would also welcome advice about whether or not I should attempt to parser a language like Elm using Pest. Here is a complete example of the syntax: https://github.com/pdamoc/elm-syntax-sscce/blob/master/src/Main.elm
It is whitespace dependent but I'm not sure it is as regular (not in the technical sense) as say simple markdown lists. From what I can tell from reading different issues, it might be possible to make it work with Pest but perhaps I would be better off with nom or combine or something? I would certainly welcome your input
I might return to pest to re-asses it but I couldn't initially see how to handle complex indentation dependent code. It is interesting that it produces tokens rather than ASTs though. I hadn't realised that.
And when I pointed out the issue with the fiddle, I meant that various links to the fiddle, from various Github issues, would load the fiddle but without any actual code in it. The links were like "look at this example" and you'd click on it and see nothing.
Is there a best-practice way to prevent identifiers from being keywords in a Pest grammar? Right now, I'm using the following design:
kw_bool = { "bool" }
kw_int = { "int" }
kw_void = { "void" }
keywords = { kw_bool | kw_int | kw_void }
id = @{ !(keywords ~ (WHITESPACE|EOI)) ~ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }but this feels a little heavy-handed, to me.
Mo | Tu | We | Th | Fr as weekday and Sa | Suas weekend -- and use those rules in other rules -- but want to end up with a token called day that I can reference, what's the best way to do it? Or am I barking up the wrong tree?
I don't see why you can't just use
weekday = _{ "Mo" | "Tu" | "We" | "Th" | "Fr" }
weekend = _{ "Sa" | "Su" }
day = { weekday | weekend }You can always access the actual text of the "day" rule by calling the my_day_pair.as_str(), and, if needed, you can make decisions based upon the type of day with
match my_day_pair.into_inner().next().unwrap().as_rule() {
Rule::weekday => { ... },
Rule::weekend => { ... }
}No?
strict_rule adds additional requirements to the weekday/weekend rules. If my "top level" rule calls something ambiguous like day, it's not going to be able to differentiate between the two types of day. Of course, I could change my day rule to match your day instead of using char ranges and that's fine.
weekday/weekend rules, but also group them into a final output token.
match example, but I think that's unnecessarily "dumb" of the parser... if you consider tokens that are output as being the "API" of a parser, I feel that I should be able to scope it down to what matters.
Something "Serde-like" would also be cool, similar to this:
#[pest(parser = MyParser, entrypoint = some_rule)]
struct MyStruct {
#[pest(rule = day)]
day String,
#[pest(rule = day_of_month)]
day_of_month u8,
#[pest(rule = month)]
month MyMonth,
}With the macros implying something like a MyStruct::parse(&str) -> Result<MyStruct>
match clauses deeper into the rule unpacking as you figure out what you're dealing with.... since, in essence, I think we're talking about sub-type polymorphism, here, so you kinda need a way to discover the particular subclass you're dealing with or enforce some kind of standard characteristics among the subclasses.
tag day = { weekend | weekday } and if either of those rules is produced, that "tag" gets populated as well?