Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Actually I think the sequence of evaluations does not matter for my particular application - so I may just roll with right recursion for this. However... I'm kinda interested now. How are people tackling this issue in Nom?
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Hmmmmmmmmmm Maybe fold_many will work for me...
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Oh man, I'm so close to having this fold_many implementation working. I'm tripping up on some lifetime stuff.
    let (input, init) = operand(input)?;
    fold_many0(
        preceded(
            tag("&"),
            cut(specific(Specific::ExpectAndExpression, operand)),
        ),
        ||init, // Lifetime error here
        |a, b| Token::And(Box::new(a), Box::new(b)),
    )(input)
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
It works if I use clone(), but since I know ownership of init should be moved, cloning would be a waste. It would also work if the init parameter of fold_many() were FnOnce instead of FnMut, although I don't know if that would break some combinator usage.
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Ok init parameter being FnOnce wouldn't work because return type of fold_many is FnMut. Hmmmm
xiretza
@xiretza:xiretza.xyz
[m]
yeah that's a frequent pain point with nom's combinators, even if you only need the resultant parser once, the combinator still needs to return a "FnNonOnce", so all inputs need to be Clone/"FnNonOnce"
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Ironically, this near exact pattern is used in nom/tests/arithmetic
Sadly that's using i64, so it's copy and cloneable
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
I can probably solve this by making a container object to manage the token structure
Specifically for use in the fold functions. It's just a shame as it's extra boilerplate
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
omg I think I have a solution
You're going to vomit
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
fn and_term<'a, E>(input: &'a str) -> IResult<&'a str, Token, E>
where
    E: nom::error::ParseError<&'a str> + SpecificError<&'a str>,
{
    let (input, init) = term_or_not_term(input)?;
    let (input, mut root) = fold_many0(
        preceded(
            ws_tag("&"),
            cut(specific(Specific::ExpectAndExpression, term_or_not_term)),
        ),
        || Token::Bool(true),
        |a, b| Token::And(Box::new(a), Box::new(b)),
    )(input)?;

    // Well well well, what is this hot mess?
    // Because the left side is always either And or Bool, we can
    // simply loop until we find the bool then replace it with init
    //
    // #NOTE This is necessary because it's not possible to move
    // init into the closures passed to fold_many0 as they expect
    // FnMut, not FnOnce
    let mut t = &mut root;
    while let Token::And(a, _) = t {
        t = a.as_mut();
    }
    debug_assert!(matches!(t, Token::Bool(_)));
    *t = init;

    Ok((input, root))
}
This passes the tests, but it's so dirty
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Someone please tell me if there's some closure magic that will convince rust to move init directly into fold_many() instead of this filth you see above
tanriol
@tanriol:matrix.org
[m]
@AWildAudioNerd_twitter: I'd try let init = Some(init) outside and || init.take().unwrap() as the initializing closure.
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
WTF that worked (with let mut init = Some(init) outside and move || init.take().unwrap() is the initializing closure)
tanriol
@tanriol:matrix.org
[m]
Sorry, I missed these details :-) the basic idea here is to be explicit: "take the ownership the first time the closure is called; you're free to panic if it's called again"
Alex Blunt ✊🏿 BLM
@AWildAudioNerd_twitter
Ok I understand now
The option allows the ownership transfer to be explicit. If the closure were to be called again (which it won't in this instance), the option would be None and the method would panic
Ok, that's also dirty. But it's less dirty than my implementation.
Thank you! You folks have helped me level up in Rust during this process too
Jozef Miklos
@sparatko_gitlab

hello, any nom_locate users? i have trouble wrapping my mind around the error i am getting...
Original &str input parser:

type MyResult<'a, T> = nom::IResult<&str, T>;
fn yang_version_arg(input: &str) -> MyResult<&str> {
    tag("1.1")(input)
}

seems to work fine...

Then i try to use nom_locate and follow the gitbuh readme with Span 'type':

type MyResult<'a, T> = nom::IResult<Span<'a>, T>;
fn yang_version_arg(input: Span) -> MyResult<&str> {
    tag("1.1")(input)
}

but i get errors across whole app codebase wherever i try to use any nom combinator/parser:

error[E0308]: mismatched types
  --> src\parser\rfc7950\yang_version_stmt.rs:29:5
   |
28 | fn yang_version_arg(input: Span) -> MyResult<&str> {
   |                                     -------------- expected `Result<(LocatedSpan<&str>, &str), nom::Err<nom::error::Error<LocatedSpan<&str>>>>` because of return type
29 |     tag("1.1")(input)
   |     ^^^^^^^^^^^^^^^^^ expected `&str`, found struct `LocatedSpan`
   |
   = note: expected enum `Result<(LocatedSpan<&str>, &str), nom::Err<nom::error::Error<LocatedSpan<&str>>>>`
              found enum `Result<(LocatedSpan<&str>, LocatedSpan<&str>), nom::Err<_>>`

do i miss some piece of puzzle? isn't some deref supposed to happen to allow usage of all pre-build combinators in nom?

tanriol
@tanriol:matrix.org
[m]
Looks like tag here returns LocatedSpan<&str> instead of &str, you may want to change the return type or convert it somehow.
Jozef Miklos
@sparatko_gitlab
my understanding was the nom_locate modifies the input for combinators, not the output, but yes, the message seems opposite/ like you say
tanriol
@tanriol:matrix.org
[m]
I'd guess tag and take are a special case here because their input and output are the same type?..
Jozef Miklos
@sparatko_gitlab
there may be some pattern i am not following / is not very greatly explained in the nom_locate "tutorial", as i get similar hundreds of errors for basically any combinator i use across my parser from nom :( (with the same rustc error pattern)
Jozef Miklos
@sparatko_gitlab
please excuse possibly stupid query, but, i seem to be beaten a bit by the error handling... is it possible to create/return explicitly some trivial error, no custom error implementaitons? in the combinator/parser code... all the examples keep showing custom structs/implementations, etc., i just want to plainly fail with no custom message, or, worst case, simple &str msg...
in the code, when all goes ok, i can simply do Ok(rest, someval), but i fail to comperehend how to do the opposite -> Err(???), i may be dumbed down after hours or related experiments, but i fail to find the easy/short way on what to put into Err() enum for IResult<> not to complain about some types/fn-s,/etc.
tanriol
@tanriol:matrix.org
[m]
If you're using nom::error::Error, you can use Error::new
If your code is generic over error types, there's ParseError::from_error_kind
Jozef Miklos
@sparatko_gitlab
:thumbsup: aaargh, thank you good sir/madam, Error:new indeed works, but i've been missing the return from fn vs match arm return value...
tanriol
@tanriol:matrix.org
[m]
Yeah, there are a couple levels of wrapping here - you probably want Err(nom::Err::Error(Error::new(...)))
Jozef Miklos
@sparatko_gitlab
what is idiomatic way to change failure of combinator to success if it happens (but still keep original success path)?
regular match ok/err on combinator result, or is there some wrapper combinator? none of the ones mentioned in "choosing combinators' seems to fit scenario directly
xiretza
@xiretza:xiretza.xyz
[m]
so you want the combinator to succeed unconditionally? what's your use case for this?
just from a type compatibility standpoint, that's opt()
Jozef Miklos
@sparatko_gitlab
i have a combinator that tries to peek a keyword, and return it if found, but return successful None if not, ignoring the rest of input
xiretza
@xiretza:xiretza.xyz
[m]
opt(peek(keyword))?
Jozef Miklos
@sparatko_gitlab
facepalm, right, i tried to overcomplicate things a bit and this straightforward way didnt occur to me, thanks :)
Aram Peres
@aramperes
Hey folks! What's the proper way to do the equivalent of delimited(start, content, end)that includes both start and end in the final output?
I have delimited(char('['), is_not("]"), char(']'))("[::1]") that returns ("", "::1")but I would like it to return [::1] instead (without having to format! the output if possible)
Aram Peres
@aramperes
Looks like wrapping with recognize is what I want
Jozef Miklos
@sparatko_gitlab
right, recognize() should do the trick...
Mikko Murto
@mmurto
How should I parse a word that is alphanum and optionally can have one specific characeter as its last character?
I'd like it to return the str including the last character if it exists
tanriol
@tanriol:matrix.org
[m]
Sounds like recognize again :-)
Mikko Murto
@mmurto
That's apparently something I need to familiarize myself with :) Thanks!
Jozef Miklos
@sparatko_gitlab
this is possibly an obvious/intended pattern, i use recognize() a lot for "stringifying" a composite combinator/sub-parser outputs into single value with a structure prescribed by the mentioned sub-parsers
xiretza
@xiretza:xiretza.xyz
[m]
yep, that's exactly what it's for ;)
simpsoneric
@simpsoneric
I don't know if it's helpful, but I've typed up some pseudo-code for what I'm trying to do: https://gist.github.com/rust-play/3a8121101770f0ee5e76847fccf8f684 . Rather than an ad-hoc parser, I like the structured nom approach, but I'm not sure how to describe this type of streaming parser. Does anyone have examples for similar problems?
Jozef Miklos
@sparatko_gitlab
rather phylosophical question... when parsing an input according to some/a grammar, did you ever find it useful to "delay" the statement count restrictions to higher level/AST? e.g. for batch error identification, not to interrupt parser on first problem occurence? (forcing the user to re-iterate with each input parsed text fixes again and again before he progressively encounters all the errors)
1 reply