Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
twitchyliquid64
@twitchyliquid64:matrix.org
[m]
You might be able to use take_until for line 5 instead of many0 too
twitchyliquid64
@twitchyliquid64:matrix.org
[m]
One last general comment, capturing a ton of bytes inside parsers (ie: terminated(many0(is_not("*")), tag("*")),) is usually frowned upon and rarely what you want for languages which can 'nest' features/syntax in a tree. In this case, you want to recurse back to some parser instead of consuming the bytes.
Sean Carroll
@seancarroll
good advise. i have an inline text parser where I believe i'm doing that, if I understand the recommendation, but didnt think to apply that to emphasis
4 replies
twitchyliquid64
@twitchyliquid64:matrix.org
[m]
By way of example, I have a wierd parser I built for a language which can represent a sequence of <geometry>, but an array of geometry (ie: [4]<geometry>) is also valid. To parse this, I have a nom parser which matches and parses the [4] thing, but then recurses to get the nested parse fragment.
Sean Carroll
@seancarroll
@CSUwangj , while doing some research I stumbled upon this. https://github.com/chipsenkbeil/vimwiki-rs. I thought you might find it helpful
Atmaram Naik
@atmnk
use nom_parser_derive::*;
use nom_parser::Parsable;
use nom_parser::ParseResult;
use nom::combinator::{opt};
use nom::sequence::{tuple};
use nom::bytes::complete::{tag};
use nom::character::complete::{ digit1};
#[derive(Parser,PartialEq,Clone,Debug)]
pub enum BinaryExpression{
    #[nom(tuple(field(0),tag(" + "),field(1)))]
    Add(Number,Number),
    #[nom(tuple(field(0),tag(" - "),field(1)))]
    Subtract(Number,Number),
    #[nom(tuple(field(0),tag(" * "),field(1)))]
    Multiply(Number,Number),
    #[nom(tuple(field(0),tag(" / "),field(1)))]
    Divide(Number,Number)
}
impl BinaryExpression{
    pub fn apply(&self)->Number{
        match self {
            Self::Add(one,two)=>{
                return Number(one.0+two.0)
            },
            Self::Subtract(one,two)=>{
                return Number(one.0-two.0)
            },
            Self::Multiply(one,two)=>{
                return Number(one.0*two.0)
            },
            Self::Divide(one,two)=>{
                return Number(one.0/two.0)
            }
        }
    }
}
#[derive(Parser,PartialEq,Clone,Debug)]
pub struct Number(#[nom(func(parse_isize))]isize);
fn parse_isize<'a>(input:&'a str) ->ParseResult<'a,isize>{
    let mut num = tuple((opt(tag("-")),digit1));
    let (i,(sign,nums)) = num(input)?;
    let str_num=format!("{}",nums);
    let f_num=str_num.parse::<isize>().unwrap();
    if let Some(_)=sign{
        Ok((i,(f_num * -1)))
    } else {
        Ok((i,f_num))
    }
}
fn main() {
    let (_,expr) = BinaryExpression::parse("1 + 2").unwrap();
    assert_eq!(expr,BinaryExpression::Add(Number(1),Number(2)));
    assert_eq!(expr.apply(),Number(3));

    let (_,expr) = BinaryExpression::parse("1 - 2").unwrap();
    assert_eq!(expr,BinaryExpression::Subtract(Number(1),Number(2)));
    assert_eq!(expr.apply(),Number(-1));

    let (_,expr) = BinaryExpression::parse("4 * 2").unwrap();
    assert_eq!(expr,BinaryExpression::Multiply(Number(4),Number(2)));
    assert_eq!(expr.apply(),Number(8));

    let (_,expr) = BinaryExpression::parse("4 / 3").unwrap();
    assert_eq!(expr,BinaryExpression::Divide(Number(4),Number(3)));
    assert_eq!(expr.apply(),Number(1));

}
I was able to come up with derive macro to parse AST, above is simple example.
Atmaram Naik
@atmnk
use nom_parser_derive::*;
use nom_parser::Parsable;
use nom_parser::ParseResult;
use nom::sequence::{tuple};
use nom::branch::alt;
use nom::character::complete::{alpha1, alphanumeric1};
use nom::multi::many0;
use nom::bytes::complete::tag;

#[derive(Parser,PartialEq,Debug,Clone)]
#[nom(tuple(tag("fn"),field(identifier),tag("("),field(args),tag(")"),tag("{"),field(statements),tag("}")))]
pub struct Function{
    identifier:Identifier,
    #[nom(separated_list0(tag(","),field))]
    args:Vec<Argument>,
    #[nom(many0(terminated(field,tag(";"))))]
    statements:Vec<Statement>
}
#[derive(Parser,PartialEq,Debug,Clone)]
#[nom(tuple(field(name),field(field_type)))]
pub struct Argument{
    name:Identifier,
    #[nom(opt(tuple(tag(":"),field)))]
    field_type:Option<Type>
}
#[derive(Parser,PartialEq,Debug,Clone)]
pub enum Statement{
    #[nom(tag("Hello"))]
    Hello,
    #[nom(tag("World"))]
    World
}
#[derive(Parser,PartialEq,Debug,Clone)]
pub struct Identifier(#[nom(func(ident))]String);
pub fn ident<'a>(input: &'a str) -> ParseResult<'a,String> {
    nom::combinator::map(tuple((
        alt((alpha1,tag("_"))),
        many0(alphanumeric1))),|(a,b)|{format!("{}{}",a,b.join(""))})
        (input)
}
#[derive(Parser,PartialEq,Debug,Clone)]
pub enum Type{
    #[nom(tag("Int"))]
    Int,
    #[nom(tag("String"))]
    String
}
fn main() {
    let (_,val) = Function::parse("fn hello( a : String , b ){ Hello; World; } ").unwrap();
    assert_eq!(val,Function{
        identifier:Identifier("hello".to_string()),
        args:vec![Argument{
            name:Identifier("a".to_string()),
            field_type:Some(Type::String)
        },
                  Argument{
                      name:Identifier("b".to_string()),
                      field_type:None
                  }
        ],
        statements:vec![Statement::Hello,Statement::World]
    });
}
Here is an example of How this reduces so much code. Simple Function Parser
望将
@CSUwangj

@CSUwangj , while doing some research I stumbled upon this. https://github.com/chipsenkbeil/vimwiki-rs. I thought you might find it helpful

thanks! I'll check it

Filippos Vasilakis
@vasilakisfil
is there any way, other than using regex, to tell to nom, eat the string until you find a specific substr ?
or is the regex the way to do it ?
I just feel that using regex is a poor man's job but maybe it's just my idea
Filippos Vasilakis
@vasilakisfil
I actually now do take_until(substr) and then tag(substr) to get to the interesting point
I am wondering how I can combine take_until(substr) and tag(substr) so I can get to the interesting point in one call
望将
@CSUwangj

is there any way, other than using regex, to tell to nom, eat the string until you find a specific substr ?

Actually, I've written a parser combinator called take_except which eat string until given parser success, maybe u could just use it

it may run with bad performance, it just tests every position if given parser succeeds. But I use it for markdown, I don't care about performance until I have completed it.
望将
@CSUwangj
oh, I misunderstand your question, I think all you need is a tuple((take_until(substr), tag(substr))(input)
Filippos Vasilakis
@vasilakisfil
thanks very helpful !!
Filippos Vasilakis
@vasilakisfil
I am wondering: why there are no functions like double for other kind of numbers? Atm I can see that there is double and float, but if I want to parse a u8 (which could be 1, 2 or 3 chars) I need to first parse it as a double/float and then convert it to u8 (like num = double(remaining)?; num = num as u8). Am I missing something ?
tanriol
@tanriol:matrix.org
[m]
@vasilakisfil: It's probably more correct to use digits1 and parse_to
Filippos Vasilakis
@vasilakisfil
ok I see
Jason Heeris
@detly
I'm trying to figure out how to avoid repetition where I have FromStr implemented for an enum. I want to be able to build up a more complex parser without a bunch of tag(...) calls that duplicate the strings in FromStr. Here's a small example of what I mean: https://gist.github.com/detly/1e03c0adcbdfc27eb1de9650519b923a
Basically I want to be able to do let result: ParseResult<Flip> = after_slash(/* what goes here? */)(input) so I can build up more and more complex parsers with combinators. I'm aware of parse_to!() but it requires the input right there in the macro call, so I can't see how to use it to build things up.
tanriol
@tanriol:matrix.org
[m]
You can probably take the next word with alpha1 and try FromStr on it.
Jason Heeris
@detly
I don't get how I can "try FromStr on it" though - that's the sticking point. I can capture it easy enough, but I can't actually validate it against the list of valid strings without copy-pasting the whole FromStr implementation into a set of tag expressions
Since asking that I've gotten as far as after_slash(<&str>::parse_to) but because parse_to() returns an Option (???) and everything else in nom expects a Result I can't just slot it in.
tanriol
@tanriol:matrix.org
[m]
@detly: Look at the examples of map_res and/or map_opt
Something like map_res(alpha1, |word| Flip::from_str(word)) should work
Jason Heeris
@detly
Ah that got me on the right track! In fact, map_res(..., Flip::from_str) (no closure) worked fine!
Jason Heeris
@detly
Is there a combinator that works like separated_list1() but with different types? If I have, say, TypeA|TypeB|1|2|etc, is there something like things_separated_by("|", (TypeA::from_str, TypeB::from_str, digits1, ...))?
So far I'm using preceded(char('|', map_res(is_not(","), TypeA::from_str))) but there's a lot of them.
tanriol
@tanriol:matrix.org
[m]
Don't think so.
Jason Heeris
@detly
Ah well
Tom Prince
@tomprince
Is there a way to re-use a parser in a contaminator expression? The naive approach in https://gist.github.com/a586f02edd50d27adbd4fa60564cce6f gives me use of moved value: 'item'
tanriol
@tanriol:matrix.org
[m]
&mut item might help...
Tom Prince
@tomprince
I don't think so. After various permutations, I get cannot borrow 'item' as mutable more than once at a time.
Filippos Vasilakis
@vasilakisfil
is there a better way to do the following?
    let part = "[a]";
    let a = delimited::<_, _, _, _, VerboseError<&str>, _, _, _>(tag("["), take_until("]"), tag("]"))(part);
I am fine doing it that way, I am just asking if there is something better/more clever that I might missing
r4v3n6101
@r4v3n6101
Hi everyone . How can I alternate input? (I need getting A or B val if A1 or B1 are present)
Kinda cond or may be alt with cond
Filippos Vasilakis
@vasilakisfil
I am wondering the same thing actually, I need to take different paths depending on what I find and I am not sure what's the best way to do this
r4v3n6101
@r4v3n6101
@vasilakisfil I'm doing same way, working like a charm (https://github.com/r4v3n6101/hlbsp/blob/hlbsp_viewer/file/src/map.rs#L22 delimiting { XXX } blocks)
Dmitry
@neokril

Hi everyone. I'm trying to move from nom 5 to 6 and got some issues with context. My old code which works ok with v5 is this:

fn my_number<'a, E: ParseError<&'a str>>(i: &'a str) -> IResult<&'a str, MyToken, E> {
  let (i, num) = context("my_number", take_while1(move |c| c >= '0' && c <= '9'))(i)?;

Compiles with the following error when I use v6:

the trait bound `E: nom::error::ContextError<&str>` is not satisfied
the trait `nom::error::ContextError<&str>` is not implemented for `E`rustc(E0277)

For now the single solution I came up with is just to remove context. Is there a way to fix this error without removing it?

2 replies
tyler
@tyler:hallada.net
[m]
How do I convert a Result to a nom::IResult?
I know map_res can do that, but it doesn't makes sense to use it in my case because I'm not parsing anything (I'm decompressing bytes already parsed)
tanriol
@tanriol:matrix.org
[m]
tyler: I think the FromExternalError trait is intended for that...
1 reply
tyler
@tyler:hallada.net
[m]
I'm really stuck trying to parse a compressed field. I need to decompress it before parsing it, which requires me to make a new Vec<u8> buffer, decompress into it, and then pass it to the parsing function. But, all nom parsing functions return a reference to the input (the remaining input slice) so I keep getting returns a value referencing data owned by the current function error. Is there a pattern for this that I'm missing?
tanriol
@tanriol:matrix.org
[m]
tyler: You've seen how map_res does that, right?