Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
  • Nov 25 20:48
    mtelka opened #474
  • Nov 24 04:42
    alecthomas closed #473
  • Nov 24 04:00
    Ashley123456789 opened #473
  • Nov 21 22:30
    lmittmann closed #282
  • Nov 21 22:30
    lmittmann edited #282
  • Nov 19 02:38
    willfaught closed #153
  • Nov 19 02:38
    willfaught commented #153
  • Nov 18 21:43
    alecthomas closed #284
  • Nov 18 21:43
    alecthomas locked #284
  • Nov 18 20:55
    Jmoore1127 edited #284
  • Nov 18 20:53
    Jmoore1127 opened #284
  • Nov 18 18:01
    petee-d commented #263
  • Nov 16 17:06
    petee-d synchronize #263
  • Nov 16 13:24
    alecthomas commented #263
  • Nov 16 13:02

    alecthomas on peeking-lexer-pointers


  • Nov 16 13:02
    alecthomas closed #283
  • Nov 16 13:02

    alecthomas on master

    refactor: change PeekingLexer t… (compare)

  • Nov 16 13:02
    alecthomas commented #263
  • Nov 16 13:01
    alecthomas ready_for_review #283
  • Nov 16 13:01
    alecthomas synchronize #283
Alec Thomas
i think the validator you're looking for is ExactSequence
fwiw :)
Naveen Nathan
Ok I have a better understanding now of how a basic list works - it's just a list of unordered valid inputs - so ['a', 'b'] in a schema will accept ['b']*100. So it will not validate against the 'a', but will validate against the 'b' for each item in the list. Now going based on my above example - I still don't know why it failed the assertion (I get I want an ExactSequence and not a list of objects that are valid inputs), but i'm pretty sure it was receiving a valid input - and that the input - specifically the timing section, should've matched the valid input list for the timings in the schema above:
  "success": true,
  "response": { "unnecessary": "crap" },
  "timing": [
      "duration_in_ms": 629,
      "source": "backend_api_call"
      "duration_in_ms": 655,
      "source": "create"
  "messages": [
    "Customer created",
  "time": "2019-08-12T03:53:41.070455Z"
Alec Thomas
no, it will also validate b
it failed because {...} matched, but then failed on a sub-key
voluptuous will not backtrack, so once it enters a sub-schema (ie. {...}) it will not exit
Naveen Nathan
The brain is ticking again. I get it -- had to read the (lack of) backtrack explanation in README, but makes sense now.
Ryan Gonzalez
Is there a way to have posix-style parsing, where option parsing stops after the first argument?
E.g. if I run myapp -a b -c, b -c are non-options
I'm using kingpin for a Go tool since it's pretty much the sanest Go option parsing library by a long shot, but I really need to support this...
Alec Thomas
No that is not supported. Kingpin supports -- to separate remaining args.
Mmm actually there is an interspersed option that I forgot about
Ryan Gonzalez
Oof that sucks...would there be e.g. a PR accepted to add this somehow? GLib's GOptionContext calls it "struct posix mode", so maybe a .StrictPosix() on a Command instance?
Alec Thomas
Did you try interspersed?
Ryan Gonzalez
Ooo that might actually be what I need, I'll try it tomorrow
Ryan Gonzalez
Sorry for all the noise, but out of curiosity was there a reason alecthomas/kingpin#245 was never merged?
Nitish Malhotra
@alecthomas How can I Capture Time formatted values using participle. I am trying to capture [2016-04-15T20:17:00.310Z]. Not sure how to proceed ?!
Alec Thomas
@nitishm if you're using the regex lexer (for example) you'll need to match and emit a token. eg. (?P<Time>\[\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d\.\d{3}Z\])
then you'll need to implement the participle.Capture interface on a custom type
eg. something like this
type Time time.Time

func (t *Time) Capture(tokens []string) error {
  s, err := time.Parse("...", tokens[0])
  *t = s
  return nil
Alec Thomas
you could also omit the second step and just capture the string
Nitish Malhotra
Yup I ended up creating a lexer.Regexp() for this. It's a fascinating tool. Am still trying to wrap my head around it 😂. But every iteration it seems to make more sense.
I am trying to write a custom log parser for envoy access logs.
Alec Thomas
/all I'm moving this to the Gopher's Slack, as Gitter is effectively dead/unmaintained.
@/all I'm moving this to the Gopher's Slack, as Gitter is effectively dead/unmaintained.
Tariq Ibrahim
Gitter effectively dead?
Hey guys.Is there any book recommended for me to understand ECS?
Hey , i need some assistance with an issue im having. For EntityX, i have a system thats recieving ComponentRemoved Event. However, when i destroy an entity, it doesnt seem like the system is recieving the ComponentRemovedEvent.
The component is attached to the entity
Hello there. I have a question about voluptuous. Is there any way to restrict a name of a field
something like:
schema = Schema({
    lambda name: '.' in name: str
Aurélien Gasser
Hi Alec, thanks a lot for your Go serializer benchmark!
I'm trying to compress/decompress data - that data will only be used by my Go program.
I'm having a hard time understanding the "Totals" section of https://github.com/alecthomas/go_serialization_benchmarks. What do the columns mean? In particular, I'm interested in: speed of compression / speed of decompression / compressed payload size. Which columns should I look at ? How can I compare the profobuf row with the gob row?
Thank you!
Michael Pfaff
I'm interested in something similar to Aurélien, but I have a slight focus on deserialization speed
re chroma: hi Alec, any chance of a version bump on chroma so can hugo can pick up the latest?
Иван Сердюк
Hi. Got such a bug:

$ go build -i -v -x
mkdir -p $WORK/b017/
mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/alecthomas/
cp /home/oceanfish81/.cache/go-build/3f/3f17f09c67a788726b59d58fcca2958f8e89f05f49c82be2d1c6e75183bf139a-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/alecthomas/libunits.a
mkdir -p $WORK/b020/
mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/nicksnyder/go-i18n/i18n/
cp /home/oceanfish81/.cache/go-build/1d/1d82d5034089a023b328cb96570d592006677007d9489fa77d5a6a64c584b247-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/nicksnyder/go-i18n/i18n/liblanguage.a
rm -r $WORK/b017/
rm -r $WORK/b020/
mkdir -p $WORK/b022/
cp /home/oceanfish81/.cache/go-build/00/007e3ade50871872acfa250f6dcd292a0f846c61c8739ab18f79bccfcea98082-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/nicksnyder/go-i18n/i18n/libtranslation.a
rm -r $WORK/b022/
mkdir -p $WORK/b025/
mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pelletier/
cp /home/oceanfish81/.cache/go-build/1e/1e1e4b85589ef24d39f9fc768b9bc7f8719ca407fca7c59fe6f650ae7bc96d43-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/pelletier/libgo-toml.a
mkdir -p $WORK/b035/
mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/gopkg.in/
cp /home/oceanfish81/.cache/go-build/17/17b6e6437480a8d8489dcaf96914181618b1641928828f8227fa59e5c2b5fc0b-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/gopkg.in/libyaml.v2.a
rm -r $WORK/b025/
rm -r $WORK/b035/
mkdir -p $WORK/b019/
cp /home/oceanfish81/.cache/go-build/cf/cf0ac62a87259aadf849489ca255fad6bf03bf9a1a31bb4df5dcf87ce8232e45-d /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/nicksnyder/go-i18n/i18n/libbundle.a
rm -r $WORK/b019/
mkdir -p $WORK/b018/
mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/nicksnyder/go-i18n/
cp /home/oceanfish81/.cache/go-build/26/267db29519d8a5d352c042db18c08aa7a0f3e408c4b8720cc52aa9195c0951ab-d /home/oceanfish81/go/pkg/gccgo_linuxamd64/github.com/nicksnyder/go-i18n/libi18n.a
rm -r $WORK/b018/
mkdir -p $WORK/b001/
cd $WORK
/home/oceanfish81/gollvm_dist/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true
mkdir -p $WORK/b001/importcfgroot/github.com/google
ln -s /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/google/libshlex.a $WORK/b001/importcfgroot/github.com/google/libshlex.a
mkdir -p $WORK/b001/importcfgroot/gopkg.in/alecthomas
ln -s /home/oceanfish81/go/pkg/gccgo_linux_amd64/gopkg.in/alecthomas/libkingpin.v3-unstable.a $WORK/b001/importcfgroot/gopkg.in/alecthomas/libkingpin.v3-unstable.a
cd /home/oceanfish81/go_projects/gometalinter
/home/oceanfish81/gollvmdist/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-relative-import-path=/home/oceanfish81/go_projects/gometalinter -o $WORK/b001/go.o -I $WORK/b001/importcfgroot ./aggregate.go ./checkstyle.go ./config.go ./directives.go ./execute.go ./issue.go ./linters.go ./main.go ./partition.go ./stringset.go


./main.go:36:95: error: argument 1 has incompatible type (different parameter types)
./main.go:38:101: error: argument 1 has incompatible type (different parameter types)
./main.go:39:100: error: argument 1 has incompatible type (different parameter types)
./main.go:40:84: error: argument 1 has incompatible type (different parameter types)
./main.go:43:57: error: argument 1 has incompatible type (different parameter types)
./main.go:44:55: error: argument 1 has incompatible type (different parameter types)
./main.go:200:13: error: argument 1 has incompatible type (different parameter types)


@ https://github.com/alecthomas/participle


I appreciate the work on github.com/alecthomas/participle library :thumbsup: !

I'm writing a dynamic parser/lexer for EBNF grammar. Currently, I'm trying to use Go EBNF grammar that I derived from Go specification HTML (https://gist.github.com/vellotis/d59980f924309e0258222f26c02218fb). For parsing EBNF grammar I used Go's https://github.com/golang/exp/blob/master/ebnf/parser.go plus a bit of instrumentation of the grammar productions so it would fulfill my needs.
I have almost finished with my 1st working version of the lexer as well. But with a few complications with the Go's EBNF grammar. Probably the problem is how I handle which of the EBNF alternations should be resolved if multiple of them return a value.


Go's short variable declaration as input
simple := "value"

        type testableShortVarDecl struct {
            ShortVarDecl `@@ term`

        // Collect instrumented version of github.com/golang/exp/ebnf/parser.go returned grammar
        grammar := GoGrammar()
        // Putting together my lexer with a starting production of "StatementList"
        lexerDefinition, err := NewLexer(grammar, ebnf.InitialProduction("StatementList"))
        require.NoError(t, err)

        // Build a participle parser
        parser, err := participle.Build(&testableShortVarDecl{},
        require.NoError(t, err)

        shortVarDecl := &testableShortVarDecl{}

        err = parser.ParseString("", `single := "value"`, shortVarDecl)

        assert.NoError(t, err)

The problem is that for production "SimpleStmt" it finds a matching lexical token for ExpressionStmt -> Expression -> UnaryExpr -> PrimaryExpr -> Operand -> OperandName -> identifier(= "simple") and resolves the whole alternation and doesn't reach to ShortVarDecl at all.

  • StatementList = { Statement ";" } .
  • Statement =
    Declaration | LabeledStmt | SimpleStmt |
    GoStmt | ReturnStmt | BreakStmt | ContinueStmt | GotoStmt |
    FallthroughStmt | Block | IfStmt | SwitchStmt | SelectStmt | ForStmt |
    DeferStmt .
  • SimpleStmt = EmptyStmt | ExpressionStmt | SendStmt | IncDecStmt | Assignment | ShortVarDecl .
  • ExpressionStmt = Expression .
  • Expression = UnaryExpr | Expression binary_op Expression .
  • UnaryExpr = PrimaryExpr | unary_op UnaryExpr .
  • PrimaryExpr =
    Operand |
    Conversion |
    MethodExpr |
    PrimaryExpr Selector |
    PrimaryExpr Index |
    PrimaryExpr Slice |
    PrimaryExpr TypeAssertion |
    PrimaryExpr Arguments .
  • Operand = Literal | OperandName | "(" Expression ")" .
  • OperandName = identifier | QualifiedIdent .
    = "simple"

Does any of you have any expertise in EBNF and picking a resolving alternate element when many of them resolve?
Or could you point me to another discussion board? I know my last option is StackOverflow, but I would like to have a discussion, not just a solution at first.


I have currently implemented the alternation resolving as a sequential matching of the elements. The first one that matches will be selected.
I know that the EBNF specification states that there is no specific order for how the alternation elements matching should be visited. But how to understand it in terms of the selection of the true matching element?
Discussing by myself. I probably need some Backtracking


Backtracking occurs in the Sequence expression. If e1 succeeds and consumes
some input, and then e2 fails, the cursor is moved back to where it was before
trying e1. If this Sequence was called as the first alternative in a Choice, Choice
has an opportunity to try another alternative on the same input. However, the
backtracking is limited: once e1 in the Choice e1|e2 succeeded, e2 will never be
tried on the same input, even if the parse fails later on.

Rubber ducking worked! Thanks! :smile:

trying to use participle to parse fluent bit config files.
im using https://github.com/calyptia/go-fluentbit-config/blob/master/parser.go as a base and
an example input file is

    Flush     5
    Daemon    off
    Log_Level debug

    Name  cpu
    Tag   my_cpu

    Name  stdout
    Match my*cpu

I dont want as many complex "value" fields. i just want to parse everything as a string and I can't work out how to do it.

for instances i can have ${PATH} or <tags> and i just want to parse it as a string
i guess what im really asking is how do i parse
  key value

  key value
  key2 value2

entityx is using component ids as indexes

but i dont understand something

the entityx uses similar to this type of code:

using CompTypeId = std::size_t;
CompTypeId newCompId()
    static CompTypeId idCount{0u};
    return idCount++;

template<class CompType>
CompTypeId getComponentId()
    // Static & const: to return the same id whenever same the type given
    static const CompTypeId uniqueId{newCompId()}; 
    return uniqueId;

to create and use component ids but this can increase the id count without registering the component pools

which means it cant be used as index
because there are a lot of functions that use similar functionality like getComponentId which can increase the counter if the component doesnt exist

am i missing something?

@alecthomas ?