Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 13 21:54
    dependabot-preview[bot] labeled #436
  • Sep 13 21:54
    dependabot-preview[bot] opened #436
  • Sep 13 21:54

    dependabot-preview[bot] on cargo

    Bump tracing from 0.1.8 to 0.1.… (compare)

  • Sep 13 18:23
    dependabot-preview[bot] labeled #435
  • Sep 13 18:23
    dependabot-preview[bot] opened #435
  • Sep 13 18:23

    dependabot-preview[bot] on cargo

    Bump hyper from 0.12.34 to 0.12… (compare)

  • Sep 13 16:00
    dependabot-preview[bot] labeled #434
  • Sep 13 16:00
    dependabot-preview[bot] opened #434
  • Sep 13 16:00

    dependabot-preview[bot] on cargo

    Bump rand from 0.7.0 to 0.7.1 … (compare)

  • Sep 08 02:06
    dependabot-preview[bot] labeled #433
  • Sep 08 02:06
    dependabot-preview[bot] opened #433
  • Sep 08 02:06

    dependabot-preview[bot] on cargo

    Bump serde from 1.0.99 to 1.0.1… (compare)

  • Sep 06 13:26

    dependabot-preview[bot] on cargo

    (compare)

  • Sep 06 13:26
    dependabot-preview[bot] closed #423
  • Sep 06 13:26
    dependabot-preview[bot] edited #423
  • Sep 06 13:26
    dependabot-preview[bot] commented #423
  • Sep 06 13:26

    dependabot-preview[bot] on cargo

    (compare)

  • Sep 06 13:26

    dependabot-preview[bot] on cargo

    (compare)

  • Sep 06 13:26
    dependabot-preview[bot] closed #424
  • Sep 06 13:26
    dependabot-preview[bot] edited #424
blitzerr
@blitzerr
👍
blitzerr
@blitzerr
Hi @hntd187, I am going through the code. You have done quite a lot single handedly. kudos. I am wondering how do you debug or step through.
Stephen Carman
@hntd187
Depends what I am working with
Some of the futures stuff is somewhat...difficult to step through at times
I've kind of learned how to debug it by seeing it fail a million times
but maybe you could be more specific
blitzerr
@blitzerr
Let's say, you want to step through the code. It is a single machine system at the moment, so that might be doable, right ?
I don't have a specific use case. I was just wondering how to.
Stephen Carman
@hntd187
Oh yea, my unit tests give me pretty good insight into individual parts
but I run locally with high debugging
or I have a lot of extra hardware to run clusters on in my apartment
blitzerr
@blitzerr
thanks a lot
Stephen Carman
@hntd187
and if you're working on something and need help investigating things myself or Paul the creator of tantivy has been incredibly helpful to me
blitzerr
@blitzerr
thanks a lot @hntd187
orangesoup
@orangesoup
Hey! It's probably been asked a lot, but... is there a way to use a custom tokenizer?
Stephen Carman
@hntd187
Actually it's not @orangesoup but no there is no current way, but I'm open to suggestions for how you might want to do this in toshi
and by that I mean how you would want to define it during index creation or something of the sort
blitzerr
@blitzerr
@hntd187 do you nhave
Sorry did not mean to hit send yet
Stephen Carman
@hntd187
Yes I have it
blitzerr
@blitzerr
:)
I was thinking to making Toshi scale and be more elastic than elastic search and I was thinking we would need to split hot shards and move them over to different machines etc. I was thinking that for this I would have to know the Tantivy internals well. So I was wondering if you have something on that. Their github does not contain any design docs
Stephen Carman
@hntd187
So I don’t know about design docs but I know a bit about it from actual usage of tantivy. The segments in tantivy the api for it is somewhat locked down a bit I want to talk to Paul about opening the api up a bit
I think toshi “shuffling” shards is kinda hard right now
blitzerr
@blitzerr
hmm..
Stephen Carman
@hntd187
Yup so I was going to do some tantivy work for their next release to open up that api a bit
so we could shuffle individual segments