Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 05:36
    dependabot-preview[bot] labeled #447
  • 05:36
    dependabot-preview[bot] opened #447
  • 05:36

    dependabot-preview[bot] on cargo

    Bump http from 0.1.18 to 0.1.19… (compare)

  • Oct 14 13:27

    hntd187 on experimental

    Dumping failure in use of anyho… (compare)

  • Oct 13 06:07

    hntd187 on experimental

    After 1000 years I was finally … Still need to do some work on t… (compare)

  • Oct 12 21:22
    hntd187 closed #443
  • Oct 12 21:22
    hntd187 commented #443
  • Oct 12 03:30
    dependabot-preview[bot] labeled #446
  • Oct 12 03:30
    dependabot-preview[bot] opened #446
  • Oct 12 03:30

    dependabot-preview[bot] on cargo

    Bump failure from 0.1.5 to 0.1.… (compare)

  • Oct 11 22:47
    dependabot-preview[bot] labeled #445
  • Oct 11 22:47
    dependabot-preview[bot] opened #445
  • Oct 11 22:47

    dependabot-preview[bot] on cargo

    Bump tower-buffer from 0.1.1 to… (compare)

  • Oct 11 20:07

    hntd187 on experimental

    cargofmt (compare)

  • Oct 11 20:05

    hntd187 on experimental

    I somehow forgot this? (compare)

  • Oct 11 20:03

    hntd187 on experimental

    I somehow forgot this? (compare)

  • Oct 11 13:29

    hntd187 on experimental

    Minor dep upgrades (compare)

  • Oct 10 04:48

    hntd187 on experimental

    Bulk fix (#432) * Most changes… Dep updates, started on some ag… Added github actions and 12 more (compare)

  • Oct 10 04:00
    Chopinsky commented #112
  • Oct 10 03:58
    Chopinsky commented #112
blitzerr
@blitzerr
Ya, it would be good to not have a feature bloat like ES
Stephen Carman
@hntd187
and as tantivy adds features to it's library toshi will adopt the ones that make sense
and I think its' a veyr achievable goal to do that
and so the first part is done there
toshi functions as a single node document store perfectly fine on a single machine
blitzerr
@blitzerr
Few things, but really really realiable, scalable way
Stephen Carman
@hntd187
the first pass of clustering/networking is almost finished, it's shit, but it's a start
blitzerr
@blitzerr
That's awesome. I would like to contribute to make it distributed
Stephen Carman
@hntd187
so with that nearing it's completion it's basically start to give it the reliability and accurate error handling
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
So the two things i have to still hookup are deleting terms from an index
and distributed index summaries
I've written I think the client side for both
I think both need their serverside rpc calls
blitzerr
@blitzerr
okay
oh and the bulk insert too
I just stubbed it out I didn't think about it at all yet
but I'd imagine it's the harder of what's left but it's reasonable
blitzerr
@blitzerr
Thanks a lot for the code pointer.
Stephen Carman
@hntd187
I mean I'm just open to wherever people wanna dig in it's really up to anyone
blitzerr
@blitzerr
Is there a patch/commit you can point me to ?
For some of what you mentioned ?
Stephen Carman
@hntd187
what specifically ar eyou looking for?
blitzerr
@blitzerr
I built and ran toshi it just works. Awesome.
@hntd187 I will browse the source and ask you questions (probably a lot :) )
Stephen Carman
@hntd187
Okay no probs just lemme know how you fare
blitzerr
@blitzerr
👍
blitzerr
@blitzerr
Hi @hntd187, I am going through the code. You have done quite a lot single handedly. kudos. I am wondering how do you debug or step through.
Stephen Carman
@hntd187
Depends what I am working with
Some of the futures stuff is somewhat...difficult to step through at times
I've kind of learned how to debug it by seeing it fail a million times
but maybe you could be more specific
blitzerr
@blitzerr
Let's say, you want to step through the code. It is a single machine system at the moment, so that might be doable, right ?
I don't have a specific use case. I was just wondering how to.
Stephen Carman
@hntd187
Oh yea, my unit tests give me pretty good insight into individual parts
but I run locally with high debugging
or I have a lot of extra hardware to run clusters on in my apartment
blitzerr
@blitzerr
thanks a lot
Stephen Carman
@hntd187
and if you're working on something and need help investigating things myself or Paul the creator of tantivy has been incredibly helpful to me
blitzerr
@blitzerr
thanks a lot @hntd187
orangesoup
@orangesoup
Hey! It's probably been asked a lot, but... is there a way to use a custom tokenizer?
Stephen Carman
@hntd187
Actually it's not @orangesoup but no there is no current way, but I'm open to suggestions for how you might want to do this in toshi
and by that I mean how you would want to define it during index creation or something of the sort
blitzerr
@blitzerr
@hntd187 do you nhave
Sorry did not mean to hit send yet
Stephen Carman
@hntd187
Yes I have it
blitzerr
@blitzerr
:)
I was thinking to making Toshi scale and be more elastic than elastic search and I was thinking we would need to split hot shards and move them over to different machines etc. I was thinking that for this I would have to know the Tantivy internals well. So I was wondering if you have something on that. Their github does not contain any design docs
Stephen Carman
@hntd187
So I don’t know about design docs but I know a bit about it from actual usage of tantivy. The segments in tantivy the api for it is somewhat locked down a bit I want to talk to Paul about opening the api up a bit