Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 00:24
    dependabot-preview[bot] labeled #448
  • 00:24
    dependabot-preview[bot] opened #448
  • 00:24

    dependabot-preview[bot] on cargo

    Bump uuid from 0.7.4 to 0.8.0 … (compare)

  • Oct 16 05:36
    dependabot-preview[bot] labeled #447
  • Oct 16 05:36
    dependabot-preview[bot] opened #447
  • Oct 16 05:36

    dependabot-preview[bot] on cargo

    Bump http from 0.1.18 to 0.1.19… (compare)

  • Oct 14 13:27

    hntd187 on experimental

    Dumping failure in use of anyho… (compare)

  • Oct 13 06:07

    hntd187 on experimental

    After 1000 years I was finally … Still need to do some work on t… (compare)

  • Oct 12 21:22
    hntd187 closed #443
  • Oct 12 21:22
    hntd187 commented #443
  • Oct 12 03:30
    dependabot-preview[bot] labeled #446
  • Oct 12 03:30
    dependabot-preview[bot] opened #446
  • Oct 12 03:30

    dependabot-preview[bot] on cargo

    Bump failure from 0.1.5 to 0.1.… (compare)

  • Oct 11 22:47
    dependabot-preview[bot] labeled #445
  • Oct 11 22:47
    dependabot-preview[bot] opened #445
  • Oct 11 22:47

    dependabot-preview[bot] on cargo

    Bump tower-buffer from 0.1.1 to… (compare)

  • Oct 11 20:07

    hntd187 on experimental

    cargofmt (compare)

  • Oct 11 20:05

    hntd187 on experimental

    I somehow forgot this? (compare)

  • Oct 11 20:03

    hntd187 on experimental

    I somehow forgot this? (compare)

  • Oct 11 13:29

    hntd187 on experimental

    Minor dep upgrades (compare)

Stephen Carman
@hntd187
and adding functionality to let the master reason about the health of it's nodes and what to do about it
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
the problem with a lot of my code is I've written almost all of it in a vacuum so introducing someone else to it is like introducing someone to your weird hobby
blitzerr
@blitzerr
:)
Stephen Carman
@hntd187
but for the most part even simple things like better error handling, better error messages, cleaning up error types maybe better more clear http responses, small enhancements that a beginner could make but are very helpful
blitzerr
@blitzerr
Its been all you this long ?
Stephen Carman
@hntd187
even code organization, naming of stuff
No I've gotten some good help from people, it's mostly what sold me on using tower-grpc for the cluster rpc communications
but yea, I think I've written about 94% of the code
blitzerr
@blitzerr
that's awesome
Stephen Carman
@hntd187
oh also Paul the author of Tantivy has been instrumental in making everything work well
blitzerr
@blitzerr
How did you decide on using Tantivity ?
Stephen Carman
@hntd187
both by making tantivy in the first place and also by helping me be slightly less bad at rust
blitzerr
@blitzerr
Well I can beat you to being bad at rust :)
Stephen Carman
@hntd187
I chose tantivy because it was the only library at the time, there apparently is a lucene rust port available now, but I like the path tanttivy is going
tantivy isn't trying to be lucene and I don't think toshi should try to be ES
blitzerr
@blitzerr
absolutely !
Stephen Carman
@hntd187
but I started the project basically from getting mad at work and sick of the constant GC thrashing of large elastic clusters
So I started this as a pet project to just build my own, just to be clear it's not affiliated with where I work at all
blitzerr
@blitzerr
On what respects do you envision Toshi to be different from ES ?
Stephen Carman
@hntd187
I think Toshi's use of rust def puts it in a position to do a lot more with a lot less resource wise than any ES cluster
also ES's functionality has grown in a lot of directions, not always in a good way
I'd like to keep toshi very focused, and very complete in that focus
blitzerr
@blitzerr
I would like Toshi to scale better than ES as well. Like supporting several 100 clusters
Stephen Carman
@hntd187
I was surprised when I dug into ES's functionality at just how much stuff elastic can do
some of it was just terrifying to find out it could be done
blitzerr
@blitzerr
and touch the realm of a thousand
that's great. So you already know in where Toshi can be better :)
What do you think your focus will be ?
Stephen Carman
@hntd187
So I figured chasing ES parity was a fool's errand so I figured i'd get the basic document store functionality finished, get clustering finish
and then focus on making toshi really stable and reliable
blitzerr
@blitzerr
Ya, it would be good to not have a feature bloat like ES
Stephen Carman
@hntd187
and as tantivy adds features to it's library toshi will adopt the ones that make sense
and I think its' a veyr achievable goal to do that
and so the first part is done there
toshi functions as a single node document store perfectly fine on a single machine
blitzerr
@blitzerr
Few things, but really really realiable, scalable way
Stephen Carman
@hntd187
the first pass of clustering/networking is almost finished, it's shit, but it's a start
blitzerr
@blitzerr
That's awesome. I would like to contribute to make it distributed
Stephen Carman
@hntd187
so with that nearing it's completion it's basically start to give it the reliability and accurate error handling
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
So the two things i have to still hookup are deleting terms from an index
and distributed index summaries
I've written I think the client side for both
I think both need their serverside rpc calls
blitzerr
@blitzerr
okay
oh and the bulk insert too
I just stubbed it out I didn't think about it at all yet
but I'd imagine it's the harder of what's left but it's reasonable