Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 18 22:22

    dependabot-preview[bot] on cargo

    (compare)

  • Oct 18 22:22
    dependabot-preview[bot] closed #448
  • Oct 18 22:22
    dependabot-preview[bot] commented #448
  • Oct 18 22:22
    dependabot-preview[bot] labeled #449
  • Oct 18 22:22
    dependabot-preview[bot] opened #449
  • Oct 18 22:22

    dependabot-preview[bot] on cargo

    Bump uuid from 0.7.4 to 0.8.1 … (compare)

  • Oct 18 00:24
    dependabot-preview[bot] labeled #448
  • Oct 18 00:24
    dependabot-preview[bot] opened #448
  • Oct 18 00:24

    dependabot-preview[bot] on cargo

    Bump uuid from 0.7.4 to 0.8.0 … (compare)

  • Oct 16 05:36
    dependabot-preview[bot] labeled #447
  • Oct 16 05:36
    dependabot-preview[bot] opened #447
  • Oct 16 05:36

    dependabot-preview[bot] on cargo

    Bump http from 0.1.18 to 0.1.19… (compare)

  • Oct 14 13:27

    hntd187 on experimental

    Dumping failure in use of anyho… (compare)

  • Oct 13 06:07

    hntd187 on experimental

    After 1000 years I was finally … Still need to do some work on t… (compare)

  • Oct 12 21:22
    hntd187 closed #443
  • Oct 12 21:22
    hntd187 commented #443
  • Oct 12 03:30
    dependabot-preview[bot] labeled #446
  • Oct 12 03:30
    dependabot-preview[bot] opened #446
  • Oct 12 03:30

    dependabot-preview[bot] on cargo

    Bump failure from 0.1.5 to 0.1.… (compare)

  • Oct 11 22:47
    dependabot-preview[bot] labeled #445
Stephen Carman
@hntd187
those are all less exciting things that just need to be grinded out
blitzerr
@blitzerr
What would it take to stabilize ?
Do you already have an unmerged PR ?
Stephen Carman
@hntd187
No most of my work is already merged
most of it deals with health of other members of the clusters
and more real cluster operations not just "I have connections to these other machines, hope they don't fail ever"
blitzerr
@blitzerr
Okay, so you have already built the fault tolerance part
What does stabilizing entail at this point if they are already merged ?
Stephen Carman
@hntd187
I built the ability to do index operations over a network
so you can search an index on a remote machine, add docs all those types of things
stabilizing the functionality would essentially be making those operations more reliable
and adding functionality to let the master reason about the health of it's nodes and what to do about it
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
the problem with a lot of my code is I've written almost all of it in a vacuum so introducing someone else to it is like introducing someone to your weird hobby
blitzerr
@blitzerr
:)
Stephen Carman
@hntd187
but for the most part even simple things like better error handling, better error messages, cleaning up error types maybe better more clear http responses, small enhancements that a beginner could make but are very helpful
blitzerr
@blitzerr
Its been all you this long ?
Stephen Carman
@hntd187
even code organization, naming of stuff
No I've gotten some good help from people, it's mostly what sold me on using tower-grpc for the cluster rpc communications
but yea, I think I've written about 94% of the code
blitzerr
@blitzerr
that's awesome
Stephen Carman
@hntd187
oh also Paul the author of Tantivy has been instrumental in making everything work well
blitzerr
@blitzerr
How did you decide on using Tantivity ?
Stephen Carman
@hntd187
both by making tantivy in the first place and also by helping me be slightly less bad at rust
blitzerr
@blitzerr
Well I can beat you to being bad at rust :)
Stephen Carman
@hntd187
I chose tantivy because it was the only library at the time, there apparently is a lucene rust port available now, but I like the path tanttivy is going
tantivy isn't trying to be lucene and I don't think toshi should try to be ES
blitzerr
@blitzerr
absolutely !
Stephen Carman
@hntd187
but I started the project basically from getting mad at work and sick of the constant GC thrashing of large elastic clusters
So I started this as a pet project to just build my own, just to be clear it's not affiliated with where I work at all
blitzerr
@blitzerr
On what respects do you envision Toshi to be different from ES ?
Stephen Carman
@hntd187
I think Toshi's use of rust def puts it in a position to do a lot more with a lot less resource wise than any ES cluster
also ES's functionality has grown in a lot of directions, not always in a good way
I'd like to keep toshi very focused, and very complete in that focus
blitzerr
@blitzerr
I would like Toshi to scale better than ES as well. Like supporting several 100 clusters
Stephen Carman
@hntd187
I was surprised when I dug into ES's functionality at just how much stuff elastic can do
some of it was just terrifying to find out it could be done
blitzerr
@blitzerr
and touch the realm of a thousand
that's great. So you already know in where Toshi can be better :)
What do you think your focus will be ?
Stephen Carman
@hntd187
So I figured chasing ES parity was a fool's errand so I figured i'd get the basic document store functionality finished, get clustering finish
and then focus on making toshi really stable and reliable
blitzerr
@blitzerr
Ya, it would be good to not have a feature bloat like ES
Stephen Carman
@hntd187
and as tantivy adds features to it's library toshi will adopt the ones that make sense
and I think its' a veyr achievable goal to do that
and so the first part is done there
toshi functions as a single node document store perfectly fine on a single machine
blitzerr
@blitzerr
Few things, but really really realiable, scalable way
Stephen Carman
@hntd187
the first pass of clustering/networking is almost finished, it's shit, but it's a start
blitzerr
@blitzerr
That's awesome. I would like to contribute to make it distributed