Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 08:31
    mysticaltech closed #469
  • 08:31
    mysticaltech commented #469
  • Nov 11 15:28
    hntd187 commented #469
  • Nov 11 13:41
    mysticaltech commented #469
  • Nov 11 05:46
    hntd187 commented #469
  • Nov 10 21:02
    mysticaltech opened #469
  • Nov 10 04:53
    dependabot-preview[bot] labeled #468
  • Nov 10 04:53
    dependabot-preview[bot] opened #468
  • Nov 10 04:53

    dependabot-preview[bot] on cargo

    Bump tantivy from 0.10.2 to 0.1… (compare)

  • Nov 10 02:42
    dependabot-preview[bot] labeled #467
  • Nov 10 02:42
    dependabot-preview[bot] opened #467
  • Nov 10 02:42

    dependabot-preview[bot] on cargo

    Bump thiserror from 1.0.4 to 1.… (compare)

  • Nov 09 07:52
    dependabot-preview[bot] labeled #466
  • Nov 09 07:52
    dependabot-preview[bot] opened #466
  • Nov 09 07:52

    dependabot-preview[bot] on cargo

    Bump hostname from 0.1.5 to 0.2… (compare)

  • Nov 08 03:37
    dependabot-preview[bot] labeled #465
  • Nov 08 03:37
    dependabot-preview[bot] opened #465
  • Nov 08 03:37

    dependabot-preview[bot] on cargo

    Bump isahc from 0.7.5 to 0.7.6 … (compare)

  • Nov 07 18:42

    dependabot-preview[bot] on cargo

    (compare)

  • Nov 07 18:42
    dependabot-preview[bot] closed #462
Stephen Carman
@hntd187
I implemented it awhile back but something odd about them caused me to remove them I don’t remember exactly why
blitzerr
@blitzerr
hello everyone
Stephen Carman
@hntd187
hi! @blitzerr welcome!
blitzerr
@blitzerr
Thanks @hntd187
blitzerr
@blitzerr
@hntd187 I would like to contribute to Toshi.
How should I begin ? Anything upcoming ?
Stephen Carman
@hntd187
I have a lot of the naive clustering things worked out @blitzerr it'd help to be able to stabilize that and make it more feature full was my next task I think
I also wanted to do real release setup, rewrite the docs, make the website not shit
those are all less exciting things that just need to be grinded out
blitzerr
@blitzerr
What would it take to stabilize ?
Do you already have an unmerged PR ?
Stephen Carman
@hntd187
No most of my work is already merged
most of it deals with health of other members of the clusters
and more real cluster operations not just "I have connections to these other machines, hope they don't fail ever"
blitzerr
@blitzerr
Okay, so you have already built the fault tolerance part
What does stabilizing entail at this point if they are already merged ?
Stephen Carman
@hntd187
I built the ability to do index operations over a network
so you can search an index on a remote machine, add docs all those types of things
stabilizing the functionality would essentially be making those operations more reliable
and adding functionality to let the master reason about the health of it's nodes and what to do about it
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
the problem with a lot of my code is I've written almost all of it in a vacuum so introducing someone else to it is like introducing someone to your weird hobby
blitzerr
@blitzerr
:)
Stephen Carman
@hntd187
but for the most part even simple things like better error handling, better error messages, cleaning up error types maybe better more clear http responses, small enhancements that a beginner could make but are very helpful
blitzerr
@blitzerr
Its been all you this long ?
Stephen Carman
@hntd187
even code organization, naming of stuff
No I've gotten some good help from people, it's mostly what sold me on using tower-grpc for the cluster rpc communications
but yea, I think I've written about 94% of the code
blitzerr
@blitzerr
that's awesome
Stephen Carman
@hntd187
oh also Paul the author of Tantivy has been instrumental in making everything work well
blitzerr
@blitzerr
How did you decide on using Tantivity ?
Stephen Carman
@hntd187
both by making tantivy in the first place and also by helping me be slightly less bad at rust
blitzerr
@blitzerr
Well I can beat you to being bad at rust :)
Stephen Carman
@hntd187
I chose tantivy because it was the only library at the time, there apparently is a lucene rust port available now, but I like the path tanttivy is going
tantivy isn't trying to be lucene and I don't think toshi should try to be ES
blitzerr
@blitzerr
absolutely !
Stephen Carman
@hntd187
but I started the project basically from getting mad at work and sick of the constant GC thrashing of large elastic clusters
So I started this as a pet project to just build my own, just to be clear it's not affiliated with where I work at all
blitzerr
@blitzerr
On what respects do you envision Toshi to be different from ES ?
Stephen Carman
@hntd187
I think Toshi's use of rust def puts it in a position to do a lot more with a lot less resource wise than any ES cluster
also ES's functionality has grown in a lot of directions, not always in a good way
I'd like to keep toshi very focused, and very complete in that focus
blitzerr
@blitzerr
I would like Toshi to scale better than ES as well. Like supporting several 100 clusters
Stephen Carman
@hntd187
I was surprised when I dug into ES's functionality at just how much stuff elastic can do
some of it was just terrifying to find out it could be done
blitzerr
@blitzerr
and touch the realm of a thousand
that's great. So you already know in where Toshi can be better :)
What do you think your focus will be ?
Stephen Carman
@hntd187
So I figured chasing ES parity was a fool's errand so I figured i'd get the basic document store functionality finished, get clustering finish
and then focus on making toshi really stable and reliable