Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 10:35

    dependabot-preview[bot] on cargo

    (compare)

  • 10:35
    dependabot-preview[bot] closed #433
  • 10:35
    dependabot-preview[bot] commented #433
  • 10:35
    dependabot-preview[bot] labeled #437
  • 10:35
    dependabot-preview[bot] opened #437
  • 10:35

    dependabot-preview[bot] on cargo

    Bump serde from 1.0.99 to 1.0.1… (compare)

  • Sep 13 21:54
    dependabot-preview[bot] labeled #436
  • Sep 13 21:54
    dependabot-preview[bot] opened #436
  • Sep 13 21:54

    dependabot-preview[bot] on cargo

    Bump tracing from 0.1.8 to 0.1.… (compare)

  • Sep 13 18:23
    dependabot-preview[bot] labeled #435
  • Sep 13 18:23
    dependabot-preview[bot] opened #435
  • Sep 13 18:23

    dependabot-preview[bot] on cargo

    Bump hyper from 0.12.34 to 0.12… (compare)

  • Sep 13 16:00
    dependabot-preview[bot] labeled #434
  • Sep 13 16:00
    dependabot-preview[bot] opened #434
  • Sep 13 16:00

    dependabot-preview[bot] on cargo

    Bump rand from 0.7.0 to 0.7.1 … (compare)

  • Sep 08 02:06
    dependabot-preview[bot] labeled #433
  • Sep 08 02:06
    dependabot-preview[bot] opened #433
  • Sep 08 02:06

    dependabot-preview[bot] on cargo

    Bump serde from 1.0.99 to 1.0.1… (compare)

  • Sep 06 13:26

    dependabot-preview[bot] on cargo

    (compare)

  • Sep 06 13:26
    dependabot-preview[bot] closed #423
Stephen Carman
@hntd187
but I started the project basically from getting mad at work and sick of the constant GC thrashing of large elastic clusters
So I started this as a pet project to just build my own, just to be clear it's not affiliated with where I work at all
blitzerr
@blitzerr
On what respects do you envision Toshi to be different from ES ?
Stephen Carman
@hntd187
I think Toshi's use of rust def puts it in a position to do a lot more with a lot less resource wise than any ES cluster
also ES's functionality has grown in a lot of directions, not always in a good way
I'd like to keep toshi very focused, and very complete in that focus
blitzerr
@blitzerr
I would like Toshi to scale better than ES as well. Like supporting several 100 clusters
Stephen Carman
@hntd187
I was surprised when I dug into ES's functionality at just how much stuff elastic can do
some of it was just terrifying to find out it could be done
blitzerr
@blitzerr
and touch the realm of a thousand
that's great. So you already know in where Toshi can be better :)
What do you think your focus will be ?
Stephen Carman
@hntd187
So I figured chasing ES parity was a fool's errand so I figured i'd get the basic document store functionality finished, get clustering finish
and then focus on making toshi really stable and reliable
blitzerr
@blitzerr
Ya, it would be good to not have a feature bloat like ES
Stephen Carman
@hntd187
and as tantivy adds features to it's library toshi will adopt the ones that make sense
and I think its' a veyr achievable goal to do that
and so the first part is done there
toshi functions as a single node document store perfectly fine on a single machine
blitzerr
@blitzerr
Few things, but really really realiable, scalable way
Stephen Carman
@hntd187
the first pass of clustering/networking is almost finished, it's shit, but it's a start
blitzerr
@blitzerr
That's awesome. I would like to contribute to make it distributed
Stephen Carman
@hntd187
so with that nearing it's completion it's basically start to give it the reliability and accurate error handling
blitzerr
@blitzerr
makes sense
Stephen Carman
@hntd187
So the two things i have to still hookup are deleting terms from an index
and distributed index summaries
I've written I think the client side for both
I think both need their serverside rpc calls
blitzerr
@blitzerr
okay
oh and the bulk insert too
I just stubbed it out I didn't think about it at all yet
but I'd imagine it's the harder of what's left but it's reasonable
blitzerr
@blitzerr
Thanks a lot for the code pointer.
Stephen Carman
@hntd187
I mean I'm just open to wherever people wanna dig in it's really up to anyone
blitzerr
@blitzerr
Is there a patch/commit you can point me to ?
For some of what you mentioned ?
Stephen Carman
@hntd187
what specifically ar eyou looking for?
blitzerr
@blitzerr
I built and ran toshi it just works. Awesome.
@hntd187 I will browse the source and ask you questions (probably a lot :) )
Stephen Carman
@hntd187
Okay no probs just lemme know how you fare
blitzerr
@blitzerr
👍
blitzerr
@blitzerr
Hi @hntd187, I am going through the code. You have done quite a lot single handedly. kudos. I am wondering how do you debug or step through.
Stephen Carman
@hntd187
Depends what I am working with
Some of the futures stuff is somewhat...difficult to step through at times
I've kind of learned how to debug it by seeing it fail a million times
but maybe you could be more specific
blitzerr
@blitzerr
Let's say, you want to step through the code. It is a single machine system at the moment, so that might be doable, right ?
I don't have a specific use case. I was just wondering how to.
Stephen Carman
@hntd187
Oh yea, my unit tests give me pretty good insight into individual parts