Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    David Stemmer
    @weefbellington
    looks like buffer supports sampling as well so that might be what you want
    @eyeced ^
    Abhinav Solan
    @eyeced
    So is should I just create common observable and merge in all the incoming observables in that .. and use buffer with that ...
    David Stemmer
    @weefbellington
    yep
    Abhinav Solan
    @eyeced
    thanks a lot @weefbellington
    David Stemmer
    @weefbellington
    sure
    Abhinav Solan
    @eyeced
    I did a setup for cassandra cluster which was giving around 60k inserts performance .. when I used single insertes it came down 20k inserts .. I will try out with this suggestion .. this might put it back to original performance
    60k / sec
    David Stemmer
    @weefbellington
    @eyeced you could use a PublishSubject as your source observable if you are OK with your observable being "hot"
    each time you need to push an event to the stream, you would call PublishSubject#onNext
    Abhinav Solan
    @eyeced
    Ok i will try out with that too ..
    David Stemmer
    @weefbellington
    personally I don't recommend Subjects ("hot" observables) in the general case (they can be problematic because you can start pushing data to them before anything is subscribed to them) but for some scenarios they can be useful
    Dorus
    @Dorus
    I could swear Sample only returns the latest value (if any) every time interval, while Buffer and Window allow you to group all events in a certain timespan together (Probably buffer in this case, as that returns the collected values in a collection, while window keeps them in a nested observable so you would still need to aggregate them). But i'm coming from Rx.NET, so it might be different here with RxJava.
    David Stemmer
    @weefbellington
    @Dorus yeah looks like you're right, buffer is what he wants here
    I usually use sample to manage backpressure when I don't care about receiving every update
    Dorus
    @Dorus
    yeah exactly
    Dorus
    @Dorus
    Also, combining streams is simply Merge, as long as the clients dont emit errors. I see there is even a mergeDelayError in case you want to wait for all other clients to complete first. If bulk insert from a single client is easier than combining events from different clients, do the buffer operation before you merge.
    Or connect to the clients inside the sequence with flatMap.
    Abhinav Solan
    @eyeced
    I am exposing an API so I would not be able to control the client .. they are just going to call "put(Observable<T> t) " and in our case it would be just one value most of the time so while doing the insert in put call I want to merge all the incoming observables.
    Dorus
    @Dorus
    If you work with the previous mentioned PublishSubject<Observable<T>>, you can just call PublishSubject.OnNext(t)inside put. Do first subscribe to the PublishSubject with PublishSubject.asObservable().Merge().Subscribe(...) (doesn't strictly need the asObservable if you subscribe locally). Just make sure you don't expose the Subject anywhere.
    Abhinav Solan
    @eyeced
    PublishSubject seems the way ..
    Dorus
    @Dorus
    The general concession is to avoid Subjects, but i still haven't found a way to avoid them in these situations, just as long as you keep the Subject private and local it shouldn't become a mess.
    Dorus
    @Dorus
    I mean, i would love to call Observer.Create(), but that doesn't allow you to call merge anymore. So you kinda need
    private Observer ob;
    public constructor() {
        PublishSubject<Observable<T>> sub = PublishSubject.create();
        sub.merge().subscribe(...);
        ob = sub.asObserver();
    }
    public void put(Observable<T> t) {
        ob.onNext(t);
    }
    (Please anybody correct me if i'm wrong)
    Abhinav Solan
    @eyeced
    Thanks @Dorus this worked beautifully here is the implementation I did

    public class SubjectTest {
    private static final SubjectTest INSTANCE = new SubjectTest();
    private PublishSubject<Integer> subject;

    public SubjectTest() {
        subject = PublishSubject.create();
        subject.buffer(10).subscribe(System.out::println);
    }
    
    void put(Observable<Integer> obs) {
        obs.subscribe(integer -> subject.onNext(integer));
    }
    
    public static void main(String[] args) {
        ExecutorService executor = Executors.newWorkStealingPool(10);
        Observable.range(1, 100).subscribe(integer -> {
            executor.execute(() -> INSTANCE.put(Observable.just(integer)));
        });
    }

    }

    Dorus
    @Dorus
    you're missing a few spaces in front.
    (you can edit)
    Anyway looks good, but arn't you missing a merge() before buffer?
    Abhinav Solan
    @eyeced
    in RxJava there is no merge() call for the subject
    Dorus
    @Dorus
    really?
    Abhinav Solan
    @eyeced
    There is a mergeWith(Observable<T> t) but I don't think that's what you meant
    Dorus
    @Dorus
    http://reactivex.io/RxJava/javadoc/rx/subjects/PublishSubject.html -> Methods inherited from class rx.Observable -> merge.
    no i'm looking for the merge() that goes from Observable<Observable<Integer>> to Observable<Integer>. Mmm
    I do see merge(Observable<? extends Observable<? extends T>> source), but i'm not 100% sure how you call it.
    Abhinav Solan
    @eyeced
    ohh that's on class level only
    Dorus
    @Dorus
    The docs i'm reading here are pretty clear you can do merge() on a observable of observables. Did you find it?

    http://reactivex.io/documentation/operators/merge.html

    Instead of passing multiple Observables (up to nine) into merge, you could also pass in a List<> (or other Iterable) of Observables, an Array of Observables, or even an Observable that emits Observables, and merge will merge their output into the output of a single Observable:
    If you pass in an Observable of Observables, you have the option of also passing in a value indicating to merge the maximum number of those Observables it should attempt to be subscribed to simultaneously. Once it reaches this maximum subscription count, it will refrain from subscribing to any other Observables emitted by the source Observable until such time as one of the already-subscribed-to Observables issues an onCompleted notification.

    Javadoc: merge(Observable<Observable>)
    Javadoc: merge(Observable<Observable>,int)

    Abhinav Solan
    @eyeced
    Ok let me try it using this then .. but from the code snippet you send out you were calling merge only on the created subject instance only .. these are static methods not for the created instance
    Dorus
    @Dorus
    In C# it's a extension method. Those are static but work on instances. Not 100% sure how that flies in Java, don't have it set up here to test.
    Dorus
    @Dorus
    I think it's rx.Observable.merge(subject).buffer(10).subscribe(System.out::println);
    Indeed, just a static call.
    That was me writing in C# style before :sweat_smile:
    Abhinav Solan
    @eyeced
    Using the previous method I am able to print out all numbers .. using it merge is showing up o nly 90 elements from the 100
    Dorus
    @Dorus
        public class App {
            private Observer<Observable<Integer>> ob;
    
            public App() {
                PublishSubject<Observable<Integer>> sub = PublishSubject.create();
                rx.Observable.merge(sub, 10).subscribe(System.out::println);
                ob = sub;
            }
    
            public void put(Observable<Integer> t) {
                ob.onNext(t);
            }
    
            public static void main(String[] args) {
           App app = new App();
            Observable.range(1, 100).subscribe(integer -> {
                app.put(Observable.just(integer));
            });
            }
        }
    That works for me
    Abhinav Solan
    @eyeced
    yup .. seems instead of buffer merge with count would work here
    Dorus
    @Dorus
    public class App {
        private Observer<Observable<Integer>> ob;
    
        public App() {
            PublishSubject<Observable<Integer>> sub = PublishSubject.create();
            rx.Observable.merge(sub).buffer(10).subscribe(System.out::println);
            ob = sub;
        }
    
        public void put(Observable<Integer> t) {
            ob.onNext(t);
        }
    
        public static void main(String[] args) {
            App app = new App();
            Observable.range(1, 100).subscribe(integer -> {
                app.put(Observable.just(integer));
            });
        }
    }
    Works also
    Abhinav Solan
    @eyeced
    yes .. thanks a lot @Dorus