Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, so, could we think a succession of events such as?: (1) the first private peer P1 (of a private network N) arrives to a public team, through the tracker T. (2) the splitter informs P1 that it the fist peer of N. (3) the second private peer P2 of N contacts T which communicates to P2 that must built a private team. (4) P2 runs a private splitter S that is feed by P1. (5) S communicates its private (and public) end-point to T. (6) P2 joins S.
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz This seems like the idea. But, I've a few questions.
    1. Does all the peer in a team have to be in the same private network ?
    2. If yes number of teams in the worst case will be 'n' (total number of nodes in the network). So, it will be quite be a overhead for the source to feed all these teams.
    3. If no We have to propose an algorithm by which trackers will decide how to assign peers into teams.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, if your question is: "Does all the peers in the "private team" have to be in the same private network?", the answer is yes.
    Anyway, we need to propose such algorithm in some way. In this case, the algorithm is straightforward: if it is P1, the team will be public. If it is about P2, the team will be the private one.
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz I think you misinterpreted what i was trying to so.
    I'm asking if all the peers in a team (not a private team) have to be on the same network.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    Not, usually each peer of a team belongs to a different LAN. In fact, this is compulsory if EMS is not being used.
    SASWAT K MISHRA
    @clicksaswat

    what i understood from the docs that the point of TCS should be that tracker would be able to do efficient clustering of peers into teams.

    Let's say we're saying that tracker will assign each peer into a separate team. Let's say P1 is such a peer. when a new peer P2 which belongs to same private network as that of P1 comes along tracker will instruct P2 to cluster with P1. That's what you're proposing.

    But, the problem is if all of the peers are of different private networks. Then total number of clusters (or teams) will be equal to number of peers. It will be too much load on the source to feed these number of teams. And, its definitely not a efficient clustering mechanism from a tracker point of view.

    SASWAT K MISHRA
    @clicksaswat

    The kind of algorithm will kind of do the following.
    When a peer contacts with tracker, it will do the following.
    for each team in its team list, it will calculate the following.

    • locality - i.e number of peers in that team which happens to be on the same private network as the incoming node.
    • avg. bandwidth (optional) - if somehow we are keeping track of peer's physical topology and upstream/bandwidth we can somehow use this data to calculate if the incoming peer is suitable for this team or not.
      After, each team has been analysed we will assign the peer to the team which will be the closest match according to the parameters as described. If we found the none of the teams has any peer in the same private network as the incoming peer, we will assign it to any one of the teams.

    Of course when initial team list is empty, tracker will just assign each peer to a separate team unless they're from same private network. But, it will continue to do so until a max. number of teams(this parameter will be decided based on sources capability to feed simultaneous connections) has been reached. After that, any incoming peer will be added to one of the teams using algorithm above mentioned.

    SASWAT K MISHRA
    @clicksaswat
    I think TCS should be implemented as a derivable (interface) class. So, that all the initial peer and tracker communication, then tracker's communication with the peer instructing it to join a team, then peer's communication with splitter will be implemented in the base class and each derived class will implement its own version of Tracker's Cluster Selection algorithm. In this case which happens to consider "locality" of peers for team selection.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, thanks for all these ideas. I agree that the tracker should run some clustering algorithm in order to redirect the incoming peers to the most suitable team.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    However, I think that there is a situation you have misunderstood: the teams do not need to be fed by a source, alto it can be done by a peer. It's like we think that a team can be built with teams, recursively. For this reason, in my first example, P1 and P2 are in different teams (P1 in the "level X" team and P2 in the "level X+1" team).
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz i don't understand it properly. Can you please explain a bit elaborately? I've looked through the documentation still in doubt.
    SASWAT K MISHRA
    @clicksaswat
    i understand that p2psp is mesh-based topology but what you're suggesting sounds more like a tree based topology.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, yes, I propose a tree. It has a clear disadvantage (if a node fails, the tree which is fed by it is out) but has also advantages (it can use a push based transmission model). What did you propose?
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz actually i hadn't thought of the architecture much until you brought it up. what i was imagining was a simple structure where source feeds a set of teams where each team has a splitter which is directly connected with the source.
    But, now when i think of it i can see that it has a major disadvantage. It's not scalable. So, i think what you're proposing might be a better idea. Anyway, i think i will have to walk through some of your papers to get more idea.
    SASWAT K MISHRA
    @clicksaswat
    anyway, if you think you're approach is good enough we can directly start working on writing the code.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, I think that the idea is close to be mature, but it should be clear first how we are going to implement the functionality to achieve that if a user run a peer (without knowing that his peer is not the first peer in his NATed LAN), this action runs effectively his peer but connected to a local splitter whose source is other (the first) peer in his LAN. Graphically:
    Before:
    
                       NAT
                       +-+
    +------------+     | |     +-----+      +--------+
    | team - P_i |<----| |---->| P_i |----->| Player |
    +------------+     | |     +-----+      +--------+
                       +-+
    
    After:
    
                       NAT
                       +-+
    +------------+     | |     +-----+      +--------+
    | team - P_i |<----| |---->| P_i |----->| Player |
    +------------+     | |     +--+--+      +--------+
                       +-+        |
                                  v
                               +-----+
                               |  S  | S = Splitter
                               +-----+
                                  |
                                  v
                               +-----+      +--------+
                               | P_j |----->| Player |
                               +-----+      +--------+
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    So, I think that the first step we should do is to provide serving concurrency to the peers (at this moment, only one TCP client can be served by a peer).
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz i'm extremely sorry for being so late. had some urgent family work so i was out for town for some days. anyway, i got your point now. so, basically as soon as a local peer arrives the peer should be able to function like splitter.
    SASWAT K MISHRA
    @clicksaswat
    so, the approach you're talking about will give the peer splitter like power so that it would be listening to a tcp socket to which the local peers will try to connect to and the peer will serve them with chunks. but, i point i want to mention that what we're trying to achieve is just a duplication of what a splitter already does. Can't we just fork another process which will run splitter_ims (or any version of splitter for that matter) ??? then we don't have to worry about porting any further changes in splitter code onto the peer code.
    @vicente-gonzalez-ruiz i think we should start working on it as soon as we can. i may be out of town again in some near future and i won't be able to communicate at that time. so, please reply as soon as you're available.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, now I'm sorry for the delay ...
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    I think you did not understand me. The only modification I propose is to incorporate to the peer the possibility of serving to more than one client. In the last example, the peer P_i servers the Player and the Splitter.
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz ok. i was actually thinking something way out of line here. anyway, i got your point now. so, how do you propose to implement this ? i have a thing or two in mind but first i would like to listen to your approach.
    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, I would implement the concurrent service in the peers using the traditional approach (a thread serving each client). You will find that we have to modify peer_ims. However, I would like to extract this code from the core and make the corresponding implementation of this functionality in the side (repository) of the final target. I mean, for example, in the p2psp-console repository we would implement a service that uses sockets for serving the clients, in the p2psp-ios repository (which now I don't know) we should use the resources more adequate for this, in the future p2psp-android the same. In the core should remain only the most general behavior (may be using abstract classes) and implementing this in the final products. What do you think?
    SASWAT K MISHRA
    @clicksaswat

    @vicente-gonzalez-ruiz I've been doing some reading about how to go about implementing this into code. And, you're right. We should probably decouple this from core and make platform specific implementation. This will let us use platform specific libraries for socket programming. So, let's start with p2psp-console first.

    So, we're going to serve multiple consumers. Should we implement it in a way such that each consumer will have it's own TCP socket? Currently, there's only one UDP serving socket for serving chunks to the peers. But, that's UDP and it's stateless. But, if we're going to take the same approach for TCP it will add to a lot of overheads. (for each new connection we're gonna have SYN/ACKs). Because TCP connections are session-oriented we can just use individual socket per consumer and pass on the chunk to it when it's received. And also we can make this serving individual socket multi-threaded too.

    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, TCP is most common for streaming (we can use mplayer, vlc, the web browser, etc.). I would try to use TCP (as now) but sending through so many sockets as consumers and no extra threads. I mean:
    • A chunk arrives to the peer.
    • The chunks is placed in the buffer.
    • The chunk is sent to all the consumers using a non-blocking send.
    I have been working on p2psp-console for a while and I think that we could implement this last step in the p2psp-console code. I'm going to create a issue in that repo, only for the sake of following the common procedures :-)
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz using non-blocking i/o instead of threads feels like a nice idea. should we start working on it ?
    SASWAT K MISHRA
    @clicksaswat

    @vicente-gonzalez-ruiz I've some proposals that i would like to share if we're going to implement this.
    1) the base class (peer_ims) should implement the following methods.

    • FeedConsumers(chunk)
    • FeedPeers(chunk)
      Currently, the feeding to the peers is done inside ProcessNextMessage() which makes it unusually lengthy and clumsy. Isolating them in a different function I think would make it more readable.
      Implementing FeedConsumers() will also override ConsumeChunk() method.
      2) additionally each peer should maintain a consumer list just like the peer list but instead of maintaining end point IP address this list should hold the reference to the corresponding socket object made for the end point.

    what do you think of it? Let me know if you want any changes or addition.

    Vicente González Ruiz
    @vicente-gonzalez-ruiz
    @clicksaswat, I think that your ideas are fine. So, do you want to start coding?
    SASWAT K MISHRA
    @clicksaswat
    @vicente-gonzalez-ruiz definitely yes. i'm on it.