Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jun 01 2018 15:53

    cristobalmedinalopez on master

    License added (compare)

  • Nov 14 2017 07:10
    agauniyal closed #1
JP
@jpgarciaortiz
And have u tested it? I mean, have u been able to execute a splitter, a peer, etc?
Abhinav Gauniyal
@agauniyal
yes, I took the commands from I-want-to.. repo, putted them inside a .sh file and executed it. everything worked perfect and I saw that bunny video and nice stats on command line
Infact I was initially applying for
Abhinav Gauniyal
@agauniyal
Peer behavior idea so I'm already somewhat familiar with c++ code
JP
@jpgarciaortiz
do you have experience with node.js + express?
Abhinav Gauniyal
@agauniyal
yes, I've a lot of projects written in node, with and without express. I've interned at 2 places in last year where I worked primarily on node+express+mongoose backend. Currently I was writing a side project where express seemed a bloat so I wrote my own in-house framework, you can see that here - https://github.com/agauniyal/mw-server though it's still under active development and things are messy atm because I switched from sqlite to postgres backend but you'll get the idea :).
And I'm active in Express's gitter channel too - https://gitter.im/expressjs/express occasionally helping others with their queries!
JP
@jpgarciaortiz
have you already thought about the strategy to create/manage splitters from the media server?
Abhinav Gauniyal
@agauniyal
I haven't finalized it yet as right now I'm actively researching about it. I've got two ways to do this right now, first one requires no to little change in c++ sources since mediaserver will handle the strategy entirely by itself, other requires significant change to develop communication system b/w mediaserver and p2psp sources and in this the mediaserver will only act as record-keeping+notifying agent and new splitters will be launched/killed by p2psp server itself.
However as I mentioned, I'm still researching about it so these ways might change or other ideas might crop up as I proceed further.
Abhinav Gauniyal
@agauniyal
Feel free to share any ideas/suggestions you've got about this!
JP
@jpgarciaortiz
well, for instance, in order to create a new "channel": which approach do you think would be best?
it's required to define a URL schema, used to call the media server
with that, it'd be necessary to check if there already is a channel with that name; if not, a new one is created, a new splitter is launched, and the connection information (to connect to this splitter) would be returned
Abhinav Gauniyal
@agauniyal

well, for instance, in order to create a new "channel": which approach do you think would be best?

The choice of approach depends on where do we intend to put up mediaserver. Like if it's on the same machine as where you want to launch splitter processes, the first approach ie. 'mediaserver maintaining everything' works very well. It can monitor processes, relaunch them if they fail/close down etc.

However, if mediaserver can be put on entirely different machine than the one to be having new splitter processes, the mediaserver server here cannot launch new splitters by itself because they would be launched on this machine only. Note that we can always launch splitter on mediaserver machine but point them to source at other machine since splitters accept source's address as argument. What we can do is, change p2psp code so that it can communicate with mediasources server and launch/kill new splitters when asked to do so.

Abhinav Gauniyal
@agauniyal

Let me provide pseudo examples -

cd bin
wget https://upload.wikimedia.org/wikipedia/commons/7/79/Big_Buck_Bunny_small.ogv
cvlc Big_Buck_Bunny_small.ogv --sout "#http{mux=ogg,dst=:8080/BBB.ogv}" :sout-keep &

^ Source is ready

node mediasources --src 'sourceIP:port' &

^ mediaserver launched and monitoring source, also reverse is possible too where mediasources is launched first and sources register themselves with it.

./monitor --splitter-url 'http://niceurl/abcd1234' > /dev/null &
vlc http://localhost:9999 &
./peer --splitter-url 'http://niceurl/abcd1234' --player_port 10000 &
vlc http://localhost:10000 &

Note that no splitter creation command is necessary, peers contact http url where they're provided with splitter's ip:port

^ This is for 1st case where mediaserver launches splitter on it's own machine only
Abhinav Gauniyal
@agauniyal

For 2nd case, where mediaserver can be on entirely different server -

cd bin
wget https://upload.wikimedia.org/wikipedia/commons/7/79/Big_Buck_Bunny_small.ogv
cvlc Big_Buck_Bunny_small.ogv --sout "#http{mux=ogg,dst=:8080/BBB.ogv}" :sout-keep &

^ Source is ready

node mediasources --src 'sourceIP:port' &

^ mediaserver launched and monitoring source, also reverse is possible too where mediasources is launched first and sources register themselves with it.

./splitter-manager --mediasources-url 'http://someUrl.ToSendRequests/'
./splitter-manager --mediasources-url '127.0.0.1:3000'  # works locally too since address is valid
./monitor --splitter-url 'http://niceurl/abcd1234' > /dev/null &
vlc http://localhost:9999 &
./peer --splitter-url 'http://niceurl/abcd1234' --player_port 10000 &
vlc http://localhost:10000 &

Note that no splitter creation command is necessary, peers contact http url where they're provided with splitter's ip:port.
In this case, splitter-manager is launching/maintaining splitter processes since mediaserver might be on different machine

Infact ./splitter-manager doesn't need to be cpp program at all, it can be same mediaserver launched in client mode where it sits on required machine and contacts real mediaserver to get list of splitters/add new splitter/get address of all channels/ get address of particular channel etc.. and based on response it gets from server, launches and manages splitter processes.
Abhinav Gauniyal
@agauniyal

it's required to define a URL schema, used to call the media server
with that, it'd be necessary to check if there already is a channel with that name; if not, a new one is created, a new splitter is launched, and the connection information (to connect to this splitter) would be returned

By URL schema, do you mean certain http endpoints fixed for either GETing responses or POSTing them?
In second line are you talking about a scenario where same channel name is created for new source too? Or where a new peer/client hits for a channel but the channel wasn't there so a new channel is created - new splitters launched for it and their address returned to peer/client. In this case, what if source for that channel doesn't exists or has been closed? Actually I was under the impression that a channel can only be created when a source is present for it, so there is always going to be a valid channel for valid source

if there already is a channel with that name; if not, a new one is created

so the 'if not' condition would never happen here because source would not be present for a non-existent channel name.

I might be speculating it wrong too, so feel free to correct me!

JP
@jpgarciaortiz
IMHO, the media server should run on the same machine where the splitters (or splitters manager) is running, so it can create/check/whatever directly
I had imagined the media server as "router" point: if I wanted to create a new channel, I use a call of the REST API, so that by means of a HTTP POST I send the video, indicating the channel name, video parameters, etc.
JP
@jpgarciaortiz
the media server would create the splitter for that new channel and would return information about the connection; this information could be queried later (e.g., if someone get the list of channels, etc.)
the media server could also "link" the video stream to the splitter, this would be great
another easier approach would be that, in order to create a new channel, the API call would be as "hey media server, I'm a video source, and I want to create a channel; give me a splitter and a connection info"
the problem with this approach is that the media server can not control when the video source stops/break
Abhinav Gauniyal
@agauniyal

IMHO, the media server should run on the same machine where the splitters (or splitters manager) is running, so it can create/check/whatever directly

Great, this will make a lot of things easier eg. authenticity of data, securing communication, process maintenance etc. We don't even need the first two right now!

I had imagined the media server as "router" point: if I wanted to create a new channel, I use a call of the REST API, so that by means of a HTTP POST I send the video, indicating the channel name, video parameters, etc.

the media server would create the splitter for that new channel and would return information about the connection; this information could be queried later (e.g., if someone get the list of channels, etc.)

Yes, this is exactly what I had in my mind too! One more thing though, how do you plan to send HTTP POST request? I mean client could be anything ranging from curl, some separate client to p2psp, p2psp qt based GUI, to a simple HTML form to create that channel, Media server won't care and definitely handle it well, but do you have something in mind from where HTTP POST request comes for while I'll be working on it?

Abhinav Gauniyal
@agauniyal

the media server could also "link" the video stream to the splitter, this would be great

Could you explain what you mean by linking the video stream to splitter? Since a sample splitter command is like this -

./splitter --source_addr 127.0.0.1 --source_port 8080 --splitter_port 8001 --channel BBB.ogv --header_size 30000

Do you mean by linking it is automatically passing source address and port to splitter process? Or do you mean MediaServer will consume the Video Stream itself first, eg buffering up to some length and then give it to the splitter process, which IMO is unnecessary level of indirection :/. Or is it entirely other thing that I'm missing out?

Abhinav Gauniyal
@agauniyal

another easier approach would be that, in order to create a new channel, the API call would be as "hey media server, I'm a video source, and I want to create a channel; give me a splitter and a connection info"

the problem with this approach is that the media server can not control when the video source stops/break

Well there are ways to detect if a video broke down/source is unresponsive and not sending data, via 1. polling - which means simply checking video source for streaming data, if it is sending anything at all! Eg. the video source might be transmitting udp/tcp/ or through some different protocol and we can keep checking in a time interval, very small amount of data and then discarding packets since we don't need them. After a certain timeout, if the source still isn't sending anything, we can assume it's broken down! 2. Watching for splitter process to write something on stdin/stderr pipes, we can offload this task to splitter processes we've launched for that particular video source, and when they write something preferably on stderr, our MediaServer could easily catch it and detect that the video has broke down.. This seems more apt to me since stderr is also attached to terminal so in case no MediaServer is present, it still displays the error code, but both are viable ways to detect it.

JP
@jpgarciaortiz
One more thing though, how do you plan to send HTTP POST request?
I had thought to use POST requests with chunked encoding
Do you mean by linking it is automatically passing source address and port to splitter process? Or do you mean MediaServer will consume the Video Stream itself first, eg buffering up to some length and then give it to the splitter process, which IMO is unnecessary level of indirection :confused:. Or is it entirely other thing that I'm missing out?
yeah, it's an additional level of indirection, but it's the easiest way that the media server can control the stream. There is another more complex, and efficient way: to be able to pass the socket to the splitter. This can be done in Unix machines, but I'm not sure if this might be achieved with node.js...
JP
@jpgarciaortiz
Well there are ways to detect if a video broke down/source is unresponsive and not sending data, via 1. polling - which means simply checking video source for streaming data, if it is sending anything at all! Eg. the video source might be transmitting udp/tcp/ or through some different protocol and we can keep checking in a time interval, very small amount of data and then discarding packets since we don't need them. After a certain timeout, if the source still isn't sending anything, we can assume it's broken down! 2. Watching for splitter process to write something on stdin/stderr pipes, we can offload this task to splitter processes we've launched for that particular video source, and when they write something preferably on stderr, our MediaServer could easily catch it and detect that the video has broke down.. This seems more apt to me since stderr is also attached to terminal so in case no MediaServer is present, it still displays the error code, but both are viable ways to detect it.
well... I like idea 2! it's a good idea actually!
Abhinav Gauniyal
@agauniyal

I had thought to use POST requests with chunked encoding

what I meant was, which program would be the potential client to use this API? Is there already such a client ready/or work in progress?

but it's the easiest way that the media server can control the stream.

Could you tell me what would we do by controlling the stream? I mean what would be the difference if we directly set source address on the splitter process without taking any packets inside Mediaserver?

There is another more complex, and efficient way: to be able to pass the socket to the splitter.

By passing the socket, you mean passing the socket's integer value right? Because socket in UNIX sense is just an integer returned by underlying socket() call. If yes, I'll see if there's a way to get that value within from nodejs, though it seems there won't be any.

I was basing my assumptions on -

Well, I'm not the creator of this idea, but, what I would implement is: (1) a source contact the media-server, (2) the media server creates a splitter in a "friendly URL", (3) the media-server send this URL to the source, (4) the source sends the stream to the splitter.

Abhinav Gauniyal
@agauniyal
I'm not getting the idea behind getting source packets inside Mediaserver (not saying it's not possible, it is) but how would it help?
JP
@jpgarciaortiz
what I meant was, which program would be the potential client to use this API? Is there already such a client ready/or work in progress?
if I remember well, there was an iOS video source quite advanced that already sent POST requests to send the video stream
By passing the socket, you mean passing the socket's integer value right? Because socket in UNIX sense is just an integer returned by underlying socket() call. If yes, I'll see if there's a way to get that value within from nodejs, though it seems there won't be any.
not, that's not the idea; a process can send a copy of a socket (not just the integer value) to another process
but nevermind, as I said, I like your idea, so we can keep the media server as an intermediate, but checking that the video source is alive
Abhinav Gauniyal
@agauniyal

not, that's not the idea; a process can send a copy of a socket (not just the integer value) to another process

Oh, you meant SCM_RIGHTS way! I'll see if node provides it but again this is even more unlikely.

Abhinav Gauniyal
@agauniyal
It seems the only way to pass socket to another process in node is to fork current process, which is not exactly suitable for us.
Abhinav Gauniyal
@agauniyal
Thankyou for selecting me for implementing MediaSources server, I'm immediately setting up my development environment and will start proceeding as per my schedule :smiley_cat:
Vicente González Ruiz
@vicente-gonzalez-ruiz
Congratulations, @agauniyal :-)
JP
@jpgarciaortiz
Congratulations!
Sushil khanchi
@khanchi97
Congratulation :smile: @agauniyal
Abhinav Gauniyal
@agauniyal
Thanks :smirk_cat: