These are chat archives for paypal/squbs

23rd
Nov 2016
Anil Gursel
@anilgursel
Nov 23 2016 00:40
@akara Please see the first cut of http client documentation paypal/squbs#336. I am pretty sure it will go through a few iterations..
Akara Sucharitakul
@akara
Nov 23 2016 04:25
@anilgursel Perhaps I'm missing something, but I don't seem to find the use of the cipher suites in AkkaSSLConfig. The firstSession seems to consume the enabledCipherSuites. Perhaps I'm missing something. Even in TLS, I cannot find access to AkkaSSLConfig.configureCipherSuites Perhaps I have not reviewed their code enough.
Akara Sucharitakul
@akara
Nov 23 2016 05:26
OK. I've loaded the latest version of the libs in the IDE. Allows me to investigate a bit more. Will get back.
The AkkaSSLConfig cipher suites get populated into SSLEngine via line 84, through its sslEngineConfigurator.
Akara Sucharitakul
@akara
Nov 23 2016 07:38
The enabledCipherSuites from ConnectionContext.https gets populated into firstSession which then is used to construct finalSessionParameters. This is applied to the SSLEngine via line 99.
In each of these places, it calls SSLEngine.setEnabledProtocols. In the first case, it is mandatory and it comes from SSLContext.getDefaultSSLParameters. In the second place (line 99) that line internally calls SSLEngine.setEnabledProtocols in case the enabledCipherSuites is not None.
Akara Sucharitakul
@akara
Nov 23 2016 07:45
So, in short, the cipher suites passed directly to ConnectionContext.https, if not None, overrides the defaults in AkkaSSLConfig.
Akara Sucharitakul
@akara
Nov 23 2016 07:58
One more point I missed: The first case on line 84 got the enabledCipherSuites from a selection between the SSLContext.getDefaultSSLParameters and the SSLConfigSettings, and in case there the latter is None, from Ciphers.recommendedCiphers. Basically, the ciphers must be in SSLContext.getDefaultSSLParameters in order to be in the list. This ensures only valid cipher suites are selected.
The protocol mechanism works in a very similar manner.
Lets summarize.
  1. Take cipher suites from the SSLConfigSettings passed to the AkkaSSLConfig.
  2. If cipher suites is not None, select only the suites available from SSLContext.getDefaultSSLParameters.
  3. If cipher suites is None, select suites from Ciphers.recommendedCiphers and take only the ones available from SSLContext.getDefaultSSLParameters.
  4. Apply those by SSLEngine.setEnabledCipherSuites
  5. Take the suites from ConnectionContext.https populated through firstSession and, if not None, apply those by calling SSLEngine.setEnabledCipherSuites, thus overriding anything from AkkaSSLConfig.
Akara Sucharitakul
@akara
Nov 23 2016 08:08
In essence, the ConnectionContext.https parameters, if set, takes precedence. Then the massaged ones in AkkaSSLConfig, if set. Finally use massaged ones fromCiphers.recommendedCiphers. The two latter only if the suite also exists in SSLContext.getDefaultSSLParameters.
Again, if you look at AkkaSSLConfig there are two places in the application.conf, one overriding each others, that can be used.
Akara Sucharitakul
@akara
Nov 23 2016 08:13
Very complicated fallback hierarchy!
Challenge is how to document all these - but I think we should refer to Akka or help them document it properly.
But to the point of our API, we need to think about providing similar options.
Anil Gursel
@anilgursel
Nov 23 2016 14:48
Actually, ConnectionContext takes SSLParameters as a parameter as well.. It could define cipherSuites as well.. If so, that would take the most precedence..
My goal with https://gist.github.com/anilgursel/bf53aa66f73ebfc2897bb107b8d24330 was to explain the fallback just to make it easier in the conversation with Akka folks. but, from your responses, I see that it did not help much and you had to follow the code through yourself as well :(
Anil Gursel
@anilgursel
Nov 23 2016 15:37
another interesting thing is that.. An SSLContext is passed to ConnectionContext.https and SSLEngine is created from this.. But, a new SSLContext object is create in https://github.com/akka/akka/blob/v2.4.14/akka-stream/src/main/scala/com/typesafe/sslconfig/akka/AkkaSSLConfig.scala#L72..
Anil Gursel
@anilgursel
Nov 23 2016 16:02
@akara I tried to clean up and simplify my gist based on your comments.. Please click on the gist link and take a review. Is it more clear now? If so, I will copy and paste it on the Akka issue to start discussion.. https://gist.github.com/anilgursel/bf53aa66f73ebfc2897bb107b8d24330
Ricardo Araújo
@ricardomga
Nov 23 2016 16:48
Hi, can someone help me?
Akara Sucharitakul
@akara
Nov 23 2016 16:48
Please post your question.
Ricardo Araújo
@ricardomga
Nov 23 2016 16:50
To make a PayPal express checkout go live what do i need?
Akara Sucharitakul
@akara
Nov 23 2016 16:50
Ah, wrong place for those kind of questions.
Ricardo Araújo
@ricardomga
Nov 23 2016 16:50
Do i need to submit it to revision or something?
Do you know a proper place to as?
Akara Sucharitakul
@akara
Nov 23 2016 16:51
Sorry we're not the right people. This is not a PayPal API support forum.
Ricardo Araújo
@ricardomga
Nov 23 2016 16:51
Ask
Akara Sucharitakul
@akara
Nov 23 2016 16:51
I just answered these kind of questions not long ago. Let me dig this out.
Ricardo Araújo
@ricardomga
Nov 23 2016 16:53
IThanks, that would be a gr
Akara Sucharitakul
@akara
Nov 23 2016 17:02
@anilgursel You're right, the sslParameters will even override the enabledCipherSuites and enabledProtocols as you can see at https://github.com/akka/akka/blob/v2.4.14/akka-stream/src/main/scala/akka/stream/impl/io/TLSActor.scala#L469
Akara Sucharitakul
@akara
Nov 23 2016 17:09
I think you can use your gist to post this problem. Looking at their docs, it defines the Lightbend SSL config quite well. But you have much more on top of that in the API.
Anil Gursel
@anilgursel
Nov 23 2016 17:10
u talking about http://typesafehub.github.io/ssl-config/ ? If so, yes, that’s a great doc. But, that focuses on ssl-config.. Does not talk about the above fallback stuff..
If u think my gist is clear enough, I will post the contents to akka/akka#21753
Akara Sucharitakul
@akara
Nov 23 2016 17:12
Yes please!
Anil Gursel
@anilgursel
Nov 23 2016 17:14
done!
Akara Sucharitakul
@akara
Nov 23 2016 21:20
Just a thought. Would circuit breaker and timeout flow qualify as BidiFlow?
Anil Gursel
@anilgursel
Nov 23 2016 21:22
why BidiFlow? It is one in - one out. Out would either be actual result or timeout result.. What would the two ports be connected to ?
Akara Sucharitakul
@akara
Nov 23 2016 21:23
You encapsulate the rest of the flow and measure it's timeout. If it were a BidiFlow, if could very well be a pipeline handler.
For Http, I1 would be HttpRequest, O1would be HttpRequest, I2 would be Try[HttpResponse], O2 would be Try[HttpResponse].
In essence, it measures the timeout of the flow connected to the back of it, and short circuits on timeout.
Akara Sucharitakul
@akara
Nov 23 2016 21:30
In essence, TimeoutStage[T, U] would be a BidiFlow[T, T, Try[U], Try[U]]
The architecture is exactly the same as you've drawn, just a different manifestation at the API level.
Anil Gursel
@anilgursel
Nov 23 2016 21:33
I think the internals would not change.. We would still have to deal with delay, dedup, etc. But, at the API level, we can try and see how it looks.. My plan was not to introduce a stage, instead provide a method that looks something like:
def withTimeout[In, Out, Mat](flow: Flow[In, Out, Mat]): Flow[In, Try[Out], Mat]
but, it would be a better option, if I could instead do Flow[String].withTimeout(2 seconds)
Akara Sucharitakul
@akara
Nov 23 2016 21:52
Yes, here we pass in a Flow that gets encapsulated into another Flow. Just thinking about the model, that would essentially constitute a BidiFlow. It also makes me think back to our commit stages with PersistentBuffer and whether that would constitute a BidiFlow. The story would get much messier with BroadcastBuffer, though. So I think we can leave it alone for the time being.
   ~> | TimeoutStage | ~> | Some Processing |
   <~ |              | <~ |                 |
Anil Gursel
@anilgursel
Nov 23 2016 22:13
I have a very quick and dirty version of timeout mechanism (super ugly at this point with many shortcuts).. But, it is good enough to reveal an issue.. If the timeout value is not optimized, delay stage would be backpressuring..
we might need to detach it from upstream..
Akara Sucharitakul
@akara
Nov 23 2016 22:18
You can't. You need to use a bigger buffer for the delay. The question is what's the right buffer size? That's hard to determine.
Anil Gursel
@anilgursel
Nov 23 2016 22:19
actually, I was wrong.. It was actually backpressured by the flow itself when timeouts happen.. which is actually the desired behaviour probalby anyway..
but, “what is the right buffer size” is a question and hard one to answer..
Akara Sucharitakul
@akara
Nov 23 2016 22:19
Flow.withAttributes(...).delay(...)
Anil Gursel
@anilgursel
Nov 23 2016 22:20
yes, we have to make it configurable.
Akara Sucharitakul
@akara
Nov 23 2016 22:20
If we were to implement the delay with mapAsync and a scheduler, this buffer size would be the parallelism.
Akara Sucharitakul
@akara
Nov 23 2016 22:31
Lets see... a 2 second timeout at 5,000 requests/sec means we can have 10,000 requests in the buffer. Of course, for each individual materialization of an Http Flow you'll be dealing with far less waiting requests. We need to decide whether to supply it with a configuration or pass this in the API.