Fast, Effect, Simple Web Framework for Crystal. Help Kemal development https://www.patreon.com/sdogruyol
anyone know how to set encoding of a raw env.request.body to UTF-8?
Stripe integration https://github.com/confact/stripe.cr loads it's helper json file "event.json" just fine, but when I substitute a file that I streamed directly from kemal as such:payload = env.request.body.as(IO).gets_to_endFile.write("payload.json", payload)
then I get a US-ASCII encoded file.
I encounter parsing problems when trying to construct a Stripe::Webhook.construct_event from my stream (IO) or file,
however, the spec's helper json file "event.json" loads just fine and makes an event.
(secret is working fine with any consistent pair with the spec's "event.json" file)
I get JSON::SerializableError
Exception: Unexpected token: , at line 55, column 4
parsing Stripe::Event::Data#object at line 7, column 5
parsing Stripe::Event#data at line 6, column 3 (JSON::SerializableError)
I'm reasonably sure that the issue is the encoding of my env.request.body, which needs to be a precise bit-copy of the raw request body sent by Stripe, due to their webhook signature comparison authentication functionality.
set_encoding on the IO, but no luck so far, as it's supposedly defaulting to UTF-8, which is what i need...https://crystal-lang.org/api/1.0.0/IO.html
yet there is something wrong with MY env.request.body not being a raw UTF-8 string... even though when I pipe my env.request.body to a file it looks the same as the passing-spec's "event.json" !
this file this spec uses works fine in my code with any secret
spec = https://github.com/confact/stripe.cr/blob/master/spec/stripe/methods/webhook_spec.cr
file = https://github.com/confact/stripe.cr/blob/master/spec/support/event.json
env.request.body.as(IO).gets_to_end gives me something broken, compared to this "event.json" file.
when I load the 2 files into the application and inspect them, they look identically formatted, yet mine gives parsing errors from_json
Exception: Unexpected token: , at line 55, column 4
parsing Stripe::Event::Data#object at line 7, column 5
parsing Stripe::Event#data at line 6, column 3 (JSON::SerializableError)
from /home/aaron/.crenv/versions/1.0.0/share/crystal/src/json/serialization.cr:159:7 in 'initialize:__pull_for_json_serializable'
from lib/stripe/src/stripe/objects/core/event.cr:4:3 in 'new_from_json_pull_parser'
from lib/stripe/src/stripe/objects/core/event.cr:4:3 in 'new'
from /home/aaron/.crenv/versions/1.0.0/share/crystal/src/json/from_json.cr:13:3 in 'from_json'
from lib/stripe/src/stripe/methods/core/webhook/webhook.cr:17:5 in 'construct_event'
and this "event.json" file parses fine and constructs an event after.
anyway, the bottom line: Stripe makes a signature as an HMAC of a "secret" and a "signed payload"
the secret is a string.
the signed-payload is a concatenation of header-timestamp, a dot/period ".", and the literal request.body string as UTF-8
then we compare our signature with the signature that Stripe sent in the env.request.header["Stripe-Signature"]
the spec's "event.json" file compares correctly.
my env.request.body.as(IO).gets_to_end does not match signatures.
when I put the contents of env.request.body.as(IO).gets_to_end in a file, it still doesn't work. it looks the same as "event.json" in every way that I can see so far...
in other words:
require "stripe"and in my webhook handler route
stripe_webhook_secret = "whatever"
headers = env.request.headers
myFileData = env.request.body.as(IO).gets_to_end
File.write("payload.json", myFileData)
timestamp = Time.utc
file1 = File.read("event.json") # the file from https://github.com/confact/stripe.cr/blob/master/spec/support/event.json
file2 = File.read("payload.json") # can try with the file, or even just the IO stream myFileData, which is apparently a String...
signature = Stripe::Webhook::Signature.compute_signature(
timestamp,
file1,
stripe_webhook_secret
)
header = Stripe::Webhook::Signature.generate_header(
timestamp,
signature,
scheme: "v1"
)
event = Stripe::Webhook.construct_event(file1, header, stripe_webhook_secret)
pp eventand file1 works OK and parses and Stripe::Webhook.construct_event is constructed fine.
file2 gets parsing errors starting at the first comma ,
but this works fine
payload = env.request.body.as(IO).gets_to_end
json = JSON.parse(payload)
pp jsonbut somehow, somehow, payload is different than the literal string Stripe emitted...
for example, comparison here gives false
(#example of stripe_signature = "t=1627478133,v1=66a99c20d015638aec8733ec609a8e1c0555011c0b2275c97e2ec6776aabddfe")
headers = env.request.headers
stripe_signature = headers["Stripe-Signature"]
sigbits = stripe_signature.split(/,+/)
t = sigbits[0].lchop("t=")
v1 = sigbits[1].lchop("v1=")
signed_payload = t + "." + payload
hmac = OpenSSL::HMAC.hexdigest(OpenSSL::Algorithm::SHA256, stripe_webhook_secret, signed_payload)
comparison = Crypto::Subtle.constant_time_compare(hmac.to_s, v1)
pp comparisonor manually verifying per https://stripe.com/docs/webhooks/signatures#verify-manually
even given fabricated headers that SHOULD be equal
i cannot get a signature match with a payload of
env.request.body.as(IO).gets_to_endbut i can get a signature match with a payload of this file https://github.com/confact/stripe.cr/blob/master/spec/support/event.json
using ANY secret (in the fake header fabrication, and the expected-signature construction)
env.request.body.as(IO).gets_to_end as far as even CR LF newlines etc. utf-8. at the moment i'm trying to see why the JSON::PullParser.read_raw can't handle a comma "," which is odd... I guess this is not a Kemal pertinent problem, sorry to drag you all through this...
You can join via https://discord.gg/prSVAZJEpz
I'm trying this:
require "kemal"
get "/hello/:name" do
name = env.params.url["name"]
"Hello, #{name}!"
end
Kemal.runBut getting this:
[renich@introdesk headers]$ crystal run src/headers.cr
Showing last frame. Use --error-trace for full trace.
In src/headers.cr:4:10
4 | name = env.params.url["name"]
^--
Error: undefined local variable or method 'env' for top-levelAny idea why?
public/index.html when a user requests /. How do I configure that? If I request /index.html then it works.
I'm starting to play with Kemal in order to compare Crystal with Ruby. What I see on my MacOSX is very strange memory consumption. I ran a very simple Kemal hello world:
require "kemal"
require "http/client"
CLASSIFIER_ENDPOINT = "http://localhost:9292"
get "/" do
response = HTTP::Client.get "#{CLASSIFIER_ENDPOINT}?sleep=1"
"Hello World!"
end
Kemal.runWhen I start the server with crystal src/app.cr the reported RSS memory is 350MB. I would expect that to be way lower. For comparision simple Sintra hello world is around 30MB in starting memory.
Do I do something wrong?
crystal src/app.cr is completely different than building the executable and running it standalone.
