Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 28 19:44
    KineticCookie commented #1592
  • Sep 27 06:17
    blast-hardcheese commented #1592
  • Sep 27 06:13

    blast-hardcheese on core-v0.75.2

    (compare)

  • Sep 27 06:05
    blast-hardcheese labeled #1539
  • Sep 27 06:03

    blast-hardcheese on master

    Update akka-http, akka-http-tes… Merge pull request #1576 from s… (compare)

  • Sep 27 06:03
    blast-hardcheese closed #1576
  • Sep 27 06:03
    codecov[bot] commented #1576
  • Sep 27 05:49
    blast-hardcheese closed #1553
  • Sep 27 05:49
    blast-hardcheese closed #1551
  • Sep 27 05:48

    blast-hardcheese on master

    Update scala-library, scala-ref… Update wartremover.sbt Update scoverage.sbt and 2 more (compare)

  • Sep 27 05:48
    blast-hardcheese closed #1589
  • Sep 27 05:47
    codecov[bot] commented #1589
  • Sep 27 05:47
    codecov[bot] commented #1589
  • Sep 27 05:47
    codecov[bot] commented #1589
  • Sep 27 05:46
    blast-hardcheese synchronize #1576
  • Sep 27 05:46
    codecov[bot] commented #1589
  • Sep 27 05:46
    codecov[bot] commented #1589
  • Sep 27 05:27
    blast-hardcheese synchronize #1589
  • Sep 27 05:11

    blast-hardcheese on master

    Trying to work around codecov b… (compare)

  • Sep 27 03:16
    blast-hardcheese closed #1584
kelnos
@kelnos:matrix.org
[m]
one real world data point is that someone at twilio was using the "legacy" behavior (required considered, nullable ignored), and expected "whatever": null on the server when required=false to end up giving them the default value in the handler.
kelnos
@kelnos:matrix.org
[m]

i think the thing i'm struggling with is if we should be interpreting the required, nullable, and default as instructions about the wire format, or about the UX of the server or client, or in some cases both. i think it's pretty safe to assume that required and nullable are first and foremost about the wire format, though they can affect the UX of the client and server. whereas default might be entirely about the UX of the client and server.... but we still have to make those UX decisions in the context of the wire format.

like if someone says required=false,nullable=true,default=foobar, are they saying "i just want to have a really permissible wire format, where you can leave the field out or set it to null, and no one cares, but the UX should never see a null/absent and should always have foobar substituted when there's no value or no field"?

or are they saying "absent field means they don't care and we should substitute the default, and null means they specifically want no value and we should pass that through"?

but then i try to think about the actual use cases. since CRUD is a big use case, i think... ok, on create, you probably mean the first thing. i can't think of a non-bonkers API where creating a resource has a field where you can either ask for the default, ask for null, or set a value. that just seems weird to me. but maybe someone wants to do that? then i think about update, and my feeling there is that a default doesn't ever really make sense. in that case absent has to mean "don't modify this field", null has to mean "set this field to null", and a value has to mean "set this value". and if someone does set a default value for a field in an update, i would say they are doing it wrong, because that doesn't make sense.

so maybe we can just say... ok, if you provide a default, we'll of course obey required/nullable when it comes to the wire format, but from the perspective of the server (when handling a request) or client (when inspecting a response), there is no such thing as absent or null. that property always has a value, and if we don't see one on the wire, that value is the default value. and yes, that means that if someone puts a default on their update operation, things might be broken, but that's kinda on them for doing something logically nonsensical.

outside of CRUD, are there places where someone might want that "absent=default, null=null, value=value" interpretation of what comes over the wire? maybe? can we come up with something concrete, or is this just navel-gazing? the problem is that if we decide that there's nothing concrete here, and don't allow that interpretation, and then someone comes along and says they need it, we can't support it without either breaking existing behavior (nope) or adding another command-line switch to enable it. but on the flip side, if we do add support for this case from the start, at least people who don't care about this distinction (and just want "absent=default, null=default, value=value") can still implement that in their services without too much trouble (but yes, to be fair, the UX there ends up being slightly sub-optimal).

oh damn, that was a huge wall of text.
blast_hardcheese
@blast_hardcheese:matrix.org
[m]

to make sure we're talking about the same possibilities, we've got...

  • the "wire format focused" solution, where UX is irrelevant and we emulate CRUD, take it or leave it. This covers the presence/null/value trinary.
  • the current implementation, wherein we always send the default value no matter what, because that value is baked into the POJO builders forever.
  • the server-side implementation of the previous solution, where the client can't introspect the default value smuggled along with the builder, but nulls and absent values all get decoded to the default.

Is that correct?

kelnos
@kelnos:matrix.org
[m]
i think so, yes
Stanislav Chetvertkov
@stanislav-chetvertkov
I been thinking about the following use case - when collecting usage metrics from both clients and servers in order to monitor and alert based on this information, one has to replace the actual call urls with user defined parts like Ids and user defined arguments into a grouped representation, so that, for example, /Numbers/12345 becomes /Numbers/number both to be able to group by the same endpoint and to avoid leaking sensitive data. I guess the question is whether someone has already brought up and maybe solved this issue in an automated way based inferred from the swagger specs.
Stanislav Chetvertkov
@stanislav-chetvertkov
On second though, perhaps this is easier to solve by parsing the swagger spec in runtime
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
Initially, we thought about having "extractor" style to emit a dto of the extracted parameters from any supplied route
it just never got prototyped and implemented
it would need to get hooked into the routing layer though, http4s has this functionality already, it's a little more manual in akka using the map* directives, and I don't know what it would look like for Java
the challenge comes from ensuring proper round trip for string-interpolated values, so this likely would rely on the Uri classes available for each framework
Nick Hudkins
@nickhudkins
Hi! Firstly, thank you so much for all of the work on this project. I just found it the other day and hoping to possibly begin driving adoption at work. I noticed an issue hanging out about tracing, and was curious if there was already any in-progress work. If not, I’d love to help out here, as I’m gonna need to figure it out for our use anyway.
Nick Hudkins
@nickhudkins
In my head, my approach would be to inject a parent span (not TraceBuilder) into each method of the handlers. I believe that with that, it would allow some more granular “internal” instrumentation, while continuing to allow the trace to be started (or continued in a distributed model) from request handling, and finished on the response. It’s also entirely possible that this is feasible today, and I’m not following what exists.
Nick Hudkins
@nickhudkins
Hot diggity dog I think customExtract might do the trick!
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
Yeah -- TraceBuilder is really a relic, considering the new request/response transformers. I was initially against having those, as they permit complete violation of the spec or schema, but after wrestling with TraceBuilder for so long, I was really brought around to the more pragmatic, general solution.
I'm glad you find the project helpful!
Nick Hudkins
@nickhudkins
I certainly don’t love the idea of custom extract because of the ability to violate the spec, but I think it is easier to have discipline in one place (the custom extractor) than relying on having discipline across your whole API. If I got worried about abuse, I could always create a more constrained abstraction on top to avoid it getting used “for evil” 😂
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
Yeah, certainly. Plus, it gives rise to the possibility for a guardrail-contrib library dependency that fits that function signature to add a bit more structure, in a modular way
Nick Hudkins
@nickhudkins
Y’all don’t by chance use datadog APM do you? If so, I’d be happy to work together on guardrail-contrib’s first lib! (datadog apm)
Otherwise I’ll test it out here in a project and if it works I’ll extract it!
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
The TraceBuilder stuff was modeled around some abstractions we had in Twilio for integrating with Lightstep, datadog APM wasn't available to use at that time
I'd be happy to see it though. I've got a toddler running around, but I'm trying to carve out some time to finish the 1.0.0 guardrail release, (switching the packages from com.twilio to dev.guardrail, no other API changes other than that)
Nick Hudkins
@nickhudkins
Well feel free to toss things my way, my kids are old enough that they don’t want anything to do with me mostly (except when they do). Yeah we migrated off of Lightstep as well. I might actually have time to do the package move if you haven’t started.
blast_hardcheese
@blast_hardcheese:matrix.org
[m]

The migration has already been merged, (0.64.4 was actually already accidentally released under dev.guardrail with the package changes), the only follow-up changes are to sbt-guardrail, guardrail-maven-plugin, and guardrail-gradle-plugin. If any of those strike your fancy, that would be tremendously helpful.

Once the new plugins under the new org are available, my plan is to cut a hotfix on the 0.64.x branch in the old org that includes a log line to advise consumers to switch over to the new org in order to continue getting updates via scala-steward or dependabot

Nick Hudkins
@nickhudkins
sbt-guardrail PR open :)
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
lol, spent an hour trying to figure out why sbt-sonatype wasn't picking up my credentials before I discovered that it sneakily caches the result of sonatypeProfiles in target/, without any sort of eviction if you fix the problem.
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
sbt-guardrail 0.64.4 released, matching the dev.guardrail package shift.
For the next few releases, I'm going to override the organization back to com.twilio and publish the same version to both, to give an opportunity for scala-steward to prompt for upgrades to a version that also prints a warning about the organization change
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
kelnos: What would you say to splitting the guardrail library into different individual modules? Something like guardrail-circe, guardrail-http4s, guardrail-dropwizard, or otherwise? A problem I'm trying to solve is to stop the seeming runaway train that is attempted semver split across so many subprojects
guardrail core really doesn't change too terribly much, and splitting the modules out would reduce the number of no-op PRs opened by scala-steward and friends
like http4s may not change at all, but akka changes a bunch or vice versa, and the other party is just going along with the updates because it might impact them
It might make sense to do guardrail-${language}-${library} as well
with the further added benefit of making each individual codebase more tractable for contributors.
kelnos
@kelnos:matrix.org
[m]
blast_hardcheese: sounds like that could help, but i'm worried about it all being more annoying to release, and also during development when making core changes that are then used in the generators
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
kelnos: Yeah, maintainability needs to continue to be good. What about a version.sbt per module, all as subprojects in the main guardrail repo?
some magicks for checking to see if the current version has already been published and skipping if so
kelnos
@kelnos:matrix.org
[m]
if that's something you can do, sure
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
I believe that's possible to do with sbt
trying to think of what the implications of the current release flow through the github UI would be though
I really like the release-notes -> publish workflow
might need to write a custom github action :\
also, all this is moot if I can't come up with a good UX for consumption from sbt/maven/gradle
kelnos
@kelnos:matrix.org
[m]
also maybe a little worried about what kind of soup the git tags will look like
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
mmyeah. It'd end up turning into a grid of gems
a plus would be extensions to guardrail core would just be able to be natively supported directly in the build tooling though
something like <modules><module><name>http4s</name><clientClass>dev.guardrail.generators.ScalaGenerator.Http4sClientGenerator.ClientTermInterp</clientClass>... where the fqcn's are able to be specified (with sensible defaults) in the pom.xml, and so long as the codegen phase has those dependencies on the classpath everything comes together
kelnos
@kelnos:matrix.org
[m]
that'd be kinda cool
blast_hardcheese
@blast_hardcheese:matrix.org
[m]
thinking ahead, that could open modules up to the community without having to choose which build tool plugin has the features you want.
hah, for instance, circe-java8 could be spun out without just hanging on in the core forever