Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 11 14:55

    michielbdejong on main

    Mention https://people.apache.o… (compare)

  • Sep 28 12:50

    michielbdejong on main

    Update README.md (compare)

  • Sep 28 12:48

    michielbdejong on main

    shorten stage.graphmetrix.net U… (compare)

  • Sep 21 11:56

    dependabot[bot] on npm_and_yarn

    (compare)

  • Sep 21 11:56

    michielbdejong on main

    Bump tmpl from 1.0.4 to 1.0.5 i… Merge pull request #128 from so… (compare)

  • Sep 21 11:56
    michielbdejong closed #128
  • Sep 21 03:33
    dependabot[bot] labeled #128
  • Sep 21 03:33
    dependabot[bot] opened #128
  • Sep 21 03:33

    dependabot[bot] on npm_and_yarn

    Bump tmpl from 1.0.4 to 1.0.5 i… (compare)

  • Aug 31 13:45

    michielbdejong on main

    Report status of https://github… (compare)

  • Aug 31 13:38

    michielbdejong on split-out-CON

    (compare)

  • Aug 31 13:38

    michielbdejong on main

    temp more edits and 4 more (compare)

  • Aug 31 13:38
    michielbdejong closed #127
  • Aug 30 19:12
    michielbdejong commented #127
  • Aug 30 19:07
    michielbdejong synchronize #127
  • Aug 30 19:07

    michielbdejong on split-out-CON

    paragraph headers (compare)

  • Aug 30 19:05
    michielbdejong opened #127
  • Aug 30 19:05

    michielbdejong on split-out-CON

    temp more edits and 2 more (compare)

  • Aug 16 14:46
    dependabot[bot] labeled #126
  • Aug 16 14:46
    dependabot[bot] opened #126
Michiel de Jong
@michielbdejong
yes!
are you saying this in general or do you have a specific one failing?
we should add more links - the specs are now also a lot more complete than when we first wrote all the tests, so there is more to link to now
Fred Gibson
@gibsonf1
I think @edwardsph was working on some testing that returned spec references which looked really promising
Michiel de Jong
@michielbdejong
:+1:
Pete Edwards
@edwardsph
@gibsonf1 How are you doing with https://stage.graphmetrix.net? I wanted to check a new approach to access control related tests against TrinPod but it does not appear to be available yet.
Fred Gibson
@gibsonf1
We are in the middle of an upgrade - should be back up in about an hour - I'll text here when back
Pete Edwards
@edwardsph
Perfect - thanks
Aaron Coburn
@acoburn
The agenda for next week's AuthN Panel is available at https://hackmd.io/TqfZNJF-RU-I6xD-NV9l_g. The first agenda item is the test suite discussion.
Fred Gibson
@gibsonf1
@edwardsph It was a long hour, but the new build is up on stage now and ready
It was a Pi hour - I forgot to multiply by Pi in the estimate
Fred Gibson
@gibsonf1
Actually @edwardsph We're not quite there, still have a bug we're chasing in this build that just failed a test...
Pete Edwards
@edwardsph
OK. I will send you a report of a failure on a single test I just ran in case it is a different situation.
Fred Gibson
@gibsonf1
Just checked the log, the issue you had might be specific to the test user pod, checking into it
Fred Gibson
@gibsonf1
Thanks @edwardsph Just got your email. The issue we see in the log matches that, so thank you. I'll let you know when next fixed build is up.
Fred Gibson
@gibsonf1
@edwardsph It's looking like the issue is in trinpod turtle parsing where [] is used as a new blank node...
Fred Gibson
@gibsonf1
It looks like although we have support for complex nestings in [], we didn't have the simple case of using [] alone as a blank node
Pete Edwards
@edwardsph
Glad you found it so quickly - I hope the fix is equally quick for you
Fred Gibson
@gibsonf1
Doing a build now - should be ready on stage in just a few minutes...
Ok @edwardsph The new build on stage is ready for testing
Fred Gibson
@gibsonf1
Sorry @edwardsph I realized that the update assumed that [] was labeled and gave it the [] label so that all occurrences would have received the same node, so that fix is now up in the current build (it looks like you were running a test just as it was updating - sorry)
Pete Edwards
@edwardsph
@gibsonf1 It wasn't the conformance test harness running at that time, it was probably the Jest based test-suite. You can see when I am running tests as the user agent is defined as: User-Agent: Solid-Conformance-Test-Suite. I just re-ran the single test and it is failing all sub tests now. I haven't got time to look in detail but on first glance it appears the ACL link header is not being returned as expected. Could that be true?
Fred Gibson
@gibsonf1
We are still chasing a bug that is affecting many things - I'll let you know when we pass all our tests.
Michiel de Jong
@michielbdejong
Submitted my second batch of hours for the independent test suite (up from 10 to 17 cummulatively) https://opencollective.com/independent-solid-test-suite
Sarven Capadisli
@csarven
Michiel de Jong
@michielbdejong
Thanks! That's useful
Michiel de Jong
@michielbdejong
@edwardsph congrats on your v1.0 release, looks great! https://inrupt.com/blog/interoperability-tests-for-Solid-developers
I see the CSS team are running it in their CI, but not passing yet, apparently https://github.com/solid/community-server/actions/workflows/schedule.yml
How does CTH deal with solid/web-access-control-tests#41 ?
Michiel de Jong
@michielbdejong
Pete Edwards
@edwardsph
@michielbdejong Thanks. The tests were running fine in CSS until something in the 2.0.0 release related to registering the test accounts (it creates test users on the fly). We are going to look into that. We may not have the test coverage for the issue you refer to. @kjetilk has converted his old tests to run with this harness and is now going through the specs to see how well aligned everything is. Those converted tests are not normally run at present as they push some of the edge cases were there is still uncertainty about what the spec mandates for response codes. Whilst I focus on the CTH abilities, he is focussing more on the test cases.
Michiel de Jong
@michielbdejong
ah cool
Michiel de Jong
@michielbdejong
@edwardsph have you tried running CTH against PSS? I think it should work because the parameters of the login page are identical. pdsinterop/test-suites#47
2 replies
Fred Gibson
@gibsonf1
@michielbdejong and @edwardsph We have a new build up on https://stage.graphmetrix.net of trinpod ready for testing
Michiel de Jong
@michielbdejong
:+1:
Michiel de Jong
@michielbdejong
NSS is failing a number of tests from specification-tests:
  • status code was: 401, expected: 400 in 'PUT/PATCH/POST requests without Content-Type'
  • status code 201 instead of [409, 415] in 'PUT container/resource, then try resource with same name'
  • status code 201 instead of [204, 205] in 'Bob can PUT to the resource but gets nothing back since he cannot read'
2 replies
did someone open github issues for these yet?
Fred Gibson
@gibsonf1
We just found a last issue with our read event futures - will be all set after that in a few hours
1 reply
Michiel de Jong
@michielbdejong

I guess we can distinguish three kinds of incompatibilities between servers:

Dispute

Implementers agree their implementations are incompatible with each other but can't agree on which behaviour is the correct one.
This can happen when different implementers interpret the spec differently, or when the spec is not conclusive.

In this case we should probably (temporarily) not test the disputed behaviour, until the spec dispute is resolved

Vote with your Code

Implementers agree their implementations are incompatible with each other, and also agree on which behaviour is the correct one.
However, implementers may prefer the deviating behaviour over the spec, and instead of correcting their implementation,
they proposes a spec change. They keep their implementation incompatible on purpose until the proposal has been fully discussed.

In this case we should probably keep testing the behaviour, so that users know about the incompatibility.

Bug

Implementer agrees their server's behaviour is incompatible with the spec.

In this case we should obviously also keep testing the behaviour, so that users know about the incompatibility.

I guess the "vote with your code" case is where the independent test suite and CTH differ in approach, right?

There is maybe a further distinction between "spec addition" (adding a feature but staying compatible with the ecosystem) and "spec change" (leading to incompatibility if some implementers adopt it before it's accepted)
elf Pavlik
@elf-pavlik

Vote with your Code

I think for those we should have tracking issues and preferably an inline issue in the spec linking to it https://tabatkins.github.io/bikeshed/#remote-issues

Pete Edwards
@edwardsph

@michielbdejong Those 3 categories are a good summary.
I'm not sure what you mean here though - can you elaborate?

I guess the "vote with your code" case is where the independent test suite and CTH differ in approach, right?

Michiel de Jong
@michielbdejong
Well, take the example of "containment triple creation permissions". I interpreted https://gitter.im/solid/test-suite?at=61658c429d20982e4fb6e4e7 to mean you're not testing that until the discussion about the spec change proposal is decided.
But maybe I overinterpreted what you said?
Pete Edwards
@edwardsph
I meant that no-one has got around to writing the test rather than it is deliberately not being written yet. If I were to write it now, I would interpret the spec the same way as you did (you cannot create a resource unless the container allows the containment triple to be added). If the test existed it would be marked as unreviewed (as all are at present). I'm adding a mode to CTH to allow it to only run tests approved by spec editors (and we need to get the existing test reviewed ASAP). In the case you refer to, I guess they would need to decide whether to approve it as a correct interpretation of the spec as it stands, or leave it under review whilst there is an open discussion on whether the spec needs to change. Either way, something else we are planning to add to the CTH is the option to reference discussions and comments on test cases and to pass the details through to the reports so that when reviewing the report an implementer can see the context and determine which of your categories the result falls into.
Michiel de Jong
@michielbdejong
Ah I see, so the spec editors will decide this but haven't done so yet. Yes, so then if they decide "approve it as a correct interpretation of the spec as it stands" then our approaches would match, and if they decide "leave it under review whilst there is an open discussion on whether the spec needs to change" then our approaches would differ. References to discussions are of course always a good thing, in both cases!
So do I understand correctly that CTH is now published as a v1.0, but specification-tests is not yet?
Pete Edwards
@edwardsph
That's correct - sorry that wasn't clearer. There will need to be some versioning for the specification-tests repo and it will be independent to the version of CTH. At present the current tests (HEAD) are included in the docker image but that is only as a convenience and it is always possible to run against a local copy of the tests or a version that the test script can pull when it runs.
Michiel de Jong
@michielbdejong
:+1:
Sarven Capadisli
@csarven
I've updated the channel topic. It'd be great if we can at least keep the Code of Conduct up there going forward. See also https://github.com/solid/deit/issues/13#issuecomment-944710055