Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Asko Soukka
    @datakurre

    @iFlameing Sounds good. So, after MVP, let's figure out a way to test all these before any cleanup refactoring.

    tests for fetchUrl and fetchPlone provide an example how Plone could be mocked for testing https://github.com/collective/gatsby-source-plone/blob/master/src/__tests__/utils.tests.js#L90

    At first, the use of http library should be parameterized so that any call to the library could be mocked with test fixture. This is already done for fetchUrl and fetchPlone, but not for any higher level functions.

    Once it is possible to call more higher level functions with mock http interface, it should be possible to make that mock to serve the current JSON fixtures (from tests/gatsby-starter-default/fixture

    But first things first. That is the MVP :)

    As an almost unrelated thing, we just released the biggest GatsbyJS site so far: https://studyguide.jyu.fi/ (about 4000 pages)

    There is no Plone content there yet (current content comes Hasura-enhanced PostgreSQL database), but the roadmap includes also CMS-content and Plone is the most probable solution to manage that.

    Asko Soukka
    @datakurre

    Good that you manage with the TOKEN environment variable. I got good progress with adding security for the websocket support, but still more work to do.

    But this week I will have more time later, so I really hope this to be done by the next week.

    Alok Kumar
    @iFlameing

    As an almost unrelated thing, we just released the biggest GatsbyJS site so far: https://studyguide.jyu.fi/ (about 4000 pages)

    This is great :)

    Asko Soukka
    @datakurre

    @iFlameing Good news. You don't run out of work too soon.

    I added a couple of new tickets related to MVP.

    The first one is pretty easy and you can already simulate it by dropping the token.

    It is a valid use case that fetching URL from Plone returns Unauthorized or Forbidden. In gatsby develop this should result in deleting that node if that exists (so that pages changed to private would be removed from the site).

    The other one is trickier and needs more work. Currently Collection pages may "change" indirectly so there is no way to get update events for those from Plone. Updating those require scheduling (setTimeout) so that when ever there has been updates, after a small amount of time all known collections should be updated. (That means that whenever a new timeout is scheduled, the existing one should be cancelled so that update is run only once after a short period of time no more events from Plone).

    Alok Kumar
    @iFlameing
    @datakurre I have submitted the form of first evaluation :)
    Nilesh
    @nileshgulia1
    Hey @iFlameing , Congrats on first evaluation!
    Keep up the good work!
    Alok Kumar
    @iFlameing
    @nileshgulia1 thanks!
    @datakurre Since all the issue related to MVP is closed now, I am thinking that MVP is done now :tada: :tada: .
    Asko Soukka
    @datakurre
    @iFlameing Yes. Congratulations!
    So, what should you do next...
    Alok Kumar
    @iFlameing
    @datakurre I should start writing test for the added feature as we discuss earlier.
    can you guide me?
    How I can get started with it?
    Asko Soukka
    @datakurre
    @iFlameing Could you start by going through a few popular/official GatsbyJS source plugins to learn how they do their testing?
    I would guess that plugins they support for GatsbyJS Preview should be good quality enough to have some tests https://www.gatsbyjs.com/docs/#using-gatsby-preview
    Or maybe Wordpress one.
    All of those probably live inside the main GatsbyJS repository.
    Currently we have only a few simple unit tests for utils.js.
    https://github.com/collective/gatsby-source-plone/blob/master/src/__tests__/utils.tests.js
    Alok Kumar
    @iFlameing
    @datakurre ok!
    Asko Soukka
    @datakurre
    We are using jest https://jestjs.io/docs/en/getting-started but at least currently not snapshot tests. Just regular asserts.
    Snapshot tests are meant to test React component rendering.
    I assume we could continue to use jest for integration tests of gatsby-node API.
    Asko Soukka
    @datakurre

    @iFlameing So, please, use this week to review how the existing source plugins have been tested.

    Only then we should really decide, what kind of testing makes sense here.

    We already have high level acceptance tests for all the old features. I take the responsibility for upgrading that to support testing changes through websocket events. (That would be mainly bash scripting and Robot Framework, so for you that would be quite a step outside current JavaScript focus.)
    Most obvious choice would be to keep moving code from gatsby-node.js to utils.js or some other module and test those funtions with jest unit tests.
    But integration tests for the GatsbyJS API hooks in gatsby-node.js are still missing and there I would like GatsbyJS to provide some example or framework to do it right (and easily).
    Then again, another way for better quality would be to convert the project into typescript. It would probably have larger impact than having integration tests, but the amount of work in that is hard to estimate.
    Asko Soukka
    @datakurre
    @iFlameing So, because this is now only the first week of the second part of GSOC, I think it would be good to use a week to just learn the other plugins and then decide. If you run out of work, look into typescript also. This seemed to be a plugin using it https://github.com/sanity-io/gatsby-source-sanity
    Alok Kumar
    @iFlameing
    @datakurre okay! I will see the test written for different Gatsby-preview CMS and interact with GatsbyJS community for How we can write test for GatsbyJS API :)
    Asko Soukka
    @datakurre
    @iFlameing Thanks! After this week I would expect you to have an opinion on which way would make most sense (more testing, which kind of testing or just typescript).
    Alok Kumar
    @iFlameing
    @datakurre I will give my best In researching on these topics and develop my opinion about these different options.
    Asko Soukka
    @datakurre
    Great! Thanks!
    Alok Kumar
    @iFlameing
    @datakurre I read the test written for utils.js. I also read the documentation of jest. So, in my opinion we should write unit test first because it confirms that our logic is correct. Whether we use typescript we need to write unit test because typescript only ensure that whatever we returning from the function is of correct form but it doesn't ensure that our logic is correct. I asked Jason Lengstorf for integration tests for the GatsbyJs API and he replied "Each of the Gatsby APIs is a function, so you can unit test by importing and stubbing out Gatsby helpers. For integration testing, it would be easier to test the output (e.g. ensure pages were created by createPages). " I don't think I understand it fully. what should I do now?
    Asko Soukka
    @datakurre
    @iFlameing Thank you for your research.
    @iFlameing We do already have acceptance tests to test that pages are created and I'm on my way to extend that approach to also support testing that your code creates, updates and deletes page on "gatsby develop".
    @iFlameing So I can agree with you that focusing on unit tests might be the best next step.
    Asko Soukka
    @datakurre
    Possibly you should add gatsby-node.tests.js here https://github.com/collective/gatsby-source-plone/tree/master/src/__tests__ and start trying to unit-test functions, starting from the seemingly easiest ones.
    This should be interesting reference test case https://github.com/collective/gatsby-source-plone/blob/f332e32784e15609d8d0d488cbb3d62a25089138/src/__tests__/utils.tests.js#L90
    To be able to test that fetchPlone fetches all pages from paged REST API endpoint, I needed to refactor fetchPlone to take the used HTTP client library as an argument. As seen in utils.js it defaults to the imported library, but for testing it allows to pass a mock library, to test the function without making actual HTTP calls.
    mocking = stubbing
    Asko Soukka
    @datakurre
    Similar approach should work for testing all functions, but usually requires touching the function to make them accept the mock/stub as an argument.
    What could a stub be for a GatsbyJS helper? E.g. something that simply counts how many times it has been called and then the test assert just checks that it has been called correct amount of times.
    Asko Soukka
    @datakurre
    Meanwhile I did update versions in the master branch. Changes in GatsbyJS required to replace the use of yarn link with manually symlinking the plugin into test project's node_modules. If you face strange issues, a fresh clone should help. They added some GatsbyJS themes related autodiscovery that broke yarn linked plugins, because GatsbyJS auto-discovers iteself from linked plugin's node_modules...
    To summarize:
    • Try unit-testing functions in src/gatsby-node.js in src/tests/gatsby-node.tests.js
    • I will update acceptance tests to test the new code (and that mostly covers integration tests)
    • If you have issues with master due to my changes, try a fresh clone
    I'm sorry that I'm online weird hours now in July. I'm actually on vacation and I'm mostly on keyboard when the rest of my family is already asleep.
    Alok Kumar
    @iFlameing
    @datakurre thanks for explaining everything in so details.

    I'm sorry that I'm online weird hours now in July. I'm actually on vacation and I'm mostly on keyboard when the rest of my family is already asleep.

    That’s absolutely fine :)

    Asko Soukka
    @datakurre

    @iFlameing To make some heavy manual testing of your work:

    Leave "make watch" running and run "make init-backend" to trigger a lot of content updates. Delete all content from Plone and then start again with "make init-backend" to re-create everything.