Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Alok Kumar
    @iFlameing
    @nileshgulia1 thanks!
    @datakurre Since all the issue related to MVP is closed now, I am thinking that MVP is done now :tada: :tada: .
    Asko Soukka
    @datakurre
    @iFlameing Yes. Congratulations!
    So, what should you do next...
    Alok Kumar
    @iFlameing
    @datakurre I should start writing test for the added feature as we discuss earlier.
    can you guide me?
    How I can get started with it?
    Asko Soukka
    @datakurre
    @iFlameing Could you start by going through a few popular/official GatsbyJS source plugins to learn how they do their testing?
    I would guess that plugins they support for GatsbyJS Preview should be good quality enough to have some tests https://www.gatsbyjs.com/docs/#using-gatsby-preview
    Or maybe Wordpress one.
    All of those probably live inside the main GatsbyJS repository.
    Currently we have only a few simple unit tests for utils.js.
    Alok Kumar
    @iFlameing
    @datakurre ok!
    Asko Soukka
    @datakurre
    We are using jest https://jestjs.io/docs/en/getting-started but at least currently not snapshot tests. Just regular asserts.
    Snapshot tests are meant to test React component rendering.
    I assume we could continue to use jest for integration tests of gatsby-node API.
    Asko Soukka
    @datakurre

    @iFlameing So, please, use this week to review how the existing source plugins have been tested.

    Only then we should really decide, what kind of testing makes sense here.

    We already have high level acceptance tests for all the old features. I take the responsibility for upgrading that to support testing changes through websocket events. (That would be mainly bash scripting and Robot Framework, so for you that would be quite a step outside current JavaScript focus.)
    Most obvious choice would be to keep moving code from gatsby-node.js to utils.js or some other module and test those funtions with jest unit tests.
    But integration tests for the GatsbyJS API hooks in gatsby-node.js are still missing and there I would like GatsbyJS to provide some example or framework to do it right (and easily).
    Then again, another way for better quality would be to convert the project into typescript. It would probably have larger impact than having integration tests, but the amount of work in that is hard to estimate.
    Asko Soukka
    @datakurre
    @iFlameing So, because this is now only the first week of the second part of GSOC, I think it would be good to use a week to just learn the other plugins and then decide. If you run out of work, look into typescript also. This seemed to be a plugin using it https://github.com/sanity-io/gatsby-source-sanity
    Alok Kumar
    @iFlameing
    @datakurre okay! I will see the test written for different Gatsby-preview CMS and interact with GatsbyJS community for How we can write test for GatsbyJS API :)
    Asko Soukka
    @datakurre
    @iFlameing Thanks! After this week I would expect you to have an opinion on which way would make most sense (more testing, which kind of testing or just typescript).
    Alok Kumar
    @iFlameing
    @datakurre I will give my best In researching on these topics and develop my opinion about these different options.
    Asko Soukka
    @datakurre
    Great! Thanks!
    Alok Kumar
    @iFlameing
    @datakurre I read the test written for utils.js. I also read the documentation of jest. So, in my opinion we should write unit test first because it confirms that our logic is correct. Whether we use typescript we need to write unit test because typescript only ensure that whatever we returning from the function is of correct form but it doesn't ensure that our logic is correct. I asked Jason Lengstorf for integration tests for the GatsbyJs API and he replied "Each of the Gatsby APIs is a function, so you can unit test by importing and stubbing out Gatsby helpers. For integration testing, it would be easier to test the output (e.g. ensure pages were created by createPages). " I don't think I understand it fully. what should I do now?
    Asko Soukka
    @datakurre
    @iFlameing Thank you for your research.
    @iFlameing We do already have acceptance tests to test that pages are created and I'm on my way to extend that approach to also support testing that your code creates, updates and deletes page on "gatsby develop".
    @iFlameing So I can agree with you that focusing on unit tests might be the best next step.
    Asko Soukka
    @datakurre
    Possibly you should add gatsby-node.tests.js here https://github.com/collective/gatsby-source-plone/tree/master/src/__tests__ and start trying to unit-test functions, starting from the seemingly easiest ones.
    To be able to test that fetchPlone fetches all pages from paged REST API endpoint, I needed to refactor fetchPlone to take the used HTTP client library as an argument. As seen in utils.js it defaults to the imported library, but for testing it allows to pass a mock library, to test the function without making actual HTTP calls.
    mocking = stubbing
    Asko Soukka
    @datakurre
    Similar approach should work for testing all functions, but usually requires touching the function to make them accept the mock/stub as an argument.
    What could a stub be for a GatsbyJS helper? E.g. something that simply counts how many times it has been called and then the test assert just checks that it has been called correct amount of times.
    Asko Soukka
    @datakurre
    Meanwhile I did update versions in the master branch. Changes in GatsbyJS required to replace the use of yarn link with manually symlinking the plugin into test project's node_modules. If you face strange issues, a fresh clone should help. They added some GatsbyJS themes related autodiscovery that broke yarn linked plugins, because GatsbyJS auto-discovers iteself from linked plugin's node_modules...
    To summarize:
    • Try unit-testing functions in src/gatsby-node.js in src/tests/gatsby-node.tests.js
    • I will update acceptance tests to test the new code (and that mostly covers integration tests)
    • If you have issues with master due to my changes, try a fresh clone
    I'm sorry that I'm online weird hours now in July. I'm actually on vacation and I'm mostly on keyboard when the rest of my family is already asleep.
    Alok Kumar
    @iFlameing
    @datakurre thanks for explaining everything in so details.

    I'm sorry that I'm online weird hours now in July. I'm actually on vacation and I'm mostly on keyboard when the rest of my family is already asleep.

    That’s absolutely fine :)

    Asko Soukka
    @datakurre

    @iFlameing To make some heavy manual testing of your work:

    Leave "make watch" running and run "make init-backend" to trigger a lot of content updates. Delete all content from Plone and then start again with "make init-backend" to re-create everything.

    Alok Kumar
    @iFlameing
    @datakurre ok! I am trying that right now and update you with the result.
    Asko Soukka
    @datakurre

    I'm once again working on top-level acceptance tests for your work and I'm seeing those random errors that might actually be GatsbyJS issues. I haven't properly reproduced those yet.

    Those might also be issues caused by gatsby-source-plone asynchronously processing multiple conflicting notifications.

    I wonder if we should wrap notification handlers with a lock https://www.npmjs.com/package/async-lock

    @iFlameing Are you familiar with concept of locks (usually familiar from threaded programming)?
    Alok Kumar
    @iFlameing
    @datakurre No, But I will learn
    Asko Soukka
    @datakurre

    @iFlameing Ok. I believe the README of https://www.npmjs.com/package/async-lock explains it well.

    I've never used that package before, but I've neither ever used locks with JavaScript, so that simple the first Google result.

    @iFlameing I believe that currently every "await" within a single handling of ws.onmessage allows NodeJS to switch execution context.

    For example, whenever we are waiting for the results of call to search-endpoint (for example, to update breadcrumbs or navigation), NodeJS is allowed to start processing next ws.onmessage event.

    Alok Kumar
    @iFlameing
    @datakurre I am thinking that it works like this if any async function is running and if we call it again we have to wait for earlier function call to complete. what do you think?