Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 16:04
    mvastarelli commented #789
  • 04:10
    SergeiGolos review_requested #789
  • 04:06
    SergeiGolos synchronize #789
  • 01:12
    SergeiGolos synchronize #789
  • 01:07
    SergeiGolos synchronize #789
  • Jan 18 22:30
    SergeiGolos synchronize #789
  • Jan 18 22:29
    SergeiGolos synchronize #789
  • Jan 15 22:18
    SergeiGolos synchronize #789
  • Jan 15 15:49
    SergeiGolos synchronize #789
  • Jan 15 02:27
    SergeiGolos opened #789
  • Dec 21 2019 17:44
    jeremydmiller reopened #788
  • Dec 21 2019 17:44
    jeremydmiller commented #788
  • Dec 21 2019 17:44
    jeremydmiller closed #788
  • Dec 21 2019 17:44
    jeremydmiller commented #788
  • Dec 19 2019 19:23
    jeremydmiller opened #788
  • Dec 19 2019 14:42
    jeremydmiller milestoned #787
  • Dec 19 2019 14:42
    jeremydmiller labeled #787
  • Dec 19 2019 14:42
    jeremydmiller opened #787
  • Nov 06 2019 21:38
    dependabot[bot] labeled #786
  • Nov 06 2019 21:38
    dependabot[bot] opened #786
Jarrod
@JarrodJ83
@jeremydmiller is there a timeout of some sort for a grammar to execute. I'm running into an issue where I have a grammar that just does awaits a Task.Delay. If I make it for any longer than 60 seconds the UI will not continue processing when the wait completes. If I make it less than 60 there doesn't seem to be any issues. Any thoughts? Am I missing something obvious?
Right around 60 seconds I notice the UI changes as well and all the check marks next to the steps that have already been executed disappear
Jeremy D. Miller
@jeremydmiller
The specification itself has a timeout that defaults to 60 seconds. Check the docs, there’s ways of overwriting that.
Already thought that we’d surface that better in the UI for 6.0.
You have to have a timeout to prevent runaway tests. Especially if naive Selenium tests are anywhere.
Jarrod
@JarrodJ83
Sorry I must have missed that in the docs. I'll check it out. Thanks for the info!
Jarrod
@JarrodJ83
@jeremydmiller the only thing I see is the --timeout flag that can be passed to the storyteller run command. Doesn't seem like there is a way to specify the timeout on a per-spec basis or globally while using the editor. Am I missing something?
Jeremy D. Miller
@jeremydmiller
Doh! Maybe so.
Is that not an input flag on the dotnet storyteller command? Like dotnet storyteller —timeout #?
Jarrod
@JarrodJ83
Looks like it is only on RunInput (not OpenInput)
Jeremy D. Miller
@jeremydmiller
That’s a big miss, okay.
For v6, I want that to be somethign that’s surfaced in the UI
Jarrod
@JarrodJ83
That'd be nice
In my particular case I just need it at a grammar level. Were you thinking of keeping at the spec level or possibly having it set at multiple levels?
Jeremy D. Miller
@jeremydmiller
At the spec level. You can reinforce “performance thresholds” at the grammar level, but that just fails the build. Doesn’t stop the execution
You’d have to do that one yourself
Jarrod
@JarrodJ83
Just some info on my use case: I'm testing some authn stuff so have a grammar that basically waits for an access token to expire. In my case any spec that uses it would need to have a longer timeout or it wouldn't work right so that is why I was curious about a grammar level settings
Sergei Golos
@SergeiGolos

@jeremydmiller i have been looking at the code for a day or two, and i think that I have misunderstandings about the role of the IGrammar method on a fixture.

I am using storyteller to test the validity of a system installation (smoke test), but have a variable number of configurations during install that would effect which subsystems are running and require testing. So I was thinking that the I could create a method that created different Facts based on what we see in the installed configuration files. But I think by doing this, I am bumping elbows with the logical separation of Fixture from Specification and execution contexts. I see the the this.Context object which exposes services to my fixutres is not yet defined when the function is executed.

So my question, am I doing something completely not intended or is this kind of what you were thinking when you allowed dynamic IGrammar generation? And a follow up, if it was the design idea behind the IGrammar methods, is there an obvious way i am missing to get service injection at method execution time (since the generated grammars are executed later)?

Jeremy D. Miller
@jeremydmiller
@SergeiGolos You might be trying to use Storyteller in a place where xUnit.Net is more appropriate
Fixture.Context would only be available at the time the specification is being executed
Sergei Golos
@SergeiGolos
We are looking at stoyteller as a way of spot checking QA and OPS installations of our software and if errors are found generating reports that reduce my teams time to troubleshoot. This seems a bit of a stretch for xUnit, and the storyteller reports seem to really fit the bill for detailed feedback on generated logs and system configuration.
Jeremy D. Miller
@jeremydmiller
Are you getting tripped up on the Fixture class’s lifecycle then?
Sergei Golos
@SergeiGolos
Kind of, what i am trying to first figure out is if my usecase of "optional" steps is outside of the expected storyteller workflow. (assuming the user is dumb and can't select appropriate tags to select the tests we want to run.)
Jeremy D. Miller
@jeremydmiller
Storyteller isn’t going to do any kind of optional step like that, no.
You could do something in the Fixture code that short circuits the check I suppose and stubs in a success
Sergei Golos
@SergeiGolos
Well from looking at the code, that's kind of what i though the methods resulting in IGrammar were for, but it seems that at that point in time you don't have access to the system at test.
Jeremy D. Miller
@jeremydmiller
Correct. “IGrammar” is kind of like a recipe for an action or an assertion that will happen at specification time. You’re creating a syntax to be used within specifications
Sergei Golos
@SergeiGolos
And all the Fixtures are loaded and compiled into specifications before the system under test is loaded?
Jeremy D. Miller
@jeremydmiller
Not quite. The Fixtures are built independently of loading either the system or the specifications. As part of execution, specifications are “planned” where the steps are attached to all the right IGrammars, but that’s meant to be an implementation detail
Sergei Golos
@SergeiGolos
Right, that makes sense. So is there any other way to get service resolution when the IGrammer method is compiled? It seems because of the Fixture composition features, we can't really do DI on fixture, and have a static singleton DI container seems bad.
Jeremy D. Miller
@jeremydmiller
Nope, you’d need to make everything lazy.
And as of Storyteller 3.0, it purposely does not do dependency injection against Fixtures because that became a performance issue to rebuild Fixture objects for every test
Sergei Golos
@SergeiGolos
ok thank you for your time, seems like i need to go back to the drawing board here
Jeremy D. Miller
@jeremydmiller
Yeah. I don’t think Storyteller is gonna be your guy here unless you can live within its normal idioms
Udemy free courses
@ayoub50706635_twitter
Hello has anyone tried the IGrammer Method ?
Sergei Golos
@SergeiGolos
Hey Jeremy!!! quick question, i have been crawling around in the Storyteller runner logic for the past couple of hours, and i am wondering if there is any specific way to hijack the document creation process. Basically, i want to add some scripts tags with helpful javascript methods for linking and other dynamic functionality. I see that most of the work on report generation is done by BatchResultsWriter, which is a static class. Is there any way around that?
Seems like my only option with code as is, is basically post processing on the generated file?
Sergei Golos
@SergeiGolos
    public static HtmlDocument BuildResults(BatchRunResponse results, IEnumerable<IDocumentBuilder> sideloadedBuilders = null)
    {
        var document = new HtmlDocument
        {
            Title = "Storyteller Batch Results for {0}: {1}".ToFormat(results.system, results.suite)
        };

        var cssFiles = new[]
        {
            new ResourceFileLoader("Storyteller.stylesheets.bootstrap.min.css"),
            new ResourceFileLoader("StoryTeller.stylesheets.storyteller.css"),
            new ResourceFileLoader("StoryTeller.stylesheets.fixed-data-table.min.css"),
        };
        var jsFile = new ResourceFileLoader("StoryTeller.batch-bundle.js");

        var builders = new IDocumentBuilder[] {
            new StylePartBuilder(cssFiles),
            new ScriptPartBuilder(jsFile),
            new ReportPartBuilder(),
        };

        foreach (var builder in builders.Concat(sideloadedBuilders))
        {
            builder.Apply(document, results);
        }

        return document;
    }     
Jeremy D. Miller
@jeremydmiller
Is that the existing code, or a suggestion about changing it? I don’t think it’d be that bad to add some extensibility there
Sergei Golos
@SergeiGolos
yeah this is what i am playing around with for changing it
no "new" code, just moving some stuff around and hiding it behind interface classes
interfaced* classes
Jeremy D. Miller
@jeremydmiller
Cool, thought so. Looks good to me. Maybe do something a little more formal for v6, but that’s gonna be awhile
Sergei Golos
@SergeiGolos
well, the issue that i am running into, is while this method would allow side loading of additional IDocumentBuilder(s), i think the call to this is behind the "RunCommand" object and that I don't know how to inject the sideloadedBuilders there
i mean i guess the hackish way would be to expose a static property we modify in the program file.. but as an engineer that bothers me :(
Jeremy D. Miller
@jeremydmiller
Cheat and make it a static, so you could do it in your ISystem. The static makes it easy to avoid breaking backward compatibility.
And yeah, but we could revisit it with v6;-)
Sergei Golos
@SergeiGolos
so i should make a pull request? sorry this is popping my open source cherry :)
Jeremy D. Miller
@jeremydmiller
Ah, cool, yes please.
Sergei Golos
@SergeiGolos
Any suggestions on how you do faking/mocking in the tests project? Do you do mocking/faking?