Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 10 17:39
    dependabot[bot] labeled #137
  • Aug 10 17:39
    dependabot[bot] opened #137
  • Aug 10 17:39

    dependabot[bot] on npm_and_yarn

    Bump path-parse from 1.0.5 to 1… (compare)

  • Jun 10 16:37
    btburton42 commented #115
  • Jun 07 21:51
    jkelvie commented #115
  • May 09 23:41
    dependabot[bot] labeled #136
  • May 09 23:41
    dependabot[bot] opened #136
  • May 09 23:41

    dependabot[bot] on npm_and_yarn

    Bump hosted-git-info from 2.7.1… (compare)

  • May 07 01:17
    dependabot[bot] labeled #135
  • May 07 01:17
    dependabot[bot] opened #135
  • May 07 01:17

    dependabot[bot] on npm_and_yarn

    Bump underscore from 1.9.1 to 1… (compare)

  • May 06 20:48

    dependabot[bot] on npm_and_yarn

    (compare)

  • May 06 20:48
    dependabot[bot] closed #125
  • May 06 20:48
    dependabot[bot] commented #125
  • May 06 20:48
    dependabot[bot] labeled #134
  • May 06 20:48
    dependabot[bot] opened #134
  • May 06 20:48

    dependabot[bot] on npm_and_yarn

    Bump lodash from 4.17.15 to 4.1… (compare)

  • May 06 17:25
    dependabot[bot] labeled #133
  • May 06 17:25
    dependabot[bot] opened #133
  • May 06 17:25

    dependabot[bot] on npm_and_yarn

    Bump handlebars from 4.5.3 to 4… (compare)

ash-at-github
@ash-at-github
We again started seeing "Invalid response: 400 Message: " for our virtual alexa scripts executed via Jenkins. We saw this before and we increased the timestamp validation limit to 150 seconds which is the max value suggested by Amazon (https://developer.amazon.com/docs/custom-skills/host-a-custom-skill-as-a-web-service.html#timestamp), which resolved that issue but it again started happening. Any suggestions?
Juan Perata
@jperata
Hi @ash-at-github , have you had any recent change in your jenkins environment or development server. A likely culprit, if it was working before, is that the timestamp has issues due the development server and jenkins being in different machines with their clocks on different times.
ash-at-github
@ash-at-github
no changes as far as we are aware, is there anyway to debug this on virtual alexa end?
Juan Perata
@jperata
if you are using virtual alexa directly as a Javascript library, you can use the filter property to log the requests and verify the timestamp.
If you are using virtual Alexa through YML tests, then you enable the "trace" property and it will print out the complete requests and responses for each interaction.
ash-at-github
@ash-at-github
@jperata Have used Trace option before, but how will it help in debugging this issue? to get the timestamps? We are getting timestamps right now after the error even without trace. And they seem ok. Anything else we need to use from trace?
Juan Perata
@jperata
yes, to get the timestamp that is being sent inside the request, and validate it against the server time in your jenkins server
Adam Elmore
@adamelmore
Hi Bespoken Team! Is it possible to use the virtual-alexa library to test interaction models only (for skills that don't use lambdas, and instead rely on external services)? I'm hoping to write unit tests to test the interaction model only; basically a bunch of "utter" tests where I'm asserting on intents and slots. Make sense?
John Kelvie
@jkelvie
Hi @adamelmore - it's likely possible with a bit of tweaking. But a quick question - what are you trying to test? Is it primarily making sure the interaction model is configured correctly? Or are you trying to make sure that the speech recognition and NLU are working right?
Adam Elmore
@adamelmore
I'll be looking into e2e later (with bespoken) to test speech reco, but right now I'm looking to build tests that assert that given an utterance, a specific intent is resolved with specific slot values. Maybe this isn't advised given that there may be a delta between what virtual-alexa resolves and what the actual Alexa would resolve?
John Kelvie
@jkelvie
Yes, there will definitely be a delta - our resolution mechanism is simplistic, and is only meant as a convenience mechanism
Adam Elmore
@adamelmore
One hack I'm considering is adding a fake handler that just parrots back the request as the response. In that way, I could assert on response.intent, etc.
John Kelvie
@jkelvie
It would be useful for ensuring all phrases you think are associated with an intent are actually associated with it - but that's about it. It won't tell you anything about ASR or NLU performance
Adam Elmore
@adamelmore
Cool, maybe I should just rely on e2e tests to validate that my interaction model is good and hasn't regressed.
John Kelvie
@jkelvie
The e2e is for full regression testing of code and AI - we actually have a new product that is squarely focused on testing just interaction models, in a very complete way
Adam Elmore
@adamelmore
oh, can you you point me to the interaction model product?
John Kelvie
@jkelvie
It's called Usability Performance Testing - our most recent case study on it is here: https://bespoken.io/blog/the-mars-agency-case-study/
Adam Elmore
@adamelmore
:tada: thanks a ton!
John Kelvie
@jkelvie
And a general overview is on our website: https://bespoken.io/usability-testing/
Our pleasure - and of course reach out if you have any additional questions
Adam Elmore
@adamelmore
Is there pricing information for usability-testing that I'm not seeing?
Or is it custom and you provide a quote?
John Kelvie
@jkelvie
It's the same pricing as for our subscription product: https://bespoken.io/pricing/ - that includes the usability performance testing
All of our products rely on interactions with Alexa/Google Assistant at their core, whether it is for end-to-end testing, monitoring or usability testing
xcobbler
@xcobbler
@jperata, amazon has acknowledged an issue with upsells when using certain models. my ugly workaround is to have 1 model for testing and 1 model for deployment :/
https://forums.developer.amazon.com/questions/207372/upsell-not-responding.html?childToView=210963#answer-210963
https://forums.developer.amazon.com/questions/208737/alexa-does-not-respond-to-request-for-upsell.html
Juan Perata
@jperata
thanks for keeping us informed @xcobbler , hopefully they will resolve the issue soon. But for now those links can help someone with the same issue in the future
Seth Lemoine
@mrseth01
I wrote a data library, and I'd like to write tests for it without emulating Alexa. I want to write tests that pull data from and endpoint and verify the data came back etc. How do I go about doing that?
I'm very new to testing. noob.
const NBA = require("../lambda/custom/nba-voice-library/NBA");

test("gets schedule from endpoint", async () => {
  expect.assertions(1);
  const data = await NBA.schedule();
  expect(data.length).toBe(4);
});
Juan Perata
@jperata
Hi @mrseth01 , your way sounds about right. Since you are not testing the Alexa section you don't need to introduce virtual-alexa in that section. We recommend that you also add some tests with our library to emulate Alexa too, it will interact with your endpoint unless you do some steps to mock it.
Mina Saleeb
@saleebm
Hey I'm getting an error about my network connection every time. I even tried following the node example but got this same error:
```
Something went wrong. Please check your network connection and try again.
Mina Saleeb
@saleebm
I'm using node v10, stack is RequestError: Error: read ECONNRESET\n at new RequestError ... t TLSSocket.socketErrorListener (_http_client.js:392:9)\n at TLSSocket.emit with macOS
John Kelvie
@jkelvie
Hi @saleebm - what tool are you using? And what command are you sending?
Mina Saleeb
@saleebm
Hey thank you for the quick response, I sent the command bst init and then I also tried node ./node_modules/bespoken-tools/bin/bst-proxy.js lambda ./src/index.ts
I installed bespoken-tools globally with npm v10.18.1 and also tried doing it in a local project,
Is it maybe because I use node version manager and I have my default set to v13?
Juan Perata
@jperata
Hi @saleebm - to use the proxy with the lambda parameters you would need to point to the ".js" file instead of ".ts" one.
Also, are you behind a corporate proxy?
Mina Saleeb
@saleebm
Oh ok, I see, and yes I actually might be, I'm at a college. I'll have to try again and see later, thank you
Mark Tucker
@rmtuckerphx
What is the best way to build a Connections.Response? It could include values for Buy, Upsell, Cancel, Setup, Charge & ACCEPTED, DECLINED, ALREADY_PURCHASED, ERROR
pintonos
@pintonos
Hi there, i have problems while mocking dynamoDB. Running it locally works fine, but when using it in the GitLab CI following error occurs: Uncaught exception: AskSdk.DynamoDbPersistenceAdapter Error: Could not create table (...): The security token included in the request is invalid. Any ideas?
John Kelvie
@jkelvie
Hmm...@pintonos - do you have a sample you can share with us?
pintonos
@pintonos
@jkelvie not really. I just used the alexa.dynamoDB().mock(); Do you ever have tested it in a CI (GitLab) environment? Perhaps this produces some new problems, like this one.
John Kelvie
@jkelvie
Yes we use CI environments regularly, including Gitlab. But I have not seen that particular error before.
akhil1296
@akhil1296
Hi, I am made alexa skill that uses third party APIs as well as few HTTP apps. I am not understanding how to do the unit testing for such cases. Any kind of help from you people will be highly appreciated. Thanks
allthepies
@allthepies

Hi, I am made alexa skill that uses third party APIs as well as few HTTP apps. I am not understanding how to do the unit testing for such cases. Any kind of help from you people will be highly appreciated. Thanks

Hi, I use nock to mock the API /HTTP calls.

Juan Perata
@jperata
Hi @akhil1296 , our full documentation for unit testing in Alexa using YML scripts is here. For your specific question if you are using YML, we set up an environment variable that you can use to change to get specific responses.
Since you are asking in the Virtual Alexa channel maybe you are using Virtual Alexa directly with Jest or Mocha, if that's the case then Nock is a great solution to mock API calls as @allthepies suggest.
spham1
@spham1_gitlab
hi
i have an issue after lauch bst test
Cannot find module '/home/sylvain/spham/alexa/skill-sample-nodejs-hello-world/index.js' from 'ModuleInvoker.js'
allthepies
@allthepies
Hi Guys. I 'm using the virtual-alexa dynamoDB mock and want to pre-populate the mocked dynamo table with a set of attributes at the start of a test. Is this possible ?
John Kelvie
@jkelvie
Hi @allthepies - if you activate the mock, and then make call to DynamoDB put - either directly via the API or via the AWS SDK, it will set the state of the table