Most of our customers test their applications while they are still in development, before they have been released to the public. We can also test skills that are in production, after they have been made public.
You may need to test in production due to skill certification timelines, or to troubleshoot errors reported with an app you have already released.
In order to have a successful beta test, you’ll need to make some preparations before testing.
- Ensure you’ve logged into your Amazon Developer account through your Pulse Labs account to authorize SMAPI. In addition to skill metadata, this lets our system automatically add panelists to the skill beta and remove them when testing is done.
- If you plan to test a development version of a skill that is already in production, create the development version under the same skill ID, or, if you need to create a separate skill, give it a different invocation name. If two skills have the same invocation, Alexa sometimes prioritizes the production version, which can cause issues during testing.
- If inviting your own panelists, keep in mind that Amazon has a limit of 500 beta participants at a time.
- If you are running a beta test and need to test a skill in a different developer account, wait until the first test is done. If you change mid-test, this will disrupt the SMAPI connection and prevent panelists from being able to test.
- Amazon does not allow beta participants to be added while a skill is undergoing certification, so you will need to complete testing before or after certification.
If you have an Enterprise or Agency account, you can set up testing for Google Actions.
- If your action is in beta and you are testing with your own panelists, be sure to add their email addresses to your beta at least three hours before you plan to submit your test.
- If you plan to test in beta with Pulse Labs Panelists, contact your account manager, and our operations team will get you email addresses of the panelists who will be testing so you can add them to the beta.
- Include the beta link in the test overview so panelists can enable the action.
- For more details about Google’s beta enablement process, see their documentation.
- You can also do alpha testing in a similar fashion, but Google limits this to 20 participants.