Creating your account and testing your Alexa Skill is a breeze with our easy online platform! Our system will walk you through setup step-by-step, but here are some things that are helpful to know before you get started.
First time account setup: 4 minutes
- Register for a Pulse Labs Customer Account
- Connect your Amazon Developer account for easy test setup and beta enablement
Before testing a new Skill: 20-30 minutes
- Add our SDK to your Skill's code to get the most out of your test
- If you previously tested the Skill and left the SDK in your code, you don't have to do this again!
Set up your test: At your own pace
- Provide Skill information
- Decide if you want to use your own panelists or Pulse Panelists, and if you have any specific demographic needs
- Write test instructions to give panelists an overview of what they will be testing, and 2-5 tasks for panelists to complete during the test
- Decide on 10 to 15 post-test questions
Want more detail? Keep reading!
Step 1 of 7: Register for a Pulse Labs Customer Account
Takes about 2 minutes
If this is your first test, or if you haven't tested with us since before April 2019, you'll need to create an account.
Step 2 of 7: Connect your Amazon Developer Account for easy beta enablement
Takes about 2 minutes
- Make sure that you have Admin or Developer privileges to the Amazon Developer account you use for skill development.
- If you have access to multiple consoles, you'll be able to choose skills from either console.
- If you'll be testing a Skill that is in development, make sure you've set up a Beta Test for it in your Alexa Developer console, otherwise we won't be able to test it.
Step 3 of 7: Add our SDK to your Skill's code
Get a 360-degree view of each testing session by instrumenting your Skill's code with our SDK (Software Development Kit). We'll use this to combine data from your Skill with user-side information to deliver unparalleled insights about each test session. Available for all dev environments, you can integrate this into your Skill’s code in about 20 minutes.
You can leave it in after testing to track your Skill’s performance over time as we continue to add more tools for long-term analytics.
Learn how to integrate the SDK for your skill.
Step 4 of 7: Provide Skill information
Most of this will be auto-populated if you log into your Amazon Developer account and choose one of your skills from the drop-down menu.
If your Skill requires account linking, be sure to check that option and provide instructions about how panelists should create and/or link their accounts. We suggest that you provide panelists with dummy data when possible so they do not have to share their personal information. We're happy to work with you if you have any unique account linking needs related to your development environment!
Step 5 of 7: Decide who should test your Skill
You can select gender and age, and write in any other custom demographic requirements. We can accommodate a wide variety of requests, but recruiting for custom demographic attributes can take time, so notify your account manager as soon as you decide what your needs are.
Step 6 of 7: Write test instructions and tasks
The instructions gives a summary to panelists about what they'll be testing, so they know what to expect when they get started. Tasks are steps they complete during testing.
- Tasks can be as broad or specific as you'd like.
- We recommend keeping them limited to 2-5 concise instructions.
- If you're early in development or wanting to test general usability, you may want to ask panelists to use the skill naturally, exit and re-enter, ask for help, or complete other basic tasks.
- If you're later in the development cycle or testing a problem area, you may want to ask users to try specific pathways like complete a purchase, make an appointment, or other areas where you want to focus on analysis.
Step 7 of 7: Come up with 10 to 15 post-test questions
There may be scenarios where you need to ask a few additional questions, and we can accommodate this. However; we've noticed that when asked more than about 15 questions, panelist start to exhibit 'test fatigue' and give lower quality feedback on later questions, especially if there are many that require a lengthy written response.
We provide 5 standard questions you can use or remove:
You can also add your own questions in the following formats:
- Yes or No
- Rating (1-5)
- Multiple Choice
- Short Answer
We know that coming up with great questions can be tough, so your account manager and our Ops team will be happy to help you come up with targeted questions to help you find the insights on the things that are most important to you!
Congratulations you've made it to the end of test setup!
Pulse Labs Review
Before we release your test to panelists, we'll carefully review all the information you provided for accuracy. We will also run through the instructions while using the skill to make sure test tasks are clear and that there are no critical skill issues that would prevent panelists from testing. If we find ways to optimize your test setup, we will let you know.