How to use the command line client.
By default the test-runner will do nothing if no projects are configured. If you have projects setup in your dashboard. The test-runner will run all tests on all projects. For this reason it's advisable to setup your CI with a projects environment variable so you only run the projects specified.
It can run multiple projects tests, or just one. By simply providing a comma separated list of projects, the test runner will fetch the tests associated and run them. Environment variables are supported as well as a command line flags for the test runner.
Lets see running multiple projects with an environment variable.
SUPERADMIN_PROJECTS="Engineering, Metrics" superadmin-cli
You can also set the projects list as a command line flag.
superadmin-cli --projects "Engineering, Metrics"
Note: Make sure if you have multiple projects or a project with a space in the name, you use quotes of some kind so the shell interpreter doesn't separate the value.
You can also set the projects argument as a keymap, to override the domain of a project.
For example. Let's say you set the domain of a project to http://myapp.example. And when running tests, you're running a test server at http://localhost:4000. You want to run tests during CI, before you deploy to production. You can do so with the following command line flag.
superadmin-cli --projects "Myapp=http://localhost:4000"
Now all of your tests will be run against
http://localhost:4000 instead of
myapp.example. You can do the same with the environment variable
Note: To avoid ambiguity a protocol is required, whether
https:. Otherwise a hostname override will error out. If the protocol is specified as
https: the certificate is not validated.
Here are some acceptable hostname overrides:
Here are some incorrect urls:
localhostis resolved by the dns resolver and will typically use the Kernal loopback interface to arrive at
127.0.0.1. You can also specify
127.0.0.1directly to avoid the dns resolution step.
mylocalhostis not a valid dns resolution for
localhostand wouldn't resolve correctly.
0.0.0.0is typically valid as well. Learn more
An API Key can be passed as a command line flag or as an environment variable. We recommend passing secret keys through the environment so they are not stored or saved in your code.
Lets look at how to pass authentication keys in our CI.
SUPERADMIN_API_KEY=eyJhbGc... npm run test
You'll need to generate an API Key from the dashboard if you do not have one. Learn More
Here's an example of how to pass an authentication key as a command line flag.
superadmin-cli --projects Engineering --api-key eyJhbGc...
You can tag your test runs so it is clear what environment you are testing against when the tests are run. You may for example want to label certain tests as staging to indicate they were run in the staging environment. You can do so easily.
superadmin-cli --tags staging
All tests run will get the label staging. You can also add multiple tags such as in the following example.
superadmin-cli -t "staging, testing"
Note: You can also specify the environement variable
SUPERADMIN_TAGS similar to the command line flag.
SUPERADMIN_API_KEY=ey... superadmin-cli -P "Engineering=http://localhost:4000" -t staging
Run only tests for the project "Engineering" at
localhost:4000 and set the build tag to
SUPERADMIN_API_KEY=ey... superadmin-cli -t production
Run all tests for all projects at their set domain and set the build tag to