======= Testing ======= To get an overall status of your cluster, run the tests from the command line. There are two types of tests: `testinfra `__ tests verify the setup, and end-to-end tests that check the webservice availability. Testinfra tests =============== Testinfra tests are split into two groups, lets call them *blackbox* and *clearbox* tests. The blackbox tests run on your provisioning machine and test the Stackspin cluster from the outside. For example, the certificate check will check if the Stackspin returns valid certificates for the provided services. The clearbox tests run on the Stackspin host and check i.e. if docker is installed in the right version etc. Our testinfra tests are a combination of blackbox and clearbox tests. If you have installed the Stackspin python requirements inside a :ref:`virtual environment ` then you need to activate it before testing: .. code:: console $ . env/bin/activate Then enter the *test* directory in the Git repository **on your provisioning machine**. .. code:: console $ cd test To run the test against your cluster, first export the *CLUSTER_DIR* environment variable with the location of your cluster config directory (replace *stackspin.example.org* with your cluster name): Then export the *KUBECONFIG* variable that lets ``kubectl`` talk to your cluster. Thirdly, export a variable that contains the password for the Prometheus HTTP endpoint. .. code:: console $ export CLUSTER_DIR="${PWD}/../clusters/stackspin.example.org" $ export KUBECONFIG=${CLUSTER_DIR}/kube_config_cluster.yml $ export BASIC_AUTH_PW=$(kubectl get secret -n stackspin stackspin-prometheus-basic-auth --template '{{ .data.pass }}' | base64 -d) $ Run all tests ------------- .. code:: console $ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' Test all applications --------------------- This will check for: * The applications return proper certificates * All helm releases are successfully installed * All app pods are running and healthy (this test includes all optional applications) These tests includes all optional applications and will fail for optional applications that are not installed. .. code:: console $ pytest -s -m 'app' --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' Tests a specific application ---------------------------- .. code:: console $ pytest -s -m 'app' --app="wordpress" --connection=ansible --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' Advanced usage of Testinfra tests --------------------------------- Specify host manually: .. code:: console $ py.test -s --hosts='ssh://root@example.stackspin.net' Run only tests tagged with *prometheus*: .. code:: console $ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m prometheus Run cert test manually using the ansible inventory file: .. code:: console $ py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m certs Run cert test manually against a different cluster, not configured in any ansible inventory file, either by using pytest: .. code:: console $ FQDN='example.stackspin.net' py.test -sv -m 'certs' or directly: .. code:: console $ FQDN='example.stackspin.net' pytest/test_certs.py Running Testinfra tests with local gitlab-runner docker executor Export the following environment variables like this: .. code:: console $ export CI_REGISTRY_IMAGE='open.greenhost.net:4567/stackspin/stackspin' $ export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_stackspin_ci)" $ export COSMOS_API_TOKEN='…' then: .. code:: console $ gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap Known Issues ------------ The default ssh backend for testinfra tests is *paramiko*, which does not work out of the box. It fails to connect to the host because the *ed25519* hostkey is not verified. Therefore we need to force plain *ssh://* with either ``connection=ssh`` or ``--hosts=ssh://…`` End to end tests ================ End to end tests run in a browser to test if all the web services are up and running and functioning correctly. Cypress tests ------------- Cypress tests are *not yet* integrated in the ``stackspin`` CLI command suite. Used for testing these applications: * Grafana * Hedgedoc * Nextcloud * Wekan * Wordpress * Zulip To install `Cypress`_ on your provisioning machine go to the *test/* directory and run: .. code:: console $ npm install -g cypress Test Grafana '''''''''''' .. code:: console $ export CYPRESS_BASE_URL='https://grafana.stackspin.example.net' $ export CYPRESS_USE_SSO_LOGIN="true" $ export CYPRESS_SSO_USER='admin' $ export CYPRESS_SSO_PW='...' $ cypress run --headed Test nextcloud '''''''''''''' The cypress end to end test is not included in the main Stackspin repository, but in the `Nextcloud helm chart repository `__. .. code:: console $ git clone https://open.greenhost.net/stackspin/nextcloud.git $ export CYPRESS_BASE_URL='https://files.stackspin.example.net' $ export CYPRESS_USE_SSO_LOGIN="true" $ export CYPRESS_SSO_USER='admin' $ export CYPRESS_SSO_PW='...' $ cypress run --headed .. _Cypress: https://go.cypress.io