From: Petr Štetiar <ynezz@true.cz>
Date: Fri, 22 Dec 2023 09:46:01 +0000 (+0000)
Subject: ci: improve QA with cram based tests
X-Git-Tag: v11~1
X-Git-Url: http://git.openwrt.org/?p=buildbot.git;a=commitdiff_plain;h=f94c6dd611a4b8147c84f55e5b52677b6dc74a43

ci: improve QA with cram based tests

Currently we've broken master container but we're not aware about it as
current tests are very lame, so lets improve it a bit with some more
reliable, extensible solution.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
---

diff --git a/.github/workflows/build-push.yml b/.github/workflows/build-push.yml
index 9a2eed4..f568ed6 100644
--- a/.github/workflows/build-push.yml
+++ b/.github/workflows/build-push.yml
@@ -83,8 +83,8 @@ jobs:
         run: |
           docker run --detach ${{ matrix.container_test_command }} --name test-${{ matrix.container_flavor }} local/${{ matrix.container_flavor }}
           sleep 5
-          docker logs test-${{ matrix.container_flavor }} | tee ${{ matrix.container_flavor }}.log
-          grep "${{ matrix.container_verify_string }}" ${{ matrix.container_flavor }}.log
+          pip install cram
+          cram --verbose "tests/cram/${{ matrix.container_flavor }}"
 
   deploy:
     name: Push Container
diff --git a/.gitignore b/.gitignore
index a94d1d2..0d4e15f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,3 +25,5 @@ phase[12]/twistd.*
 !.github/**/*
 !requirements-dev.txt
 !.ruff.toml
+!tests
+!tests/**/*
diff --git a/requirements-dev.txt b/requirements-dev.txt
index a14cc03..e1cb65f 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -1,2 +1,3 @@
+cram==0.7
 black==23.3.0
 ruff==0.0.267
diff --git a/tests/cram/master/01-logs.t b/tests/cram/master/01-logs.t
new file mode 100644
index 0000000..b6c5c81
--- /dev/null
+++ b/tests/cram/master/01-logs.t
@@ -0,0 +1,7 @@
+Check that logs have expected content after container startup:
+
+  $ docker logs test-master
+  updating existing installation
+  creating /master/master.cfg.sample
+  creating database (sqlite:///state.sqlite)
+  buildmaster configured in /master
diff --git a/tests/cram/worker/01-logs.t b/tests/cram/worker/01-logs.t
new file mode 100644
index 0000000..7afdd4e
--- /dev/null
+++ b/tests/cram/worker/01-logs.t
@@ -0,0 +1,10 @@
+Check that logs have expected content after container startup:
+
+  $ docker logs test-worker
+  updating existing installation
+  mkdir /builder/info
+  Creating info/admin, you need to edit it appropriately.
+  Creating info/host, you need to edit it appropriately.
+  Not creating info/access_uri - add it if you wish
+  Please edit the files in /builder/info appropriately.
+  worker configured in /builder