Fix issues with new versions of docker-compose

.github/actions/run-tests/README.md

@@ -26,18 +26,22 @@ A complete run will all tests would consist of the following steps:
 1. Pull all helper images like mysql, postgres, apache and nginx.
 2. Build a set of images to run the tests within.
 3. Start the database containers.
-4. Install the nextcloud server and enable the cookbook app within.
-5. Save the environemnt (data files and DB dump).
-6. Start of additional containers like Apache and nginx container for integration testing.
-7. Run the tests themselves within the docker container.
-8. Shutdown all containers no more needed.
+4. Install the nextcloud server.
+5. Create a `plain` dump for testing of migrations.
+6. Enable the cookbook app within the Nextcloud server.
+7. Save the environemnt (data files and DB dump).
+8. Start of additional containers like Apache and nginx container for integration testing.
+9. Run the tests themselves within the docker container.
+10. Shutdown all containers no more needed.
 
 There is a script called `run-locally.sh` that handles all these steps plus some housekeeping. It takes command line options to specify which steps to carry out. When in doubt, you can always execute `./run-locally.sh --help` to see an list of the options.
 
 ## Manual invocation of the tests
 
 The order of the parameters of the script is not of importance. The order of the steps to be carried out is fixed and chosen such that a typical test run can be carried out quickly.
 
+If you need a special ordering for whatever reason, please call the script multiple times.
+
 ### Handling of the docker images
 
 The following options are available to manage the docker images:
@@ -50,7 +54,7 @@ The following options are available to manage the docker images:
 
 Typically, one will use the `--create-images` flag which will pull any images and rebuild the existing images if anything has changed.
 
-It is hightly suggested to set the environment variable `COMPOSE_DOCKER_CLI_BUILD` to `1`. This will avoid building the images from scratch as a online cached version (nightly build) can be used instead. If your setup does not break, you can simply call `export COMPOSE_DOCKER_CLI_BUILD=1` before the first invocation of `run-locally.sh` once in each terminal you open.
+By default, the image built for testing is a customized version of the `php` image. The customized version is built once per night and can be repulled with `--pull`. If you encounter issues with a dated PHP version or dependencies, you could refetch/rebuild the image locally.
 
 To push images credentials are needed. This is only required to speed up the the build prucess for other users. You will not have to do this unless you know exacly what it does.
 
@@ -66,9 +70,15 @@ You can ask the script to start the containers and stop them at the termination
 
 The environemnt is the databse and the file structure. During integration tests these can change and need to be reset using a fixute. These parameters allow to create such a fixture easily.
 
-Creation of an environment means to set up a new nextcloud instance in the existing folder stucture. This might take some time and also some download bandwith as the server needs to be downloaded.
+Creation of an environment means to set up a new nextcloud instance in the existing folder stucture. This might take some time (in the range of 5-10 min) and also some download bandwith as the server source code needs to be downloaded.
+
+To speed things up, fixtures can be saved on the local disc and replayed later during the integration tests if needed. Once a fixture is saved, the corresponding name is reserved. Creating another fixture with the same name is prohibited by default. This behavior can be overridden by an explicit flag `--overwrite-env-dump`.
+
+There are two fixtures involved:
+- The `plain` fixture is the state just after successfully install the Nextcloud core. It can be used for testing migrations.
+- The `default` fixture (can be renamed with `--env-dump-path`) with the cookbook app just installed.
 
-To speed things up, fixtures can be saved on the local disc and replayed later during the integration tests if needed. Once a fixture is saved, the corresponding name is reserved. Creating another fixture with the same name is prohibited by default. This behavior can be overridden by an explicit flag `--overwrite-env-dump`. The name of the fixture is by default `default` but can be changed using `--env-dump-path`. When given multiple times, the last occurence will take precedence.
+The name of the fixture is by default `default` but can be changed using `--env-dump-path`. When given multiple times, the last occurence will take precedence.
 ```
   --setup-environment <BRANCH>      Setup a development environment in current folder. BRANCH dictates the branch of the server to use (e.g. stable20).
   --drop-environment                Reset the development environment and remove any files from it.
@@ -77,8 +87,12 @@ To speed things up, fixtures can be saved on the local disc and replayed later d
   --drop-env-dump                   Remove a backup from an environment
   --overwrite-env-dump              Allow to overwrite a backup of an environment
   --env-dump-path <PATH>            The name of the environment to save. Multiple environment backups are possible.
+  --copy-env-dump <SRC> <DST>       Copy the environment dump in <SRC> to <DST>
+  --list-env-dumps                  List all environment dumps stored in the docker volume
 ```
 
+Using the `--list-env-dumps` and `--copy-env-dump` the user might be able to create a bunch of env dumps representing different versions to test (e.g. different Nextcloud core versions or different PHP versions). Just make sure that the current one to use is called `plain` and `default`.
+
 ### Running the tests
 
 The tests are separated into unit tests and integration tests. These can be run separately or together with `--run-tests`.
@@ -93,12 +107,45 @@ To tune the running of the tests, some more flags are present:
   --extract-code-coverage           Output the code coverage reports into the folder volumes/coverage/.
   --install-composer-deps           Install composer dependencies
   --build-npm                       Install and build js packages
-  --filter <FILTER>                 Pass the FILTER to the testing framework for filtering.
 ```
 
 You can create a code coverage report that might be useful to check your tests.
-Using the `--filter` flag you can pass filters to the phpunit in order to resttrict the test cases if you are actively developing and solving a bug.
-Further you can command composer to update all dependencies before the tests are run in order to simulate a CI run.
+Further you can command composer and/or NPM to update all dependencies before the tests are run in order to simulate a CI run.
+
+### Debugging in test mode
+
+It might make sense to step-debug the tests in case the result is not as intended. There are a few settings prepared to get this running quickly.
+
+```
+  --debug                           Enable step debugging during the testing
+  --debug-port <PORT>               Select the port on the host machine to attach during debugging sessions using xdebug (default 9000)
+  --debug-host <HOST>               Host to connect the debugging session to (default to local docker host)
+  --debug-up-error                  Enable the debugger in case of an error (see xdebug's start_upon_error configuration)
+  --debug-start-with-request <MODE> Set the starting mode of xdebug to <MODE> (see xdebug's start_with_request configuration)
+  --xdebug-log-level <LEVEL>        Set the log level of xdebug to <LEVEL>
+  --enable-tracing                  Enable the tracing feature of xdebug
+  --trace-format <FORMAT>           Set the trace format to the <FORMAT> (see xdebug's trace_format configuration)
+  --enable-profiling                Enable the profiling function of xdebug
+```
+
+The flags `--debug`, `--enable-tracing`, and `--enable-profiling` activate the corresponding features for step-debugging, tracing of PHP code and time-profiling the tests. As tracing and profiling tend to generate big files, these should be used with care.
+
+For step-debugging, there needs to be a matich IDE that listens for incoming connections by the PHP debugger [xdebug](https://xdebug.org/). You can configure the port and host using `--debug-port` and `--debug-host`. By default these are set such that a locally running IDE on port 9000 is accessed.
+
+The other flags allow fine-tuning. In case of problems with the IDE , one might consider increasing the log level of xdebug using `--xdebug-log-level` to 10 (debug). There will be a log file written in the folder `volumes/output`.
+
+### Custom options to phpunit and filtering
+
+It is possible to pass arbitrary parameters to the phpunit process. This is especially useful to filter only certain tests. The same parameters are passed to both unit and integration testing. So, if you need different parameters, you should consider calling them separately.
+
+The parameters to the `run-locally.sh` script are separated by `--`. If no `--` is found, the second part is considered empty.The first part of the parameters are the named parameters as described above. The second part are passed 1-to-1 to the PHPUnit process. For possible options see the man page, the internet documentation of PHPUnit or even `./run-locally.sh --run-unit-tests -- --help`.
+
+
+Using the `--filter` flag of PHPUnit you can pass filters to the phpunit in order to resttrict the test cases if you are actively developing and solving a bug. For example to only run the tests for the tests in `OCA\Cookbook\tests\Unit\Foo` namespace, you could run
+```
+./run-locally.sh --run-unit-tests -- --filter '^OCA\\Cookbook\\tests\\Unit\\Foo'
+```
+(Note: The filter string is a regex agains the FQDN of the class. Therefore the `\` must be escaped even inside the `''` quotes. More fancy filters are of course possible.)
 
 ## Typical steps involved to run tests locally
 

.github/actions/run-tests/run-locally.sh

@@ -32,10 +32,10 @@ Possible options:
   --debug-port <PORT>               Select the port on the host machine to attach during debugging sessions using xdebug (default 9000)
   --debug-host <HOST>               Host to connect the debugging session to (default to local docker host)
   --debug-up-error                  Enable the debugger in case of an error (see xdebug's start_upon_error configuration)
-  --debug-start-with-request <MODE> Set the starting mode of xdebug to <MODE> (see xdebug's start_with_request configuration)
+  --debug-start-with-request <MODE> Set the starting mode of xdebug to <MODE> (see xdebug's start_with_request configuration, 'yes' by default)
   --xdebug-log-level <LEVEL>        Set the log level of xdebug to <LEVEL>
   --enable-tracing                  Enable the tracing feature of xdebug
-  --trace-format <FORMAT>           Set the trace format to the <FORMAT> (see xdebug's trace_format configuration)
+  --trace-format <FORMAT>           Set the trace format to the <FORMAT> (see xdebug's trace_format configuration, default 1)
   --enable-profiling                Enable the profiling function of xdebug
   --help                            Show this help screen
   --                                Pass any further parameters to the phpunit program
@@ -420,7 +420,7 @@ run_tests() {
 	echo "Staring container to run the unit tests."
 	echo "Parameters for container: $PARAMS"
 	echo "Additional parameters for phpunit: $@"
-	docker-compose run --rm -T dut $PARAMS -- "$@"
+	docker-compose run --rm dut $PARAMS -- "$@"
 	echo 'Test runs finished.'
 }
 
@@ -498,6 +498,9 @@ source postgres.env
 
 RSYNC_PARAMS="--delete --delete-delay --delete-excluded --archive"
 
+# Use old styled docker-compose names with underscores instead of dashes
+export COMPOSE_COMPATIBILITY=true
+
 ##### Extract CLI parameters into internal variables
 
 if [ $# -eq 0 ]; then

CHANGELOG.md

@@ -3,6 +3,8 @@
 ### Fixed
 - CI build always builds docker images from scratch
   [#823](https://github.com./nextcloud/cookbook/pull/823) @christianlupus
+- Fix test script after update in docker-compose
+  [#833](https://github.com./nextcloud/cookbook/pull/833) @christianlupus
 
 
 ## 0.9.6 - 2021-10-18