15th Meeting of the Joomla Automated Testing working Group
wednesday 21th of October
Meeting was attended by: Javier Gomez, Tito Alvarez, Puneet Kala, Milton Bryant, Yves Hoppe, Roland Dalmulder, David Jardin, Michael Babker, Niels Braczek, Robert Deutz
Absent: Tristan Bailey, Viktor Vogel, Michiel Verhoef, Hervé Boinnard, Kshitij Sharma, Jack Raknailuang, Valerie Isaksen, Dalpat Singh, Victor Escobar, ...
Previous meeting minutes: https://volunteers.joomla.org/reports/121-14th-meeting-automated-testing-working-group
Topics Covered in the Meeting:
## Testing Code Sprint
The code sprint will happen from the 28th of October (arriving on the afternoon) until the 2nd of November (departure day).
### The goals are:
This are the tasks that we would like to cover during the two days:
Goal 1: Create all the tests for com_weblinks
- Add all the missing tests. See https://github.com/joomla-extensions/weblinks/issues/
- Update to use Codeception 2.1 and it’s new features
- Make travis report errors to the affected pull requests
- Fix travis to run PHP 5.5 and other versions (now running 5.3)
Goal 2: Prepare an action plan to migrate all of the current system Tests
- Creating Admin class
- Article Manager Tests
- Category Manager Tests
- Improve Joomla-Browser
- Run tests in a Docker container
- Run tests in parallel
Goal 3: Prepare Unit Tests for PHP7 compatibility
Goal 4: Create the REST api tests for https://github.com/joomla-projects/webservices/
Goal 5: Enhance the quality of our tests by doing a training with Davert, the creator of Codeception Testing Framework.
The final list of participants is: David Jardin, Javier Gomez, Viktor Vogel, Robert Deutz , Puneet Kala , Kshitij Sharma, Yves Hoppe, Niels Braczek, Daniel Dimitrov, Michael Bodnarchuk (virtually), Niels Nübel, Francesco Abeni
## Status of System Tests, and updates on the GSOC Project (Kshitij & Puneet)
https://github.com/joomla/joomla-cms/pull/6765 pending, but Kshitij will do a complete run to check the current status.
## Running system tests on Docker Containers (Tito)
Tito is working on running Docker in TravisCI and also in Jenkins. See plans at: https://docs.google.com/document/d/1-gB3036PvCi97-_im6Fn0InT0UAnXv8XTq4tz_83Z5A/edit#heading=h.a6yli2kxlfto
Around 60% of the points explained in the concept document are done, using a couple of base Docker images (server / client) and the first real parallel tests are being prepared.
Also TravisCI has been prepared so it can run our Docker environment.
Next step is putting some limitation to the parallel containers running, to avoid overloading the system, and also a new script for allowing parallel tests in the same environment is to be created (to speed up single environment tests).
Docker image will be published at: https://github.com/joomla-projects/docker-joomla-automated-testing
The tool continues evolving https://github.com/joomla-projects/joomla-browser/pulse/monthly. We will be celebrating the 8000 downloads in the next days: https://packagist.org/packages/joomla-projects/joomla-browser
## Codeception tests for Weblinks (Javier)
Daniel found issue installing joomla https://github.com/joomla-projects/joomla-browser/issues/70 it should be fixed during the code sprint
Travis is running PHP 5.3 instead of 5.5. Javier will fix it
More updates are expected during the code sprint
## Codeception tests for Webservices (Javier)
A proposal for testing the Webservices API was sent and is now ready to merge https://github.com/joomla-projects/webservices/pull/25 . Javier will speak with Chris Davenport, release leader of Joomla 3.6.
## Documentation (Javier, Kshitij and Valerie)
No updates. Work in progress: https://docs.joomla.org/Testing_Joomla_Extensions_with_Codeception.
## Easy tool to run the tests
Daniel confirms that the Robo script to run the tests is working on Windows. We will review during the Code Sprint
## Unit Tests status (Michael)
Michael: regarding “Goal 3: Prepare Unit Tests for PHP7 compatibility“ that’s been done for months, 3.5 branch and FW repos have had minimal or no PHP 7 issues for months, you don’t need to invest time in that task unless you intend on breaking some stuff along the way
## Tests in Joomla 4 (Niels)
regarding tests in Joomla 4: Currently, the test base didn't evolve very much. A lot of the existent unit tests will probably go, because the tested code gets removed. To make progress, J!4 urgently needs integration tests, e.g., for the complete routing from request to component call. These tests need to be built in and for J!3, so we can be sure to break as little as possible. This, btw, is something I'd like to achieve during the code sprint.
## QA Scripts: Code Style
Niels: I'm working on the coding standard to make it cover the current state of PHP, PSR-1, PSR-2, and PSR-5. My goal is to load the standard to CS2 with Composer, and provide the auto-fixes.
Robert Deutz: there is a related PR for this https://github.com/joomla/coding-standards/pull/109
Robert Deutz: that would mean we have to update the whole codebase, I would make two steps:
1) move to PHPCS2.0
2) include new language parts
Niels Braczek: Yes, makes sense. Thus, CS2 can be used for 3.x, while the extended rules could be J!4+ only.
## Next meeting will be in person during the Code sprint
29th of October