Report of the Automated Testing Code Sprint 2015
This report contains a list of tasks done by the individuals that participated in the Automated Testing Code Sprint that took place in the city of Karlsruhe (Germany) from October 29th until November 1st. More details will be published at the Joomla Magazine on Decembers edition.
Michael Bodnarchuck (Davert)
Michael did a training for the participants about general concepts in testing and the usage of Codeception testing framework. Slides are available at:
- Refactored the tests in weblinks to standardise them
- Created the test scenarios in text format
- Investigated about the usage of the Recorder Codeception extension to document the tests https://github.com/joomla-extensions/weblinks/issues/159 (see video)
- Fix bug in Travis CI builds using PHP 5.3 instead of the selected PHP version in .travis.yml (thanks tito). https://github.com/joomla-extensions/weblinks/pull/115
- worked on a error reporter when an error is identified in the tests
- generate the task list
- tested PRs and and merges
- Improved RoboFile with a local configuration file and some additional behaviours: https://github.com/joomla-extensions/weblinks/pull/104 , https://github.com/joomla-extensions/weblinks/pull/107 , https://github.com/joomla-extensions/weblinks/pull/124
- Fixed a bug in weblinks component, discovered thanks to the automated tests: https://github.com/joomla-extensions/weblinks/pull/153
- Improved RoboFile with environments and named based options: https://github.com/joomla-extensions/weblinks/pull/158
- Add two new tests for hits counter: https://github.com/joomla-extensions/weblinks/pull/127
- Coordinate the task list
- Checked and tested several PRs
- Tried unsuccessfully to convert the team to the use of the timebox technique :-)
- Ate too much :-)
Currently, Joomla is missing integration (functional) tests. This lack is an obstacle for the development of Joomla!4, because there’s no feedback on, where backward compatibility is touched. To resolve that, and because the Automated Testing Working Group has decided to use CodeCeption for system tests, I moved the current unit tests to CodeCeption (in joomla-pythagoras repository) as a preparation. The test results seem not to be identical, so this has to be investigated and can and will be fixed later. However, CodeCeption seems to be the way to go, especially, since support for behavioral tests using Gherkin in CodeCeption are on the horizon.
The JBuild Robo.li tool by Yves has been moved (forked) to joomla-projects/jorobo, where it will be extended to be a full fledged build and test tool. My pull request PR#4 contains the quality assurance part of that tool. It generates a lot of metrics reports.
Most of them are well-known already, but phpmetrics.html may be new to many. It is an HTML report with nice diagrams, showing many aspects of the code quality:
jorobo will be available and recommended for general Joomla! extension development, when it is released.
- Installed Drone.io on one of our server and configured my weblinks fork to run tests on drone
- Find out that drone is not as stable as we need it for testing Joomla and Weblinks
- We faced a situation that looks at the first try that codeception wasn’t working as expected. Checked the situation we with on the issue reduced code and finally found that codeception is doing well.
- Helped to create the jorobo package
- Fix current issue when creating a category in weblinks
- Helped Yves to create the jorobo package ( https://github.com/joomla-projects/jorobo )
- Test PRs in github
- Worked on creating a release script for github
- Helped Robert setting up drone.io
- Worked on a automated Release tool
- Worked on integrating the new robo.li based build system JoRobo (based on jBuild)
- Improvements in Joomla-Browser and Weblinks
- Tested PRs
- Helping Robert setting up drone.io
- Integrated new folder structure for Weblinks
- Helping on running tests on all platforms
- Refactoring and improving RoboFile
- 1st level technical pc support ;)
- Worked on improving the descriptions for Windows machines
- Rewrote the readme file with exact instructions for Windows users
- Made sure that the build command to generate AcceptanceTester on Windows machines are running properly
- Tested and merged PR’s in the Weblinks repository https://github.com/joomla-extensions/weblinks/commits/master?author=Kubik-Rubik
- Tested and merged PR’s by Niels in the jorobo repository https://github.com/joomla-projects/jorobo/commits/master?author=Kubik-Rubik
- Created the masterpiece of “Thug Life Javier” - https://pbs.twimg.com/media/CSqdeioWIAEHz4H.jpg - https://twitter.com/viktorvogel/status/660781722099974144
- Posted pictures and updates on social channels
- Provided food and shelter for all participants
- Refactored codeception tests in com_weblinks
- Added several new tests like deleting, moving to Trash, validating tabs on edit view.
- Updated Joomla! Browser to add more generic functions like setFilter, clickToolBarButton etc.
- Updated the Robo File in com_weblinks to add functionality to run individual method from a particular test.
- Finalized structure of tests with codeception.
- Modified function in Joomla! browser to work with different screen resolution, so tests could be executed on Mobile, Desktop, Tabs.
- Created function to kill selenium: https://github.com/joomla-extensions/weblinks/pull/116
- Created archive category tests in AdministratorCategoryCest.php https://github.com/joomla-extensions/weblinks/pull/105
- I worked on creating the tests for Multilingual Joomla
- Created tests to check the language translation on the front end
- Created category with different languages
- Built function to make an association between the categories
- Worked on writing integration tests of menu items, Weblink Category and Weblinks
- Create a menu item: Displays a list of Web Links for a category
- Submit a Web Link
- Worked on writing functions to delete menuItems, create weblinks with user specified category
- I wrote function in the step files in category.php to publish and unpublish a category
in weblink.php to delete a weblink
- I wrote test to check publish and unpublish feature of category.
- publish a category in backend and then check its existence in front-end
- unpublishing a category in backend and then verifying that it is not present in the front-end.
- com_category Error while using modal for making multilingual associations between the categories https://github.com/joomla/joomla-cms/issues/8231
Association tab not working in banners https://github.com/joomla/joomla-cms/issues/8231
as Victor and I were the only Windows users at the event we spent a lot of our time making sure that the tests can run on Windows Operating System. We've updated the documentation and fixed several bugs that were preventing the tests to work as easily on Windows as they do on other OS systems.
- Inspected the output of the Selenium Server and tried to find out why sometimes exceptions are thrown
worked on improving the chosen select function tests
- looked at the possibilities of using the Selenium IDE together with the codeception formatter for writing tests faster https://docs.joomla.org/Testing_Joomla_Extensions_with_Codeception#Thoughts_on_Selenium_IDE_and_the_codeception_Formatter