Added a new step to the CI executed upon merge request, we now will run pa11yci against a set of URLs. We are doing this by running the application within a docker container and then using docker-compose to network this and the pa11y_ci container together.
I removed the previous Dockerfiles and I simplified to a single Dockerfile to run the web server and then a separate Dockerfile-ci to run the pa11y_ci script. The set of URLs is contained in pa11y.config.json. Some of the URLs I initially tried caused the chrome browser in puppeteer to hang so those where removed. I will open a ticket to investigate why but my initial investigation was not successful.
A more complete test would need to actually use pa11y directly and probably creation of an integration test script in order to do things like test the result of actions like clicking a button.