* Pass in a YAML true value as `__bootc_validation: true` using
the --extra-vars option to ensure that `__bootc_validation` is
treated as a boolean and not a string value.
`-e "__bootc_validation: true"`
You can also use JSON format:
`-e '{"__bootc_validation": true}'`
but YAML is simpler in this case.
* Use tox-lsr version 3.11.1
* Ensure the citest bad comment works when the test was cancelled in
addition to the failure case.
* Update contributing.md documentation
* Update number of nodes to use in testing farm, if needed
* remove unnecessary ansible-lint skips
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
NOTE: This also requires upgrading to tox-lsr 3.11.0
Ansible 2.19 will be released soon and has some changes which will
require fixes in system roles. This adds 2.19 to our testing matrix
on fedora 42 so that we can start addressing these issues.
python 3.13 is now being used on some platforms.
Using ansible-core 2.18 requires using py311 for pylint and other
python checkers.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
NOTE: This also requires upgrading to tox-lsr 3.11.0
Ansible 2.19 will be released soon and has some changes which will
require fixes in system roles. This adds 2.19 to our testing matrix
on fedora 42 so that we can start addressing these issues.
python 3.13 is now being used on some platforms.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
NOTE: This also requires upgrading to tox-lsr 3.10.0, and some
hacks to workaround a podman issue in ubuntu.
These tests run the role during a bootc container image build, deploy
the container into a QEMU VM, boot that, and validate the expected
configuration there. They run in two different tox environments, and
thus have to be run in two steps (preparation in buildah, validation in
QEMU). The preparation is expected to output a qcow2 image in
`tests/tmp/TESTNAME/qcow2/disk.qcow2`, i.e. the output structure of
<https://github.com/osbuild/bootc-image-builder>.
There are two possibilities:
* Have separate bootc end-to-end tests. These are tagged with
`tests::bootc-e2` and are skipped in the normal qemu-* scenarios.
They run as part of the container-* ones.
* Modify an existing test: These need to build a qcow2 image exactly
*once* (via calling `bootc-buildah-qcow.sh`) and skip setup/cleanup
and role invocations in validation mode, i.e. when
`__bootc_validation` is true.
In the container scenario, run the QEMU validation as a separate step in
the workflow.
See https://issues.redhat.com/browse/RHEL-88396
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
Add Fedora 42 to testing farm test matrix, drop Fedora 40
Use tox-lsr 3.9.0 for the `--lsr-report-errors-url` argument.
Add the argument `--lsr-report-errors-url DEFAULT` to the qemu test so that
the errors will be written to the output log. This uses the output callback
https://github.com/linux-system-roles/auto-maintenance/blob/main/callback_plugins/lsr_report_errors.py
Use the check_logs.py script
https://github.com/linux-system-roles/auto-maintenance/blob/main/check_logs.py
with the `--github-action-format` argument to format the errors
in a github action friendly manner.
Rename the log files `-FAIL.log` or `-SUCCESS.log` depending on status.
This is compatible with the way the testing farm log files are named, and
makes it easy to tell if a test passed or failed from the log file name.
Upload README.html as artifacts of the build_docs job for debugging
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
This will make the qemu/kvm tests be tested in either
ascending or descending ASCII order. This should give
us better test coverage of clean up scenarios which may
fail depending on the order of the previous tests.
Rename the qemu/kvm tests so that the statuses are shorter
and more intuitive.
Improve qemu/kvm test failure error reporting.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
tox-lsr 3.6.0 will guarantee order of qemu test execution, which should
help make tests reproducible and help debug test failures.
Improve qemu test logging - this will help debug the qemu test
failures.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
The latest version of virtualenv does not support creating
python 2.7 virtualenvs. Change our CI tests to restrict the version
of virtualenv<20.22.0 and tox<4.15 for py27 environments
Move pylint, flake8, and black checks to the py310 environment
which is currently supported by ansible-core 2.17 and its related
checkers such as ansible-lint and ansible-test
pylint now uses ansible-core 2.17 and restricts the version of
pylint to 3.1.0 which is the version used by ansible-test 2.17
Remove `extends: default` for .yamllint.yml. The latest version
of ansible-lint will automatically incorporate local yamllint
settings unless there is an `extends:`.
The above changes require some fixes to the role code.
For more information, see
https://github.com/linux-system-roles/tox-lsr/pull/168
and
https://github.com/linux-system-roles/tox-lsr/pull/170
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
Upgrade ci tests to use tox-lsr 3.3.0
tox-lsr 3.3.0 uses ansible-test 2.17
Create the ansible-test ignore file for 2.17
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
This is fixed by tox-lsr 3.2.2 - all actions that use tox-lsr are updated to
3.2.2, not just the python unit tests, even though the fix is only related to
pytest. All roles are updated to use tox-lsr 3.2.2 for the sake of consistency
even if not affected by the pytest issue.
Something changed recently in the way github actions provisions systems which
means some of the directories are not readable by the python unit test actions.
In addition, the python unit tests were causing a lot of unnecessary directory
traversal doing collection/discovery of unit test files, because of using
`pytest -c /path/to/tox-lsr/pytest.ini` Unfortunately, with `pytest`, the
directory of the config file is the root directory for the tests and tests
discovery, and there is no way around this.
Therefore, the only solution is to copy the tox-lsr `pytest.ini` to the
`tests/unit` directory, which makes that the test root directory.
See also https://github.com/linux-system-roles/tox-lsr/pull/160
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
The old ansible-community ansible-lint is deprecated. There is a
new ansible-lint github action.
The latest Ansible repo gating tests run ansible-lint against
the collection format instead of against individual roles.
We have to convert the role to collection format before running
ansible-test.
This also requires tox-lsr 3.2.1 - bump other actions to use 3.2.1
Role developers can run this locally using
`tox -e collection,ansible-lint-collection`
See https://github.com/linux-system-roles/tox-lsr/pull/125
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
This is primarily for the update to ansible-plugin-scan to
work with the upcoming ostree changes, but also includes
some minor fixes which affect ci.
3.1.0 was released but not used due to a bug fixed in 3.1.1
See full release notes for 3.1.0 and 3.1.1
https://github.com/linux-system-roles/tox-lsr/releases
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
We use the setup-python github action. This has dropped support for
python 2.7. Instead, just manually install the python2.7 package.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
tox-lsr 3.x does not support python 2.6, so use tox-lsr 2.13 for
py26 testing. This should be fine until we can drop support for
python 2.6
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
The major version bump is because tox-lsr 3 drops support
for tox version 2. If you are using tox 2 you will need to
upgrade to tox 3 or 4.
tox-lsr 3.0.0 adds support for tox 4, commitlint, and ansible-lint-collection
See https://github.com/linux-system-roles/tox-lsr/releases/tag/3.0.0
for full release notes
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
When adding an approved PR to the merge queue, the required status
checks of the workflows are missing, which prevents the PR getting
merged. This commit is intended to trigger merge group checks with
Github Actions, so that the workflows will report the needed status
checks.
Signed-off-by: Wen Liang <liangwen12year@gmail.com>
Add a github action task to upload the coverage files
to codecov after the unit tests are complete.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
There are now separate github actions for the various tests, instead of all of
them being done as part of the tox tests in tox.yml - ansible-lint,
ansible-test, etc.
Use the officially supported github actions e.g. for ansible-lint, ansible-test,
rather than using our own from `tox-lsr` and trying to keep up with the latest
changes. Developers will still be able to use `tox-lsr` on their local
development environments to run these tests in the same way that they are run in
github actions, so that errors found when submitting PRs can be reproduced and
corrected locally without too many github UI roundtrips.
Using separate github actions, and especially the official github actions which
generally have support for in-line comments, should help greatly with
readability and troubleshooting test results.
Python tests are removed from roles that do not use python.
Python tests are now done by python-unit-tests.yml which also does the black,
flake8, and pylint tests.
Signed-off-by: Rich Megginson <rmeggins@redhat.com>
