SLES/OpenSUSE - build 15.5 and then install/upgrade test to later #608
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add worker for openeuler 24.03. Rename container generation to mathc the same name.
As per https://github.com/MariaDB/server/blob/8478a06cdbb7281e1cc2d507f18c985c736c2ffc/cmake/cpack_rpm.cmake#L374 MariaDB-compat won't be built for OpenEuler.
Clients in our binary taballs are linked dynamically with libncurses, which causes a trouble on systems where it is no longer available As per MDBF-794 only centos7 and debian 10 bintar builders need to link statically with libncurses. - for debian 10: libncurses.a is already present in /scripts/local/lib/ and -DCURSES_NEED_NCURSES=1 flag is enough to link it statically. - for centos7: I've rebuilt the image and pushed manually a dev_ tag. libncurses.a was present on the system but not in the library path: /scripts/local/lib/ Tests available at: - debian10: https://buildbot.dev.mariadb.org/#/builders/187/builds/23 - centos7: https://buildbot.dev.mariadb.org/#/builders/172/builds/292 Reference CMAKE output line: -- Found Curses: /scripts/local/lib/libncurses.a
Adjust images to 1505 notation as well.
Fedora 41 is including MariaDB 10.11 so use that as the minimum version.
Centos based have asio-devel in EPEL. SLES is omitted as it requires a subpackage module repository to be enabled. Doing so would increase the package dependencies when using the built packages. https://packagehub.suse.com/packages/asio/ (SLE-Module-PackageHub-Subpackages-Module ) OpenSUSE also has asio-devel but adds a complicated path in with the openssl libraries and boost that we'd be influcting on users if they used a non-bundled version. On RHEL - asio-devel in ubi9 EPEL but not 8.
Parts 1 and 2: Switching minor upgrade into "all" mode and extending it with additional checks (ldd, requirements, capabilities)
Post-review fixes and cosmetic changes
During the "Compile step" - create_initial_db.cmake runs a bootstrap as a sanity check to avoid spending time on test runs. It can happen that some changes on the server may cause a loop during the bootstrap process. Buildbot will terminate the parent process (dojob) when it reaches a timeout, but the MariaDB service will remain up causing new builds to fail. https://buildbot.mariadb.org/#/builders/234/builds/32911 Modified the stop_processes step so handle this.
Use ( deb | archive ).mariadb.org to download the Packages file. This file is mandatory for both the 'All' and 'Columnstore' test types, exit with error if it is not found. Apply uniq to remove duplicate values in the package list and sort the list for readability.
… old bb - .build-id files may also be present on old systems where --noartifact does not work - package dependencies need to be sorted for comparison - dependency collection hangs frequently in buildbot, enabled logging temporarily to get more output for the issue
… old bb - .build-id files may also be present on old systems where --noartifact does not work - package dependencies need to be sorted for comparison - dependency collection hangs frequently in buildbot, enabled logging temporarily to get more output for the issue
Improving build time by excluding the too many files of mariadb-test package from the ldd check when dependencies are collected.
Align code with old-bb and peform an extra restart after the previous version is installed so plugins are loaded.
The build runs --big-test which puts a lot of memory pressure on the machine, so adjusting the parallel to mitigate this.
…PM repo on {archive/mirror}.mariadb.org
Exception: centos >= 9 which should be sorted out! (relative path on mirrors centos/$version/$platform)
Build_arch is used as $arch in bash_lib.sh, rpm_setup_mariadb_mirror{} to calculate mirror/archive URL.
{opensuse155-amd64,opensuse156-amd64} on https://mirror.mariadb.org/yum/$branch
Fixing minor upgrades in test_mode = 'ALL". See: https://buildbot.dev.mariadb.org/#/builders/326/builds/14/steps/3/logs/stdio
…it as REQUIRED hence minor upgrades (all) will reflect the change during verifications. > 10.5 MariaDB releases use liburing. Q3 release was linked against libaio which accounts for the differences found during testing.
- cache location is zypp - "all" does not exist for clearing cache, instead zypper uses --all - when MariaDB.repo is first added the key should be trusted in a non-interactive manner, hence --gpg-auto-import-keys - first zypper packages -r goes to /dev/null because there is a lot of noise from trusting the key. Sudo is needed to make the cache. - second zypper packages -r runs as a normal user with no issues.
- ID_LIKE may not exist - upgrade command does not exist for zypper - set delimiter for zypper packages -r output - suppress libmecab diffs to account for old/new bb env diffs - add krb5/judy/lzo to opensuse images to allow creating rpm packages for lzo, oqgraph, and gssap
:~$ docker run --rm -it -u root quay.io/mariadb-foundation/bb-worker:dev_opensuse1505 bash 0b0ed4c04573:/home/buildbot # rpm -qa | grep libedit libedit0-3.1.snap20150325-2.12.x86_64 libedit-devel-3.1.snap20150325-2.12.x86_64 :~$ docker run --rm -it -u root quay.io/mariadb-foundation/bb-worker:dev_opensuse1506 bash 588c54d665b5:/home/buildbot # rpm -qa | grep libedit libedit0-3.1.snap20150325-2.12.x86_64
Changes: - libvirt: - add minor | columnstore builder - suggest + tags for major/minor tests - suggest adding test_mode to builder name - bash library: - fix syntax for creating columnstore structures - move here: get_columnstore_logs() - deb upgrade: - the big comment block was deleted because dependency collection was handled in bash library - columnstore package list is now static. Mainly because it will pull CMAPI and was agreed not to install / upgrade it. - rpm upgrade: - columnstore test_mode option was missing - handled extra restart if test_mode is columnstore - remove chunks of commented code. Same reason as in deb upgrade. What will this patch do: - install the server and the CS plugin. Create structures (query) - upgrade both - verify structures (query) LDD and REQS comparison are OFF for CS.
This keep our packages list smaller, less building, and works for 15.5 and 15.6 Factor the generation of lists together as what should be identical implementation. There should be only one match in the list.
|
Looks ok, but I suggest pinging Elena and Daniel to raise awareness for the Q1 2025 release. |
| if "sles-1505" in builderName or "opensuse-1505" in builderName: | ||
| builders.append(b.replace("1505", "1506")) | ||
| break | ||
| return builders |
There was a problem hiding this comment.
Nice touch here, rather than returning an empty list if the parent builder doesn't have any install / upgrade builders defined.
RazvanLiviuVarzaru
force-pushed
the
dev
branch
from
d7eb898
to
4be5d7c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This keep our packages list smaller, less building, and works for 15.5 and 15.6
Factor the generation of lists together as what should be identical implementation. There should be only one match in the list.
Template selection
Please go the the
Previewtab and select the appropriate sub-template: