feat(build-deb): Upload source package and run lintian#22

Closed

EduardGomezEscandell wants to merge 3 commits intomainfrom

build-deb-lintian

Contributor

EduardGomezEscandell commented Feb 6, 2024 •

edited

Loading

Being tested in canonical/ubuntu-pro-for-wsl#546

UDENG-2235

EduardGomezEscandell self-assigned this

EduardGomezEscandell force-pushed the build-deb-lintian branch 2 times, most recently from 781d029 to 4156a27 Compare

February 6, 2024 12:51

Contributor Author

EduardGomezEscandell commented Feb 7, 2024

Example failure: https://github.com/canonical/ubuntu-pro-for-wsl/actions/runs/7817501950/job/21325536015?pr=546

EduardGomezEscandell force-pushed the build-deb-lintian branch from ab58f16 to 7b57d13 Compare

February 9, 2024 15:51

EduardGomezEscandell added 3 commits

February 9, 2024 16:58


          Add source package to artefact

c5b8666


          Add lintian check to source package

918af21


          Make debian version compliant

9eb66d6

Fix warnings related to:
- Version number
- File too long
- No contributor name
- Extra whitespace around name

EduardGomezEscandell force-pushed the build-deb-lintian branch from 7b57d13 to 9eb66d6 Compare

February 9, 2024 15:59

3v1n0 reviewed

View reviewed changes

Contributor

3v1n0 left a comment

Thanks for starting this here, I had few other ideas that I was drafting in canonical/authd#130, but this seems a good progress.

gh-actions/common/build-debian/action.yml

+                      # Sanitize the docker name so that it stick to debian policy
+                      # https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
+                      sanitized_docker=$( echo "${{ inputs.docker-image }}" | sed -r 's/[^a-zA-Z0-9.+~]+/+/g' )

Contributor

3v1n0 Feb 23, 2024

Ideally I'd use lsb_release -r -s, but it has to run inside the target docker so that we have an incremental number (while docker names can be static or even cyclic).

gh-actions/common/build-debian/action.yml

+                      sanitized_docker=$( echo "${{ inputs.docker-image }}" | sed -r 's/[^a-zA-Z0-9.+~]+/+/g' )
+                      # Short commit to avoid "package-has-long-file-name"
+                      commit=$(echo ${{ github.sha }} | cut -c1-8)

Contributor

3v1n0 Feb 23, 2024

Why don't we use git ourselves here?

So we can ahve something fancier by using echo git+$(git rev-list --count HEAD).$(git rev-parse --short HEAD) (or similar), and we have also an incremental number together with the SHA.

gh-actions/common/build-debian/action.yml

Comment on lines +68 to +69

		export DEBFULLNAME="GitHub actions runner"
		export DEBEMAIL="noreply@github.com"

Contributor

3v1n0 Feb 23, 2024

Or maybe latest commit author?

Suggested change

      
                    export DEBFULLNAME="GitHub actions runner"
          
                    export DEBEMAIL="noreply@github.com"
          
                    export DEBFULLNAME="$(git log -1 --format='%an' HEAD) - GH Action"
          
                    export DEBEMAIL="$(git log -1 --format='%ae' HEAD)"

Plus an indication that is GH though, to make clear both things.

gh-actions/common/build-debian/action.yml

+                      export DEBFULLNAME="GitHub actions runner"
+                      export DEBEMAIL="noreply@github.com"
+                      debchange --local "~${sanitized_docker}+${commit}" "Github build. Run id: ${{ github.run_id }}. Run number: ${{ github.run_number }}."

Contributor

3v1n0 Feb 23, 2024

Suggested change

      
                    debchange --local "~${sanitized_docker}+${commit}" "Github build. Run id: ${{ github.run_id }}. Run number: ${{ github.run_number }}."
          
                    debchange --local "~${sanitized_docker}+${commit}" \
          
                      "Github build. Run id: ${{ github.run_id }}. Run number: ${{ github.run_number }}."

gh-actions/common/build-debian/action.yml

+                lintian:
+                  required: false
+                  description: Arguments to pass to lintian, if any. Set to false to skip the lintian chek.
+                  default: ''

Contributor

3v1n0 Feb 23, 2024 •

edited

Loading

This should be true IMHO (to make it clearer at the reader as I see that by default it runs).

Edit: I saw why it's unset, but as explained later I think we're maybe giving too much control to action users :)

gh-actions/common/build-debian/action.yml

+                    if: inputs.lintian != 'false'
+                    shell: bash
+                    working-directory: ${{ env.SOURCE_OUTPUT_DIR }}
+                    run: lintian ${{ inputs.lintian }} -- *.changes

Contributor

3v1n0 Feb 23, 2024

Suggested change

      
                  run: lintian ${{ inputs.lintian }} -- *.changes
          
                  run: lintian --pedantic --fail-on error ${{ inputs.lintian }} -- *.changes

So we still see what's not exactly clean. Plus, IMHO, it would be nice to have an option to control --fail-on, since ideally good packages should set it to warning

Contributor

3v1n0 Feb 23, 2024

Oh, I sorry ignore me on the --fail-on thingy... It's indeed possible to override the value.

Although, maybe this option is even too powerful, since IMHO any suppression should be in sources.

3v1n0 reviewed

View reviewed changes

gh-actions/common/build-debian/action.yml

Comment on lines +109 to +113

+                  - name: Copy source package to output dir
+                    shell: bash
+                    run: |
+                      set -eu
+                      cp -r ${{ env.SOURCE_OUTPUT_DIR }}/* ${{ env.BUILD_OUTPUT_DIR }}

Contributor

3v1n0 Feb 25, 2024

IMHO, we should do this phase just after building the source, and ensure that this happens also if binary build fails: This can help a lot to debug situations failure situations, because one can download the sources locally and build them with sbuild (or similar tools) to fix the inconsistencies.

Also in commit message, we mostly spell it as "art_i_fact" even if both words seems to be correct.

3v1n0 reviewed

View reviewed changes

gh-actions/common/build-debian/action.yml

+                    run: |
+                      set -eu
+                      cp -r ${{ env.SOURCE_OUTPUT_DIR }}/* ${{ env.BUILD_OUTPUT_DIR }}
+                  - name: Upload artefacts

Contributor

3v1n0 Feb 25, 2024

-> artifacts

3v1n0 reviewed

View reviewed changes

gh-actions/common/build-debian/action.yml

                         if [ -n "${GITHUB_TOKEN}" ]; then
                           git config --system url."https://api:${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/"
                         fi
+                  - name: Run lintian

Contributor

3v1n0 Feb 25, 2024

Ideally this should run in the same distro in which we build the package... So I'm wondering if we should fork the jtdor/build-deb-action@v1 action to support post-build-hooks or just a run-lintian option.

3v1n0 mentioned this pull request

feat(build-debian): Use docker to manually run build and prepare steps and add tests #28

Merged

Contributor Author

EduardGomezEscandell commented Feb 26, 2024

@3v1n0 I'm a bit busy at the moment, feel free to take over this PR if you can and want. I think I'll be able to work on it in a couple of weeks otherwise.

Contributor

3v1n0 commented Feb 26, 2024

@3v1n0 I'm a bit busy at the moment, feel free to take over this PR if you can and want. I think I'll be able to work on it in a couple of weeks otherwise.

Yeah, nw... I've took something of this in #28 but we can revisit it. Also I didn't add the lintian thing for now as that can probably be split as another job so that could potentially be triggered in parallel with autopkgtests for example

Contributor Author

EduardGomezEscandell commented Mar 13, 2024

Closing as I will not have time to work on this. I'll keep the branch alive in case someone wants to continue it.

EduardGomezEscandell closed this

3v1n0 added a commit that referenced this pull request


          feat(build-debian): Use docker to manually run build and prepare step…

6e15fc5

…s and add tests (#28)

As underlined in
canonical/authd#130 (comment) using
`jtdor/build-deb-action` for building debian packages has some problems
that are yet not clear, and it also does not seem to properly support
building non-native packages.

As per this, given that the code needed for handling the build inside a
simpler docker container is quite small, I've moved the build to manual
handling which allows us some more control over the steps at the cost of
being a bit more verbose.

As bonus:
- We now only build as user (not the sources, since there's no much
point)
- All the network traffic is totally blocked for the builder user (not
just HTTPS)
- ~~`DEB_BUILD_OPTIONS` can be customized for special builds (e.g.
`nocheck`)~~
 - Version uses actual distro version

I've added some commits from #22, but not included the lintian build yet
(that I've ready
[here](3v1n0@1b72205))
not to make this PR harder to check, but also because I think those
could instead be part of another action to allow more parallelization
when used.

By doing this change, not only both test cases pass, but also authd is
correctly built: canonical/authd#130 (e.g.
https://github.com/ubuntu/authd/actions/runs/8044158864)

UDENG-2439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet