Skip to content

add a CI system#10

Open
samdolt wants to merge 4 commits intodl9pf:masterfrom
samdolt:chore/github-ci
Open

add a CI system#10
samdolt wants to merge 4 commits intodl9pf:masterfrom
samdolt:chore/github-ci

Conversation

@samdolt
Copy link
Collaborator

@samdolt samdolt commented Sep 3, 2021

No description provided.

@samdolt
Copy link
Collaborator Author

samdolt commented Sep 3, 2021

You can see this github action file running here: samdolt#1

@bachp
Copy link
Collaborator

bachp commented Sep 3, 2021

@samdolt I seems the timeout from github actions is not sufficient to run the build

@samdolt
Copy link
Collaborator Author

samdolt commented Sep 6, 2021

@samdolt I seems the timeout from github actions is not sufficient to run the build

@bachp : Indeed, the maximal timeout is only 360 minutes.

We have another problem with Github Action, it's that the cache is limited to 5GB. It will be enough for the "download" directory but not fore the SSTATES folder.

But maybe we can set up something, store the SSTATE on S3 and then using that over http.

@samdolt samdolt force-pushed the chore/github-ci branch 26 times, most recently from c70c358 to 4293955 Compare September 8, 2021 07:18
@samdolt samdolt force-pushed the chore/github-ci branch 3 times, most recently from 2409b77 to ba13326 Compare September 8, 2021 07:31
@samdolt
Copy link
Collaborator Author

samdolt commented Sep 8, 2021

@bachp

So I implemted a workarround for the 360 minutes problems, by pre-generating the SSTATE on a local machine. Then I published the SSTATE online. In the CI, if the sstate folder is empty, it will automatically download the pregenerated SSTATE archive. With this scheme, the CI can build meta-codechecker in about 10 minutes.

See https://github.com/samdolt/meta-codechecker/actions/runs/1212406448

The generated report is then available as artifacts.

@samdolt samdolt marked this pull request as draft September 20, 2021 08:57
@samdolt samdolt force-pushed the chore/github-ci branch 4 times, most recently from b9e82d6 to 56b8c72 Compare September 20, 2021 11:16
@samdolt samdolt marked this pull request as ready for review September 20, 2021 15:34
@samdolt
Copy link
Collaborator Author

samdolt commented Sep 20, 2021

I added Dunfell and Gatesgarth to the CI and changed to use S3 to prefetch a locally generated SSTATE archive (needed because of Github Action limited capabilities).

@bachp @dl9pf : Can this be merged? At least it will prevent me to break Dunfell again ;-)

@bachp
Copy link
Collaborator

bachp commented Sep 20, 2021

@samdolt I don't see how we could manage this sstate in the long run. So I rather not merge this as is.

I think you already tried http://sstate.yoctoproject.org/, what was the issue with that? Clang is not part of it?

@samdolt
Copy link
Collaborator Author

samdolt commented Sep 21, 2021

@bachp : I tried the official sstate with the version of ubuntu provided by Github Action (without using docker) but it hasn't work. I think somehow the hash don't match the one from the sstate server.

I have looked at why, but I think it will be either:

  • because somehow the distribution used change the hash, and Github Action doesn't use a "vanilla" version of ubuntu
  • because adding meta-clang change the hash for all package

@bachp
Copy link
Collaborator

bachp commented Sep 21, 2021

@samdolt I looked at what others are doing:

  • meta-rust: they seem to have their own runners with sstate being stored on it
  • meta-iot2050 use normal runners but this probably works because it's not yocto but isar which is based on binary packages.

@samdolt
Copy link
Collaborator Author

samdolt commented Sep 21, 2021

@bachp Having our own runner seems like the best ways to do it. We could then also persist the download and sstate dir on the machine. Maybe some low cost virtual machine like the one provided by Scaleway

@bachp bachp force-pushed the chore/github-ci branch from 56b8c72 to 53cea81 Compare May 5, 2022 11:56
@bachp bachp force-pushed the chore/github-ci branch from 53cea81 to 5428fe8 Compare May 5, 2022 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants