Docker build error and how to use host container

[yushoyamaguchi]

Thank you for developing.
I'm using this repository in research activity from Japan.

In my environment in ubuntu22, when executing

sudo docker build -t plugsched_host .

This error is occurrd.
Should I edit Dockerfile?

Errors during downloading metadata for repository 'epel':
  - Curl error (6): Couldn't resolve host name for http://mirrors.cloud.aliyuncs.com/epel/8/Everything/x86_64/repodata/repomd.xml [Could not resolve host: mirrors.cloud.aliyuncs.com]
Error: Failed to download metadata for repo 'epel': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Second question, Can I use the container by this Dockerfile as host of plugsched?
I imagine that it is difficult to use kernel module from a container.

[dtcccc]

For the first question, I'm sorry that we found "mirrors.cloud.aliyuncs.com" is an internal address that only ECS from aliyun can access. You may edit Dockerfile to change the address to "mirrors.aliyun.com" in /etc/yum.repos.d/epel.repo after installing epel-aliyuncs-release. Or directly use our image(plugsched-registry.cn-hangzhou.cr.aliyuncs.com/plugsched/plugsched-sdk).

For the second question, are you trying to install the plugsched rpm on ubuntu? We have not supported deb, but may try to provide you a simulated way to install plugsched on host(not container).

[dtcccc]

First, create a working dir:
/var/plugsched/$(uname -r)

Then, move necessary files into it:
install -m 755 working/symbol_resolve/symbol_resolve /var/plugsched/$(uname -r)/symbol_resolve
install -m 755 kernel/sched/mod/scheduler.ko /var/plugsched/$(uname -r)/scheduler.ko
install -m 444 working/tainted_functions /var/plugsched/$(uname -r)/tainted_functions
install -m 755 working/scheduler-installer /var/plugsched/$(uname -r)/scheduler-installer
install -m 755 working/hotfix_conflict_check /var/plugsched/$(uname -r)/hotfix_conflict_check

Last, run the installer script:
/var/plugsched/$(uname -r)/scheduler-installer install

I've not tested with this, so feel free to report any problem you meet.

[yushoyamaguchi]

Thank you for your replying.
That means the purpose of conatainer is setting up necessary files (and copying to the host), right?

If I use centOS as the host, can I run new scheduler actually in this way?

In addition, I have one more question.
We can implement new scheduler by editing files under kernel/sched/mod/ after executing bounday analyzer(plugsched-cli init ?), right?
How to build this new scheduler as kernel module?
Source files under src/ (like main.c , sched_rebuild.c) are related?

[dtcccc]

Thank you for your replying. That means the purpose of conatainer is setting up necessary files (and copying to the host), right?

Yes

If I use centOS as the host, can I run new scheduler actually in this way?

Yes. After copying the rpm from container to host, you can use "rpm -i" to install the new scheduler directly on host.

In addition, I have one more question. We can implement new scheduler by editing files under kernel/sched/mod/ after executing bounday analyzer(plugsched-cli init ?), right? How to build this new scheduler as kernel module? Source files under src/ (like main.c/sched_rebuild.c) are related?

main.c and sched_rebuild.c under src/ will be copied to kernel/sched/mod/ automatically. They are related.

You can refer to cmd_build() in cli.py to see how it work in detail. The key file is kernel/sched/mod/Makefile (which is copied from src/Makefile), and the kernel module will be built as kernel/sched/mod/scheduler.ko

Directly insmod scheduler.ko may fail because we still need some other works. See scheduler-installer.

[yushoyamaguchi]

Thank you very much.

[yushoyamaguchi]

When executing
plugsched-cli init $(uname -r) ./kernel ./scheduler
in the podman container, this error was generated.

I use a docker image build by myself using Dockerfile.
Also, I use /work as working dir instead of /tmp/work because of tmpfs capacity.

If you have some ideas about this error, please tell me.

[dtcccc]

I'm sorry that we found our anolisos:latest has updated gcc minor version recently, so gcc-python-plugin needs rebuild. We will fix this problem soon.

To workaround, please downgrade the gcc version (using yum install):
gcc-8.5.0-10.1.0.3.an8
gcc-c++-8.5.0-10.1.0.3.an8
gcc-plugin-devel-8.5.0-10.1.0.3.an8
libstdc++-static-8.5.0-10.1.0.3.an8
gcc-python-plugin-0.17-1.4.an8

What's more, we found there may be something wrong with "pip3 install pyyaml".

We suggest just replacing:
RUN yum install epel-aliyuncs-release -y &&
with:
RUN yum install epel-release -y &&
in Dockerfile. This will use the source from mirrors.fedoraproject.org directly, and then "yum install python3-pyyaml".

[yushoyamaguchi]

As I modified two points(gcc version and epel-release), which you teach, gcc-python-plugin error is fixed. Thank you.
(Modified Dockerfile is this)
However, another error was generated.

ImportError: cannot import name 'CLoader'
Is this the pyyaml error that you told?

What's more, we found there may be something wrong with "pip3 install pyyaml".

What should I do?

[dtcccc]

Do not use
pip3 install pyyaml

Use
yum install python3-pyyaml
instead

[yushoyamaguchi]

I've overlooked.
Thank you.

[yushoyamaguchi]

In my environment, executing plugsched-cli init $(uname -r) ./kernel ./scheduler in a container has not been finished in 3 hours. (Current my environment is not good.)
Is this so time-consuming process, right?
For example, which is more time-consuming than compiling the Linux kernel?

[dtcccc]

Is this so time-consuming process, right?

Yes, it is time-consuming, but...
Hmm... It should be faster than compiling the whole Linux kernel, I think.

Is your terminal printing info continuously? Or is stuck at one step?

[dtcccc]

We usually work on servers with 100+ cpus, so in our environment "init" will only cost several minutes.

After "init", the "build" step is very fast.

We suggest making the "init" task to background. When the task is done, maybe you can make "./scheduler" folder a backup if you want to develop different branches.

[yushoyamaguchi]

Before Pressing Ctrl-C, CC/LD/AR is printed continuously.

By the way, I use kernel 6.46.
However, content of boundary.yaml is same as it of kernel 5.10, because I didn't find how to modify.
Is this related to unfinished process?

[yushoyamaguchi]

For parallel processing, do we need some options?