View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0006628 | GNUnet | build process | public | 2020-10-28 17:57 | 2023-06-02 19:16 |
| Reporter | juga | Assigned To | schanzen | ||
| Priority | normal | Severity | feature | Reproducibility | always |
| Status | closed | Resolution | suspended | ||
| Product Version | Git master | ||||
| Summary | 0006628: Build GNUnet docker image | ||||
| Description | For the pEp mixnet project we are GNUnet docker images to run our tests. I've thought it might be useful to create the docker images directly upstream, ie. to have the Docker file in "official" GNUnet repo. There is no need to publish docker images, or images could be published in a way GNUnet decides. I use a lot Gitlab CI because it not only allows you to create the docker images, but also publish them in the "container registry". Though they are not secure for a production system, they're very useful for running tests in them in the same Gitlab CI. I found there's a GNUnet clone in gitlab.com that does not allow merge requests, but i could base my fork on it. So i created the branch call `docker` in my fork https://gitlab.com/juga0/gnunet/-/tree/docker, where i add the Dockerfile, plus a docker-compose.yml to test the image as a service an create a "friend", plus the .gitlab-ci.yml file that build the image, publish it and run it. Even thought GNUnet canonical repo is not in Gitlab CI, it might be useful for some other people using docker and/or Gitlab instances. Feel free to comment on things to change, merge the branch, or don't :) | ||||
| Steps To Reproduce | git clone https://gitlab.com/juga0/gnunet/ git checkout docker docker-compose up | ||||
| Tags | No tags attached. | ||||
|
|
Ah, you can see the Gitlab CI pipeline here: https://gitlab.com/juga0/gnunet/-/pipelines/205327100 |
|
|
Hi, we manage https://gitlab.com/gnunet primarily as a mirror. The source repository does not have issues enabled so that bugs are reported here. It also prohibits merge requests because in order to contribute, you need to sign a copyright assignment: https://gnunet.org/en/copyright.html Also, we do not really want to monitor the gitlab mirror as the primary reposity is on gnunet.org. Also, the gitlab project does already include a docker build: https://gitlab.com/gnunet/docker-build It also pushes into the gitlab registry: https://gitlab.com/gnunet/docker-build/container_registry We trigger the build from the gnunet.org repository in a hook. |
|
|
Thanks for your answer. > > we manage https://gitlab.com/gnunet primarily as a mirror. > The source repository does not have issues enabled so that bugs are reported here. So i did :) > It also prohibits merge requests because in order to contribute, you need to sign a copyright assignment: https://gnunet.org/en/copyright.html It says "Minor contributions (basically, anyone without Git access) do not require copyright assignment". I think this is a minor contribution, i can sign it anyway. > Also, we do not really want to monitor the gitlab mirror as the primary reposity is on gnunet.org. I understand that, you could still have an automatic post-hook in git.gnunet.org or `pushurl` in your local repo. > > Also, the gitlab project does already include a docker build: https://gitlab.com/gnunet/docker-build I saw that, but here a few differences: - it's based on alpine, not debian - docker-build doesn't build using multi-stages so that it run faster and different images can be used for testing (with sources) or production (without) - docker-build doesn't give an option on which UID would run the process - docker-build doesn't give an option on which target path to install it - docker-build doesn't show gnunet logs - docker-bukld doesn't have an example docker-compose file and a test that actually test the images and the compose. - docker-build does have a .gitlab-ci.yml file that build the image in gitlab.com but it currently fails (https://gitlab.com/gnunet/docker-build/-/jobs/256745166) Those are reasons why i thought could be helpful to create one and include it upstream in gnunet repo. > It also pushes into the gitlab registry: https://gitlab.com/gnunet/docker-build/container_registry > We trigger the build from the gnunet.org repository in a hook. If you prefer to have the docker related files in a difference repository (though i think it's better all sources for the some software are in the same repo), i could also adapt my branch to be merged in the docker-build repo instead of the gnunet repo and i would live an option to build the alpine or debian image. Thanks for your time! |
|
|
The problem ist not the docker-related files. Those we could put into contrib/ The issue is the .gitlab-ci file. Which is completely foreign to our CI (buildbot.gnunet.org) So I think we shoul modify build-docker to also build from a debial base image and push it using a different tag. |
|
|
> The problem ist not the docker-related files. Those we could put into contrib/ I've only realized with this sentence that there's actually already docker files in contrib/docker/, but it actually fails to build with: /bin/sh: 1: ./bootstrap: not found The command '/bin/sh -c cd /gnunet && ./bootstrap && ./configure --with-nssdir=/lib --prefix="$GNUNET_PREFIX" --enable-logging=verbose && make -j3 && make install && ldconfig && cd - && rm -fr /gnunet' returned a non-zero code: 127 I'd need to look what to change there and change my branch according to that. > The issue is the .gitlab-ci file. Which is completely foreign to our CI (buildbot.gnunet.org) I see other gnunet repos have .gitlab-ci.yml, like https://git.gnunet.org/gnunet-python.git/tree/. The file doesn't hurt in and that would make the gitlab runner in https://gitlab.com/gnunet/gnunet to automatically run, build the image, store it in https://gitlab.com/gnunet/gnunet/container_registry, and test it with docker compose, which i can't do with buildbot. The official docker image could be hosted in other place. Which is actually the place where it is?, i don't see a docker image in https://buildbot.gnunet.org. > So I think we shoul modify build-docker to also build from a debial base image and push it using a different tag. If the docker image can be built directly from gnunet repo with the docker files in contrib, i think that that the repo https://gitlab.com/gnunet/docker-build README should be updated pointing to the gnunet repo and maybe archive it, so that other people do not get confused about where are the docker files used to build the gnunet docker image. |
|
|
There is no "official" gitlab.com repository. We will also not likely move to it, see https://www.gnu.org/software/repo-criteria-evaluation.html So for now, the gitlab.com mirror is a semi-official place to build docker images and have a docker registry. I have no idea why the gnunet-python repo contains a .gitlab-ci. It should not. Putting a .gitlab-ci in the repo is confusing because it is unclear which gitlab instance is used because we use our own dedicated source code repository. Will the CI be run on our own instance? gitlab.com? Which variables can/should be used? The advantages for the developer of using gitlab fade when this is unclear and ambiguous. |
|
|
> There is no "official" gitlab.com repository. We will also not likely move to it, see https://www.gnu.org/software/repo-criteria-evaluation.html > Sorry i might not have explain myself well, i'm not saying to move gnunet repo officially to gitlab.com > So for now, the gitlab.com mirror is a semi-official place to build docker images and have a docker registry. This is what i meant. But there're not docker images in the gitlab.com container registry: https://gitlab.com/gnunet/gnunet/container_registry. The way i put docker images in gitlab.com is using .gitlab-ci.yml to build them and test them automatically, but maybe there's another different way to do that. > I have no idea why the gnunet-python repo contains a .gitlab-ci. It should not. > Putting a .gitlab-ci in the repo is confusing because it is unclear which gitlab instance is used because we use our own dedicated source code repository. That can be specified in the summary of the repo in gitlab.com, main README or the docker README. Actually i modified the last one in my branch https://gitlab.com/juga0/gnunet/-/commit/32bb7c687659d515145e778dcfde9c62cb770220. > Will the CI be run on our own instance? gitlab.com? Which variables can/should be used? That should be up to you and documented somewhere. > The advantages for the developer of using gitlab fade when this is unclear and ambiguous. I'm with you, it was already confusing for me to find https://gitlab.com/gnunet/gnunet/-/tree/cadet_option/contrib/docker, https://gitlab.com/gnunet/docker-build/ and https://hub.docker.com/search?q=gnunet&type=image. But i believe that can be fixed with documentation. In any case, i've modified my docker branch in gitlab.com: https://gitlab.com/juga0/gnunet/-/tree/docker. Now all the docker files are inside contrib/docker and there's no .gitlab-ci.yml file. I was talking with dvn and t3sseract and maybe they'd like to have a look to the branch, dunno how to put them in copy here. |
|
|
Bump. Can we close this? I do not really see an action item at this time? |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2020-10-28 17:57 | juga | New Issue | |
| 2020-10-28 17:59 | juga | Note Added: 0017046 | |
| 2020-10-28 19:18 | schanzen | Note Added: 0017047 | |
| 2020-10-29 11:15 | schanzen | Assigned To | => schanzen |
| 2020-10-29 11:15 | schanzen | Status | new => feedback |
| 2020-10-31 16:29 | juga | Note Added: 0017074 | |
| 2020-10-31 16:29 | juga | Status | feedback => assigned |
| 2020-10-31 20:14 | schanzen | Note Added: 0017078 | |
| 2020-11-03 21:14 | juga | Note Added: 0017094 | |
| 2020-11-11 10:57 | schanzen | Note Added: 0017120 | |
| 2020-11-11 10:58 | schanzen | Note Edited: 0017120 | |
| 2020-11-11 15:10 | juga | Note Added: 0017122 | |
| 2022-09-28 15:42 | schanzen | Note Added: 0019195 | |
| 2022-10-19 09:25 | schanzen | Status | assigned => resolved |
| 2022-10-19 09:25 | schanzen | Resolution | open => suspended |
| 2023-06-02 19:16 | schanzen | Status | resolved => closed |
