View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0008877 | Taler | deployment and operations | public | 2024-05-28 14:19 | 2025-05-10 02:11 |
| Reporter | dvn | Assigned To | dvn | Resolution deadline | |
| Implementer | Reviewer | ||||
| Release signed by | Deployed by | ||||
| Verifier | Communicator | ||||
| Data protection authority notification required | No | ||||
| CSIRT notification required | No | ||||
| Estimated effort (hours) | Actual effort (hours) | 0 | |||
| Fast-track | No | ||||
| Work package | none | ||||
| Priority | normal | Severity | feature | Reproducibility | have not tried |
| Status | closed | Resolution | fixed | ||
| Product Version | git (master) | ||||
| Target Version | 1.0 | Fixed in Version | 1.0 | ||
| Summary | 0008877: Make CI containers use ramdisk/tmpfs [estimate: 10h] | ||||
| Description | In order to lessen I/O pressure we want the CI containers to use ramdisks instead of writing to the HDD. | ||||
| Tags | No tags attached. | ||||
| Javier recommended passing these arguments to podman: `-d --tmpfs /tmp:rw,size=787448k,mode=1777` | |
| The most simple and stable way to approach this is to mount a tmpfs into a path inside the container. Podman supports this natively (see previous comment). This will require some reworking of the way we are using and managing our workspaces inside the container. Buildbot needs to be able to setup the workspace based on the git checkout associated with a given build. Currently we mount this directory from the host into the container under /workdir. This allows buildbot on the host to manage the workspace. However, if we use a ramdisk for the workspace we will need to either copy over the mounted directory or fetch it from within the container. | |
| Starting with podman 4.4 there is an option "--transient-storage". This makes all the container metadata state which incurs lots of file locking happen under /run/ which is a tmpfs. I've updated podman on firefly to the version provided by trixie/testing, and made the buildbot container-worker invoke the transient storage option. It's been over 24 hours now, and all of the alerts about high disk read rate and high context switching have not fired again. So I'm closing this as I think the main issue is resolved between the transient metadata storage and the ramdisk for the workspace. Though there are still some disk operations that can happen in containers such as when new packages are installed, for example. | |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2024-05-28 14:19 | dvn | New Issue | |
| 2024-05-28 14:19 | dvn | Status | new => assigned |
| 2024-05-28 14:19 | dvn | Assigned To | => javier.sepulveda |
| 2024-05-28 14:19 | dvn | Note Added: 0022482 | |
| 2024-05-28 15:56 | dvn | Note Added: 0022484 | |
| 2024-05-29 08:56 | javier.sepulveda | Status | assigned => acknowledged |
| 2024-05-29 08:56 | javier.sepulveda | Assigned To | javier.sepulveda => dvn |
| 2024-06-24 00:49 | dvn | Summary | Make CI containers use ramdisk/tmpfs => Make CI containers use ramdisk/tmpfs [estimate: 5h] |
| 2024-07-28 19:49 | Christian Grothoff | Severity | minor => feature |
| 2024-07-28 20:13 | Christian Grothoff | Target Version | => post-1.0 |
| 2025-03-03 12:55 | dvn | Summary | Make CI containers use ramdisk/tmpfs [estimate: 5h] => Make CI containers use ramdisk/tmpfs [estimate: 10h] |
| 2025-03-11 14:41 | dvn | Status | acknowledged => informing |
| 2025-03-11 14:41 | dvn | Resolution | open => fixed |
| 2025-03-11 14:41 | dvn | Note Added: 0024177 | |
| 2025-03-21 12:02 | Christian Grothoff | Product Version | => git (master) |
| 2025-03-21 12:02 | Christian Grothoff | Fixed in Version | => 1.0 |
| 2025-03-21 12:02 | Christian Grothoff | Target Version | post-1.0 => 1.0 |
| 2025-05-10 02:11 | Christian Grothoff | Status | informing => closed |