View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0009284 | Taler | exchange | public | 2024-10-21 12:30 | 2025-08-31 17:09 |
| Reporter | fefe | Assigned To | Christian Grothoff | Resolution deadline | |
| Implementer | Reviewer | ||||
| Release signed by | Deployed by | ||||
| Verifier | Communicator | ||||
| Data protection authority notification required | No | ||||
| CSIRT notification required | No | ||||
| Estimated effort (hours) | Actual effort (hours) | 0 | |||
| Fast-track | No | ||||
| Work package | none | ||||
| Priority | none | Severity | trivial | Reproducibility | have not tried |
| Status | closed | Resolution | open | ||
| Product Version | git (master) | ||||
| Target Version | 1.0 stretch goals | Fixed in Version | 1.0 stretch goals | ||
| Summary | 0009284: inefficient web serving in exchange mhd_spa | ||||
| Description | The function build_webui opens a file, reads a copy of it into malloced memory (without size restrictions), then calls zlib compress on the buffer. This appears to happen also for pre-compressed files like jpeg and png. This is very inefficient. Proposal 1: This is all for static files, so why not pre-compress them? Put foo.html.gz next to foo.html and if the browser supports gzip compression, just serve that file instead. No CPU cycles wasted on compression. Proposal 2: At least use mmap instead of malloc+read. Doing malloc+read puts unnecessary pressure on the virtual memory system for no gain. The only situation where malloc+read has an advantage is if you expect a filesystem race where someone truncates a file while you have it mmapped. This does not appear to be a realistic scenario here. Downside: you need to track how to free the buffer then and can't just call free on it. | ||||
| Tags | No tags attached. | ||||
| mmap always uses whole pages. That's actually inefficient for small files. Here, we do often have rather small files. So if we do this it'd mostly make sense for the one large file. We could indeed have pre-compressed files on disk, but that'd put more requirements on the installation process. You'd either have to put both compressed and uncompressed into say the Debian package, or have the installer include extra steps to do the compression. A bigger benefit with compression-on-disk would be that we could actually use sendfile() and avoid mmap/malloc entirely. But, regardless, optimization issue of no practical relevance in the medium term IMO. | |
| https://packages.debian.org/bookworm/pigz can be used to generate zlib output (deflate). | |
| dev/grothoff/bug9284 contains a preliminary patch. Needs more testing, and Debian packages should be modified to install pre-compressed resource files for the SPAs. | |
| Testing done, looks fine, added postinst steps to challenger, exchange (incl. auditor) and merchant to compress SPA resources after unpacking (so the Deb packages don't increase in size from the compressed versions). | |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2024-10-21 12:30 | fefe | New Issue | |
| 2024-10-21 12:30 | fefe | Status | new => assigned |
| 2024-10-21 12:30 | fefe | Assigned To | => Christian Grothoff |
| 2024-10-22 03:53 | Christian Grothoff | Note Added: 0023555 | |
| 2024-10-22 03:53 | Christian Grothoff | Priority | normal => none |
| 2024-10-22 03:53 | Christian Grothoff | Severity | minor => trivial |
| 2024-10-22 03:53 | Christian Grothoff | Product Version | => git (master) |
| 2024-10-22 03:53 | Christian Grothoff | Target Version | => post-1.0 |
| 2024-10-22 04:55 | Christian Grothoff | Assigned To | Christian Grothoff => |
| 2024-10-22 04:55 | Christian Grothoff | Status | assigned => implementing |
| 2024-11-12 22:56 | Christian Grothoff | Target Version | post-1.0 => 1.4 |
| 2024-12-07 22:03 | Christian Grothoff | Target Version | 1.4 => 1.3 |
| 2025-04-17 20:49 | Christian Grothoff | Target Version | 1.3 => post-1.0 |
| 2025-06-21 22:21 | Christian Grothoff | Assigned To | => Christian Grothoff |
| 2025-06-21 22:21 | Christian Grothoff | Status | implementing => assigned |
| 2025-06-21 22:42 | Christian Grothoff | Note Added: 0025299 | |
| 2025-06-22 21:29 | Christian Grothoff | Note Added: 0025311 | |
| 2025-07-03 09:26 | Christian Grothoff | Note Added: 0025391 | |
| 2025-07-03 09:26 | Christian Grothoff | Status | assigned => informing |
| 2025-07-03 09:26 | Christian Grothoff | Fixed in Version | => 1.0 stretch goals |
| 2025-07-03 09:26 | Christian Grothoff | Target Version | post-1.0 => 1.0 stretch goals |
| 2025-08-31 17:09 | Christian Grothoff | Status | informing => closed |