View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0009284 | Taler | exchange | public | 2024-10-21 14:30 | 2025-06-22 23:29 |
| Reporter | fefe | Assigned To | Christian Grothoff | ||
| Priority | none | Severity | trivial | Reproducibility | have not tried |
| Status | assigned | Resolution | open | ||
| Product Version | git (master) | ||||
| Target Version | post-1.0 | ||||
| Summary | 0009284: inefficient web serving in exchange mhd_spa | ||||
| Description | The function build_webui opens a file, reads a copy of it into malloced memory (without size restrictions), then calls zlib compress on the buffer. This appears to happen also for pre-compressed files like jpeg and png. This is very inefficient. Proposal 1: This is all for static files, so why not pre-compress them? Put foo.html.gz next to foo.html and if the browser supports gzip compression, just serve that file instead. No CPU cycles wasted on compression. Proposal 2: At least use mmap instead of malloc+read. Doing malloc+read puts unnecessary pressure on the virtual memory system for no gain. The only situation where malloc+read has an advantage is if you expect a filesystem race where someone truncates a file while you have it mmapped. This does not appear to be a realistic scenario here. Downside: you need to track how to free the buffer then and can't just call free on it. | ||||
| Tags | No tags attached. | ||||
|
|
mmap always uses whole pages. That's actually inefficient for small files. Here, we do often have rather small files. So if we do this it'd mostly make sense for the one large file. We could indeed have pre-compressed files on disk, but that'd put more requirements on the installation process. You'd either have to put both compressed and uncompressed into say the Debian package, or have the installer include extra steps to do the compression. A bigger benefit with compression-on-disk would be that we could actually use sendfile() and avoid mmap/malloc entirely. But, regardless, optimization issue of no practical relevance in the medium term IMO. |
|
|
https://packages.debian.org/bookworm/pigz can be used to generate zlib output (deflate). |
|
|
dev/grothoff/bug9284 contains a preliminary patch. Needs more testing, and Debian packages should be modified to install pre-compressed resource files for the SPAs. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2024-10-21 14:30 | fefe | New Issue | |
| 2024-10-21 14:30 | fefe | Status | new => assigned |
| 2024-10-21 14:30 | fefe | Assigned To | => Christian Grothoff |
| 2024-10-22 05:53 | Christian Grothoff | Note Added: 0023555 | |
| 2024-10-22 05:53 | Christian Grothoff | Priority | normal => none |
| 2024-10-22 05:53 | Christian Grothoff | Severity | minor => trivial |
| 2024-10-22 05:53 | Christian Grothoff | Product Version | => git (master) |
| 2024-10-22 05:53 | Christian Grothoff | Target Version | => post-1.0 |
| 2024-10-22 06:55 | Christian Grothoff | Assigned To | Christian Grothoff => |
| 2024-10-22 06:55 | Christian Grothoff | Status | assigned => confirmed |
| 2024-11-12 23:56 | Christian Grothoff | Target Version | post-1.0 => 1.5 |
| 2024-12-07 23:03 | Christian Grothoff | Target Version | 1.5 => 1.6 |
| 2025-04-17 22:49 | Christian Grothoff | Target Version | 1.6 => post-1.0 |
| 2025-06-22 00:21 | Christian Grothoff | Assigned To | => Christian Grothoff |
| 2025-06-22 00:21 | Christian Grothoff | Status | confirmed => assigned |
| 2025-06-22 00:42 | Christian Grothoff | Note Added: 0025299 | |
| 2025-06-22 23:29 | Christian Grothoff | Note Added: 0025311 |
