|
|
This sounds simple enough. I would like to better understand the server architecture. For example "weblate.taler.net" will point to what server? I think now might be a good time to create (or inform me about, if it already exists) a three-tiered system. The development system is on the developers personal computer, or possibly a VM (preferably one that lives on a metal server that we own and can create VMs on), the stage system is one I, or whomever you wish, could have sudo/root access to and would do things like this task on. Then the third layer would be live and you or Florian would test my deployment on stage, read the docs I wrote on how I built the server, and then deploy on some live server. The stage server should always be an exact match for live. I don't want to make this simple task more complicated than it needs to be, but having this sort of infrastructure in place will make it easier to "systemitize" deployments of all kinds in the future. Anwyay, something to think about. But the shorter version of everything I just said is: Is there a testing server we could do this on first, using "test.weblate.taler.net" as the target URL? |
|
|
weblate.taler.net will be a CNAME to gv.taler.net, like most subdoamins of taler.net. gv.taler.net has a virtual hosting setup with a reverse proxy pointing to the (loopback) port with the weblate service. Given that today we only have _one_ (physical) server for taler.net, there is no proper 'stage' in your sense. stage.taler.net (for the Web-site staging) is merely the exact same setup targeting a different Git branch. I agree that systematic staging on an 'equivalent/equal' server setup might be nice, not sure when we can get there. For the Weblate, I really don't see much of a risk though: running another httpd on a loopback port under some $USER is hardly going to endanger the rest of the system ;-). |
|
|
> I agree that systematic staging on an 'equivalent/equal' server setup might be nice, not sure when we can get there I know to an extent I am just experiencing the difference between what I am used to and a new system, and soon I will be used to taler's procedures. But it's difficult for me to feel comfortable installing major subsystems, that require trial-and-error, on an existing live server. Maybe a cheap VPS? Since it would only have public/generic info, there is no security risk. We could even make access to it Tor-only so it is not available on the public internet. I am just brainstorming, and it is not directly related to this specific task. Anyway I can develop in a local VM first, so that is something. Specific to this task: exactly what is the the goal? You said "Weblate is a page to allow translators to contribute translations. We should setup weblate.taler.net for this...." Okay, but what exactly will be translated? What do you expect to *see* when this task is complete? |
|
|
You're overcomplicating things. There also is no risk or need for a VPS. You'll get an SSH account (and local Postgres access) and can configure Weblate to bind against some (free) TCP port on loopback. You write a systemd service file with how to launch the service under that $USER. That's it. Result: Weblate instance running on weblate.taler.net, where we then can import po files, add users, manage translations, etc. A few of us setup as 'administrators' on the sub-domain. This should be fast and easy, maybe a 2h task. |
|
|
DNS/TLS and SSH access to weblate@taler.net for buck have been setup. However, I still need to run the 'apt install' steps from the Weblate instructions before Buck can take over here. |
|
|
> You're overcomplicating things. Probably. That is common in the absence of details, or the institutional knowledge of how you like to handle these things. > You'll get an SSH account (and local Postgres access) / SSH access to weblate@taler.net for buck have been setup Aha! That's good useful information that was missing from the spec. Now I see why you are not worried about the risk to the broader system. Hopefully you understand that without knowing about this sandbox (weblate user), I was right to worry. But these are the parameters to proceed under. Great! I love definition. Please provide the Postgres details when you have them. (https://docs.weblate.org/en/latest/admin/install.html#database-setup) > and can configure Weblate to bind against some (free) TCP port on loopback Aha! That is more specific-to-this-system, very useful, but very unguessable information that was not in the spec. Should be no problem! > You write a systemd service file with how to launch the service under that $USER. That's it. Aha! Do I need to say the next part? :) > There also is no risk or need for a VPS. Well there is always a need to be able to test a system before running live, and I can't do that if I don't have control over the web server, DB server, firewall, etc (unless it all installs with no debugging necessary). So I'll build this out in a VM first. No problem! > maybe a 2h task. If we had a test server, less. But since I will be creating a test VM from scratch, probably slightly more. Questions: - How do you feel about using virtualenv as recommended in the weblate docs? - Do you want me to run celery? https://docs.weblate.org/en/latest/admin/install.html#celery - Do you expect to need any of the optional dependencies? https://docs.weblate.org/en/latest/admin/install.html#optional-deps - Are there any other assumptions you are making that are not explicitly stated? Any standard procedures in how you handle this server that are not universals? (In this case, I doubt it as everything stated so far paints the picture of how the system will run, which was a blank slate previously.) - What are you expecting to *see* and *do* with weblate when it's installed? If it's sandboxed under the weblate user, it won't talk to other applications on the server, right? |
|
|
I've now setup a database 'weblate' which the 'weblate' user owns. So you can create/drop/insert/delete/update/etc. on that database as needed. No credentials, use the UNIX domain socket to connect to Postgres and your UID is used to check permissions. For example: $ psql weblate psql> |
|
|
As for 'testing', I see no reason why you can't test it already as prescribed: you bind against a free port on loopback. That's only accessible to those few with an account ON the system. You can use SSH port forwarding to forward the loopback port to your client system and run a browser against it. So bound-to-loopback *is* your firewall. As the weblate database is used for nothing else, feel free to use it for experiments. virtualenv: sure, why not? Celery: it reads like it is needed. We could use a systemd user service for this. Optional dependencies: none are needed. Other assumptions: hah, no idea ;-). See/do: Taler core team (incl. you, but not only you) will import our .po and .xml files from the various Taler components (software, website), and then allow volunteers from the community (or professional translators, if needed and affordable) to sign up and contribute translations. Taler core team (incl. maybe you) will then export the translations and use them when packaging releases of the software or deploy to the Web site. See, for example, https://taler.net/de/. |
|
|
Weblate is tested working on a VM. Install took about 30 minutes once the VM was ready. Ten more for systemd. REQUESTS: 1. > However, I still need to run the 'apt install' steps from the Weblate instructions before Buck can take over here. Please do this when it is convenient, then please post back when you're done and I'll install weblate in a screen session to test. 2. (for later) Then there will be an intermediary step when I ask you to import two systemd files (celery-weblate and weblate.) Celery intructions for root are here: https://docs.weblate.org/en/latest/admin/install.html#background-tasks-using-celery Sections: Running Celery as system service Environment configuration to be placed as `/etc/default/celery-weblate`: systemd file for weblate: --/etc/systemd/system/weblate.service-- [Unit] Description=Weblate Server After=multi-user.target celery-weblate.service [Service] Type=simple Restart=always RestartSec=1 User=user ExecStart=/home/weblate/weblate-env/bin/python3 /home/weblate/weblate-env/bin/weblate runserver [Install] WantedBy=multi-user.target ---- BUT please do not enable the systemd services before I install celery and weblate, which I will do when you take care of the pre-reqs (1). 3. Please verify that I can run this on port 8000. As `weblate` user I can not (be sure if I) see all used ports. (Note It is non-trivial to use a port other than 8000.) MISC: > As for 'testing', I see no reason why you can't test it already as prescribed: you bind against a free port on loopback. Well that's fine, right? Different sysadmins do things differently, may or may not see the other's reasons, and anyway what is "prescribed" to me is of course limited to *what* the task is, not *how* I prefer to accomplish it. It so happens that I will not deploy something on a new, live server until I have tested the something first. You know this server intimately, and are willing to take more chances. I am being cautious, and it's my call to make when I am responsible for the task. But I will try to answer: Binding against a free port on loopback is (i) rather difficult when one is not root since only root knows what ports are free (see above) (ii) actually *doing* the thing I want to test before doing it, so it's beside the point (which was running a VM to test before going to the live server.) > You can use SSH port forwarding to forward the loopback port to your client system and run a browser against it. Sure I could but doing so would not be "testing" since production will not run this way. It's a good debug step if testing fails, but so far that didn't happen. (But FYI I am adept with SSH tunnels for the many times they will be useful on this or any other projects.) > See/do: Okay from your answer it sounds like this will be a follow-up task so I will wait for that. Next: Weblate UI has a password. How do you want me to get this password to you? |
|
|
Debian packages for weblate are installing as I write this. Port 8000 is free, I'm not sure why it is non-trivial to use another port, I'd have thought this to be a simple configuration option in any sane software package. But, whatever. 8000 is free. Also, any (non-root) user can see which ports are in use: the command is "netstat -ntl". (-n no DNS, 't' for TCP, '-l' for listen). I was proposing SSH only for your testing. Anyway, I've now configured nginx, so weblate.taler.net points to port 8000 now. Note that we may need to revise the Content Security Policy before it works properly with this setup (so for testing, you may still want to use SSH). |
|
|
> Debian packages for weblate are installing as I write this. Great! But it looks like I need root's help with psql. `weblate migrate` fails with "django.db.utils.ProgrammingError: permission denied to create extension "pg_trgm" " So can you please install pw_trgm? When I tried manually: ``` weblate@gv:~$ psql -U weblate psql (12.2 (Debian 12.2-4), server 11.6 (Debian 11.6-2~sid1)) Type "help" for help. weblate=> \c weblate psql (12.2 (Debian 12.2-4), server 11.6 (Debian 11.6-2~sid1)) You are now connected to database "weblate" as user "weblate". weblate=> CREATE EXTENSION pg_trgm; ERROR: permission denied to create extension "pg_trgm" HINT: Must be superuser to create this extension. weblate=> ``` > I'd have thought this to be a simple configuration option in any sane software package. Me too but it does not appear so unless I am missing something, which is always possible. > Also, any (non-root) user can see which ports are in use: the command is "netstat -ntl". (-n no DNS, 't' for TCP, '-l' for listen). Well sure, you as root know that a non-root user can see all the used ports because you can compare them. But a non-root user does not know what he does not know. And it is directly applicable. For example, running the command you suggested shows: `Active Internet connections (only servers)` So I must ask myself: what about things that aren't servers, like (in theory) a root-owned node app on port 8000? But running the command I use (netstat -utlpn) is more explicit: ``` (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (only servers) ``` So I needed to verify with you as it would be completely reckless for me to run something on a port like 8000, on a live server, when there is even a *small* chance of conflict. > I was proposing SSH only for your testing. Right. I confused things by using "testing" to mean two different things: 1 - Testing (staging) a new subsystem (weblate) before deploying it on a live server. 2 - Testing (demoing/debugging) the actual deployment (for which SSH will be great, good idea) |
|
|
Ah! From the install page (https://docs.weblate.org/en/latest/admin/install/venv-debian.html): If you don’t want to make Weblate user a superuser in PostgreSQL, you can omit that. In that case you will have to perform some of the migration steps manually as a PostgreSQL superuser: CREATE EXTENSION IF NOT EXISTS pg_trgm; |
|
|
CREATE EXTENSION pg_trgm; executed. |
|
|
Great! Weblate is up. Currently running in a `screen` session under `weblate` user, port :8000, verified with "ssh -N -L 8000:localhost:8000 weblate@taler.net". Alternatively you could install w3m if you only want to test locally (then "w3m http://127.0.0.1:8000" should work.) Celery is not running but that depends on the celery-weblate service, so please start this when you are ready to go live. The weblate systemd service also can be started at any time. I don't know how you want me to send you the admin password but I have it available when you're ready. weblateWorking.png (9,843 bytes) |
|
|
Admin PW: just GnuPG e-mail it to me and Florian. |
|
|
Done. Closing issue. Please reopen if there are any problems with the systemd service, etc. |
|
|
systemd services delivered weblate running on port 8000 (currently/temporarily in screen session) |
|
|
Is celery also running? Note that you can reach the site under https://weblate.taler.net/ already. |
|
|
I've tried to invite myself as a "normal" user, but did not get the invitation e-mail. Similarly, I created weblate@taler.net and tried to add that address for the admin account, but that didn't work either. => I suspect somehow the setup is unable to sent emails? All of the (i) links to further documentation go to 404 pages (seems, eh, unusual). Also, /manage/performance/ lists a bunch of errors/warnings. |
|
|
So point (7) of this issue is still wide open. |
|
|
> Is celery also running? Not on my end. It will be handled by the celery-weblate systemd service once you set that up and create /etc/default/celery-weblate:. Here is the link again: https://docs.weblate.org/en/latest/admin/install.html#background-tasks-using-celery > I've tried to invite myself as a "normal" user, but did not get the invitation e-mail. > => I suspect somehow the setup is unable to sent emails? Right. I don't recall seeing e-mail setup in the specs(?) and I assume you did not expect me to do this as you did not provide the requisite information I would need (meaning, the e-mail server settings to use.) Here is the doc: https://docs.weblate.org/en/latest/admin/install.html#configuring-outgoing-e-mail If you want to provide the values for those variables, I can add them to the settings.py file. But that wouldn't make sense because I do not have access to the e-mail server/accounts, so I could not test/debug and we'd end up playing a silly game of telephone (ex. "I just sent a test e-mail, did it come through?" "no, try again" etc.) This is more appropriately a root task because root controls the e-mail, no? But if you want to provide the settings, I will add them. > All of the (i) links to further documentation go to 404 pages (seems, eh, unusual). It does seem unusual. But without specifics (ex - which link(s)) I can't say anything specific about why you might be experiencing that. The first thing I noticed is that the (i) links I found go to URLs on docs.weblate.org, so those errors are unrelated to this install. Example: http://127.0.0.1:8000/accounts/profile/#notifications (to the right of Notification settings) https://docs.weblate.org/en/weblate-4.0.1/user/profile.html#subscriptions If the URLs you have trouble with are not off-site, the next thing I would do is test using an SSH tunnel and 127.0.0.1:8000 vs weblate.taler.net. If it fails only on weblat.taler.net I would look at the nginx logs. I can do this comparison if you want to provide a specific bug report, but of course if the problem is only on weblate.taler.net I won't be able to debug the nginx config. This is another example of why it is not a great idea to have non-priv users working on systems like this, and why stage servers are so valuable. > So point (7) of this issue is still wide open. Of course it is. That reads: "7) test, integrate, import existing po files, document, etc." That line was not usable on its own (ex: test *what*?, integrate *what* with *what?, import *which* po files, etc.) so I asked some of those questions. And you clarified that the deliverable was "Result: Weblate instance running on weblate.taler.net, where we then can import po files, add users, manage translations, etc. A few of us setup as 'administrators' on the sub-domain. This should be fast and easy, maybe a 2h task." You have that, although it will require root-level info for setting up e-mail for certain components to work correctly. I can set that up once I have more info. And as far as "A few of us setup as 'administrators' " goes, well obviously that is not actionable in the absence of exactly who to set up as administrators. When you have that info, please feel free to send it and (assuming e-mail is done) I can add some administrators if you'd like. And of course there may be the 404 bug to deal with, and I am happy to do that when I have more info. |
|
|
Resolving this issue as replaced by these tasks: https://bugs.gnunet.org/view.php?id=6201 (Christian) https://bugs.gnunet.org/view.php?id=6202 (buck) https://bugs.gnunet.org/view.php?id=6203 (Christian) https://bugs.gnunet.org/view.php?id=6204 (buck) https://bugs.gnunet.org/view.php?id=6205 (buck) |
- Anonymous
