View Issue Details

IDProjectCategoryView StatusLast Update
0006223Talerdeployment and operationspublic2020-07-24 11:56
ReporterChristian Grothoff Assigned ToChristian Grothoff  
PrioritynormalSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Platformi7OSDebian GNU/LinuxOS Versionsqueeze
Product Versiongit (master) 
Target Version0.8Fixed in Version0.8 
Summary0006223: inform us about sphinx warnings
DescriptionThere is an existing buildbot job that runs 'sphinx' on the documentation (docs.git) to generate the documentation page. The underlying CI job should be enhanced to inform the last committer if Sphinx generates warnings (i.e. about syntax errors or bad links).
TagsNo tags attached.

Activities

buckE

2020-05-11 09:05

reporter   ~0015878

> There is an existing buildbot job that runs 'sphinx' on the documentation

Where *exactly* can I find this "buildbot job?"

Christian Grothoff

2020-05-11 09:52

manager   ~0015882

deployment.git/buildbot/master.cfg, the "DOC_FACTORY" (and related: the doc-worker, DOC_BUILDER and doc-builder, and DOC_SCHEDULER).

buckE

2020-06-10 07:24

reporter   ~0016247

Ah, I'm starting to see this. Honestly it was not well written, and the above info was not the answer to the question I asked. But from what I've learned about buildbot and using the above as indirect hints, I think the task is actually something like this:

```
Buildbot calls deployment/buildbot/build-docs.sh on the docs.git repo's file to build (make) the documentation for sphinx. When it runs, it should e-mail the most recent committer to Git in the event of an error. Task: investigate sphinx and buildbot to determine appropriate BASH scripting to do this. Deliverable: check in enhanced build-docs.sh.
```

Does that sound accurate?

I could set up a local demo sphinx environment so I can learn how to grab any errors that sphinx generates (so I can send those in the e-mail). And I need to learn how buildbot stores the most recent commit info so I can reference that in the bash script (unless you know, and can share the info). Once I understand those things, the actual script should not be difficult but that learning curve is not trivial.

Alternative: what user is this job (and therefore build-docs.sh) run under? And can I have permission to that account? Because if I can, then perhaps I could manually run the build-docs.sh file as that user, inputting bad docs, and view the errors on the console. Does that make sense to you as a way to test *breaking* a sphinx build, so that I can work with the errors? (I'll probably also just do the development of the script in that user's home directory in that case. Much faster than building a test environment, if you think it safe enough.)

Christian Grothoff

2020-06-10 11:28

manager   ~0016251

Last edited: 2020-06-10 11:29

View 2 revisions

Yes, what you write would be accurate, modulo that it is not just in the event of an error (hard failure), but also in the event of a warning.

The account in question is docbuilder@taler.net, you can now SSH into that (for testing, as you proposed).

buckE

2020-06-16 06:27

reporter   ~0016273

Okay I have this on my to-do list after the test project and linkchecker, which I am working on concurrently.

buckE

2020-06-18 07:30

reporter   ~0016296

Starting on this next, as linkchecker is in beta release.

buckE

2020-06-18 08:13

reporter   ~0016297

STATUS:

 - reporter created in master.cfg; e-mail verified working (sending to me)
- Makefile options convert warnings into errors but also sphinx will `--keep-going` with build on warnings. (ie - Buildbot shows failure on sphinx warnings, but sphinx continues, and writes to /home/docbuilder/warnings.log)

QUESTION #1:

Same as with the linkchecker, how do you want to provide the "warnings.log" file to interested parties?

(Maybe there is a way for buildbot to send arbitrary logs, but I don't think so. I think only bb logs. Text reads:

addLogs
(boolean). If True, include all build logs as attachments to the messages. These can be quite large. This can also be set to a list of log names, to send a subset of the logs. Defaults to False.

)

QUESTION:

Current reporter sends to me (for testing) and also " sendToInterestedUsers=True." The manual says "Interested Users are authors of changes and users from the owners build property." Is this acceptable? When I check in change now, all "InterestedUsers" should get the e-mail. Let's see who complains.

Christian Grothoff

2020-06-18 14:36

manager   ~0016300

Q #1: I would usually want to see them in the build logs of the build job on the buildbot web site *AND* in the body of the notification e-mail.
Q #2: You're unlikely to immediately get complaints, but it would be good if we could limit notifications to the last committer.

buckE

2020-06-19 08:24

reporter   ~0016314

> Q #1: I would usually want to see them in the build logs of the build job on the buildbot web site *AND* in the body of the notification e-mail.

Done, using your suggestion from the linkchecker. Simple, elegant. Log arrives as attachment to e-mail and is also available in BB UI.

> Q #2: You're unlikely to immediately get complaints, but it would be good if we could limit notifications to the last committer.

At this moment the function sends to recent committers and also "owners" but I'm not sure where "owners" is defined. We can overwrite the function, but maybe there are other uses of this function that require the owners to be included. It's a little foolish that there is no "sendToLastCommitter" but there isn't. Anyway by including "owners" we will see who is an owner, and if they should be, and maybe learn what "owners" means. Maybe you will remember where it is defined? (I didn't search hard but it's not in master.cfg.)

Below is the overwrite reference, so I can start from there if you choose to re-open.

https://docs.buildbot.net/latest/manual/configuration/reporters.html?highlight=addlogs#mailnotifier-arguments
sendToInterestedUsers
(boolean). If True (the default), send mail to all of the Interested Users. Interested Users are authors of changes and users from the owners build property. Override MailNotifier getResponsibleUsersForBuild method to change that. If False, only send mail to the extraRecipients list.

Issue History

Date Modified Username Field Change
2020-04-26 01:17 Christian Grothoff New Issue
2020-04-26 01:17 Christian Grothoff Status new => assigned
2020-04-26 01:17 Christian Grothoff Assigned To => buckE
2020-05-11 09:05 buckE Note Added: 0015878
2020-05-11 09:52 Christian Grothoff Note Added: 0015882
2020-06-10 07:24 buckE Note Added: 0016247
2020-06-10 11:28 Christian Grothoff Note Added: 0016251
2020-06-10 11:29 Christian Grothoff Note Edited: 0016251 View Revisions
2020-06-16 06:27 buckE Note Added: 0016273
2020-06-18 07:30 buckE Note Added: 0016296
2020-06-18 08:13 buckE Note Added: 0016297
2020-06-18 14:36 Christian Grothoff Note Added: 0016300
2020-06-19 08:24 buckE Assigned To buckE => Christian Grothoff
2020-06-19 08:24 buckE Status assigned => resolved
2020-06-19 08:24 buckE Resolution open => fixed
2020-06-19 08:24 buckE Note Added: 0016314
2020-07-24 11:56 Christian Grothoff Target Version => 0.8
2020-07-24 11:56 Christian Grothoff Fixed in Version => 0.8