View Issue Details

IDProjectCategoryView StatusLast Update
0008186TalerWeb site(s)public2024-03-07 19:52
Reporterdvn Assigned Todvn Resolution deadline
ImplementerReviewer 
Release signed byDeployed by 
VerifierCommunicator 
Data protection authority notification requiredNo  
CSIRT notification requiredNo  
Estimated effort (hours)Actual effort (hours)0 
Fast-trackNo
Work packagenone
PrioritynormalSeveritytweakReproducibilityalways
Status closedResolutionfixed 
Fixed in Version0.9.4 
Summary0008186: linkchecker failing on demo shop links
DescriptionLinkchecker always fails on demo.taler.net because it checks /orders/ links. These links need to be ignored.
TagsNo tags attached.

Required notifications

Address or linkRemarkAdded byNotifiedActions

Activities

Stefan

2024-01-22 13:59

reporter   ~0020954

Thank you for reporting, Devan. May I suggest forwarding this issue - also as report only - to Florian? To my latest knowledge he wanted to care about the demo. Please update me if this would not be the case. Thanks.

Florian Dold

2024-01-22 21:04

manager   ~0020966

Should indeed be assigned to Devan.

dvn

2024-01-22 23:03

reporter   ~0020974

Florian and I discussed this issue. After attempting and not succeeding to implement an "ignore-list" to our current wget-based script I will be looking into replacing it with this utility: https://linkcheck.github.io/linkchecker/

dvn

2024-02-18 20:21

reporter   ~0021346

Switched to the aforementioned "linkchecker" utility, which has functionality for ignoring URLs based on regex.

Stefan

2024-02-19 07:39

reporter   ~0021360

Sorry to say that we still get BuildBot messages stating that some demo links fail:
https://buildbot.taler.net/#builders/7/builds/360
https://buildbot.taler.net/#builders/7/builds/359
https://buildbot.taler.net/#builders/7/builds/358
https://buildbot.taler.net/#builders/7/builds/357

I cannot help it. Therefore, I would like to request your feedback for this issue.
Warm regards,
Stefan

dvn

2024-02-19 09:17

reporter   ~0021363

Okay, so a few comments I think are relevant:

1. let's focus only on the *latest* run of the linkchecker, which at this time is: https://buildbot.taler.net/#/builders/7/builds/360

2. I understand that it's failing on demo links, but are these links which should be ignored, or fixed? If you don't know, then who does?

3. You know to click the `view all 398 lines` toggle on the build page, so you can see the whole log, right? See: https://buildbot.taler.net/#/builders/7/builds/360/steps/2/logs/stdio

Stefan

2024-02-19 10:31

reporter   ~0021365

Last edited: 2024-02-19 10:32

I see three really bad links in the report https://buildbot.taler.net/#/builders/7/builds/360
at line 91: https://docs.taler.net/doxygen/exchange --> 404
at line 99: https://docs.taler.net/doxygen/merchant/ --> 404
and at line 382: https://donations.demo.taler.net/en/checkout --> parameter 'donation_amount' required

The other links are working but leading to a closed FSF bookshop and some webarchives that do not count too much.

So, who is going to rectify the three links?
Cheers,
S.

dvn

2024-02-19 10:33

reporter   ~0021366

Florian, shall I add https://donations.demo.taler.net/en/checkout to the ignore list?

Florian Dold

2024-02-19 10:35

manager   ~0021367

Yes, as we'll also get a payment required status code for it.

dvn

2024-02-19 10:47

reporter   ~0021368

Okay, I will add it to the ignore list.

Then we have these three, originating from https://shop.demo.taler.net/

https://web.archive.org/web/20120118201902/http://www.gnu.org/ - Error: 503 Service Temporarily Unavailable
https://www.researchgate.net/publication/4980956_The_Case_Against_Intellectual_Property - Error: 403 Forbidden
https://shop.fsf.org/product/free-software-free-society-2 - Error: 503 Service Temporarily Unavailable

These all appear to be up (except, arguably the FSF shop), but I guess they don't like the crawler.

Stefan

2024-02-19 12:01

reporter   ~0021370

The FSF shop predicts to stay up and be retrievable again as a shop at a certain period in time in 2024. You can leave this link, I assume.

dvn

2024-02-19 12:22

reporter   ~0021371

Okay, I've added those to the ignore list. https://git.taler.net/deployment.git/commit/?id=290bc0d46f1a44e1654e7e6b02318395d222294a

dvn

2024-02-19 12:23

reporter   ~0021372

Demo sites pass after adding links we don't want to check to the ignore list.

Issue History

Date ModifiedUsernameFieldChange
2024-01-22 13:31dvnNew Issue
2024-01-22 13:31dvnStatusnew => assigned
2024-01-22 13:31dvnAssigned To => Stefan
2024-01-22 13:59StefanNote Added: 0020954
2024-01-22 14:00StefanAssigned ToStefan => Florian Dold
2024-01-22 21:03Florian DoldAssigned ToFlorian Dold => dvn
2024-01-22 21:04Florian DoldNote Added: 0020966
2024-01-22 23:03dvnNote Added: 0020974
2024-01-29 23:49Christian GrothoffSeveritymajor => tweak
2024-02-18 20:21dvnStatusassigned => informing
2024-02-18 20:21dvnResolutionopen => fixed
2024-02-18 20:21dvnNote Added: 0021346
2024-02-19 07:39StefanStatusinforming => feedback
2024-02-19 07:39StefanResolutionfixed => reopened
2024-02-19 07:39StefanNote Added: 0021360
2024-02-19 09:17dvnNote Added: 0021363
2024-02-19 10:31StefanNote Added: 0021365
2024-02-19 10:32StefanNote Edited: 0021365
2024-02-19 10:33dvnNote Added: 0021366
2024-02-19 10:35Florian DoldNote Added: 0021367
2024-02-19 10:47dvnNote Added: 0021368
2024-02-19 12:01StefanNote Added: 0021370
2024-02-19 12:22dvnNote Added: 0021371
2024-02-19 12:23dvnStatusfeedback => informing
2024-02-19 12:23dvnResolutionreopened => fixed
2024-02-19 12:23dvnNote Added: 0021372
2024-03-07 19:52Christian GrothoffFixed in Version => 0.9.4
2024-03-07 19:52Christian GrothoffStatusinforming => closed