View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0006906 | Taler | wallet-core | public | 2021-06-17 14:22 | 2021-08-24 16:22 |
Reporter | sebasjm | Assigned To | Florian Dold | ||
Priority | high | Severity | major | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Product Version | git (master) | ||||
Target Version | 0.8 | Fixed in Version | 0.8 | ||
Summary | 0006906: problems using wallet-core in a project with webpack | ||||
Description | Trying to use storybook with wallet-web-extension and the latest code I get 3 error in the build process [1] that seems to be related to the module format. After exploring for some time I found that webpack has some issues with the new format [2] Using webpack 5 seems to have to module format parsed correctly but I get some other problems with the import that are easily addressable. [3] Storybook use webpack 4 but there is a migration to webpack 5 still in beta. I have created a repo that reproduce the problem, but should we add this as a integration test into the wallet-core repo? This example does not reproduce 2 of 3 errors (related to using the break) which I think it may be related to a webpack configuration. https://github.com/sebasjm/wallet-core-webpack-packaging To reproduce: * pnpm install * pnpm install $WALLET_CORE_FOLDER/packages/taler-wallet-core/ * pnpx webpack $ pnpx --version 6.2.1 [1] 3 errors found in storybook build process ERROR in ../taler-wallet-core/lib/crypto/primitives/nacl-fast.js 14:44 Module parse failed: Unexpected token (14:44) File was processed with these loaders: * ../../node_modules/.pnpm/babel-loader@8.2.2_105470dfa535e5f605a4a54ef7d81e87/node_modules/babel-loader/lib/index.js You may need an additional loader to handle the result of these loaders. | // We need this require function to synchronously | // import the "crypto" module in the CSPRNG initialization. > require = mod.default.createRequire(import.meta.url); | } | @ ../taler-wallet-core/lib/crypto/talerCrypto.js 36:0-50 54:9-25 170:13-39 174:9-29 177:11-20 184:10-25 185:9-18 188:22-56 189:10-25 190:9-18 318:18-34 326:18-34 334:9-18 337:13-39 338:9-27 341:9-34 344:13-27 @ ../taler-wallet-core/lib/index.js @ ./src/wxApi.ts @ ./src/wallet/Withdraw.tsx @ ./src/wallet/Withdraw.stories.tsx @ ./src sync ^\.(?:(?:^|\/|(?:(?:(?!(?:^|\/)\.).)*?)\/)(?!\.)(?=.)[^/]*?\.stories\.tsx)$ @ ./.storybook/generated-stories-entry.js ERROR in ../taler-wallet-core/lib/operations/reserves.js 1136:20 Module parse failed: Unsyntactic break (1136:20) File was processed with these loaders: * ../../node_modules/.pnpm/babel-loader@8.2.2_105470dfa535e5f605a4a54ef7d81e87/node_modules/babel-loader/lib/index.js You may need an additional loader to handle the result of these loaders. | return Promise.resolve(processReserveImpl(ws, reservePub, true)).then($return, $error); | } else { > break; | } | } catch ($boundEx) { @ ../taler-wallet-core/lib/wallet.js 43:0-119 128:31-45 697:29-42 710:19-38 849:21-47 884:14-28 @ ../taler-wallet-core/lib/index.js @ ./src/wxApi.ts @ ./src/wallet/Withdraw.tsx @ ./src/wallet/Withdraw.stories.tsx @ ./src sync ^\.(?:(?:^|\/|(?:(?:(?!(?:^|\/)\.).)*?)\/)(?!\.)(?=.)[^/]*?\.stories\.tsx)$ @ ./.storybook/generated-stories-entry.js ERROR in ../taler-wallet-core/lib/operations/transactions.js 491:12 Module parse failed: Unsyntactic break (491:12) File was processed with these loaders: * ../../node_modules/.pnpm/babel-loader@8.2.2_105470dfa535e5f605a4a54ef7d81e87/node_modules/babel-loader/lib/index.js You may need an additional loader to handle the result of these loaders. | return Promise.resolve(processWithdrawGroup(ws, withdrawalGroupId, true)).then(function ($await_27) { | try { > break; | } catch ($boundEx) { | return $error($boundEx); @ ../taler-wallet-core/lib/wallet.js 46:0-97 1154:31-46 1363:31-48 1373:31-47 @ ../taler-wallet-core/lib/index.js @ ./src/wxApi.ts @ ./src/wallet/Withdraw.tsx @ ./src/wallet/Withdraw.stories.tsx @ ./src sync ^\.(?:(?:^|\/|(?:(?:(?!(?:^|\/)\.).)*?)\/)(?!\.)(?=.)[^/]*?\.stories\.tsx)$ @ ./.storybook/generated-stories-entry.js [2] https://www.npmjs.com/package/@open-wc/webpack-import-meta-loader https://github.com/webpack/webpack/issues/6719 [3] errors found in webpack 5, hiding repeated errors ERROR in ../../taler/wallet-core/packages/taler-wallet-core/lib/crypto/primitives/nacl-fast.js 7:0-30 Module not found: Error: Can't resolve 'module' in '/home/sebasjm/Work/taler/wallet-core/packages/taler-wallet-core/lib/crypto/primitives' @ ../../taler/wallet-core/packages/taler-wallet-core/lib/crypto/talerCrypto.js 19:0-50 33:11-27 129:17-43 133:11-31 136:15-24 141:14-29 142:11-20 145:26-60 146:14-29 147:11-20 259:22-38 264:22-38 269:11-20 272:17-43 273:11-29 276:11-36 279:15-29 @ ../../taler/wallet-core/packages/taler-wallet-core/lib/index.js 35:0-40 35:0-40 @ ./src/index.js 3:0-70 ERROR in ../../taler/wallet-core/packages/taler-wallet-core/lib/crypto/workers/cryptoImplementation.js 29:0-42 Module not found: Error: Can't resolve '../../util/timer' in '/home/sebasjm/Work/taler/wallet-core/packages/taler-wallet-core/lib/crypto/workers' Did you mean 'timer.js'? BREAKING CHANGE: The request '../../util/timer' failed to resolve only because it was resolved as fully specified (probably because the origin is a '*.mjs' file or a '*.js' file where the package.json contains '"type": "module"'). The extension in the request is mandatory for it to be fully specified. Add the extension to the request. @ ../../taler/wallet-core/packages/taler-wallet-core/lib/index.js 33:0-80 33:0-80 @ ./src/index.js 3:0-70 ERROR in ../../taler/wallet-core/packages/taler-wallet-core/lib/crypto/workers/nodeThreadWorker.js 16:0-20 Module not found: Error: Can't resolve 'os' in '/home/sebasjm/Work/taler/wallet-core/packages/taler-wallet-core/lib/crypto/workers' BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default. This is no longer the case. Verify if you need this module and configure a polyfill for it. If you want to include a polyfill, you need to: - add a fallback 'resolve.fallback: { "os": require.resolve("os-browserify/browser") }' - install 'os-browserify' If you don't want to include a polyfill, you can use an empty module like this: resolve.fallback: { "os": false } @ ../../taler/wallet-core/packages/taler-wallet-core/lib/index.js 32:0-53 32:0-53 @ ./src/index.js 3:0-70 | ||||
Tags | No tags attached. | ||||
|
I think the errors we get with wp5 are due to import statements sometimes not using the ".js" extension. That is, we should always write import { runBackupCycle } from "./operations/backup.js"; and *not* import { runBackupCycle } from "./operations/backup"; Sebastian: Any idea what the issues in wp4 are, besides the missing .js extensions? |
|
webpack 4 doesnt like this line require = mod.default.createRequire(import.meta.url); here https://git.taler.net/wallet-core.git/tree/packages/taler-wallet-core/src/crypto/primitives/nacl-fast.ts#n15 |
|
> I think the errors we get with wp5 are due to import statements sometimes not using the ".js" extension. Yes, a rename of the imports will work. There are still two others problems: * Can't resolve 'module' here https://git.taler.net/wallet-core.git/tree/packages/taler-wallet-core/src/crypto/primitives/nacl-fast.ts#n8 Which I think we may replace with a dynamic import (maybe, not sure) * Can't resolve 'os' here https://git.taler.net/wallet-core.git/tree/packages/taler-wallet-core/src/crypto/workers/nodeThreadWorker.ts Same as above i think but I don't know how and where it is used to decide if we should use a polyfill or just dynamic import to handle the absence. Other strategy could be to have 1 entry point for nodejs environment and other for browser environment which IMHO will end up in better code. |
|
Can you elaborate how you would do the multiple entry points? The problem with using dynamic ES6-style imports is that: * import(...) is asynchronous * rollup (and AFAICT also webpack) doesn't support top-level await, so we'd have to restructure the code that accesses crypto functions, as the random number generator would have to be initialized asynchronously. Let me know if you have any other ideas, too! |
|
> Can you elaborate how you would do the multiple entry points? Using the browsing field in the package json seems promising https://docs.npmjs.com/cli/v7/configuring-npm/package-json#browser Although I don't see the "type" and "module" fields there, so this behavior seems to be decided with the bundler. If this browser field thing in the package_json does not work, an ugly but easy alternative could be browser should import: import { someApi } from '@gnu-taler/taler-wallet-core/browser' nodejs apps should import: import { someApi } from '@gnu-taler/taler-wallet-core/node' this folders will have the index.js file with different setup but exporting the same API. |
|
So an action plan could be: * rename imports to have the .js * add browser field in package json with different entry point * splint index file in wallet-core: browser-index.ts, node-index.ts and common-index.ts * configure rollup * refactor code to move require of "os" and "crypto" to node-index and equivalent into browser-index.ts * profit |
|
FYI https://github.com/webpack/webpack/issues/4674#issuecomment-355853969 |
|
profit achieved I will fix imports and push into master if no comments, I still having this problem with webpack 4 that I will try to fix it after push Module parse failed: Unsyntactic break (491:12) File was processed with these loaders: * ../../node_modules/.pnpm/babel-loader@8.2.2_105470dfa535e5f605a4a54ef7d81e87/node_modules/babel-loader/lib/index.js You may need an additional loader to handle the result of these loaders. | return Promise.resolve(processWithdrawGroup(ws, withdrawalGroupId, true)).then(function ($await_27) { | try { > break; | } catch ($boundEx) { | return $error($boundEx); |
|
fixed in 46f3fcbb..1c7423db |
|
The missing issue with "Unsyntactic break" was related to babel. Any project that use babel will not able to load wallet-core if wallet-core contains code that use break statement inside a block scoped case of a switch....case statement. Very rare. https://github.com/babel/babel/issues/8709 Fixed in a4902de8..954ed239 |
Date Modified | Username | Field | Change |
---|---|---|---|
2021-06-17 14:22 | sebasjm | New Issue | |
2021-06-17 14:22 | sebasjm | Status | new => assigned |
2021-06-17 14:22 | sebasjm | Assigned To | => Florian Dold |
2021-06-17 14:33 | Florian Dold | Note Added: 0017963 | |
2021-06-17 14:40 | sebasjm | Note Added: 0017964 | |
2021-06-17 14:48 | sebasjm | Note Added: 0017965 | |
2021-06-17 15:02 | Florian Dold | Note Added: 0017966 | |
2021-06-17 15:31 | sebasjm | Note Added: 0017967 | |
2021-06-17 15:36 | sebasjm | Note Added: 0017968 | |
2021-06-17 16:02 | sebasjm | Note Added: 0017969 | |
2021-06-17 17:06 | sebasjm | Note Added: 0017970 | |
2021-06-17 17:53 | sebasjm | Note Added: 0017972 | |
2021-06-17 18:22 | sebasjm | Note Added: 0017973 | |
2021-06-18 13:30 | sebasjm | Status | assigned => resolved |
2021-06-18 13:30 | sebasjm | Resolution | open => fixed |
2021-07-30 13:56 | Christian Grothoff | Fixed in Version | => 0.8 |
2021-07-30 13:59 | Christian Grothoff | Target Version | git (master) => 0.8.1 |
2021-07-30 14:01 | Christian Grothoff | Target Version | 0.8.1 => 0.8 |
2021-08-24 16:22 | Christian Grothoff | Status | resolved => closed |
2023-04-13 20:36 | Florian Dold | Category | wallet (TS core) => wallet-core |