Skip to content

Commit 56bd9a8

Browse files
GeoffreyBoothtargos
authored andcommitted
esm: --experimental-default-type flag to flip module defaults
PR-URL: #49869 Backport-PR-URL: #50669 Reviewed-By: Guy Bedford <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>
1 parent 72644d6 commit 56bd9a8

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+538
-40
lines changed

doc/api/cli.md

+31
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,36 @@ added: v17.6.0
366366

367367
Expose the [Web Crypto API][] on the global scope.
368368

369+
### `--experimental-default-type=type`
370+
371+
<!-- YAML
372+
added:
373+
- REPLACEME
374+
-->
375+
376+
> Stability: 1.0 - Early development
377+
378+
Define which module system, `module` or `commonjs`, to use for the following:
379+
380+
* String input provided via `--eval` or STDIN, if `--input-type` is unspecified.
381+
382+
* Files ending in `.js` or with no extension, if there is no `package.json` file
383+
present in the same folder or any parent folder.
384+
385+
* Files ending in `.js` or with no extension, if the nearest parent
386+
`package.json` field lacks a `"type"` field; unless the `package.json` folder
387+
or any parent folder is inside a `node_modules` folder.
388+
389+
In other words, `--experimental-default-type=module` flips all the places where
390+
Node.js currently defaults to CommonJS to instead default to ECMAScript modules,
391+
with the exception of folders and subfolders below `node_modules`, for backward
392+
compatibility.
393+
394+
Under `--experimental-default-type=module` and `--experimental-wasm-modules`,
395+
files with no extension will be treated as WebAssembly if they begin with the
396+
WebAssembly magic number (`\0asm`); otherwise they will be treated as ES module
397+
JavaScript.
398+
369399
### `--experimental-import-meta-resolve`
370400

371401
<!-- YAML
@@ -1923,6 +1953,7 @@ Node.js options that are allowed are:
19231953
* `--enable-network-family-autoselection`
19241954
* `--enable-source-maps`
19251955
* `--experimental-abortcontroller`
1956+
* `--experimental-default-type`
19261957
* `--experimental-global-customevent`
19271958
* `--experimental-global-webcrypto`
19281959
* `--experimental-import-meta-resolve`

doc/api/esm.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -105,10 +105,11 @@ provides interoperability between them and its original module format,
105105

106106
Node.js has two module systems: [CommonJS][] modules and ECMAScript modules.
107107

108-
Authors can tell Node.js to use the ECMAScript modules loader
109-
via the `.mjs` file extension, the `package.json` [`"type"`][] field, or the
110-
[`--input-type`][] flag. Outside of those cases, Node.js will use the CommonJS
111-
module loader. See [Determining module system][] for more details.
108+
Authors can tell Node.js to use the ECMAScript modules loader via the `.mjs`
109+
file extension, the `package.json` [`"type"`][] field, the [`--input-type`][]
110+
flag, or the [`--experimental-default-type`][] flag. Outside of those cases,
111+
Node.js will use the CommonJS module loader. See [Determining module system][]
112+
for more details.
112113

113114
<!-- Anchors to make sure old links find a target -->
114115

@@ -1080,6 +1081,7 @@ success!
10801081
[URL]: https://url.spec.whatwg.org/
10811082
[`"exports"`]: packages.md#exports
10821083
[`"type"`]: packages.md#type
1084+
[`--experimental-default-type`]: cli.md#--experimental-default-typetype
10831085
[`--input-type`]: cli.md#--input-typetype
10841086
[`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
10851087
[`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export

doc/api/packages.md

+30-13
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ along with a reference for the [`package.json`][] fields defined by Node.js.
5555

5656
## Determining module system
5757

58+
### Introduction
59+
5860
Node.js will treat the following as [ES modules][] when passed to `node` as the
5961
initial input, or when referenced by `import` statements or `import()`
6062
expressions:
@@ -67,14 +69,9 @@ expressions:
6769
* Strings passed in as an argument to `--eval`, or piped to `node` via `STDIN`,
6870
with the flag `--input-type=module`.
6971

70-
Node.js will treat as [CommonJS][] all other forms of input, such as `.js` files
71-
where the nearest parent `package.json` file contains no top-level `"type"`
72-
field, or string input without the flag `--input-type`. This behavior is to
73-
preserve backward compatibility. However, now that Node.js supports both
74-
CommonJS and ES modules, it is best to be explicit whenever possible. Node.js
75-
will treat the following as CommonJS when passed to `node` as the initial input,
76-
or when referenced by `import` statements, `import()` expressions, or
77-
`require()` expressions:
72+
Node.js will treat the following as [CommonJS][] when passed to `node` as the
73+
initial input, or when referenced by `import` statements or `import()`
74+
expressions:
7875

7976
* Files with a `.cjs` extension.
8077

@@ -84,11 +81,30 @@ or when referenced by `import` statements, `import()` expressions, or
8481
* Strings passed in as an argument to `--eval` or `--print`, or piped to `node`
8582
via `STDIN`, with the flag `--input-type=commonjs`.
8683

87-
Package authors should include the [`"type"`][] field, even in packages where
88-
all sources are CommonJS. Being explicit about the `type` of the package will
89-
future-proof the package in case the default type of Node.js ever changes, and
90-
it will also make things easier for build tools and loaders to determine how the
91-
files in the package should be interpreted.
84+
Aside from these explicit cases, there are other cases where Node.js defaults to
85+
one module system or the other based on the value of the
86+
[`--experimental-default-type`][] flag:
87+
88+
* Files ending in `.js` or with no extension, if there is no `package.json` file
89+
present in the same folder or any parent folder.
90+
91+
* Files ending in `.js` or with no extension, if the nearest parent
92+
`package.json` field lacks a `"type"` field; unless the folder is inside a
93+
`node_modules` folder. (Package scopes under `node_modules` are always treated
94+
as CommonJS when the `package.json` file lacks a `"type"` field, regardless
95+
of `--experimental-default-type`, for backward compatibility.)
96+
97+
* Strings passed in as an argument to `--eval` or piped to `node` via `STDIN`,
98+
when `--input-type` is unspecified.
99+
100+
This flag currently defaults to `"commonjs"`, but it may change in the future to
101+
default to `"module"`. For this reason it is best to be explicit wherever
102+
possible; in particular, package authors should always include the [`"type"`][]
103+
field in their `package.json` files, even in packages where all sources are
104+
CommonJS. Being explicit about the `type` of the package will future-proof the
105+
package in case the default type of Node.js ever changes, and it will also make
106+
things easier for build tools and loaders to determine how the files in the
107+
package should be interpreted.
92108

93109
### Modules loaders
94110

@@ -1337,6 +1353,7 @@ This field defines [subpath imports][] for the current package.
13371353
[`"packageManager"`]: #packagemanager
13381354
[`"type"`]: #type
13391355
[`--conditions` / `-C` flag]: #resolving-user-conditions
1356+
[`--experimental-default-type`]: cli.md#--experimental-default-typetype
13401357
[`--no-addons` flag]: cli.md#--no-addons
13411358
[`ERR_PACKAGE_PATH_NOT_EXPORTED`]: errors.md#err_package_path_not_exported
13421359
[`esm`]: https://github.com./standard-things/esm#readme

doc/node.1

+5
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,11 @@ Requires Node.js to be built with
140140
.It Fl -enable-source-maps
141141
Enable Source Map V3 support for stack traces.
142142
.
143+
.It Fl -experimental-default-type Ns = Ns Ar type
144+
Interpret as either ES modules or CommonJS modules input via --eval or STDIN, when --input-type is unspecified;
145+
.js or extensionless files with no sibling or parent package.json;
146+
.js or extensionless files whose nearest parent package.json lacks a "type" field, unless under node_modules.
147+
.
143148
.It Fl -experimental-global-customevent
144149
Expose the CustomEvent on the global scope.
145150
.

lib/internal/main/check_syntax.js

+2-1
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,8 @@ function loadESMIfNeeded(cb) {
6363
async function checkSyntax(source, filename) {
6464
let isModule = true;
6565
if (filename === '[stdin]' || filename === '[eval]') {
66-
isModule = getOptionValue('--input-type') === 'module';
66+
isModule = getOptionValue('--input-type') === 'module' ||
67+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs');
6768
} else {
6869
const { defaultResolve } = require('internal/modules/esm/resolve');
6970
const { defaultGetFormat } = require('internal/modules/esm/get_format');

lib/internal/main/eval_stdin.js

+4-2
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,14 @@ readStdin((code) => {
2525

2626
const print = getOptionValue('--print');
2727
const loadESM = getOptionValue('--import').length > 0;
28-
if (getOptionValue('--input-type') === 'module')
28+
if (getOptionValue('--input-type') === 'module' ||
29+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs')) {
2930
evalModule(code, print);
30-
else
31+
} else {
3132
evalScript('[stdin]',
3233
code,
3334
getOptionValue('--inspect-brk'),
3435
print,
3536
loadESM);
37+
}
3638
});

lib/internal/main/eval_string.js

+4-2
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,13 @@ markBootstrapComplete();
2323
const source = getOptionValue('--eval');
2424
const print = getOptionValue('--print');
2525
const loadESM = getOptionValue('--import').length > 0 || getOptionValue('--experimental-loader').length > 0;
26-
if (getOptionValue('--input-type') === 'module')
26+
if (getOptionValue('--input-type') === 'module' ||
27+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs')) {
2728
evalModule(source, print);
28-
else
29+
} else {
2930
evalScript('[eval]',
3031
source,
3132
getOptionValue('--inspect-brk'),
3233
print,
3334
loadESM);
35+
}

lib/internal/modules/esm/formats.js

+28
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,11 @@
22

33
const {
44
RegExpPrototypeExec,
5+
Uint8Array,
56
} = primordials;
67
const { getOptionValue } = require('internal/options');
78

9+
const { closeSync, openSync, readSync } = require('fs');
810

911
const experimentalWasmModules = getOptionValue('--experimental-wasm-modules');
1012

@@ -49,8 +51,34 @@ function getLegacyExtensionFormat(ext) {
4951
return legacyExtensionFormatMap[ext];
5052
}
5153

54+
/**
55+
* For extensionless files in a `module` package scope, or a default `module` scope enabled by the
56+
* `--experimental-default-type` flag, we check the file contents to disambiguate between ES module JavaScript and Wasm.
57+
* We do this by taking advantage of the fact that all Wasm files start with the header `0x00 0x61 0x73 0x6d` (`_asm`).
58+
* @param {URL} url
59+
*/
60+
function getFormatOfExtensionlessFile(url) {
61+
if (!experimentalWasmModules) { return 'module'; }
62+
63+
const magic = new Uint8Array(4);
64+
let fd;
65+
try {
66+
// TODO(@anonrig): Optimize the following by having a single C++ call
67+
fd = openSync(url);
68+
readSync(fd, magic, 0, 4); // Only read the first four bytes
69+
if (magic[0] === 0x00 && magic[1] === 0x61 && magic[2] === 0x73 && magic[3] === 0x6d) {
70+
return 'wasm';
71+
}
72+
} finally {
73+
if (fd !== undefined) { closeSync(fd); }
74+
}
75+
76+
return 'module';
77+
}
78+
5279
module.exports = {
5380
extensionFormatMap,
81+
getFormatOfExtensionlessFile,
5482
getLegacyExtensionFormat,
5583
legacyExtensionFormatMap,
5684
mimeToFormat,

lib/internal/modules/esm/get_format.js

+52-7
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,19 @@
11
'use strict';
2+
23
const {
34
RegExpPrototypeExec,
45
ObjectPrototypeHasOwnProperty,
56
PromisePrototypeThen,
67
PromiseResolve,
8+
StringPrototypeIncludes,
79
StringPrototypeCharCodeAt,
810
StringPrototypeSlice,
911
} = primordials;
1012
const { basename, relative } = require('path');
1113
const { getOptionValue } = require('internal/options');
1214
const {
1315
extensionFormatMap,
16+
getFormatOfExtensionlessFile,
1417
getLegacyExtensionFormat,
1518
mimeToFormat,
1619
} = require('internal/modules/esm/formats');
@@ -19,6 +22,9 @@ const experimentalNetworkImports =
1922
getOptionValue('--experimental-network-imports');
2023
const experimentalSpecifierResolution =
2124
getOptionValue('--experimental-specifier-resolution');
25+
const defaultTypeFlag = getOptionValue('--experimental-default-type');
26+
// The next line is where we flip the default to ES modules someday.
27+
const defaultType = defaultTypeFlag === 'module' ? 'module' : 'commonjs';
2228
const { getPackageType, getPackageScopeConfig } = require('internal/modules/esm/resolve');
2329
const { fileURLToPath } = require('internal/url');
2430
const { ERR_UNKNOWN_FILE_EXTENSION } = require('internal/errors').codes;
@@ -69,6 +75,18 @@ function extname(url) {
6975
return '';
7076
}
7177

78+
/**
79+
* Determine whether the given file URL is under a `node_modules` folder.
80+
* This function assumes that the input has already been verified to be a `file:` URL,
81+
* and is a file rather than a folder.
82+
* @param {URL} url
83+
*/
84+
function underNodeModules(url) {
85+
if (url.protocol !== 'file:') { return false; } // We determine module types for other protocols based on MIME header
86+
87+
return StringPrototypeIncludes(url.pathname, '/node_modules/');
88+
}
89+
7290
/**
7391
* @param {URL} url
7492
* @param {{parentURL: string}} context
@@ -77,8 +95,37 @@ function extname(url) {
7795
*/
7896
function getFileProtocolModuleFormat(url, context, ignoreErrors) {
7997
const ext = extname(url);
98+
8099
if (ext === '.js') {
81-
return getPackageType(url) === 'module' ? 'module' : 'commonjs';
100+
const packageType = getPackageType(url);
101+
if (packageType !== 'none') {
102+
return packageType;
103+
}
104+
// The controlling `package.json` file has no `type` field.
105+
if (defaultType === 'module') {
106+
// An exception to the type flag making ESM the default everywhere is that package scopes under `node_modules`
107+
// should retain the assumption that a lack of a `type` field means CommonJS.
108+
return underNodeModules(url) ? 'commonjs' : 'module';
109+
}
110+
return 'commonjs';
111+
}
112+
113+
if (ext === '') {
114+
const packageType = getPackageType(url);
115+
if (defaultType === 'commonjs') { // Legacy behavior
116+
if (packageType === 'none' || packageType === 'commonjs') {
117+
return 'commonjs';
118+
}
119+
// If package type is `module`, fall through to the error case below
120+
} else { // Else defaultType === 'module'
121+
if (underNodeModules(url)) { // Exception for package scopes under `node_modules`
122+
return 'commonjs';
123+
}
124+
if (packageType === 'none' || packageType === 'module') {
125+
return getFormatOfExtensionlessFile(url);
126+
} // Else packageType === 'commonjs'
127+
return 'commonjs';
128+
}
82129
}
83130

84131
const format = extensionFormatMap[ext];
@@ -93,12 +140,10 @@ function getFileProtocolModuleFormat(url, context, ignoreErrors) {
93140
const config = getPackageScopeConfig(url);
94141
const fileBasename = basename(filepath);
95142
const relativePath = StringPrototypeSlice(relative(config.pjsonPath, filepath), 1);
96-
suggestion = 'Loading extensionless files is not supported inside of ' +
97-
'"type":"module" package.json contexts. The package.json file ' +
98-
`${config.pjsonPath} caused this "type":"module" context. Try ` +
99-
`changing ${filepath} to have a file extension. Note the "bin" ` +
100-
'field of package.json can point to a file with an extension, for example ' +
101-
`{"type":"module","bin":{"${fileBasename}":"${relativePath}.js"}}`;
143+
suggestion = 'Loading extensionless files is not supported inside of "type":"module" package.json contexts ' +
144+
`without --experimental-default-type=module. The package.json file ${config.pjsonPath} caused this "type":"module" ` +
145+
`context. Try changing ${filepath} to have a file extension. Note the "bin" field of package.json can point ` +
146+
`to a file with an extension, for example {"type":"module","bin":{"${fileBasename}":"${relativePath}.js"}}`;
102147
}
103148
throw new ERR_UNKNOWN_FILE_EXTENSION(ext, filepath, suggestion);
104149
}

lib/internal/modules/esm/resolve.js

+2-2
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ const preserveSymlinks = getOptionValue('--preserve-symlinks');
3636
const preserveSymlinksMain = getOptionValue('--preserve-symlinks-main');
3737
const experimentalNetworkImports =
3838
getOptionValue('--experimental-network-imports');
39-
const typeFlag = getOptionValue('--input-type');
39+
const inputTypeFlag = getOptionValue('--input-type');
4040
const { URL, pathToFileURL, fileURLToPath, isURL, toPathIfFileURL } = require('internal/url');
4141
const { canParse: URLCanParse } = internalBinding('url');
4242
const {
@@ -1183,7 +1183,7 @@ function defaultResolve(specifier, context = {}) {
11831183
// input, to avoid user confusion over how expansive the effect of the
11841184
// flag should be (i.e. entry point only, package scope surrounding the
11851185
// entry point, etc.).
1186-
if (typeFlag) { throw new ERR_INPUT_TYPE_NOT_ALLOWED(); }
1186+
if (inputTypeFlag) { throw new ERR_INPUT_TYPE_NOT_ALLOWED(); }
11871187
}
11881188

11891189
conditions = getConditionsSet(conditions);

lib/internal/modules/run_main.js

+3-2
Original file line numberDiff line numberDiff line change
@@ -52,13 +52,14 @@ function shouldUseESMLoader(mainPath) {
5252
if (esModuleSpecifierResolution === 'node') {
5353
return true;
5454
}
55-
// Determine the module format of the main
55+
// Determine the module format of the entry point.
5656
if (mainPath && StringPrototypeEndsWith(mainPath, '.mjs')) { return true; }
5757
if (!mainPath || StringPrototypeEndsWith(mainPath, '.cjs')) { return false; }
5858

5959
const { readPackageScope } = require('internal/modules/package_json_reader');
6060
const pkg = readPackageScope(mainPath);
61-
return pkg && pkg.data.type === 'module';
61+
// No need to guard `pkg` as it can only be an object or `false`.
62+
return pkg.data?.type === 'module' || getOptionValue('--experimental-default-type') === 'module';
6263
}
6364

6465
/**

0 commit comments

Comments
 (0)