This makes it possible for reverse proxies to transform 403 errors
into something like "upgrade to a premium account to access this
pad".
Also add some webaccess tests.
* `src/node/server.js` can now be run as a script (for normal
operation) or imported as a module (for tests).
* Move shutdown actions to `src/node/server.js` to be close to the
startup actions.
* Put startup and shutdown in functions so that tests can call them.
* Use `await` instead of callbacks.
* Block until the HTTP server is listening to avoid races during
test startup.
* Add a new `shutdown` hook.
* Use the `shutdown` hook to:
* close the HTTP server
* call `end()` on the stats collection to cancel its timers
* call `terminate()` on the Threads.Pool to stop the workers
* Exit with exit code 0 (instead of 1) on SIGTERM.
* Export the HTTP server so that tests can get the HTTP server's
port via `server.address().port` when `settings.port` is 0.
Before, a malicious user could bypass authorization restrictions
imposed by the authorize hook:
* Step 1: Fetch any resource that the malicious user is authorized to
access (e.g., static content).
* Step 2: Use the signed express_sid cookie generated in step 1 to
create a socket.io connection.
* Step 3: Perform the CLIENT_READY handshake for the desired pad.
* Step 4: Profit!
Now the authorization decision made by the authorize hook is
propagated to SecurityManager so that it can approve or reject
socket.io messages as appropriate.
This also sets up future support for per-user read-only and
modify-only (no create) authorization levels.
Authentication plugins almost always want to read and modify
`settings.users`. The settings can already be accessed in a few other
ways, but this is much more convenient.
The authorization logic determines whether the user has already
successfully authenticated by looking to see if `req.session.user`
exists. If an authentication plugin says that it successfully
authenticated the user but it did not create `req.session.user` then
authentication will re-run for every access, and authorization plugins
will be unable to determine whether the user has been authenticated.
Return a 500 internal server error to prevent these problems.
Before it only logged an error like this:
SyntaxError: Unexpected string in JSON at position XYZ
Now it also logs the filename, making it easier to figure out where
the bad data is:
failed to read file /path/to/etherpad-lite/src/locales/en.json: SyntaxError: Unexpected string in JSON at position XYZ
Before, anyone who could create a socket.io connection to Etherpad
could read, modify, and create pads at will without authenticating
first.
The `checkAccess` middleware in `webaccess.js` normally handles
authentication and authorization, but it does not run for `/socket.io`
requests. This means that the connection handler in `socketio.js` must
handle authentication and authorization. However, before this change:
* The handler did not require a signed `express_sid` cookie.
* After loading the express-session state, the handler did not check
to see if the user had authenticated.
Now the handler requires a signed `express_sid` cookie, and it ensures
that `socket.request.session.user` is non-null if authentication is
required. (`socket.request.session.user` is non-null if and only if
the user has authenticated.)
There are two different ways an author ID becomes associated with a
user: either bound to a token or bound to a session ID. (The token and
session ID come from the `token` and `sessionID` cookies, or, in the
case of socket.io messages, from the `token` and `sessionID` message
properties.) When `settings.requireSession` is true or the user is
accessing a group pad, the session ID should be used. Otherwise the
token should be used.
Before this change, the `/p/:pad/import` handler was always using the
token, even when `settings.requireSession` was true. This caused the
following error because a different author ID was bound to the token
versus the session ID:
> Unable to import file into ${pad}. Author ${authorID} exists but he
> never contributed to this pad
This bug was reported in issue #4006. PR #4012 worked around the
problem by binding the same author ID to the token as well as the
session ID.
This change does the following:
* Modifies the import handler to use the session ID to obtain the
author ID (when appropriate).
* Expands the documentation for the SecurityManager checkAccess
function.
* Removes the workaround from PR #4012.
* Cleans up the `bin/createUserSession.js` test script.
Not all authentication plugins require the Authorization header, so it
might not be present in subsequent attempts. (In particular, a reverse
proxy might strip it.)
* Improve the comment describing how the access check works.
* Move the `authenticate` logic to where it is used so that people
don't have to keep jumping back and forth to understand how the
access check works.
* Break up the three steps to reduce the number of indentation
levels and improve readability. This should also make it easier to
implement and review planned future changes.
In this way, if the browser sends a list of preferred languages via
Accept-Language HTTP header, Etherpad will honor that.
Before this change, Etherpad always forced on the user the language from
padOptions.lang in settings.json.
This reverts a feature that was introduced in 295672f598.
If Etherpad is hosted on Windows the frontend test URI needs to be
/tests/frontend/index.html (docs say .../frontend/), otherwise there is this
error: ERR_TOO_MANY_REDIRECTS.
Fixes#3804.
This is a departure from previous versions, which did not limit import/export
requests. Now such requests are ALWAYS rate limited. The default is 10 requests
per IP each 90 seconds, and also applies to old instances upgraded to 1.8.3.
Administrators can tune the parameters via settings.importExportRateLimiting.
Importing to a pad is allowed only if an author has a session estabilished and
has already contributed to that specific pad. This means that as long as the
user is on the pad (via the browser) then import is possible.
Note that an author session is NOT the same as a group session, which is not
required.
This setting does not apply to API requests, only to /p/$PAD$/import
This change of behaviour is introduced in Etherpad 1.8.3, and cannot be
disabled.
For some weird reason, these seem to be part of the original swagger
implementation but tests assume they're turned off.
Perhaps a difference between /rest and /api?
- Tests pass ✅
- Added openapi-backend hook
- Generating OpenAPI v3 definitions for each API version
- Definitions served /api/openapi.json /api/{version}/openapi.json
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.
The "secure" flag is set if one of these is true:
1. we are directly serving Etherpad over SSL using the native nodejs
functionality, via the "ssl" options in settings.json
2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
proxy for terminating the SSL for us;
In this case, the user has to be instructed to properly set trustProxy: true
in settings.json, and the information wheter the application is over SSL or
not will be extracted from the X-Forwarded-Proto HTTP header.
Please note that this will not be compatible with applications being served over
http and https at the same time.
The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.
Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecureCloses#3561.
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.
In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.
Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.
Let's simply nuke it.
References:
https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancinghttps://github.com/socketio/socket.io/issues/2276#issuecomment-147184662 (not totally true, actually, see above)
Do not touch vendorized files (e.g. libraries that were imported from external
projects).
No functional changes.
Command:
find . -name '*.<EXTENSION>' -type f -print0 | xargs -0 sed -i 's/[[:space:]]*$//'
Currently the version is exposed in a 'Server' http headers.
This commit allows to parameterize it in the settings. By defaults it is
not exposed.
Fixes#3423
`getPadAccess()` (src/node/padaccess.js) is now "promise only", resolving to
`true` or `false` as appropriate, and throwing an exception if there's an
error.
The two call sites (padreadonly.js and importexport.js) updated to match.
Use real `async` instead of async.js where applicable.
The `getPluginTests()` function was never truly async anyway because it only
contains calls to synchronous `fs` modules.
This change is only cosmetic. Its aim is do make it easier to understand the
async changes that are going to be merged later on. It was extracted from the
original work from Ray Bellis.
To verify that nothing has changed, you can run the following command on each
file touched by this commit:
npm install uglify-es
diff --unified <(uglify-js --beautify bracketize <BEFORE.js>) <(uglify-js --beautify bracketize <AFTER.js>)
This is a complete script that does the same automatically (works from a
mercurial clone):
```bash
#!/usr/bin/env bash
set -eu
REVISION=<THIS_REVISION>
PARENT_REV=$(hg identify --rev "${REVISION}" --template '{p1rev}')
FILE_LIST=$(hg status --no-status --change ${REVISION})
UGLIFYJS="node_modules/uglify-es/bin/uglifyjs"
for FILE_NAME in ${FILE_LIST[@]}; do
echo "Checking ${FILE_NAME}"
diff --unified \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${PARENT_REV}" "${FILE_NAME}")) \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${REVISION}" "${FILE_NAME}"))
done
```
Until Etherpad 1.7.5, process.on('SIGTERM') and process.on('SIGINT') were not
hooked up under Windows, because old nodejs versions did not support them.
This excluded the possibility of doing a graceful shutdown of the database
connection under that platform.
According to nodejs 6.x documentation, it is now safe to do so. This allows to
gracefully close the DB connection when hitting CTRL+C under Windows, for
example.
Source: https://nodejs.org/docs/latest-v6.x/api/process.html#process_signal_events
- SIGTERM is not supported on Windows, it can be listened on.
- SIGINT from the terminal is supported on all platforms, and can usually be
generated with <Ctrl>+C (though this may be configurable). It is not
generated when terminal raw mode is enabled.
It's just synctactic sugar, but it is always better than executing string
concatenations in one's mind.
Do not do this with files in src/static, because we want to keep IE 11
compatibility.
The old "static/custom" directory is replaced by "static/skins/<skinName>",
where <skinName> is taken from settings.json.
When no value is found, a default of "no-skin" is assumed, so that backward
compatibility is maintained.
The most evident security concerns have been addressed.
Closes#3471.
This file uses it for robots.txt and favicon.ico.
This makes use of the new stable settings.root introduced with #3466, and will
be modified when introducing support for custom skins.
The hostname:port of URIs used in Minify are currently bogus and refer
to localhost only for historical reasons; there's no reason to retain
them and omitting them avoids generating an invalid URI when "port" is
not an integer.
Context: settings.port is passed to express's listen; if not numeric, it
is used a filename for a Unix domain socket.
This allows e.g. starting a server to be reverse-proxied on a multi-user
system, using the filesystem to handle access control and avoiding need
to allocate port numbers.
Before this change, etherpad-lite starts without error when configured
to listen on a Unix domain socket in this manner. However, `pad.js` and
`ace2_common.js` are generated incorrecting, causing an error
"Uncaught Error: The module at "ep_etherpad-lite/static/js/rjquery" does not exist."
when loading the editor:
When settings.port is a non-numeric string, e.g. `etherpad.sock`, a URI
of the form `http://localhost:etherpad.sock/static/js/rjquery.js` is
generated and parsed to find the file needed. In this case, the file
searched for is `:etherpad.sock/static/js/rjquery.js`, rather than the
expected `static/js/rjquery.js`. No such file exists, and the required
code is silently omitted from the bundle.
As a workaround, hard-code a (meaningless) hostname which can be parsed
correctly, since the current code makes no use of it anyway.
Shut down database connection and exit the node process
when SIGTERM is encountered. This is especially important
when nodejs is run as PID1, e.g. in a docker container.
Shutting down connections to clients (browsers) is beyond
this patche's scope.
Resolves#3265
Users still cannot choose which shortcuts they want to enable/disable,
so it does not make sense (yet) to have a cookie with that preference.
This can be reverted once we create an UI to change shortcuts, but
PLEASE PLEASE PLEASE do not read the cookie every time handleKeyEvent is
called!!!
This is an adjustment to #2891.
