Commit graph

1152 commits

Author SHA1 Message Date
muxator
39f271bfed DB.js: remove else clause, since we are already exiting early
No functional changes.
2020-03-24 23:26:19 +01:00
John McLear
4eec68fbdd tests: ignore vim swap files
This helps during plugin development, which otherwise which break when a
developer is editing a test.
2020-03-24 23:24:38 +01:00
Chocobozzz
43ca44e444 minify: make conditional requests work. No HTTP/304 was ever generated and file were reminified uselessly.
By specification [0], the if-modified-since HTTP header sent by browsers does
not include milliseconds.

Before this patch, let's say a file was generate at time:
    t_real-file         = 2020-03-22T02:15:53.548Z (note the fractional seconds)

When issuing a conditional request, the browser would truncate the fractional
part, and only request an if-modified-since with this contents:
    t_if-modified-since = 2020-03-22T02:15:53.000Z

The minify() function would return HTTP/304 only if
t_if-modified-since >= t_real-file, but this would never be true unless, by
chance, a file was generated at XX.000Z.

This resulted in that file being minified/compressed again and resent to the
client for no reason. After this patch, the server correctly responds with
HTTP/304 without doing any computation, and the browser uses the cached file.

[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since
2020-03-23 11:34:01 +01:00
muxator
ca35488273 minify: upgrade clean-css 3.4.19 -> 4.2.3. Adapt to the semver major change
CleanCSS 3.4.19 had a Regex Denial of Service vulnerability and has to be
updated. The major version bump requires the following changes:

1. Disabling rebase is necessary because otherwise the URLs for the web fonts
   become wrong;

   EXAMPLE 1:
       /static/css/src/static/font/fontawesome-etherpad.woff
     instead of
       /static/font/fontawesome-etherpad.woff

   EXAMPLE 2 (this is more surprising):
       /p/src/static/font/opendyslexic.otf
     instead of
       /static/font/opendyslexic.otf

2. CleanCSS.minify() can either receive a string containing the CSS, or an array
   of strings. In that case each array element is interpreted as an absolute
   local path from which the CSS file is read.

   In version 4.x, CleanCSS API was simplified, eliminating the relativeTo
   parameter, and thus we cannot use our already loaded "content" argument, but
   we have to wrap the absolute path to the CSS in an array and ask the library
   to read it by itself.

Fixes #3616.
2020-03-22 00:33:22 +01:00
muxator
a1978d2245 Minify: on errors, generate logs instead of simply silencing them
This will help when we'll have to update clean-css.
2020-03-20 22:46:39 +01:00
John McLear
c316402d86 PadMessageHandler: use a predefined color when authorInfo.colorId is not defined
For some reason authorInfo is sometimes null, and therefore it is not possible
to get colorId from it.

This resulted in the following stack trace:
    [2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'colorId' of null
    at <BASEDIR>/src/node/handler/PadMessageHandler.js:1199:37
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Promise.all (index 0)
    at async handleClientReady (<BASEDIR>/src/node/handler/PadMessageHandler.js:1171:5)
    [2020-03-16 09:27:17.291] [ERROR] console - (node:1746) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 76)
    [2020-03-16 09:27:19.034] [WARN] message - Dropped message, USERINFO_UPDATE Session not ready.[object Object]

Which is due to a bug in Etherpad that we are not going to solve now.

As a workaround, when this happens, let's set the username to "Anonymous" (if
it is not already set), and colorId to the fixed value "#daf0b2". Warning
messages are written in the logs to signal this condition.

This is no definitive solution, but fixes #3612 (via a workaround).
2020-03-20 22:32:06 +01:00
John McLear
b480416375 toolbar: the "star" button no longer disappears when visiting a read-only pad
Before this patch, visiting the read-only URL for a random pad would remove
the "Save Revision" (the "star" icon) from all the other RW pads. The only way
to make it appear again was to restart the server.

This change does not fix the underlying bug: after visiting a read only link
the "star" button would still disapper, but it is explictly reinserted via an
ad-hoc condition.

Fixes #3702
2020-03-19 22:42:22 +01:00
John McLear
8261229323 pass file ending 2020-03-17 13:08:46 +00:00
Tudor Constantin
28102d8e1f ImportHandler: importing files with unknown extension work again when allowUnknownFileEnds is true
By specification, when settings.allowUnknownFileEnds is true and the user tries
to import a file with an unknown extension (this includes no extension),
Etherpad tries to import it as txt.

This broke in Etherpad 1.8.0, that abruptly terminates the processing with an
UnhandledPromiseRejectionWarning.

This patch restores the intended behaviour, and allows to import as text a file
with an unknown extension (on no extension).

In order to catch the UnhandledPromiseRejectionWarning we had to use
fsp_rename(), which is declared earlier in the code and is promised based
instead of fs.rename(), which is callback based.

Fixes #3710.
2020-03-17 12:41:18 +01:00
John McLear
a0579c90db APIHandler: return HTTP/404 when non existing API methods are invoked
Before this change, invoking a non existing API method would return an HTTP/200
response with a JSON payload {"code":3,"message":"no such function"}.

This commit changes the HTTP status code to 404, leaving the payload as-is.

Before:
   curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"
  < HTTP/1.1 200 OK
  < X-Powered-By: Express
  [...]
  {"code":3,"message":"no such function","data":null}

After:
   curl --verbose "http://localhost:9001/api/1/notExisting?apikey=ABCDEF"

   < HTTP/1.1 404 OK
   < X-Powered-By: Express
   [...]
   {"code":3,"message":"no such function","data":null}

Fixes #3546.
2020-03-15 09:26:44 +00:00
John McLear
467fc11b72 fix 2020-03-14 21:58:26 +01:00
muxator
3b24c97d1e db/SecurityManager.js: accessing without session a public group pad no longer causes a crash
Steps to reproduce (via HTTP API):
1. create a group via createGroup()
2. create a group pad inside that group via createGroupPad()
3. make that pad public calling setPublicStatus(true)
4. access the pad via a clean web browser (with no sessions)
5. UnhandledPromiseRejectionWarning: apierror: sessionID does not exist

This was due to an overlook in 769933786c: "apierror: sessionID does not
exist" may be a legal condition if we are also visiting a public pad. The
function that could throw that error was sessionManager.getSessionInfo(), and
thus it needed to be inside the try...catch block.

Please note that calling getText() on the pad always return the pad contents,
*even for non-public pads*, because the API bypasses the security checks and
directly talks to the DB layer.

Fixes #3600.
2019-12-26 00:30:43 +01:00
muxator
a817acbbcc security: when served over https, set the "secure" flag for "express_sid" and "language" cookie
The mechanism used for determining if the application is being served over SSL
is wrapped by the "express-session" library for "express_sid", and manual for
the "language" cookie, but it's very similar in both cases.

The "secure" flag is set if one of these is true:

1. we are directly serving Etherpad over SSL using the native nodejs
   functionality, via the "ssl" options in settings.json

2. Etherpad is being served in plaintext by nodejs, but we are using a reverse
   proxy for terminating the SSL for us;
   In this case, the user has to be instructed to properly set trustProxy: true
   in settings.json, and the information wheter the application is over SSL or
   not will be extracted from the X-Forwarded-Proto HTTP header.

Please note that this will not be compatible with applications being served over
http and https at the same time.

The change on webaccess.js amends 009b61b338, which did not work when the SSL
termination was performed by a reverse proxy.

Reference for automatic "express_sid" configuration:
https://github.com/expressjs/session/blob/v1.17.0/README.md#cookiesecure

Closes #3561.
2019-12-07 04:36:01 +01:00
muxator
b82816c774 express: reformat session configuration in preparation for the next commit
No functional changes.
2019-12-07 04:22:54 +01:00
muxator
a51684b022 security: stop setting the "io" cookie
The "io" cookie is created by socket.io, and its purpose is to offer an handle
to perform load balancing with session stickiness when the library falls back to
long polling or below.

In Etherpad's case, if an operator needs to load balance, he can use the
"express_sid" cookie, and thus "io" is of no use.

Moreover, socket.io API does not offer a way of setting the "secure" flag on it,
and thus is a liability.

Let's simply nuke it.

References:
  https://socket.io/docs/using-multiple-nodes/#Sticky-load-balancing
  https://github.com/socketio/socket.io/issues/2276#issuecomment-147184662 (not totally true, actually, see above)
2019-12-07 04:20:12 +01:00
ahmadine
0a0b90c4d0 referer: change referrer policy. Stop sending referers as much as possible
Pull request with discussion: https://github.com/ether/etherpad-lite/pull/3636

What's already there:
* `meta name=referrer`: already done in 1.6.1:
  https://github.com/ether/etherpad-lite/pull/3044

  https://caniuse.com/#feat=referrer-policy
  https://w3c.github.io/webappsec-referrer-policy/#referrer-policy-delivery-meta
  (Chrome>=78, Firefox>=70, Safari>=13, Opera>=64, ~IE[1], ~Edge[1])

The previous two commits (by @joelpurra) I backported in this batch:
* `<a rel=noreferrer>`: a pull request denied before:
  https://github.com/ether/etherpad-lite/pull/2498

  https://html.spec.whatwg.org/multipage/links.html#link-type-noreferrer
  https://developer.mozilla.org/en-US/docs/Web/HTML/Link_types
  (Firefox>=37, I can't find more info about support)

This commit adds the following:
* `<a rel="noopener">`: fixing a not-so-well-known way to extract referer
  https://html.spec.whatwg.org/multipage/links.html#link-type-noopener
  (Chrome>=49, Firefox>=52, Safari>=10.1, Opera>=36, !IE, !Edge)

* `Referrer-Policy: same-origin`: the last bastion of referrer security
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
  (Chrome>=61, Firefox>=52, Safari>=11.1, Opera>=48, !IE, !Edge)

meta name=referrer wasn't enough. I happened to leak a few referrers with my
Firefox browser, though for some browsers it could have been enough.

[1] IE>=11, Edge>=18 use a different syntax for meta name=referrer, making it
    most probably incompatible (but I may be wrong on that, they may support
    both, but I have no way to test it currently). The next Edge release will be
    based on Chromium, so for that the Chrome version applies.
2019-11-25 00:05:40 +01:00
Joel Purra
2a44c83250 referer: exported html pads no longer leak URL/location through referer header
Exported HTML can, when loaded from disk or an online server, also leak the
location. Applying the `rel="noreferrer"` HTML5 standard mitigate the problem
for compatible browsers.

https://html.spec.whatwg.org/multipage/links.html#link-type-noreferrer

This commit was originally part of https://github.com/ether/etherpad-lite/pull/2498
2019-11-25 00:05:40 +01:00
muxator
28a3bba4c1 settings: fix incorrect implementation of conditional user creation.
The change that implemented #3648 (7c099fef5e) was incorrect, and resulted
in disabling every user at startup.

The problem was twofold:
1. _.filter() on an object returns an array of the object's enumerable values
   and strips out the keys, see: https://stackoverflow.com/questions/11697702/how-to-use-underscore-js-filter-with-an-object
   To filter an object, the function that needs to be used is _.pick();

2. The logic condition on userProperties.password was plain wrong (it should
   have been an AND instead of an OR).

This change corrects 1) and 2), and writes more specific logs when something
goes wrong.

Closes #3661.
2019-11-02 22:57:52 +01:00
muxator
c4564fba4b settings: rename a parameter. No functional changes.
Please note that the logic of this functionality is incorrect: this change is in
preparation of the next commit, which fixes it.
2019-11-02 23:12:35 +01:00
muxator
283100db6b runtime: deprecate node 8.x. Require node >= 10.13.0 starting from Etherpad 1.8.3
Nodejs 8 will be EOLed on December 31th, 2019 (https://github.com/nodejs/Release).

This means any future Etherpad version released from 2020 on should require at
least the next LTS (10.13.0). Let's keep some margin and decide that the first
Etherpad version dropping node 8 compatibility will be 1.8.3.

Closes #3650.
2019-10-20 00:02:00 +02:00
muxator
312c72c364 formatting: bulk remove trailing whitespaces
Do not touch vendorized files (e.g. libraries that were imported from external
projects).

No functional changes.

Command:
    find . -name '*.<EXTENSION>' -type f -print0 | xargs -0 sed -i 's/[[:space:]]*$//'
2019-10-20 02:09:22 +02:00
Ray Bellis
fc661ee13a core: allow URL parameters and POST bodies to co-exist.
Node 8.14.0 prohibits HTTP headers that exceed 8 KB (source:
https://nodejs.org/en/blog/vulnerability/november-2018-security-releases/#denial-of-service-with-large-http-headers-cve-2018-12121).

This patch allows for the parameters within the body of an HTTP POST request to
be used in addition to those within the URL (and will override them).

Closes #3568.

---
Muxator 2019-10-19:
- this commit was cherry-picked from 882b93487f
- it was modified to include the necessary changes in the documentation
2019-06-27 00:52:53 +02:00
muxator
7c099fef5e settings: do not create a user if he has no password field, or if his password is null.
This will be used by the settings.json in the default Dockerfile to eschew
creating an admin user when no password is set.

Closes #3648.
2019-10-19 00:54:56 +02:00
Moritz Jordan
0a8e32563b Fix Unicode bug in HTML export 2019-08-12 00:41:17 +02:00
Richlv
2c9383b69e minor typo fix 2019-08-08 21:58:30 +02:00
Tristram Gräbener
357780d573 Display the version in the web interface
In the settings drop-down this adds an “About” section that also shows
the commit if "exposeVersion" is set to true.

Fixes #2968
2019-04-15 23:17:34 +00:00
Tristram Gräbener
28a6f505c5 Parameters: the version is exposed in http header only when configured
Currently the version is exposed in a 'Server' http headers.

This commit allows to parameterize it in the settings. By defaults it is
not exposed.

Fixes #3423
2019-04-15 23:17:34 +00:00
muxator
705cc6f5e4 Change everywhere the link to https://etherpad.org (it was plain http) 2019-04-16 00:54:54 +02:00
muxator
75a0f339e1 Settings.js, express.js: trivial reformatting
Future commits by Tristram Gräbener will modify them.
2019-04-16 00:17:56 +02:00
muxator
dc7e49f89d Remove trailing whitespaces
Hoping to minimize future diffs. Not touching vendorized libraries.
2019-04-16 00:34:29 +02:00
muxator
cbd393d56b handler/PadMessageHandler.js: handleMessage() got the wrong padId for read only pads
This was almost guaranteed to be broken.
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
c2d8ca212b utils/Minify.js: always call statFile() with an explicit value for "dirStatLimit"
In this way the only external call to statFile() provides an explicit value for
"dirStatLimit", and thus the initial check on "undefined" at the start of the
function could be removed (just added a comment for now).
2019-03-27 18:29:12 +01:00
muxator
cdd4978973 utils/Minify.js: removed unused parameter "next" in minify()
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
5d067406b1 utils/Minify.js: removed unused parameter "redirectCount" in requestURI()
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
b2d00ae071 db/API.js: customeError -> customError
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
aa5e302d99 db/API.js: missing "let"
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
b9e537ca4f db/Pad.js: removed unreachable return statement
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
4040813447 db/Pad.js: prototype.copy(), removed redundant callback argument
This would cause a crash when calling pad.remove().
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
53b3328b5f express/padreadonly.js: missing "let"
Found by the Typescript compiler when doing an experimental conversion.
2019-03-27 18:29:12 +01:00
muxator
b8df6ca60c handler/PadMessageHandler.js: shuffle around some comments
No functional changes
2019-03-27 18:29:12 +01:00
muxator
2955740a6e Settings.js: support syntax for default values
+---------------------------+---------------+------------------+
| Configuration string in   | Value of      | Resulting confi- |
| settings.json             | ENV_VAR       | guration value   |
|---------------------------|---------------|------------------|
| "${ENV_VAR}"              | "some_string" | "some_string"    |
| "${ENV_VAR}"              | "9001"        | 9001             |
| "${ENV_VAR}"              | undefined     | null             |
| "${ENV_VAR:some_default}" | "some_string" | "some_string"    |
| "${ENV_VAR:some_default}" | undefined     | "some_default"   |
+---------------------------+---------------+------------------+

Mention this briefly in the main README.md, also.

Closes #3578.
2019-03-21 23:32:08 +01:00
muxator
c3bca6506e Settings.js: extracted into coerceValue() the logic for string -> number|bool conversion
This will be user in a later commit for implementing support for default values
2019-03-21 23:32:08 +01:00
muxator
59b1eed4a8 Settings.js: rephrased a log message 2019-03-21 23:32:08 +01:00
muxator
21ac37170e doc: rephrase settings.json.template and Settings.js
Better document current behaviour.
In this revision, ENV_VAR are supported, default values are not.
2019-03-21 23:32:08 +01:00
David Mehren
43c4fa9c2e Await padManager.getPad in getPadLines 2019-03-16 09:07:06 +01:00
muxator
6d400050a3 Settings.js: support configuration via environment variables.
All the configuration values can be read from environment variables using the
syntax "${ENV_VAR_NAME}".
This is useful, for example, when running in a Docker container.

EXAMPLE:
   "port":     "${PORT}"
   "minify":   "${MINIFY}"
   "skinName": "${SKIN_NAME}"

Would read the configuration values for those items from the environment
variables PORT, MINIFY and SKIN_NAME.

REMARKS:
Please note that a variable substitution always needs to be quoted.
   "port":   9001,          <-- Literal values. When not using substitution,
   "minify": false              only strings must be quoted: booleans and
   "skin":   "colibris"         numbers must not.

   "port":   ${PORT}        <-- ERROR: this is not valid json
   "minify": ${MINIFY}
   "skin":   ${SKIN_NAME}

   "port":   "${PORT}"      <-- CORRECT: if you want to use a variable
   "minify": "${MINIFY}"        substitution, put quotes around its name,
   "skin":   "${SKIN_NAME}"     even if the required value is a number or a
                                boolean.
                                Etherpad will take care of rewriting it to
                                the proper type if necessary.

Resolves #3543
2019-03-11 08:11:30 +01:00
muxator
f96e139b17 Settings.js: factored out parseSettings()
No functional changes.
2019-03-11 08:11:30 +01:00
muxator
6353768256 Settings.js: factored out storeSettings()
Grouped copied & pasted code into a single function.
2019-03-11 08:11:30 +01:00
muxator
ab57edef33 Settings.js: exit gracefully if an invalid credentials.json is passed.
Before this commit, when passed a malformed credentials.json the application
crashed with a stack dump. Now we catch the error and fail in a controlled way
(like already done for settings.json).

Example of exception we no longer throw:
  MALFORMEDJSON
  ^

  SyntaxError: Unexpected token M in JSON at position 0
      at JSON.parse (<anonymous>)
      at Object.reloadSettings (<BASEDIR>/src/node/utils/Settings.js:390:24)
      at Object.<anonymous> (<BASEDIR>/src/node/utils/Settings.js:543:9)
      at Module._compile (module.js:635:30)
      at Object.Module._extensions..js (module.js:646:10)
      at Module.load (module.js:554:32)
      at tryModuleLoad (module.js:497:12)
      at Function.Module._load (module.js:489:3)
      at Module.require (module.js:579:17)
      at require (internal/module.js:11:18)
2019-03-11 08:11:30 +01:00
muxator
8fa52659f5 Settings.js: trivial rewording of abiword and soffice (libreoffice) error messages 2019-03-11 08:11:30 +01:00