Commit graph

1714 commits

Author SHA1 Message Date
Richard Hansen
d5c6a44d9c PadMessageHandler: Improve documentation of sessioninfos 2021-05-03 01:35:11 -04:00
Richard Hansen
4c4415e14a PadMessageHandler: Register activePads metric only once 2021-05-03 01:35:11 -04:00
Richard Hansen
8f236b8687 Minify: Avoid crash due to unhandled Promise rejection if stat fails 2021-05-03 01:26:49 -04:00
Richard Hansen
e8df643d75 Minify: Treat ENOTDIR like ENOENT when statting a file
This avoids an exception when require-kernel requests a path like
`existing-file.js/index.js`.
2021-05-03 01:26:48 -04:00
Richard Hansen
aaacbd3a7a Minify: Refactor requestURI() for readability 2021-05-03 01:26:48 -04:00
Richard Hansen
ea8846154f favicon: Redo favicon customization 2021-04-20 13:33:55 -04:00
Richard Hansen
92e0bff80c favicon: Refactor handler and add tests 2021-04-20 13:33:55 -04:00
Richard Hansen
d0d4b95980 favicon: Only serve from /favicon.ico 2021-04-20 13:33:54 -04:00
webzwo0i
3a5af19492 AuthorManager: await for more db methods 2021-04-20 11:27:44 -04:00
webzwo0i
35797e57fc AuthorManager: await db.set in createAuthor 2021-04-20 11:27:44 -04:00
Chocobozzz
a001a13411 fix(perf): Disable wtfnode dump by default
Consumes a lot of CPU so it's better to enable it on purpose
2021-04-13 16:01:41 +02:00
Richard Hansen
951d369e3f padaccess: Delete useless try/catch 2021-04-12 22:51:06 -04:00
Richard Hansen
329d037431 Simplify read-only pad ID checks 2021-04-12 22:51:06 -04:00
pcworld
3c71e8983b Fix read only pad access with authentication
Before this commit, webaccess.checkAccess saved the authorization in
user.padAuthorizations[padId] with padId being the read-only pad ID,
however later stages, e.g. in PadMessageHandler, use the real pad ID for
access checks. This led to authorization being denied.

This commit fixes it by only storing and comparing the real pad IDs and
not read-only pad IDs.

This fixes test case "authn user readonly pad -> 200, ok" in
src/tests/backend/specs/socketio.js.
2021-04-12 22:51:06 -04:00
Richard Hansen
1ad134a538 PadMessageHandler: Improve logging of pre-CLIENT_READY drops
This should make it easier to see what is emitting the the messages so
it can be fixed.
2021-04-09 18:43:02 +02:00
Richard Hansen
91e99c84ca import: Reduce log spam from unsupported elements 2021-04-09 18:43:02 +02:00
Richard Hansen
09c349e2a1 import: Use a Set for supported elements 2021-04-09 18:43:02 +02:00
webzwo0i
e483b91916 Don't make browsers fail on sync-xhr until require-kernel is dropped 2021-04-05 04:34:29 -04:00
Richard Hansen
eb3cff5b3a LibreOffice: Remove unnecessary callbackification
`async.queue` will do the right thing if passed an `async` function.
2021-03-20 20:29:55 +00:00
Richard Hansen
0233399fdf Abiword: Avoid calling stdoutCallback multiple times 2021-03-20 20:29:55 +00:00
Richard Hansen
3ad1d0a74f cssmanager: Refactor CSS manager creation to avoid race condition
Safari takes a while to initialize `document.styleSheets`, which
results in a race condition when loading the pad. Avoid the race
condition by accessing the CSSStyleSheet objects directly from the
HTMLStyleElement DOM objects.
2021-03-20 01:07:16 +00:00
Richard Hansen
b4e1e935e2 LibreOffice: Log conversion errors 2021-03-18 09:02:28 +00:00
Richard Hansen
b2c0837cf5 import/export: Promisify Abiword and LibreOffice conversion 2021-03-18 09:02:28 +00:00
Richard Hansen
b321267e66 LibreOffice: Use the async-provided callback to signal errors
This avoids having two callbacks, which improves readability.
2021-03-18 09:02:28 +00:00
Richard Hansen
b914a46a87 LibreOffice: Use async.series to properly handle conversion errors 2021-03-18 09:02:28 +00:00
Richard Hansen
a6d5611c80 LibreOffice: Use consistent intermediate filename 2021-03-18 09:02:28 +00:00
Richard Hansen
ad0be9d1d2 LibreOffice: Add missing fileExtension property on intermediate step 2021-03-18 09:02:28 +00:00
Richard Hansen
5eab3a123d Abiword: Use the async-provided callback to signal errors
This avoids having two callbacks, which improves readability.
2021-03-18 09:02:28 +00:00
Richard Hansen
8d32463915 Abiword: Fix logging of conversion failure 2021-03-18 09:02:28 +00:00
Richard Hansen
f015f59cfc Abiword: Reduce log spam 2021-03-18 09:02:28 +00:00
Richard Hansen
259ee4a987 Abiword: Don't call the callback if null 2021-03-18 09:02:28 +00:00
Richard Hansen
83f39289aa import/export: On export error return 500 instead of crashing 2021-03-18 09:02:28 +00:00
Richard Hansen
3a11e97758 import/export: Spelling fix: "convertor" -> "converter" 2021-03-18 09:02:28 +00:00
Richard Hansen
50fdadab7d ExportHandler: Pass the error unmodified 2021-03-18 09:02:28 +00:00
Richard Hansen
216aecd433 import/export: Use Error objects for errors, not strings 2021-03-18 09:02:28 +00:00
Richard Hansen
59c167e31b ExportHandler: Replace unnecessary exception with return 2021-03-18 09:02:28 +00:00
Richard Hansen
785b7d2b44 Abiword: Reset stdout buffer when starting abiword 2021-03-18 09:02:28 +00:00
Richard Hansen
b6c2586920 import/export: Delete unnecessary comments 2021-03-18 09:02:28 +00:00
Richard Hansen
9b82d1d37d server: Log stats (metrics) on fatal error
This might help users troubleshoot rare crashes.
2021-03-11 07:38:44 +00:00
Richard Hansen
fcf43a7089 stats: Expose ueberDB metrics 2021-03-08 22:32:39 +00:00
webzwo0i
4ca989a255
sessions: add more endpoints that do not need a session (#4921)
* add more endpoints that do not need a session

* Update src/node/hooks/express/webaccess.js

Co-authored-by: Richard Hansen <rhansen@rhansen.org>

* Update src/node/hooks/express/webaccess.js

Co-authored-by: Richard Hansen <rhansen@rhansen.org>

Co-authored-by: John McLear <john@mclear.co.uk>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
2021-03-05 07:48:33 +00:00
Richard Hansen
0aad3b74da pluginfw: Improve rendering of hook list
There are two main benefits:
  * HTML is no longer printed in the startup debug logs.
  * `require()` is no longer called on client-side files. This
    eliminates "Failed to load <file> for <plugin>: ReferenceError:
    window is not defined" errors when users visit
    `/admin/plugins/info`.
2021-03-03 11:19:37 +00:00
John McLear
f95b09e0b6
Import: Import don't show warnings for supported elements 2021-03-02 17:14:47 +00:00
Richard Hansen
797ffa5600 Minify: Avoid path.relative()
Constructing a relative pathname on Windows is problematic because the
two absolute pathnames might be on different drives (or UNC paths).
Use `path.resolve()` instead of `path.join()` where appropriate to
avoid the need to construct a relative path.
2021-03-02 06:20:10 +00:00
Richard Hansen
b0862cd030 ace: Delete all $$INCLUDE_CSS logic
The intention of the deleted code was to reduce the number of fetches,
but it only saved a single fetch due to implementation flaws. The
right way to reduce the number of fetches is to use a bundling
technology such as webpack, and this change makes it easier to do so.
2021-03-01 14:32:33 +00:00
Richard Hansen
99b3918f2c Minify: Compatibility for all vendors/*.js files 2021-02-28 11:12:30 +00:00
webzwo0i
377883db98
fix pads with spaces (#4884) 2021-02-27 22:34:43 -05:00
webzwo0i
01dd9f5440 speed up page load 2021-02-27 16:28:12 +00:00
Richard Hansen
f86df5322e CachingMiddleware: Asyncify 2021-02-27 14:03:09 +01:00
Richard Hansen
0284d49522 CachingMiddleware: Switch to ES6 class syntax 2021-02-27 14:03:09 +01:00
Richard Hansen
80af66543a lint: Move up respond() in caching_middleware.js 2021-02-27 14:03:09 +01:00
Richard Hansen
7ab3ee2121 lint: src/node/utils/caching_middleware.js 2021-02-27 14:03:09 +01:00
Richard Hansen
392d9dcfde PadMessageHandler: Fix fetching of socket.io Sockets for a pad 2021-02-27 08:46:49 +00:00
Richard Hansen
9cd67cd990 PadMessageHandler: Delete unnecessary use of Promise.then() 2021-02-27 08:46:49 +00:00
webzwo0i
15dba7d886
move underscore to its old place and remove unnecessary packages (#4876) 2021-02-27 00:10:53 -05:00
John McLear
c0ec28f781 reversecompat: underscore
Backward compatibility for plugins that were written when underscore lived at src/static/js/underscore.js.
2021-02-26 11:03:19 +00:00
webzwo0i
efd211bbc5 remove useless semicolon 2021-02-25 16:59:06 +00:00
webzwo0i
fa29858a4e avoid manually including require-kernel in ace.js 2021-02-25 16:59:06 +00:00
Richard Hansen
ba5d8369bf Minify: Consistently use path.join() to build pathnames
This defends against extraneous or missing slashes, and it might
improve the experience on Windows.
2021-02-25 10:14:48 +00:00
Richard Hansen
2d3469e3ee Minify: Improve pathname sanitization
For context, see:
https://nvd.nist.gov/vuln/detail/CVE-2015-3297
9d4e5f6e35
https://github.com/ether/etherpad-lite/issues/2614
2021-02-25 10:14:48 +00:00
Richard Hansen
0cce4ae536 Minify: Also serve jquery.js from old path for compatibility 2021-02-25 10:14:48 +00:00
Richard Hansen
dabff9be77 run_cmd: Fix PATH debug log message 2021-02-23 21:41:32 -05:00
John McLear
0f16e518ff
api: drop JSONP (#4835)
* api: drop JSONP

* docs: drop JSONP

* tests: drop JSONP

* api: remove isValidJSONPName require
2021-02-22 09:10:02 +00:00
John McLear
ce83181ac3
Lgtm bugfixes (#4838)
* code tidy up: always evaluates

* tidy up: is always true

* tidy up: remove unused code

* always true/false variables

* unused variable

* tidy up: remove unused code in caretPosition.js

* for squash: Revert "tidy up: remove unused code in caretPosition.js"

The `if` condition was previously always true, so the body should be
preserved. If the body is preserved, other logic can be deleted. I
opened PR #4845 to clean it all up.

This reverts commit 75b03e5a7d.

* for squash: simplify

* for squash: Explain that the getter is used for its side effects

It's very weird to call a getter without using its return value. Add a
comment explaining why this is done so that the reader doesn't get
confused.

* for squash: Revert "tidy up: remove unused code"

The exception test was the purpose of the code.

This reverts commit 85153b1676.

* for squash: Log the tsort results

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
2021-02-22 08:26:35 +00:00
John McLear
40d7480d5b lint: padaccess.js 2021-02-21 21:09:02 -05:00
John McLear
029729a386 lint: Settings.js 2021-02-21 21:09:02 -05:00
John McLear
435562299f lint: NodeVersion.js 2021-02-21 21:09:02 -05:00
John McLear
7352dc7571 lint: ImportEtherpad.js 2021-02-21 21:09:02 -05:00
John McLear
613c7d8545 lint: ExportTxt.js 2021-02-21 21:09:02 -05:00
John McLear
d2359be08b lint: ExportHtml.js 2021-02-21 21:09:02 -05:00
John McLear
5718c8b360 lint: AbsolutePaths.js 2021-02-21 21:09:02 -05:00
John McLear
b1614f0592 lint: i18n.js
Partial, still 3 more to do that are slightly higher hanging that can get done.
2021-02-21 21:09:02 -05:00
John McLear
586af5e16e lint: padurlsanitize.js 2021-02-21 21:06:38 -05:00
John McLear
86c938cae2 lint: openapi.js 2021-02-21 21:06:38 -05:00
John McLear
25d4faddd9 lint: SocketIORouter.js 2021-02-21 21:06:38 -05:00
John McLear
5201cb717f lint: PadMessageHandler.js 2021-02-21 21:06:38 -05:00
John McLear
d67f170c46 lint: eejs/index.js 2021-02-21 21:06:38 -05:00
John McLear
3ed4ac649c lint: PadManager.js 2021-02-21 21:02:59 -05:00
John McLear
1b8cd0747d
Move vendor libraries to /vendors folder and exclude from LGTM 2021-02-21 15:07:39 +00:00
webzwo0i
0bb3e65020 fix for caching plugin-definitions 2021-02-21 14:31:15 +00:00
John McLear
bb14775820 drop apiRoot object from build 2021-02-21 11:08:07 +00:00
John McLear
ee2b32281c
pluginfw: Warn plugins on missing plugin (#4826)
* pluginfw: Warn plugins on missing plugin

Add functionality to console.warn when a plugin is missing.  This will help admins know when people are trying to use plugins that are missing.  Resolves https://github.com/ether/etherpad-lite/issues/4730

* pluginfw: importing .etherpad can notify admins of missing plugins

Extending .etherpad imports to notify admins if a missing plugin is present

* Update ImportEtherpad.js
2021-02-21 11:07:13 +00:00
Richard Hansen
b3b5af3c3c plugins: Use npm CLI to install/uninstall plugins
Using npm as a module has long been discouraged and will stop working
with npm v7.
2021-02-18 19:18:59 +00:00
Richard Hansen
1cfbf88f7c run_cmd: Enhance with ability to return stdout as string 2021-02-18 19:18:59 +00:00
Richard Hansen
d8bb5aa009 plugins: Eliminate unnecessary run_npm.js
I had anticipated more shared logic than we actually need (the
abstraction in `run_npm.js` is YAGNI).
2021-02-18 19:18:59 +00:00
Richard Hansen
426c025127 run_cmd: Log to Etherpad logs by default 2021-02-18 19:18:59 +00:00
Richard Hansen
dcf7891316 plugins: Improve logging of plugin events
This will make it easier to troubleshoot plugin and npm issues.
2021-02-18 19:18:59 +00:00
Richard Hansen
a45e85a730 Use settings.root to anchor pathnames 2021-02-18 19:18:59 +00:00
Richard Hansen
f868788417 Remove unnecessary path.normalize() calls
`path.join()` already normalizes.
2021-02-18 19:18:59 +00:00
Richard Hansen
84c1d74f8b server: Fix Gate constructor
The ECMAScript spec for `.then()` requires Promise subclass
constructors to take an executor.
2021-02-18 19:18:34 +00:00
Richard Hansen
4c6cb53d18 server: Improve log messages when exiting 2021-02-18 19:18:34 +00:00
Richard Hansen
fb745374c3 import: Improve error logging 2021-02-18 03:42:41 -05:00
Richard Hansen
00d45e3229 Defer rate limiter creation to a hook call
This makes it possible to change the rate limiter settings via
`/admin/settings` or by modifying the appropriate settings object and
reinvoking the hook.
2021-02-16 21:13:35 -05:00
Richard Hansen
d7ed71eba0 plugins: Fix "Error: spawn npm ENOENT" error on Windows
On Windows, npm should be invoked as `npm.cmd`, not `npm`. Use a
drop-in replacement for `child_process.spawn()` that does the right
thing on Windows.
2021-02-16 22:00:20 +00:00
John McLear
b7e88cb904 security: New setting for Socket.IO maxHttpBufferSize 2021-02-15 12:45:31 -05:00
Richard Hansen
ed93ef5636 /admin/settings: Reload plugins, call loadSettings hook on restart
This should match the normal startup procedure a bit more closely.
2021-02-15 08:43:14 +00:00
Egil
9c7dcb1d0a eejs: Upgrade ejs to the latest version
The type of ejs's `__output` variable is now string instead of array
of strings, so the handling of `__output` had to change.
2021-02-14 23:36:53 -05:00
John McLear
615e47114b Revert "socketio: increase socketio limit to 1MiB"
This reverts commit 55c96e5577.
2021-02-14 16:53:48 +00:00
Richard Hansen
b711ff6acf import: Ajaxify pad import
This eliminates an inline script (good for Content Security Policy)
and improves the user experience.
2021-02-14 08:35:38 +00:00
Richard Hansen
fba55fa6cf ImportHandler: Refactor doImport() for readability 2021-02-14 08:35:38 +00:00
Richard Hansen
28b28866a2 ImportHandler: Move the logger up
Also change the name to something shorter.
2021-02-14 08:35:38 +00:00
Richard Hansen
26b5a69ccc ImportHandler: Use JSON.stringify() to properly escape characters 2021-02-14 08:35:38 +00:00
Richard Hansen
ed80883709 ImportHandler: Lint the response script sent to the browser 2021-02-14 08:35:38 +00:00
Richard Hansen
0ff131bbbb ImportHandler: Throw Errors, not strings 2021-02-14 08:35:38 +00:00
Richard Hansen
908635a1de ImportHandler: Use return reject(...) to avoid double settle 2021-02-14 08:35:38 +00:00
Richard Hansen
e01059dce5 ImportHandler: Switch to fs/promises API 2021-02-14 08:35:38 +00:00
Richard Hansen
5b1b030906 ImportHandler: Use asynchronous rename instead of fs.renameSync() 2021-02-14 08:35:38 +00:00
Richard Hansen
c7b1abebe4 ImportHandler: Avoid deprecated fs.exists() function 2021-02-14 08:35:38 +00:00
Richard Hansen
008209b0e0 ImportHandler: Delete redundant variable 2021-02-14 08:35:38 +00:00
Richard Hansen
48205c1ddb import/export: Make sure Express sees async errors
Express v4.x does not check to see if a Promise returned from a
middleware function will be rejected, so explicitly pass the Promise
rejection reason to `next()`.

We can revert this change after we upgrade to Express v5.0.

See https://expressjs.com/en/guide/error-handling.html for details.
2021-02-14 08:35:38 +00:00
Richard Hansen
e674d9789e
express: Change httpUptime to httpStartTime (#4777)
It's better to provide a primitive value and let the consumer of the
metric do math if desired.

Co-authored-by: John McLear <john@mclear.co.uk>
2021-02-14 07:50:10 +00:00
Richard Hansen
ac52fb8a9d express: New httpUptime metric 2021-02-13 10:02:28 +00:00
John McLear
483f4344c2
performance: maxAge for favicon and plugin definitions (#4761) 2021-02-13 08:13:48 +00:00
Richard Hansen
d56a02c85a express: Forcibly terminate HTTP connections when restarting
This should make restarts via `/admin` actions (e.g., plugin
installation) more reliable.
2021-02-13 07:37:22 +00:00
John McLear
4c4c7b526d
performance: i18n maxage (#4759) 2021-02-13 02:35:25 -05:00
Richard Hansen
01c83917d1 socket.io: Manually track client connections/disconnections
This change is required for socket.io 3.x because in 3.x
`io.sockets.clients()` no longer returns all client Socket objects.
2021-02-13 07:13:37 +00:00
Richard Hansen
8f2f6593be lint: Re-run eslint --fix 2021-02-13 00:31:36 -05:00
John McLear
55c96e5577 socketio: increase socketio limit to 1MiB 2021-02-12 17:56:50 -05:00
Richard Hansen
73d31b12a8 Minify: Replace deprecated url.parse() with new URL() 2021-02-12 07:08:51 +00:00
Richard Hansen
7efca7dc7d Minify: Don't ignore request headers in requestURI() 2021-02-12 07:08:51 +00:00
Richard Hansen
7a003cb9e2 Minify: Let Express render the 500 error page 2021-02-12 07:08:51 +00:00
Richard Hansen
44e420b6c5 Minify: Return Date objects from statFile() 2021-02-12 07:08:51 +00:00
Richard Hansen
aa11667ff7 Minify: Use fs.promises 2021-02-12 07:08:51 +00:00
Richard Hansen
073052ac66 Minify: Asyncify minify() 2021-02-12 07:08:51 +00:00
Richard Hansen
3eefe71834 Minify: Don't set cache headers if statFile() causes 500 2021-02-12 07:08:51 +00:00
Richard Hansen
84190793dc Minify: Asyncify getFileCompressed() 2021-02-12 07:08:51 +00:00
Richard Hansen
dd7ea1a8f9 Minify: Asyncify statFile() 2021-02-12 07:08:51 +00:00
Richard Hansen
947dc8eeed Minify: Asyncify getFile() 2021-02-12 07:08:51 +00:00
Richard Hansen
5d7c07e81c Minify: Asyncify lastModifiedDateOfEverything() 2021-02-12 07:08:51 +00:00
Richard Hansen
e573276755 Minify: Asyncify getAceFile() 2021-02-12 07:08:51 +00:00
Richard Hansen
5cc191f185 Minify: Replace async.forEach() with Promise.all() 2021-02-12 07:08:51 +00:00
Richard Hansen
0c428e068e Minify: Use Promise.all() to simplify requestURIs() 2021-02-12 07:08:51 +00:00
Richard Hansen
1ec29e0d45 Minify: Asyncify requestURI() 2021-02-12 07:08:51 +00:00
Richard Hansen
a952df2cf5 Minify: Un-export requestURI()
Nobody outside this file uses it.
2021-02-12 07:08:51 +00:00
Richard Hansen
d9607f7c66 static: Asyncify 2021-02-12 07:08:51 +00:00
Richard Hansen
7f4a7156e2 Minify: Move getTar() to static.js
`static.js` is the only file that uses it.
2021-02-12 07:08:51 +00:00
Richard Hansen
996dc81825 Minify: Move tar processing into a function
This reduces the overhead of `require()`ing the module, and it will
make it easier for a future commit to asyncify everything in
`Minify.js`.
2021-02-12 07:08:51 +00:00
Richard Hansen
50929fe7f7 express: Call expressConfigure, expressCreateServer hooks asynchronously 2021-02-12 07:08:51 +00:00
Richard Hansen
8919f63c98 lint: Replace use of underscore.js with plain ECMAScript 2021-02-12 07:08:51 +00:00
John McLear
ab127289c4 security: limit socketio to 1M chars 2021-02-11 21:01:47 -05:00
Richard Hansen
ce1b69feda plugins: Helper function to run the npm CLI 2021-02-09 22:18:35 +00:00
Richard Hansen
83a519941b /admin/plugins: Fix logging of error messages 2021-02-09 22:18:35 +00:00
Richard Hansen
ebdb2798ff server: Fix handling of errors during startup and shutdown
Before, an unhandled rejection or uncaught exception during startup
would cause `exports.exit()` to wait forever for startup completion.
Similarly, an error during shutdown would cause `exports.exit()` to
wait forever for shutdown to complete. Now any error during startup or
shutdown triggers an immediate exit.
2021-02-09 08:57:24 +00:00
Richard Hansen
5999d8cd44 server: Only pass err arg to exports.exit()
The `uncaughtException` event handlers are passed two arguments: error
and "origin". The `exports.exit()` argument doesn't expect a second
argument.
2021-02-09 08:57:24 +00:00
Richard Hansen
5b327b63ac server: Simplify gating of state transition waiters 2021-02-09 08:57:24 +00:00
Richard Hansen
105f8b0ccb server: Use a log4js logger object for logging 2021-02-09 08:57:24 +00:00
Richard Hansen
1e3f352281 openapi: Turn down logging verbosity 2021-02-09 07:24:31 +00:00
John McLear
2b112ac851
tests: Admin Frontend Test Coverage(#4717)
Covers all frontend admin operations, runs separated in CI.
2021-02-07 11:32:57 +00:00
John McLear
e02246641e
ordered lists bugfix: first line item can be 0.*, don't show 0 as undefined (#4600) 2021-02-06 20:19:05 +00:00
John McLear
c969ae58c2
stats: activePads & lastDisconnected stats 2021-02-06 19:53:52 +00:00