All the configuration values can be read from environment variables using the
syntax "${ENV_VAR_NAME}".
This is useful, for example, when running in a Docker container.
EXAMPLE:
"port": "${PORT}"
"minify": "${MINIFY}"
"skinName": "${SKIN_NAME}"
Would read the configuration values for those items from the environment
variables PORT, MINIFY and SKIN_NAME.
REMARKS:
Please note that a variable substitution always needs to be quoted.
"port": 9001, <-- Literal values. When not using substitution,
"minify": false only strings must be quoted: booleans and
"skin": "colibris" numbers must not.
"port": ${PORT} <-- ERROR: this is not valid json
"minify": ${MINIFY}
"skin": ${SKIN_NAME}
"port": "${PORT}" <-- CORRECT: if you want to use a variable
"minify": "${MINIFY}" substitution, put quotes around its name,
"skin": "${SKIN_NAME}" even if the required value is a number or a
boolean.
Etherpad will take care of rewriting it to
the proper type if necessary.
Resolves#3543
Before this commit, when passed a malformed credentials.json the application
crashed with a stack dump. Now we catch the error and fail in a controlled way
(like already done for settings.json).
Example of exception we no longer throw:
MALFORMEDJSON
^
SyntaxError: Unexpected token M in JSON at position 0
at JSON.parse (<anonymous>)
at Object.reloadSettings (<BASEDIR>/src/node/utils/Settings.js:390:24)
at Object.<anonymous> (<BASEDIR>/src/node/utils/Settings.js:543:9)
at Module._compile (module.js:635:30)
at Object.Module._extensions..js (module.js:646:10)
at Module.load (module.js:554:32)
at tryModuleLoad (module.js:497:12)
at Function.Module._load (module.js:489:3)
at Module.require (module.js:579:17)
at require (internal/module.js:11:18)
ueberDB2 can return either undefined or null for a missing key, depending on
which DB driver is used. This patch changes the promise version of the API so
that it will always return null.
some code chunks previously used `async.parallel` but if you
use `await` that forces them to be run serially. Instead,
you can initiate the operation (getting a Promise) and then
_later_ `await` the result of that Promise.
If you use `await` inside a loop it makes the loop inherently serial.
If you omit the `await` however, the tasks will all start but the loop
will finish while the tasks are still being scheduled.
So, to make a set of tasks run in parallel but then have the
code block after the loop once all the tasks have been completed
you have to get an array of Promises (one for each iteration) and
then use `Promise.all()` to wait for those promises to be resolved.
Using `Array#map` is a convenient way to go from an array of inputs
to the require array of Promises.
NB1: needs additional review and testing - no abiword available on my test bed
NB2: in ImportHandler.js, directly delete the file, and handle the eventual
error later: checking before for existence is prone to race conditions,
and does not handle any errors anyway.
- removed possible issue with failing to sanitize `padName` if `padId` was also
supplied
- removed unnecessary `try` block
- simplified API and function name matching tests
Also converted the handler functions that depend on checkAccess() into async
functions too.
NB: this commit needs specific attention to it because it touches a lot of
security related code!
This patch also contains significant refactoring relating to error checking of
arguments supplied to the functions (e.g. rev) facilitated by use of `throw`
instead of nodeback errors.
1. This module was not migrated to Promises, because it is only used via
express-session, which can't actually use promises anyway.
2. all(), clear() and length() depend on the presence of the `db.forEach()`
function, which in ueberdb2 doesn't even exist.
Fortunately those three methods are optional, so I made their existence
conditional on the presence of `db.forEach`.
3. in SessionStore.clear(), replaced a call to db.db.remove() with db.remove()
Changed two occurrences of:
process.nextTick(function() {
if (fn) fn();
});
with
if (fn) {
process.nextTick(fn);
}
i.e. such that no function even gets added to the `nextTick` queue unless
there's actually a function to be called.
Extracted from Ray's work.
`getPadAccess()` (src/node/padaccess.js) is now "promise only", resolving to
`true` or `false` as appropriate, and throwing an exception if there's an
error.
The two call sites (padreadonly.js and importexport.js) updated to match.
Use real `async` instead of async.js where applicable.
The `getPluginTests()` function was never truly async anyway because it only
contains calls to synchronous `fs` modules.
Since this code can end up loaded in browsers when using client side plugins,
avoid use of ES6 syntax features such as arrow functions until MSIE support is
finally dropped.
This change is in preparation of the future async refactoring by Ray. It tries
to extract as many changes in boolean conditions as possible, in order to make
more evident identifying eventual logic bugs in the future work.
This proved already useful in at least one case.
BEWARE: this commit exposes an incoherency in the DB API, in which, depending
on the driver used, some functions can return null or undefined. This condition
will be externally fixed by the final commit in this series ("db/DB.js: prevent
DB layer from returning undefined"). Until that commit, the code base may have
some bugs.
This change extracts the grammar correction performed on the async branch,
anticipating them in a single commit. It cannot be folded with the previous
one, as it is not purely cosmetic.
This change is only cosmetic. Its aim is do make it easier to understand the
async changes that are going to be merged later on. It was extracted from the
original work from Ray Bellis.
To verify that nothing has changed, you can run the following command on each
file touched by this commit:
npm install uglify-es
diff --unified <(uglify-js --beautify bracketize <BEFORE.js>) <(uglify-js --beautify bracketize <AFTER.js>)
This is a complete script that does the same automatically (works from a
mercurial clone):
```bash
#!/usr/bin/env bash
set -eu
REVISION=<THIS_REVISION>
PARENT_REV=$(hg identify --rev "${REVISION}" --template '{p1rev}')
FILE_LIST=$(hg status --no-status --change ${REVISION})
UGLIFYJS="node_modules/uglify-es/bin/uglifyjs"
for FILE_NAME in ${FILE_LIST[@]}; do
echo "Checking ${FILE_NAME}"
diff --unified \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${PARENT_REV}" "${FILE_NAME}")) \
<("${UGLIFYJS}" --beautify bracketize <(hg cat --rev "${REVISION}" "${FILE_NAME}"))
done
```
This is documented to be more performant.
The substitution was made on frontend code, too (i.e., the one in /static),
because Date.now() is supported since IE 9, and we are life supporting only
IE 11.
Commands:
find . -name *.js | xargs sed --in-place "s/new Date().getTime()/Date.now()/g"
find . -name *.js | xargs sed --in-place "s/(new Date()).getTime()/Date.now()/g"
Not done on jQuery.
The guard condition on count being non negative and < 100 used the wrong
boolean operator. In its form it was impossible.
This error was introduced in 2013, in 5592c4b0fe.
Fixes#3499
The HTTP API doesn't ever omit arguments, it always passes `undefined` for a
parameter that wasn't supplied in the request.
The functions that were simplified are:
- getRevisionChangeset()
- getText()
- getHTML()
- saveRevision()
The only function still supporting optional arguments is getPadSafe(), which is
only called from this module.
This function was simulating two overloads:
1. copy(destinationID, force, callback)
2. copy(destinationID, callback), in this case "force" would be assumed false
But all the call sites always used the version with arity 3.
Thus, we can remove that optionality and always assume that the funcion will be
called with three parameters. This will simplify future work.
Moving classes to html tag so it can be used to style other part of template depending on plugins like #users, #chat etc...
Rename plugin class with "plugin-" prefix, because there were conflicts with some plugins using the same .ep_font_color class to apply css rules
- path.exists() is no longer part of nodejs
- fs.exists() is deprecated (as of nodejs >= 8)
- checking a file for existence before using it is open to raca condition. It is
preferable to go ahead and use the file, and eventually handle the error
- we can afford two simple synchronous fs operations here
Next version will be Etherpad 1.8. As planned in #3424, we are going to require
NodeJS >=8.9.0 and npm >= 6.4.
This commit implements that change and updates documentation and scripts.
Subsequent changes will get rid of old idioms, dating back to node < 0.7, that
still survive in the code.
Once migrated to NodeJS 8, we will be able to start working on migrating the
code base from callbacks to async/await, greatly simplifying legibility (see
#3540).
Closes#3557
Until Etherpad 1.7.5, process.on('SIGTERM') and process.on('SIGINT') were not
hooked up under Windows, because old nodejs versions did not support them.
This excluded the possibility of doing a graceful shutdown of the database
connection under that platform.
According to nodejs 6.x documentation, it is now safe to do so. This allows to
gracefully close the DB connection when hitting CTRL+C under Windows, for
example.
Source: https://nodejs.org/docs/latest-v6.x/api/process.html#process_signal_events
- SIGTERM is not supported on Windows, it can be listened on.
- SIGINT from the terminal is supported on all platforms, and can usually be
generated with <Ctrl>+C (though this may be configurable). It is not
generated when terminal raw mode is enabled.
A Windows manual install has the same directory layout of a normal Unix one
(e.g. the nice symlink node_modules/ep_etherpad-lite -> ../src).
Only when running from the pre-built Windows package the directory layout is
different (e.g. src is physically copied into node_modules/ep_etherpad-lite).
The previous version of the code wrongly assumed that all Windows installs would
be run from the pre-built pakage.
In this version the path search is the same on all platform. If it fails, and we
are on Windows, there is a fallback for the specific case of the pre-built
package.
Fixes#3550
This commit vastly shortens (and simplifies) the version table within
handler/APIHandler.js by building each version's entry incrementally based off
the previous version.
The resulting table has been validated by comparing the "before" and "after"
output of the following loop on both versions of the code (albeit with an
intermediate "sort" step to account for the different insertion order)
for (let v in version) {
let m = version[v];
for (let [k, a] of Object.entries(m)) {
console.log(v, k, a);
}
}
The patch also fixes a few typos, and removes a duplicate definition of
getChatHistory which in each applicable version was defined with two different
parameter lists, but where only the second would be used.
Compatibility with IE11 regressed in 23eab79946 while working for #3488.
That commit made use of modern js syntax, not supported by IE11.
- Removed arrow functions, replaced with normal functions.
- Removed the spread operator (<...iterable>) and the "new Set()" construct,
replaced with _.uniq()
At some point IE11 compatibility will be dropped.
Ditching it now, for such a small gain, is not wise.
Fixes#3500.
This is just a dev dependency, so no real risks, but it's better not to scare
users.
Reported vulnerability before this change:
$ npm audit
=== npm audit security report ===
# Run npm update cryptiles --depth 4 to resolve 1 vulnerability
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High │ Insufficient Entropy │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package │ cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ wd [dev] │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path │ wd > request > hawk > cryptiles │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info │ https://nodesecurity.io/advisories/720 │
└───────────────┴──────────────────────────────────────────────────────────────┘
Etherpad-lite relies on the user's browser to generate a random pad
name, but the current solution is not safe against collisions. In order
to generate unique pad names, the following modifications are made:
* use a PRNG instead of Math.random() and ensure uniform distribution
when selecting chars.
* choose the pad name length to achieve a specific number of bits of
security.
Closes: #3516
a) these rules:
[class^="icon-"]:before
[class*=" icon-"]:before
b) were the same as this one:
[data-icon]:before
except the rules in b) had a "content: attr(data-icon)" rule, too.
This commit groups all of them together, and gets rid of the "attr(data-icon)".
The commit that introduced these rules in the first place, and that are now
partially reverted, was 9aea689438 (move tiny bit
of font awesome we actually use into pad.css) from 2014-11-19.
Preparatory work for introducing colibris skin
This commit implements the following behaviour:
1. adds a function clientPluginNames() to hooks.js (mimicking what is done in
static.js), which returns an array containing the list of currently installed
client side plugins. The array is eventually empty.
2. calls that function in pad.html at rendering time (thus server-side) to
populate a class attribute.
Example results:
- with no client-side plugins installed:
<div id="editorcontainerbox" class="">
- with some client-side plugins installed:
<div id="editorcontainerbox" class="ep_author_neat ep_adminpads">
Looking at the existing code (src/node/hooks/express/static.js#L39-L57), a
client-side plugin is defined as a plugin that implements at least a client side
hook.
NOTE: there is currently no support for notifying plugin removal/installation
to the connected clients: for now, in order to get an updated class list,
the clients will have to refresh the page.
Fixes#3488
Since the original comparison compared for truthy and not for "===", and it's
3 AM now, I blindly negated it, in order to show how fragile it was in the first
instance.
No functional changes.
This is the final commit of this refactoring series.
Get rid of an else branch to simplify code layout. No functional changes at all.
==============
This series is an attempt to reduce the control structure depth of the code
base, maintaining at the same time its exact same behaviour, bugs included. It
is, in a sense, an initial attempt at a refactoring in the spirit of its
original definition [0].
The idea beyond this refactoring is that reducing the code depth and, sometimes,
inverting some conditions, bugs and logic errors may become easier to spot, and
the code easier to read.
When looked at ignoring whitespace changes, all of these diffs should appear
trivial.
[0] https://refactoring.com/
It's just synctactic sugar, but it is always better than executing string
concatenations in one's mind.
Do not do this with files in src/static, because we want to keep IE 11
compatibility.
The old "static/custom" directory is replaced by "static/skins/<skinName>",
where <skinName> is taken from settings.json.
When no value is found, a default of "no-skin" is assumed, so that backward
compatibility is maintained.
The most evident security concerns have been addressed.
Closes#3471.
skinName must be a single string (no directory separators in it) pointing to an
existing directory under /src/static/skins.
In case these conditions are not met, its value is rewritten to "no-skin".
Also, the value of skinName if sent to the client via clientVars for allowing
its use it in the browser.
Currently, an Etherpad skin requires the existence of 6 files:
- index.{css,js}
- pad.{css,js}
- timeslider.{css,js}
In the default empty skin (in static/custom), there were 2 small placeholders
({js,css}.template) to be copied in place by the startup script in case no skin
was in use.
Now that we are moving to multiple directories (see #3471) we can simply commit
the example files and remove the copying code from the startup script.
Not performing encoding/decoding when traversing logical domains is a security
risk.
String concatenation is not great, too, but this change is just focused on
allowing the implementation of skin support.
If you edit `src/templates/export_html.html` to remove the
`<meta name="changedby" content="Etherpad">` tag[1], PDF export with
soffice has a bug: the first word of the pad is deleted and a blank page
is inserted as first page (the pad's text begins on the second page).
The `--writer` soffice option avoids that bug.
[1] you may want to delete that tag since it is inserted as a comment in
.doc or .odt soffice export.
This file uses it for robots.txt and favicon.ico.
This makes use of the new stable settings.root introduced with #3466, and will
be modified when introducing support for custom skins.
This should look to consistent locations when looking for relative paths,
without depending on current working directory.
For absolute paths, nothing changes.
This is just a function (with an ugly side effect for caching purposes) that
heuristically tries to compute the Etherpad installation path when running under
Unix and win32 (they have different file system layouts).
This path can be used by Etherpad as a base for all the relative paths, in order
to be deterministic and not depending on cwd.
This is the location that is choosen by default when Etherpad starts with no
settings.json file.
It was different than the one contained into setting.json.template.
Version 2.x is not backwards compatible with 1.x.
However, according to [0], [1] and [2], it seems that the biggest concern is
when mixing different server and client versions, and this is not Etherpad's
case.
Smoke tested (successfully) on Firefox 61, Chromium 68.
npm audit before this change:
found 12 vulnerabilities (9 low, 3 high) in 8205 scanned packages
11 vulnerabilities require semver-major dependency updates.
1 vulnerability requires manual review. See the full report for details.
npm audit after this change:
found 1 low severity vulnerability in 8196 scanned packages
1 vulnerability requires manual review. See the full report for details.
Fixes#3462
[0] https://socket.io/blog/socket-io-2-0-0/
[1] https://github.com/socketio/socket.io/issues/3007#issuecomment-336791836
[2] a0d7a794de
Written the changelog and updated package.json.
From now on, releases will be cut from develop, and merged directly into master.
Each release will be a tag on the master branch (e.g. 1.7.0).
A "release/1.7.0" branch will eventually be created only if/when a hotfix will
be needed.
Minimum supported Node version is 6.9.0, but Object.values() was introduced in
Node < 7. Let's use a polyfill if needed.
This will be removed when minimum supported Node version is raised to 8.9.0.
Fixes#3459
When installing dependencies, npm informed us that measured had been deprecated,
and renamed to measured-core. Let's follow the advice, and get rid of the
warning.
npm WARN deprecated measured@1.1.0: This package has been renamed to
measured-core, all versions of measured have been re-released under
measured-core, please update your package and consider updating to the newest
version. See https://github.com/yaorg/node-measured for latest updates.
This package is used to expose a single endpoint ("/stats"), whose output does
not change after this commit.
Fixes#3458
The hostname:port of URIs used in Minify are currently bogus and refer
to localhost only for historical reasons; there's no reason to retain
them and omitting them avoids generating an invalid URI when "port" is
not an integer.
Context: settings.port is passed to express's listen; if not numeric, it
is used a filename for a Unix domain socket.
This allows e.g. starting a server to be reverse-proxied on a multi-user
system, using the filesystem to handle access control and avoiding need
to allocate port numbers.
Before this change, etherpad-lite starts without error when configured
to listen on a Unix domain socket in this manner. However, `pad.js` and
`ace2_common.js` are generated incorrecting, causing an error
"Uncaught Error: The module at "ep_etherpad-lite/static/js/rjquery" does not exist."
when loading the editor:
When settings.port is a non-numeric string, e.g. `etherpad.sock`, a URI
of the form `http://localhost:etherpad.sock/static/js/rjquery.js` is
generated and parsed to find the file needed. In this case, the file
searched for is `:etherpad.sock/static/js/rjquery.js`, rather than the
expected `static/js/rjquery.js`. No such file exists, and the required
code is silently omitted from the bundle.
As a workaround, hard-code a (meaningless) hostname which can be parsed
correctly, since the current code makes no use of it anyway.
Etherpad 1.6.6 does not run on node <= 5 already.
Node 6.9 is the first LTS release in the 6 series, and comes with npm 3.10.8.
Declarations in package.json are advisory unless the user has set
`engine-strict` config flag.
Updated the docs accordingly.
Without this change, lines that haven't ever been edited will have either
an empty class or, in the case of list start lines, a class that begins
with a space (because the `ace-line` before the space never got added).
These are the remaining non-whitespace changes needed to normalize package.json
formatting, bringing it in line with the npm 6.1.0 default format.
Future edits to this file should follow this default format, in order to
minimize churn.
Only cosmetic changes to make it easier to understand what changes in the other
commits.
This command:
git diff this-commit-hash^! --ignore-all-space
should give an empty output on this commit.
When npm saves packages.json, it sorts the dependencies alphabetically. This
change reorders them.
Its aim, togheter with the next ones, is to have a diff that is inspectable.
Moreover, the mutation of package.json by installDeps.sh will be disabled with
a future change.
When comparing original content with the changes made by the user, we
need to ignore some line attribs that are added by content collector,
otherwise we would consider the change started on the first char of the
line -- the '*' that is added when line has line attribs.
In order to be able to handle both #3354 and #3118, we need to take into
account both the styles attribs (to fix#3354) and the line attribs
defined by any of the plugins (to fix#3118), but we can ignore those
extra line attribs that are added by Etherpad and do not add any
functionality (`'lmkr', 'insertorder', 'start'`).
This change partially reverts 0a9d02562d, which got released in 1.6.4
due to #3280.
Text size and line alignment are now reverted back to their 1.6.3
appearance (thus stay non customizable, for now).
Fixes#3378
When using LibreOffice to convert pads to doc, we got `Error: no export filter for /tmp/xxxx.doc` (tested with LO 5 and 6). Maybe it's a regression from LO. Anyway, converting HTML to odt, then to doc works.
Thx to lpagliari for her review!
* Add scroll when it edits a line out of viewport
By default, when there is an edition of a line, which is out of the
viewport, Etherpad scrolls the minimum necessary to make this line
visible. This makes that the line stays either on the top or the bottom
of the viewport. With this commit, we add a setting to make possible to
scroll to a position x% pixels from the viewport. Besides of that, we
add a setting to make an animation of this scroll.
If nothing is changed on settings.json the Etherpad default behavior is
kept
Shut down database connection and exit the node process
when SIGTERM is encountered. This is especially important
when nodejs is run as PID1, e.g. in a docker container.
Shutting down connections to clients (browsers) is beyond
this patche's scope.
Resolves#3265
* Added support for the Montserrat Font as discussed in #3201
I chose a thin version of the Font (namely Montserrat Light) as the regular font and Montserrat Regular as the bold version. The thin fonts just look better in my opinion.
I added RobotoMono-Regular and RobotoMono-Bold as TrueTypeFont files and included them into the list. I tred to stay in alphabetical order where possible.
The author of #3201 was asking for a more modern monospaced font.
Fix previous commit. As "force reconnect" buttons have all the same id
on DOM, on the previous commit we were only disallowing the first button
with that id on DOM -- "userdup" -- to be closed by a click on editor.
Casually the tests were using the same error to simulate a "force
reconnect", so even the tests were not getting the issue.
Users still cannot choose which shortcuts they want to enable/disable,
so it does not make sense (yet) to have a cookie with that preference.
This can be reverted once we create an UI to change shortcuts, but
PLEASE PLEASE PLEASE do not read the cookie every time handleKeyEvent is
called!!!
This is an adjustment to #2891.
If a "force reconnect" message is displayed to the user, it means the
only way to go back to a healthy state is to reload the pad. So we
cannot hide this kind of message, like what is done with other modals
(eg: "settings").
On some erros that display a modal with "Force reconnect" button, allow
Etherpad to automatically reload pad after a few seconds. Amount of
seconds is defined on settings.json.
Still need to create tests for this feature, and implement i18n.
When checking the places on text that are identical between content
before and after a changeset, we were considering only the standard
(Etherpad-core) attributes, and not taking into account attributes
created by the plugins. One consequence was that the '*' marker of lines
with line attribs were being kept, even when the new lines have
different line attribs. See #3118 for more details.
Fix#3118.
