Commit graph

1498 commits

Author SHA1 Message Date
Thibaut Girka 412218af2e Merge branch 'master' into glitch-soc/merge-upstream 2020-11-07 18:23:02 +01:00
Takeshi Umeda d6fe0c94ca
Add account sensitized (#14361)
* Add account sensitized

* Fix i18n normalize

* Fix description and spec

* Fix spec

* Fix wording
2020-11-04 20:45:01 +01:00
Thibaut Girka 2eb054c07f Remove glitch-soc specific and obsolete /api/v1/mutes/details 2020-10-21 21:28:47 +02:00
Thibaut Girka ec49aa8175 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, we deleted it to not be flooded by Depandabot.
  Kept deleted.
- `Gemfile.lock`:
  Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
  Upstream updated the serializer to support timed mutes, while
  glitch-soc added a custom API ages ago to get information that
  is already available elsewhere.
  Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
  🤷
- `app/models/user.rb`:
  New user setting added upstream while we have glitch-soc-specific
  user settings. Added upstream's user setting.
- `config/settings.yml`:
  Upstream added a new user setting close to a user setting we had
  changed the defaults for. Added the new upstream setting.
- `package.json`:
  Upstream dependency updated “too close” to a glitch-soc-specific
  dependency. No real conflict. Updated the dependency.
2020-10-21 19:10:50 +02:00
ThibG ca56527140
Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
Thibaut Girka 311f457430 Fix crash when autolinking an invalid URI in Markdown 2020-10-13 14:35:07 +02:00
Eugen Rochko 5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko 7d985f2aac
Remove dependency on goldfinger gem (#14919)
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091
2020-10-08 00:34:57 +02:00
Thibaut Girka 785852eb15 Fix glitch-soc specific test 2020-09-28 17:56:16 +02:00
Thibaut Girka a7aedebc31 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
Eugen Rochko 974b1b79ce
Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
kawaguchi 5d3c8baa9a
Fix validates :sign_count of WebauthnCredential (#14806) 2020-09-16 20:16:46 +02:00
Eugen Rochko ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
ThibG cd4ec7cd74
Do not serve account actors at all in limited federation mode (#14800)
* Do not serve account actors at all in limited federation mode

When an account is fetched without a signature from an allowed instance,
return an error.

This isn't really an improvement in security, as the only information that was
previously returned was required protocol-level info, and the only personal bit
was the existence of the account. The existence of the account can still be
checked by issuing a webfinger query, as those are accepted without signatures.

However, this change makes it so that unallowed instances won't create account
records on their end when they find a reference to an unknown account.

The previous behavior of rendering a limited list of fields, instead of not
rendering the actor at all, was in order to prevent situations in which two
instances in Authorized Fetch mode or Limited Federation mode would fail to
reach each other because resolving an account would require a signed query…
from an account which can only be fetched with a signed query itself. However,
this should now be fine as fetching accounts is done by signing on behalf of
the special instance actor, which does not require any kind of valid signature
to be fetched.

* Fix tests
2020-09-14 13:04:29 +02:00
Eugen Rochko 4e4b3a0c8e
Refactor settings controllers (#14767)
- Disallow suspended accounts from revoking sessions and apps
- Allow suspended accounts to access exports
2020-09-11 20:56:35 +02:00
Thibaut Girka 9748f074a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb

All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
2020-09-08 16:26:47 +02:00
Eugen Rochko 65760f59df
Refactor feed manager (#14761) 2020-09-08 03:41:16 +02:00
ThibG 517af45e32
Fix multiple boosts of a same toot erroneously appearing in TL (#14759)
* Check for and record reblog info atomically

Instead of using ZREVRANK to determine whether a reblog is a new reblog or not,
use ZADD's NX option to perform the check/addition option atomically.

* Replace ZREVRANK call with ZSCORE key which is more efficient

* Make tests a bit stricter

* Fix off-by-one
2020-09-07 18:00:15 +02:00
Eugen Rochko e8bc187845
Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
Thibaut Girka e5f934ddf0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict caused because we have additional code to make sure pinned
  local-only toots don't get rendered on the ActivityPub endpoints.
  Ported upstream changes.
2020-09-07 09:21:38 +02:00
Eugen Rochko 68d3b160de
Fix various warnings in rspec (#14729) 2020-09-04 20:22:26 +02:00
ThibG 79305428a7
Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
Thibaut Girka 8c3c27bf06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream change too close to a glitch-soc change related to
  instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Minor conflict due to glitch-soc's handling of Direct Messages,
  merged upstream changes.
- `yarn.lock`:
  Not really a conflict, caused by glitch-soc-only dependencies
  being textually too close to updated upstream dependencies.
  Merged upstream changes.
2020-08-30 16:13:08 +02:00
Eugen Rochko 52157fdcba
Add support for dereferencing objects through bearcaps (#14683) 2020-08-30 12:34:20 +02:00
santiagorodriguez96 9cadd40cf4
refactor: add email previews for WebAuthn emails (#14658)
This is a leftover for the work done in #14466.
2020-08-25 01:21:11 +02:00
santiagorodriguez96 e8d41bc2fe
Add WebAuthn as an alternative 2FA method (#14466)
* feat: add possibility of adding WebAuthn security keys to use as 2FA

This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add request for WebAuthn as second factor at login if enabled

This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add possibility of deleting WebAuthn Credentials

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: disable WebAuthn when an Admin disables 2FA for a user

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA

Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.

* We had to change a little the flow for setting up TOTP, given that now
  it's possible to setting up again if you already had TOTP, in order to
  let users modify their authenticator app – given that now it's not
  possible for them to disable TOTP and set it up again with another
  authenticator app.
  So, basically, now instead of storing the new `otp_secret` in the
  user, we store it in the session until the process of set up is
  finished.
  This was because, as it was before, when users clicked on 'Edit' in
  the new two-factor methods lists page, but then went back without
  finishing the flow, their `otp_secret` had been changed therefore
  invalidating their previous authenticator app, making them unable to
  log in again using TOTP.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* refactor: fix eslint errors

The PR build was failing given that linting returning some errors.
This commit attempts to fix them.

* refactor: normalize i18n translations

The build was failing given that i18n translations files were not
normalized.
This commits fixes that.

* refactor: avoid having the webauthn gem locked to a specific version

* refactor: use symbols for routes without '/'

* refactor: avoid sending webauthn disabled email when 2FA is disabled

When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.

* refactor: avoid creating new env variable for webauthn_origin config

* refactor: improve flash error messages for webauthn pages

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
2020-08-24 16:46:27 +02:00
ThibG 720214feb0
Add support for inlined objects in activity audience (#14514)
* Add support for inlined objects in activity audience

* Add tests
2020-08-24 14:11:47 +02:00
Thibaut Girka 74a9affdcc Merge branch 'master' into glitch-soc/merge-upstream 2020-08-19 19:16:41 +02:00
ThibG f6a82cb2cd
Fix not being able to unbookmark toots when blocked by their author (#14604)
* Fix not being able to unbookmark toots when blocked by their author

* Add tests
2020-08-19 19:02:06 +02:00
Thibaut Girka da62e350e0 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-13 22:17:29 +02:00
ThibG 8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
Fire Demon 13501f7393 Add tests for direct message assertions 2020-08-04 19:43:19 +02:00
Thibaut Girka 23cd5f2c15 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-02 14:30:28 +02:00
ThibG a1412491b7
Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-08-02 11:21:10 +02:00
Thibaut Girka 2cfa142961 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `yarn.lock`:
  No real conflict, just a glitch-soc-only dependency textually too close to
  one updated upstream. Ported upstream changes.
2020-08-01 20:24:23 +02:00
ThibG bfd5aea206
Fix handling of Reject Follow when a matching follow relationship exists (#14479)
* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199
2020-08-01 18:20:37 +02:00
Thibaut Girka b9f351d845 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
  Not a “real” conflict, but change too close to a change we made to
  fix the vanilla WebUI locally pushing authored local-only toots in the
  public TL view.
2020-07-22 13:38:17 +02:00
ThibG 5d9acc0ce4
Fix not handling Undo on some activity types when they aren't inlined (#14346)
* Fix not handling Undo on some activity types when they aren't inlined

When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.

Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.

Accepts cannot be handled at all because we don't record their URI at any
point.

Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.

Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.

* Add tests

* Make newer blocks overwrite older ones

Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.
2020-07-22 11:45:35 +02:00
ThibG f55dd193f9
Fix RSS feeds not being cachable (#14368)
* Add tests for some cachable responses

This only covers responses that we should have managed to make cachable
so far. It's not the case of all responses that should be cachable in
the end.

* Fix RSS feeds not being cachable
2020-07-22 11:44:02 +02:00
ThibG 322d74fc2a
Fix boosted toots from blocked account not being retroactively removed from TL (#14339)
* Fix boosted toots from blocked account not being retroactively removed from TL

Fixes #14301

* Add test for clear_from_timeline
2020-07-17 07:07:54 +02:00
Thibaut Girka 1c6c40d17c Merge branch 'master' into glitch-soc/merge-upstream 2020-07-16 14:43:50 +02:00
ThibG d658af7ff8
Fix removing allowed domains being done synchronously (#14302)
* Fix removing allowed domains being done synchronously

* Add tests
2020-07-15 21:08:19 +02:00
Thibaut Girka 3f60b096b5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/routes.rb`:
  Upstream disabled E2EE routes, which we did earlier, but slightly
  differently. Took upstream's version.
2020-07-15 15:32:40 +02:00
ThibG 0a8a3fe595
Fix being unable to unboost when blocked by their author (#14308)
Fixes #14307
2020-07-15 14:43:19 +02:00
Thibaut Girka e9ad99bc93 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just some glitch-soc-specific dependency
  too close to an upstream-updated one.
2020-07-07 15:58:45 +02:00
Eugen Rochko 6e25574ce5
Fix media attachments enumeration (#14254)
* Fix media attachment enumeration

* Switch media_attachments id to snowflake ids

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-07-07 15:26:51 +02:00
ThibG e96e9cae62
Add test for removing endorsed accounts on account deletion/suspension (#14241) 2020-07-07 02:01:13 +02:00
Thibaut Girka 45deca65b3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Conflict because part of that file has been split to
  `app/javascript/core/settings.js`. Ported those changes
  there.
2020-07-01 19:23:14 +02:00
ThibG 35cedc922c
Change move handler to carry blocks over (#14144)
* Change move handler to carry blocks and mutes over

When user A blocks user B and B moves to a new account C, make A block C
accordingly.

Note that it only works if A's instance is aware of the Move, that is,
if B is on A's instance or has followers there.

* Also notify instances with known people blocking you when moving

* Add automatic account notes when blocking/muting an account that had no note
2020-07-01 13:51:15 +02:00
Thibaut Girka f186e0c91c Add support for “start” and “reversed” attributes to ol element
Fixes #1367
2020-07-01 01:02:27 +02:00