Commit graph

370 commits

Author SHA1 Message Date
Claire 0ad919b192 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  glitch-soc change because of having changed the default number of
  available poll options.
  Applied upstream's changes while keeping glitch-soc's default number of
  poll options.
- `public/oops.png`:
  We had a minor graphics change, probably not worth diverging from upstream.
  Took upstream version.
2022-11-06 09:50:41 +01:00
Claire bb89f83cc0
Fix additional issues with status cache hydration (#19747)
* Spare one SQL query when hydrating polls

* Improve tests

* Fix more discrepancies

* Fix possible crash when the status has no application set
2022-11-04 20:01:33 +01:00
Claire 03b991de6c
Fix various issues with store hydration (#19746)
- Improve tests
- Fix possible crash when application of a reblogged post isn't set
- Fix discrepancies around favourited and reblogged attributes
- Fix discrepancies around pinned attribute
- Fix polls not being hydrated
2022-11-04 19:33:16 +01:00
Eugen Rochko 5f9e47be34
Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Claire cb19be67d1 Merge branch 'main' into glitch-soc/merge-upstream 2022-10-28 19:23:58 +02:00
Eugen Rochko bf0ab3e0fa
Fix vacuum scheduler missing lock, locks never expiring (#19458)
Remove vacuuming of orphaned preview cards
2022-10-26 12:10:48 +02:00
Eugen Rochko 839f893168
Change public accounts pages to mount the web UI (#19319)
* Change public accounts pages to mount the web UI

* Fix handling of remote usernames in routes

- When logged in, serve web app
- When logged out, redirect to permalink
- Fix `app-body` class not being set sometimes due to name conflict

* Fix missing `multiColumn` prop

* Fix failing test

* Use `discoverable` attribute to control indexing directives

* Fix `<ColumnLoading />` not using `multiColumn`

* Add `noindex` to accounts in REST API

* Change noindex directive to not be rendered by default before a route is mounted

* Add loading indicator for detailed status in web UI

* Fix missing indicator appearing while account is loading in web UI
2022-10-20 14:35:29 +02:00
Eugen Rochko d86cb4cab8 [Glitch] Allow non-logged users to access /web
Port 43b5d5e38d to glitch-soc

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-10-09 18:23:15 +02:00
Claire e7ab476db2 Revert server-side part of "Add logged-out access to the web UI" 2022-10-02 19:36:28 +02:00
Claire 221580a3af Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/home_controller.rb`:
  Upstream made it so `/web` is available to non-logged-in users
  and `/` redirects to `/web` instead of `/about`.
  Kept our version since glitch-soc's WebUI doesn't have what's
  needed yet and I think /about is still a much better landing
  page anyway.
- `app/models/form/admin_settings.rb`:
  Upstream added new settings, and glitch-soc had an extra setting.
  Not really a conflict.
  Added upstream's new settings.
- `app/serializers/initial_state_serializer.rb`:
  Upstream added a new `server` initial state object.
  Not really a conflict.
  Merged upstream's changes.
- `app/views/admin/settings/edit.html.haml`:
  Upstream added new settings.
  Not really a conflict.
  Merged upstream's changes.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
  Upstream refactored that part and removed the file.
  Ported our relevant changes into `app/lib/vacuum/feeds_vacuum.rb`
- `config/settings.yml`:
  Upstream added new settings.
  Not a real conflict.
  Added upstream's new settings.
2022-10-02 18:46:22 +02:00
Eugen Rochko 43b5d5e38d
Add logged-out access to the web UI (#18961) 2022-09-29 04:39:33 +02:00
Eugen Rochko 5c9abdeff1
Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00
Claire 8cf7006d4e
Refactor ActivityPub handling to prepare for non-Account actors (#19212)
* Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService

ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is
specifically required to be an Account

* Refactor SignatureVerification to allow non-Account actors

* fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService

* Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors

* Refactor inbound ActivityPub payload processing to accept non-Account actors

* Refactor inbound ActivityPub processing to accept activities relayed through non-Account

* Refactor how Account key URIs are built

* Refactor Request and drop unused key_id_format parameter

* Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`
2022-09-21 22:45:57 +02:00
Eugen Rochko 50948b46aa
Add ability to filter followed accounts' posts by language (#19095) 2022-09-20 23:51:21 +02:00
luzpaz 4aa3b9bd01
Fix typos (#18604)
* Fix typos

Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,pixelx,ro`

* Follow-up typo fix
2022-08-28 17:44:34 +02:00
Claire cd87d7dcef Merge branch 'main' into glitch-soc/merge-upstream
- `.env.production.sample`:
  Our sample config file is very different from upstream since it is much more
  complete. Upstream added documentation for a few env variables.
  Copied the new variables/documentation from upstream.
- `app/lib/feed_manager.rb`:
  Upstream added a timeline type (hashtags), while glitch-soc already had an
  extra one (direct messages). Not really a conflict but textually close
  changes.
  Ported upstream's changes.
- `app/models/custom_emoji.rb`:
  Upstream upped the custom emoji size limit, while glitch-soc had configurable
  limits.
  Upped the default limits accordingly.
- `streaming/index.js`:
  Upstream reworked how hastags were normalized. Minor conflict due to
  glitch-soc's handling of instance-local posts.
  Ported upstream's changes.
2022-07-17 22:07:20 +02:00
Eugen Rochko e7aa2be828
Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Claire 7cc76b823a Merge branch 'main' into glitch-soc/merge-upstream 2022-07-05 08:49:22 +02:00
Claire 1b4054256f
Fix crash when a remote Flag activity mentions a private post (#18760)
* Add tests

* Fix crash when a remote Flag activity mentions a private post
2022-07-04 11:08:30 +02:00
Claire fe5f6bc7ed Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Fix erroneous deletion in a previous merge.
- `Gemfile`:
  Conflict caused by glitch-soc-only hCaptcha dependency
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/controllers/filters_controller.rb`:
  Minor conflict due to glitch-soc's theming system.
- `app/serializers/rest/status_serializer.rb`:
  Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28 11:11:18 +02:00
Claire 02851848e9
Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Claire 55ba907108 Add test 2022-05-24 10:26:04 +02:00
Claire 5fd8780b14 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, upstream updated a dependency textually adjacent to a
  glitch-soc-only one.
  Updated the dependency as upstream did.
2022-05-11 09:37:48 +02:00
Eugen Rochko 2b8dc58b7f
Change RSS feeds (#18356)
* Change RSS feeds

- Use date and time for titles instead of ellipsized text
- Use full content in body, even when there is a content warning
- Use media extensions

* Change feed icons and add width and height attributes to custom emojis

* Fix custom emoji animate on hover breaking

* Fix tests
2022-05-09 07:43:08 +02:00
Claire f23f784f18 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-28 18:16:42 +02:00
Eugen Rochko 3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire fc1dd81ea5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not a real conflict, upstream dependency updated textually too close to a
  glitch-soc-only dependency.
  Updated the upstream dependency.
2022-04-27 08:48:21 +02:00
Claire 0360135d4d
Fix PeerTube videos appearing with an erroneous “Edited at” marker (#18100)
* Fix PeerTube videos appearing with an erroneous “Edited at” marker

PeerTube videos have an `updated` field equal to `published`.
When processing an incoming activity that has the same value for `updated` and
`published`, assume this doesn't represent an actual edit.

* Please CodeClimate
2022-04-26 21:25:26 +02:00
Claire f2b2614d0a Fix link sanitization for outgoing text/html and text/markdown toots
Fixes #1739
2022-04-11 09:06:25 +02:00
Claire b368c75029 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Upstream removed this file, while glitch-soc had changes to deal with
  its theming system.
  Removed the file like upstream did.
2022-04-06 21:10:23 +02:00
Eugen Rochko 6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Claire 8d6f3f8a37 Fix code blocks processing 2022-03-28 23:40:45 +02:00
Claire 61cefbebf7 Add advanced text formatting back into glitch-soc 2022-03-28 22:21:37 +02:00
Claire f313bf3e97 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-28 16:22:28 +02:00
Claire 30658924a8
Fix test-related issues (#17888)
* Remove obsolete RSS::Serializer test

Since #17828, RSS::Serializer no longer has specific code for deleted statuses,
but it is never called on deleted statuses anyway.

* Rename erroneously-named test files

* Fix failing test

* Fix test deprecation warnings

* Update CircleCI Ruby orb

1.4.0 has a bug that does not match all the test files due to incorrect
globbing
2022-03-28 12:43:58 +02:00
Claire aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko cefa526c6d
Refactor formatter (#17828)
* Refactor formatter

* Move custom emoji pre-rendering logic to view helpers

* Move more methods out of Formatter

* Fix code style issues

* Remove Formatter

* Add inline poll options to RSS feeds

* Remove unused helper method

* Fix code style issues

* Various fixes and improvements

* Fix test
2022-03-26 02:53:34 +01:00
Eugen Rochko b58db8f12e
Add workaround for YouTube Shorts links (#17869)
* Add workaround for YouTube Shorts links

* Update link_details_extractor_spec.rb
2022-03-25 19:31:35 +01:00
Claire 1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Josh Soref b5329e0035
Spelling (#17705)
* spelling: account

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: affiliated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: appearance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: autosuggest

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cacheable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: component

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: domain.example

Clarify what's distinct and use RFC friendly domain space.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeds

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: functional

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inefficiency

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: not

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: notifications

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurring

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: position

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: progress

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: promotable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reblogging

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: resolve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: saturated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: similar

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: strategies

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: success

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: targeting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unauthorized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsensitizes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: various

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-06 22:51:40 +01:00
Claire 425a6c90c4 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-02 18:02:48 +01:00
Eugen Rochko 50ea54b3ed
Change authorized applications page (#17656)
* Change authorized applications page

* Hide revoke button for superapps and suspended accounts

* Clean up db/schema.rb
2022-03-01 16:48:58 +01:00
Claire f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire 8f537a1168
Change relays handling to not record boosts (#17571)
* Change relays handling to not record boosts

* Update tests
2022-02-16 14:36:44 +01:00
Claire 692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko f1f6ddd536
Fix structured data parsing from links choking on bad data (#17403)
* Fix structured data parsing from links choking on bad data

- Fix og:url meta tag being prioritized over canonical link tag
- Fix structured data parsing choking on commented-out CDATA declarations
- Fix HTML entities in title, description, provider_name, author_name
- Change structured data parsing to attempt every JSON-LD script tag

* Remove unnecessary slash escapes from CDATA regex pattern
2022-02-07 18:16:31 +01:00
Claire ad6ddb9bdd Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `config/environments/production.rb`:
  Upstream changed a header but we had different default headers.
  Applied the same change, and also dropped HSTS headers redundant with
  Rails'.
2022-01-26 22:32:21 +01:00
Eugen Rochko 6505b39e5d
Fix poll updates being saved as status edits (#17373)
Fix #17344
2022-01-26 18:05:39 +01:00
Claire fe89554a54 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
  Upstream refactored how `Create` activities are handled and how values are
  extracted from `Create`d objects. This conflicted with how glitch-soc
  supported the `directMessage` flag to explicitly distinguish between
  limited and direct messages.
  Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Upstream largely refactored that file and changed some of the logic.
  This conflicted with glitch-soc's handling of the direct timeline and
  the options to allow replies and boosts in public feeds.
  Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored to move mention-related ActivityPub deliveries to
  `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
  to not send local-only toots to remote mentioned users.
  Took upstream's version, as the check is not needed anymore, since it is
  performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
  Upstream added support for `update` toot events, while glitch-soc had
  support for an extra timeline support, `direct`.
  Ported upstream changes and extended them to the `direct` timeline.

Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
  Added code to handle the `directMessage` flag and take it into account
  to compute visibility.
- `app/lib/feed_manager.rb`:
  Extended upstream's support of `update` toot events to glitch-soc's
  `direct` timeline.
2022-01-19 23:52:48 +01:00