Commit graph

1762 commits

Author SHA1 Message Date
Claire f3a4d57be1 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Discarded upstream changes: we have our own README
- `app/controllers/follower_accounts_controller.rb`:
  Port upstream's minor refactoring
2022-12-15 20:25:25 +01:00
Claire 70415714f1
Add follow request banner on account header (#20785)
* Add requested_by to relationship maps

* Display whether an account has requested to follow you on their profile
2022-12-15 18:50:11 +01:00
Claire 2644a28cb3
Change remote media files to be downloaded outside of transactions (#21796) 2022-12-15 18:09:48 +01:00
Jeong Arm 3656a6b9cc
Add "disabled" user filter for admin/accounts UI (#21282) 2022-12-15 17:30:47 +01:00
Jeong Arm d412147d02
Save avatar or header correctly even if other one fails (#18465)
* Save avatar or header correctly if other one fails

* Fix test
2022-12-15 17:11:14 +01:00
Francis Murillo 5fb1c3e934
Revoke all authorized applications on password reset (#21325)
* Clear sessions on password change

* Rename User::clear_sessions to revoke_access for a clearer meaning

* Add reset paassword controller test

* Use User.find instead of User.find_for_authentication for reset password test

* Use redirect and render for better test meaning in reset password

Co-authored-by: Effy Elden <effy@effy.space>
2022-12-15 15:47:06 +01:00
Claire e877bb47b8 Merge branch 'main' into glitch-soc/main 2022-12-09 16:48:55 +01:00
Claire ed07f10ca8
Fix failure when “Require a reason to join” is set with open registrations (#22127) 2022-12-07 16:39:58 +01:00
Claire 9a3d91f629 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/concerns/domain_materializable.rb`:
  Fixed a code style issue upstream in a PR that got merged in glitch-soc
  earlier.
  Changed the code to match upstream's.
2022-12-07 09:03:42 +01:00
Claire b59fb28e90
Fix 500 error when trying to migrate to an invalid address (#21462)
* Fix 500 error when trying to migrate to an invalid address

* Add tests
2022-12-07 02:35:39 +01:00
Claire c8849d6cee
Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire 69137f4a90
Fix irreversible and whole_word parameters handling in /api/v1/filters (#21988)
Fixes #21965
2022-12-07 00:10:53 +01:00
Claire fe523a3045
Fix unbounded recursion in account discovery (#1994)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-04 21:23:19 +01:00
Claire a2e7997592 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README is completely different.
  Discarded upstream changes.
2022-11-28 11:33:34 +01:00
Claire 57b893d505
Fix spaces not being stripped in admin account search (#21324)
Fixes #21058

Regression from #18641
2022-11-27 20:47:29 +01:00
Kaspar V 47f0d7021e
refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
afontenot f17fc5742e
Clear voter count when poll is reset (#21700)
When a poll is edited, we reset the poll and remove all previous
votes. However, prior to this commit, the voter count on the poll
was not reset. This leads to incorrect percentages being shown in
poll results.

Fixes #21696
2022-11-26 23:08:25 +01:00
Skyler Hawthorne 5b2ff8d32d
fix media uploads with ffmpeg 5 (#21191) 2022-11-25 16:20:47 +01:00
Claire ff42233aae Merge branch 'main' into glitch-soc/merge-upstream 2022-11-22 10:28:43 +01:00
David Leadbeater 69378eac99
Don't allow URLs that contain non-normalized paths to be verified (#20999)
* Don't allow URLs that contain non-normalized paths to be verified

This stops things like https://example.com/otheruser/../realuser where
"/otheruser" appears to be the verified URL, but the actual URL being
verified is "/realuser" due to the "/../".

Also fix a test to use 'https', so it is testing the right thing, now
that since #20304 https is required.

* missing do
2022-11-20 19:28:13 +01:00
Claire aec61a703f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed how docker images were built, including how
  they were cached.
  I don't know much about it, so applied upstream's changes.
- `app/controllers/admin/domain_blocks_controller.rb`:
  The feature, that was in glitch-soc, got backported upstream.
  It also had a few fixes upstream, so those have been ported!
- `app/javascript/packs/admin.js`:
  Glitch-soc changes have been backported upstream. As a result,
  some code from `app/javascript/core/admin.js` got added upstream.
  Kept our version since our shared Javascript already has that feature.
- `app/models/user.rb`:
  Upstream added something to distinguish unusable and unusable-because-moved
  accounts, while glitch-soc considers moved accounts usable.
  Took upstream's code for `functional_or_moved?` and made `functional?`
  call it.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream cleaned up code style a bit, on a line that we had custom changes
  for.
  Applied upstream's change while keeping our change.
- `config/initializers/content_security_policy.rb`:
  Upstream adopted one CSP directive we already had.
  The conflict is because of our files being structurally different, but the
  change itself was already part of glitch-soc.
  Kept our version.
2022-11-17 12:01:37 +01:00
Claire ab7d99e035
Fix various issues with domain block import (#1944)
- stop using Paperclip for processing domain allow/block imports
- stop leaving temporary files
- better error handling
- assume CSV files are UTF-8-encoded
2022-11-17 11:42:20 +01:00
lenore gilbert c373148b3d
Support for import/export of instance-level domain blocks/allows for 4.x w/ additional fixes (#20597)
* Allow import/export of instance-level domain blocks/allows (#1754)

* Allow import/export of instance-level domain blocks/allows.
Fixes #15095

* Pacify circleci

* Address simple code review feedback

* Add headers to exported CSV

* Extract common import/export functionality to
AdminExportControllerConcern

* Add additional fields to instance-blocked domain export

* Address review feedback

* Split instance domain block/allow import/export into separate pages/controllers

* Address code review feedback

* Pacify DeepSource

* Work around Paperclip::HasAttachmentFile for Rails 6

* Fix deprecated API warning in export tests

* Remove after_commit workaround

(cherry picked from commit 94e98864e3)

* Add confirmation page when importing blocked domains (#1773)

* Move glitch-soc-specific strings to glitch-soc-specific locale files

* Add confirmation page when importing blocked domains

(cherry picked from commit b91196f4b7)

* Fix authorization check in domain blocks controller

(cherry picked from commit 7527937758)

* Fix error strings for domain blocks and email-domain blocks

Corrected issue with non-error message used for Mastodon:NotPermittedError in Domain Blocks
Corrected issue Domain Blocks using the Email Domain Blocks message on ActionContoller::ParameterMissing
Corrected issue with Email Domain Blocks using the not_permitted string from "custom emojii's"

* Ran i18n-tasks normalize to address test failure

* Removed unused admin.export_domain_blocks.not_permitted string

Removing unused string as indicated by Check i18n

* Fix tests

(cherry picked from commit 9094c2f52c)

* Fix domain block export not exporting blocks with only media rejection

(cherry picked from commit 26ff48ee48)

* Fix various issues with domain block import

- stop using Paperclip for processing domain allow/block imports
- stop leaving temporary files
- better error handling
- assume CSV files are UTF-8-encoded

(cherry picked from commit cad824d8f501b95377e4f0a957e5a00d517a1902)

Co-authored-by: Levi Bard <taktaktaktaktaktaktaktaktaktak@gmail.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-17 11:05:09 +01:00
Claire 00b2720ef0
Change automatic post deletion configuration to be accessible to redirected users (#20774)
Fixes #20550
2022-11-17 10:55:23 +01:00
Claire 0cc77263fc
Change batch account suspension to create a strike (#20897) 2022-11-17 10:52:51 +01:00
Claire 32c70d2f09 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/announcements/edit.html.haml`:
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
- `app/views/admin/announcements/new.html.haml`
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
2022-11-14 08:35:55 +01:00
Eugen Rochko 552d69ad96
Fix error when invalid domain name is submitted (#19474)
Fix #19175
2022-11-14 08:07:14 +01:00
Eugen Rochko b31afc6294
Fix error when passing unknown filter param in REST API (#20626)
Fix #19156
2022-11-14 08:06:06 +01:00
Eugen Rochko 5c826c408d
Fix image type not being set after conversion for convertible image types (#20624) 2022-11-14 07:13:14 +01:00
Eugen Rochko 2e2ba39abf
Fix rules with same priority being sorted non-deterministically (#20623) 2022-11-14 06:28:19 +01:00
Claire 93912f0498 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Conflict because we (glitch-soc) have disabled trending of posts without
  review.
  Discarded that upstream change.
- `app/views/admin/settings/discovery/show.html.haml`:
  Just an extra setting in glitch-soc.
  Kept that extra setting.
2022-11-12 13:28:03 +01:00
Jeong Arm c4c1bee880
Fix trendable status without review (#20214) 2022-11-11 21:24:10 +01:00
Emily Strickland 96f51e593f
Guard against error extracting body from URL (#20428)
If `Nokogiri::HTML(value).at_xpath('//body')` fails to find the `body` element, it will return `nil`. We can guard against that with an early return. Avoids calling `children` on `Nilclass` in those cases.
2022-11-11 21:22:28 +01:00
Eugen Rochko 9bc0a6c861
Fix metadata scrubbing removing color profile from images (#20389) 2022-11-11 09:20:10 +01:00
Claire 86f6631d28
Remove dead code and refactor status threading code (#20357)
* Remove dead code

* Remove unneeded/broken parameters and refactor descendant computation
2022-11-10 22:30:00 +01:00
Eugen Rochko c6c7c6223d
Change verification to only work for https links (#20304)
Fix #20242
2022-11-10 21:09:03 +01:00
Claire c118918520 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific validation.
  Applied upstream changes.
- `app/models/public_feed.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific parameter documentation.
  Applied upstream changes.
2022-11-10 09:36:47 +01:00
Eugen Rochko 9965a23b04
Change link verification to ignore IDN domains (#20295)
Fix #3833
2022-11-10 06:27:45 +01:00
James Tucker 78a6b871fe
Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Eugen Rochko 0cd0786aef
Revert filtering public timelines by locale by default (#20294) 2022-11-10 05:34:42 +01:00
Eugen Rochko e98833748e
Fix being able to spoof link verification (#20217)
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
Claire a8c854b3ea Merge branch 'main' into glitch-soc/merge-upstream 2022-11-08 15:44:57 +01:00
Postmodern ca80beb653
Micro-optimization: use if/else instead of Array#compact and Array#min (#19906)
* Technically `if`/`else` is faster than using `[value1, value2].compact.min` to find the lesser of two values, one of which may be `nil`.
2022-11-08 03:50:47 +01:00
Claire bbf74498f5
Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018)
* Fix followers count not being updated when migrating follows

Fixes #19900

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
Claire be7032b9cf Merge branch 'main' into glitch-soc/merge-upstream 2022-11-07 19:53:30 +01:00
Postmodern 106648b456
Micro-optimization: only split acct into two Strings (#19901)
* Since `acct` is split by `@` and assigned to `username` and `domain`, we only need to split `acct` into two Strings.
2022-11-07 16:17:55 +01:00
Claire 6c0a6097ff Merge branch 'main' into glitch-soc/merge-upstream 2022-11-07 07:50:47 +01:00
nightpool 54f0f1b9ef
Skip Webfinger cache during migrations as well (#19883) 2022-11-07 03:31:38 +01:00
Claire 0ad919b192 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  glitch-soc change because of having changed the default number of
  available poll options.
  Applied upstream's changes while keeping glitch-soc's default number of
  poll options.
- `public/oops.png`:
  We had a minor graphics change, probably not worth diverging from upstream.
  Took upstream version.
2022-11-06 09:50:41 +01:00
Eugen Rochko 3151b260e2
Fix not using GIN index for account search queries (#19830) 2022-11-06 06:16:34 +01:00