Commit graph

949 commits

Author SHA1 Message Date
Eugen Rochko 2354b10eb5
Add admin dashboard (#8029) 2018-07-16 01:11:53 +02:00
Eugen Rochko e55dce3176
Add federation relay support (#7998)
* Add federation relay support

* Add admin UI for managing relays

* Include actor on relay-related activities

* Fix i18n
2018-07-13 02:16:06 +02:00
mayaeh 401559c376 Fix whole-word selection and i18n: Add Japanese translation for Whole-word (#8004)
* Add Japanese translation for "Whole word" and add it's description.

* Fix to enable "Whole-word" selection.
2018-07-12 17:58:26 +02:00
ThibG 1ca4e51eb3 Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Eugen Rochko 6b9e03e002
Add API method to remove a suggestion (#7978)
DELETE /api/v1/suggestions/:account_id

When blocking, remove suggestion from both sides. Muting not affected,
since muting is supposed to be invisible to the target.
2018-07-07 21:09:54 +02:00
Eugen Rochko 9804ec3a6d
Fix missing irreversible in filters API, expires_in param (#7976) 2018-07-07 18:51:45 +02:00
Eugen Rochko f89c595ea0
Add admin setting to enable OG previews for sensitive media (#7962) 2018-07-06 02:15:56 +02:00
Eugen Rochko baff4a7ce0
If signed in, redirect autofollow invite to profile page (#7956)
Fix #7944
2018-07-05 20:57:35 +02:00
Eugen Rochko 1f6ed4f86a
Add more granular OAuth scopes (#7929)
* Add more granular OAuth scopes

* Add human-readable descriptions of the new scopes

* Ensure new scopes look good on the app UI

* Add tests

* Group scopes in screen and color-code dangerous ones

* Fix wrong extra scope
2018-07-05 18:31:35 +02:00
Eugen Rochko da8fe8079e
Re-add follow recommendations API (#7918)
* Re-add follow recommendations API

    GET /api/v1/suggestions

Removed in 8efa081f21 due to Neo4J
dependency. The algorithm uses triadic closures, takes into account
suspensions, blocks, mutes, domain blocks, excludes locked and moved
accounts, and prefers more recently updated accounts.

* Track interactions with people you don't follow

Replying to, favouriting and reblogging someone you're not following
will make them show up in follow recommendations. The interactions
have different weights:

- Replying is 1
- Favouriting is 10 (decidedly positive interaction, but private)
- Reblogging is 20

Following them, muting or blocking will remove them from the list,
obviously.

* Remove triadic closures, ensure potential friendships are trimmed
2018-07-03 01:47:56 +02:00
Eugen Rochko 2092d5c0ad
Improve embeds (#7919)
* Make embeds cacheable by reverse proxy

* Make follow button on embeds open remote follow modal

Instead of web+mastodon://, also, turn the button blue, and add a
sign up prompt to the remote follow modal
2018-07-01 04:12:34 +02:00
Marty McGuire 8fea9cc311 Typo in signature verification failure logging (#7916)
`@signature_verification_failure_reason` is used in most places but`@signed_verification_failure_reason` appears in two places. Likely those errors are not returned.
2018-06-30 13:11:38 +02:00
Eugen Rochko cdb101340a
Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
Shuhei Kitagawa 23955d956e Add tests for remote_unfollows_controller (#7879) 2018-06-24 19:55:55 +09:00
Shuhei Kitagawa 7f59206944 Replace bypass option with bypass_sign_in (#7867) 2018-06-21 10:41:49 +09:00
Eugen Rochko a58ec29631
Allow selecting default posting language instead of auto-detect (#7828)
* Allow selecting default posting language instead of auto-detect

* Enable default language setting in credentials API

* Fix form saving
2018-06-17 18:57:31 +02:00
Eugen Rochko 7eec279c7f
Change language opt-out to language opt-in (#7823)
* Switch filtered_languages to chosen_languages

* Adjust interface

* Remove unused translations
2018-06-17 13:54:02 +02:00
Eugen Rochko ca85658975
Add autofollow option to invites (#7805)
* Add autofollow option to invites

* Trigger CodeClimate rebuild
2018-06-15 18:00:23 +02:00
Shuhei Kitagawa 7086aa598b Add tests for intents_controller (#7763) 2018-06-09 22:47:50 +02:00
Eugen Rochko 10f51c9886
Fix domain hiding logic (#7765)
* Send rejections to followers when user hides domain they're on

* Use account domain blocks for "authorized followers" action

Replace soft-blocking (block & unblock) behaviour with follow rejection

* Split sync and async work of account domain blocking

Do not create domain block when removing followers by domain, that
is probably unexpected from the user's perspective.

* Adjust confirmation message for domain block

* yarn manage:translations
2018-06-09 22:46:54 +02:00
Eugen Rochko 0deb9fa6b9
Remove trending hashtags (#7711)
* Delete trends_controller.rb

* Update routes.rb

* Update trending_tags.rb

* Update index.js

* Update index.js

* Update search_results.js

* Update async-components.js

* Update index.js

* Delete trends.js

* Delete trends.js

* Delete trends_container.js

* Delete trends.js

* Update search_results.js

* Update search_results_container.js
2018-06-04 02:18:18 +02:00
Eugen Rochko 8bb74e50be
Add GET /api/v2/search which returns rich tag objects, adjust web UI (#7661) 2018-05-29 02:01:24 +02:00
tateisu b87a1229c7 optimize direct timeline (#7614)
* optimize direct timeline

* fix typo in class name

* change filter condition for direct timeline

* fix codestyle issue

* revoke index_accounts_not_silenced because direct timeline does not use it.

* revoke index_accounts_not_silenced because direct timeline does not use it.

* fix rspec test condition.

* fix rspec test condition.

* fix rspec test condition.

* revoke adding column and partial index

* (direct timeline) move merging logic to model

* fix pagination parameter

* add method arguments that switches return array of status or cache_ids

* fix order by

* returns ActiveRecord.Relation in default behavor

* fix codestyle issue
2018-05-28 11:04:06 +02:00
Eugen Rochko 9bd23dc4e5
Track trending tags (#7638)
* Track trending tags

- Half-life of 1 day
- Historical usage in daily buckets (last 7 days stored)
- GET /api/v1/trends

Fix #271

* Add trends to web UI

* Don't render compose form on search route, adjust search results header

* Disqualify tag from trends if it's in disallowed hashtags setting

* Count distinct accounts using tag, ignore silenced accounts
2018-05-27 21:45:30 +02:00
Eugen Rochko ebf2fef029
Catch ActionController::UnknownFormat and return HTTP 406 (#7621)
An error like that should not appear in production error log.
2018-05-26 01:09:30 +02:00
Eugen Rochko 4d81809f36
Yeah, it was supposed to be #create? 2018-05-22 20:05:24 +02:00
Eugen Rochko 36e47a31e3
Enforce order on authorized applications page (#7587)
Fix #7586
2018-05-22 14:44:53 +02:00
Eugen Rochko 05f8c375a2
Remove small pagination limit from context API (#7564)
Fix #7557
2018-05-21 12:43:05 +02:00
Eugen Rochko 8378b72eba
Ensure push subscription is immediately removed when application is revoked (#7548)
* Ensure push subscription is immediately removed when application is revoked

* When token is revoked from app, unsubscribe too
2018-05-19 21:05:08 +02:00
Eugen Rochko 1e02dc8715
Add preference to hide following/followers lists (#7532)
* Add preference to hide following/followers lists

- Public pages
- ActivityPub collections (does not return pages but does give total)
- REST API (unless it's your own) (does not federate)

Fix #6901

* Add preference

* Add delegation

* Fix issue

* Fix issue
2018-05-18 02:26:51 +02:00
Eugen Rochko 7293b9fc61
Ensure unfilled fields are shown when errors are shown (#7523)
Fix #7486
2018-05-17 13:00:56 +02:00
Eugen Rochko dfb6907e08
HTTP signatures spec no longer requires algorithms field (#7525)
Fix #7520
2018-05-17 04:03:28 +02:00
Akihiko Odaki 77cd6b5096 Do not use permitted_for scope when querying pinned statuses (#7510)
permitted_for scope is slow when combined with pinned status scope.
Fortunately permitted_for scope can safely be removed because a pinned
status is always public.
2018-05-16 12:30:14 +02:00
Akihiko Odaki 55fd55714a Raise Mastodon::RaceConditionError if Redis lock failed (#7511)
An explicit error allows user agents to know the error and Sidekiq to
retry.
2018-05-16 12:29:45 +02:00
Eugen Rochko 416f644505
Fix 404 for GET /api/v1/push/subscription (#7472)
My bad...
2018-05-13 21:45:18 +02:00
Eugen Rochko 97e43ec5f0
Add GET /api/v1/push/subscription REST API (#7471)
* Add Api::V1::Push::SubscriptionsController#show

* Add to routes
2018-05-13 21:07:31 +02:00
ThibG 352bae8c3e Update session activation time (fixes #5605) (#7408) 2018-05-11 13:20:58 +02:00
Shuhei Kitagawa 50491e0d92 Add tests for invites controller (#7441)
* Add tests for invites controller

* Small refactoring and fix for invites controller
2018-05-11 13:14:33 +02:00
Eugen Rochko b4fb766b23
Add REST API for Web Push Notifications subscriptions (#7445)
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
2018-05-11 11:49:12 +02:00
Eugen Rochko 42cd363542
Bot nameplates (#7391)
* Store actor type in database

* Add bot nameplate to web UI, add setting to preferences, API, AP
Fix #7365

* Fix code style issues
2018-05-07 09:31:07 +02:00
abcang db012b57c2 Fix distribute_add_activity and distribute_remove_activity (#7393) 2018-05-07 09:30:18 +02:00
Shuhei Kitagawa bd10a7e480 Add resend confirmation for admin (#7378)
* Add confirmations_controller#resend

* Add tests for confirmations_controller#resend

* Add translations
2018-05-06 10:59:03 +02:00
Eugen Rochko c7d1a2e400
Improve admin UI for accounts (#7360)
* Improve design of account statuses admin UI (consistent with reports)

* Make account moderation notes look consistent with report notes

* i18n-tasks remove-unused

* Fix code style issues

* Fix tests
2018-05-05 23:06:29 +02:00
ThibG d1aef17f9a Do not render first page of following and followers collections unless explicitly asked to (#7357) 2018-05-04 20:17:01 +02:00
Eugen Rochko 2c1f7b2ece
Better pagination for ActivityPub outbox (#7356) 2018-05-04 19:19:11 +02:00
Eugen Rochko cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
Eugen Rochko cae933510c
Allow updating bio fields via PUT /api/v1/accounts/update_credentials (#7288)
Add raw bio fields to the source attribute on GET /api/v1/accounts/verify_credentials
2018-05-02 15:57:37 +02:00
Eugen Rochko f62ee1ddb0
Disable API access when login is disabled (#7289) 2018-04-30 09:13:14 +02:00
Eugen Rochko 54f34d3f2a
Return HTTP 410 for suspended accounts in GET /api/v1/accounts/:id (#7287)
Fix #7243
2018-04-30 09:12:36 +02:00
Eugen Rochko da61352fab
Fix "Show more" URL on paginated threads for remote statuses (#7285)
* Fix URL of "Show more" link in paginated threads (ancestors side)

Increase item limits in threads

Fix #7268

* Fix "Show more" link in paginated threads (descendants side)
2018-04-30 01:59:42 +02:00
Eugen Rochko 9d4710ed00
Add RSS feeds for end-users (#7259)
* Add RSS feed for accounts

* Add RSS feeds for hashtags

* Fix code style issues

* Fix code style issues
2018-04-25 02:10:02 +02:00
Akihiko Odaki 1258efa882 Paginate descendant statuses in public page (#7148) 2018-04-23 19:27:35 +02:00
Yamagishi Kazutoshi 87e3f0a41d Fix spec for sr-Latn (#7203) 2018-04-20 18:14:31 +02:00
Eugen Rochko a9c440637c
Improve report layout (#7188)
* Use table for statuses in report

* Display reported account and reporter in the same table

* Split accounts and general report info into two tables again

* Redesign report statuses table, notes, merge notes and action log

* Remove unused translations

* Fix code style issue

* Fix code style issue

* Fix code style issue
2018-04-20 02:28:48 +02:00
Kaito Sinclaire 156b916caf Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
abcang 897199910f Improve web api protect (#6343) 2018-04-17 15:23:46 +02:00
Eugen Rochko aab5581c43
Set Referrer-Policy to origin in web UI and public pages of private toots (#7162)
Fix #7115
2018-04-17 13:51:01 +02:00
Akihiko Odaki a1049e9380 Redirect to account status page for page of status stream entry (#7104)
Commit 519119f657 missed a change for
stream entry page. Instead of duplicating the change, redirect to account
status page. It would also help crawlers (of search engines, for example)
to understand a stream entry URL and its corresponding status URL points
to the same page.
2018-04-16 10:04:31 +02:00
Eugen Rochko 78ed4ab75f
Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
Yamagishi Kazutoshi 50529cbceb Upgrade Rails to version 5.2.0 (#5898) 2018-04-12 14:45:17 +02:00
Akihiko Odaki 519119f657 Paginate ancestor statuses in public page (#7102)
This also limits the statuses returned by API, but pagination is not
implemented in Web API yet. I still expect it brings user experience
better than making a user wait to fetch all ancestor statuses and flooding
the column with them.
2018-04-11 12:35:09 +02:00
Emelia Smith d9b62e34da Feature: Improve reports ui (#7032)
* Further improvements to Reports UI

- Clean up notes display
- Clean up add new note form
- Simplify controller
- Allow reopening a report with a note
- Show created at date for reports
- Fix report details table formatting

* Show history of report using Admin::ActionLog beneath the report

* Fix incorrect log message when reopening a report

* Implement fetching of all ActionLog items that could be related to the report

* Ensure adding a report_note updates the report's updated_at

* Limit Report History to actions that happened between the report being created and the report being resolved

* Fix linting issues

* Improve report history builder

Thanks @gargron for the improvements
2018-04-10 20:27:59 +02:00
Emelia Smith 219a4423d8 Feature: Allow staff to change user emails (#7074)
* Admin: Show unconfirmed email address on account page

* Admin: Allow staff to change user email addresses

* ActionLog: On change_email, log current email address and new unconfirmed email address
2018-04-10 09:16:06 +02:00
Levi Bard cd0eaa349c Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
ThibG 1364e9e4ae Fix follow/unfollow buttons on public profile (fixes #7036) (#7040)
* Fix follow/unfollow buttons on public profile

- Present non-logged users with web+mastodon:// URLs for remote accounts
- Present logged-in users with appropriate links (authorize_follows and
  remote_unfollows) for remote accounts

* Do not cache rendered cards if user is logged in
2018-04-08 13:40:22 +02:00
Alda Marteau-Hardi b65eb00c53 Prevent admins and moderators eavesdropping in private and direct toots (#7067)
Fix #6986
2018-04-07 21:33:01 +02:00
Emelia Smith 2e59751823 Improve require_admin! and require_staff! filters (#7018)
Previously these returns 302 redirects instead of 403s, which meant posting links to admin pages in slack caused them to unfurl, rather than stay as a link. Additionally, require_admin! doesn't appear to be actively used, on require_staff!
2018-04-03 13:07:32 +02:00
Emelia Smith e85cffb236 Feature: Report improvements (#6967) (#7000)
* Implement Assignment of Reports (#6967)

* Change translation of admin.report.comment.label to "Report Comment" for clarity

As we'll soon add the ability for reports to have comments on them, this clarification makes sense.

* Implement notes for Reports

This enables moderators to leave comments about a report whilst they work on it

* Fix display of report moderation notes

* Allow reports to be reopened / marked as unresolved

* Redirect to reports listing upon resolution of report

* Implement "resolve with note" functionality

* Add inverse relationship for report notes

* Remove additional database querying when loading report notes

* Fix tests for reports

* Fix localisations for report notes / reports
2018-04-02 22:04:14 +02:00
Emelia Smith e7a1716701 Implement the ability for an Administrator or Moderator to remove an account avatar (#6998) 2018-04-02 13:45:07 +02:00
Eugen Rochko 33513753b9
Fix unpermitted parameters warning when generating pagination URLs (#6995) 2018-04-02 02:09:50 +02:00
Eugen Rochko 39f27b6cf3
If DEFAULT_LOCALE is set, enforce it instead of HTTP request locale (#6817)
Fix #6784
2018-03-18 16:57:04 +01:00
abcang 37b043d447 Improve performance of account_media_status_ids (#6729) 2018-03-10 17:44:26 +01:00
Eugen Rochko a29d409e20
If login redirects to omniauth, redirect logout to root_path (#6694)
Fix #6670
2018-03-08 11:18:26 +01:00
Akihiko Odaki a38dbd9c8a Redirect from Web tag timeline to public tag timeline if not signed in (#6633)
This is also implemented in Pawoo:
ceafdbd1bb
2018-03-05 19:29:36 +01:00
Akihiko Odaki 51d760960c Set the default locale in config (#6580)
Previously the default locale was set by Localized concern for controllers,
but it was not enforced for mailers.

config is enforced throughout the application and an appropriate place to
set the default locale.
2018-03-04 09:21:35 +01:00
Eugen Rochko 9110db41c5
Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
Eugen Rochko 68218d97c8
Add only_media param to public and hashtag timelines API (#6576) 2018-03-01 03:21:21 +01:00
Eugen Rochko fce8464077
Ensure that boolean params in the API are parsed for truthiness (#6575)
Use Rails smart boolean cast to account for values such as "f",
"0", "false", etc. Previously, if a param was present in the request,
it would count as true.
2018-03-01 02:47:59 +01:00
Eugen Rochko 47bdb9b33b
Fix #942: Seamless LDAP login (#6556) 2018-02-28 19:04:53 +01:00
Eugen Rochko 41a01bec23
Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Eugen Rochko 18513a978a
Improve public account cards (#6559)
- Add follow/unfollow/remote follow buttons
- Format the bio properly
- Always show username@domain, even for local accounts
2018-02-26 16:18:41 +01:00
Ian McCowan c33931b613 Fix prev/next links on public profile page (#6497)
* Fix prev/next links on public profile page

* Don't make pagination urls if no available statuses

* Fix empty check method

* Put left chevron before prev page link

* Add scope for pagination "starting at" a given id

* Status pagination try 2:

s/prev/older and s/next/newer
"older" on left, "newer" on right
Use new scope for "newer" link
Extract magic 20 page size to constant
Remove max_id from feed pagination as it's not respected

* Reinstate max_id for accounts atom stream

* normalize
2018-02-26 03:31:28 +01:00
Ghislain Loaec e668180044 New variable OAUTH_REDIRECT_AT_SIGN_IN + Ref #6538 (not only SAML strategies) (#6540) 2018-02-23 01:16:17 +01:00
Eugen Rochko c71aa468b5
Redesign landing page (again) (#6486)
* Redesign landing page (again)

* Move login form in small version to the right column

* Display closed registrations message

* Add site setting for the hero image

* Fix test

* Increase spacing, maximum width, change call to action section
2018-02-22 01:03:48 +01:00
Eugen Rochko d8bc64bb09
Fix #6526: Only store redirect location if not in JSON format (#6528) 2018-02-22 00:51:30 +01:00
Eugen Rochko 90f12f2e5a
Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko 4bc625166e
Fix bug in relationships API introduced by #6482 (#6527)
It was merge when it needed to be deep_merge. And added some tests
2018-02-21 23:22:12 +01:00
Eugen Rochko 61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Konrad Pozniak bb26cdda24 add parameter locked to /api/v1/update_credentials (#6506) 2018-02-18 22:57:53 +01:00
Eugen Rochko 9dbae6e8a1
Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Eugen Rochko 76f3d5d16b
Add preference to always display sensitive media (#6448) 2018-02-09 00:26:57 +01:00
abcang cf32f7da5c Fix response of signature_verification_failure_reason (#6441) 2018-02-08 05:00:45 +01:00
abcang 3f35d43222 Exclude nil from relationships array (#6427) 2018-02-04 12:32:10 +01:00
Renato "Lond" Cerqueira 4e4f1b0dcb Add option to show only local toots in timeline preview (#6292)
* Add option to show only local toots in timeline preview
Right know, toots from all the known fediverse are shown in the main
page of an instance. That however doesn't reflect the instance itself.
With this option the admin may choose to display only local toots so
that users checking the instance get a better idea of internal
conversations.

* Fix issues pointed by codeclimate and eslint

* Add default message for community timeline

* Update pl.yml
2018-02-04 06:00:10 +01:00
Eugen Rochko 26f21fd5a0
CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
ThibG ac1093256c Allow HTTP caching of atom-rendered public toots (OStatus compatibility) (#6207) 2018-02-02 10:54:04 +01:00
puckipedia 8e4cf6282b Allow retrieval of private statuses (single or in outbox) using HTTP signatures (#6225) 2018-02-02 10:19:59 +01:00
Alexander 04fef7b888 pam authentication (#5303)
* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style
2018-02-02 10:18:55 +01:00
Eugen Rochko 1cc44cba81
Fix #6331 (#6341)
UserTrackingConcern is circumvented by SessionsController#create
because it calls warden, which calls the User#update_tracked_fields!
method directly. Move returning user logic to that method.
2018-01-23 20:52:30 +01:00
Akihiko Odaki 613e7c7521 Rename ResolveRemoteAccountService to ResolveAccountService (#6327)
The service used to be named ResolveRemoteAccountService resolves local
accounts as well.
2018-01-22 14:25:09 +01:00
Aboobacker MK 112b1fa265 Redirect to 2FA creation page when otp_secret is not available (#6314) 2018-01-21 13:21:28 +01:00
Eugen Rochko 9b3b40df66
Fix regeneration marker not expiring (#6290)
* Fix regeneration key not getting expired

* Add rake task to remove old regeneration markers
2018-01-18 20:29:56 +01:00
Eugen Rochko 7badad7797
Fix home regeneration (#6251)
* Fix regeneration marker not being removed after completion

* Return HTTP 206 from /api/v1/timelines/home if regeneration in progress
Prioritize RegenerationWorker by putting it into default queue

* Display loading indicator and poll home timeline while it regenerates

* Add graphic to regeneration message

* Make "not found" indicator consistent with home regeneration
2018-01-17 23:56:03 +01:00
りんすき fbe7756da6 implement web share target (#6278)
* web share target

* fix

* fix
2018-01-17 17:08:10 +01:00
Patrick Figel 537d2939b1 Suppress CSRF token warnings (#6240)
CSRF token checking was enabled for API controllers in #6223,
producing "Can't verify CSRF token authenticity" log spam. This
disables logging of failed CSRF checks.

This also changes the protection strategy for
PushSubscriptionsController to use exceptions, making it consistent
with other controllers that use sessions.
2018-01-15 06:51:23 +01:00
nightpool c235711ffe Refactor /api/web APIs to use the centralized axios instance (#6223)
Also adds the ability to decouple the centralized axios logic from the
state dispatcher
2018-01-08 20:01:33 +01:00
Eugen Rochko dbda87c31f
Revert #5772 (#6221) 2018-01-08 10:57:52 +01:00
Yamagishi Kazutoshi 1d92b90be9 Fix force_ssl conditional (#6201) 2018-01-07 15:19:23 +01:00
Yamagishi Kazutoshi da809f9eec Fix unintended cache (#6214) 2018-01-07 15:12:59 +01:00
TheKinrar 95bd85d9e8 Represent numbers by strings in instance activity API (#6198)
Fixes #6197.
2018-01-05 22:38:33 +01:00
ThibG 3bee0996c5 Make sure private toots remain private and do not end up in HTTP caches (#6175) 2018-01-04 14:39:38 +01:00
Eugen Rochko c10f4bdb03
Cache JSON of immutable ActivityPub representations (#6171) 2018-01-04 01:21:38 +01:00
Akihiko Odaki 161c72d66d Allow to dereference Follow object for ActivityPub (#5772)
* Allow to dereference Follow object for ActivityPub

* Accept IRI as object representation for Accept activity
2018-01-03 18:08:57 +01:00
ThibG 99f962ba73 Allow HTTP caching of json view of public statuses (#6115)
* Allow HTTP caching of json view of public statuses

HTML views are not cached as they can contain private statuses as well

* Disable session cookies for ActivityPub json rendering of public toots
2018-01-03 04:57:57 +01:00
Patrick Figel 04ecf44c2f Add confirmation step for email changes (#6071)
* Add confirmation step for email changes

This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.

Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.

Fixes #3871

* Add review fixes
2018-01-02 16:55:00 +01:00
Akihiko Odaki 2c1ed5f872 Show mastodon on modal (#6129) 2018-01-02 05:07:56 +01:00
beatrix a56c4742d3 keep the same filters and page when doing custom emojo stuff (fixes #6112) (#6114) 2017-12-30 02:43:43 +01:00
Eugen Rochko 38fc1b498d
Add more instance stats APIs (#6125)
* Add GET /api/v1/instance/peers API to reveal known domains

* Add GET /api/v1/instance/activity API

* Make new APIs disableable, exclude private statuses from activity stats

* Fix code style issue

* Fix week timestamps
2017-12-29 19:52:04 +01:00
ThibG 868568d1c1 Make host_meta/webfinger replies cacheable (fixes #6100) (#6101)
* Make host_meta/webfinger replies cacheable (fixes #6100)

Drop common code for handling users and sessions as webfinger queries
are very basic, public APIs.

Also explicitly mark results as cacheable with “expires_in”.

* Add “Vary: Accept” header for caching since content-negociation is used
2017-12-27 18:21:12 +01:00
Eugen Rochko a3b2ea599d
Fix #6022 - Prevent nested migrated accounts, or migrations to self (#6026) 2017-12-14 21:35:30 +01:00
Eugen Rochko ad75ec8b5b
Add filters to admin UI for custom emojis (#6003) 2017-12-13 13:28:31 +01:00
Eugen Rochko 20a6584d2d
Clean up admin UI for accounts (#6004)
* Add staff filter to admin UI for accounts, remove obsolete columns

* Only display OStatus section in admin UI for accounts when OStatus data
2017-12-13 12:15:10 +01:00
Eugen Rochko 1356ed72cd
Fix #5953 - Add GET /api/v1/accounts/:id/lists (#5983) 2017-12-12 03:55:39 +01:00
Eugen Rochko feed07227b
Apply a 25x rate limit by IP even to authenticated requests (#5948) 2017-12-11 15:32:29 +01:00
abcang fcc0795a40 Remove unused function (#5950) 2017-12-09 23:37:31 +01:00
Eugen Rochko a865b62efc
Rate limit by user instead of IP when API user is authenticated (#5923)
* Fix #668 - Rate limit by user instead of IP when API user is authenticated

* Fix code style issue

* Use request decorator provided by Doorkeeper
2017-12-09 14:20:02 +01:00
abcang 99242b92bc Keep WebPush settings (#5879) 2017-12-09 02:31:37 +01:00
Eugen Rochko 2f4c5f504f
Limit users to 50 lists, remove pagination from lists API (#5933) 2017-12-09 01:32:29 +01:00
Yamagishi Kazutoshi b21db9bbde Using double splat operator (#5859) 2017-12-06 11:41:57 +01:00
Eugen Rochko 42bcbd36b7 Remove rabl dependency (#5894)
* Remove rabl dependency

* Replicate old Oj configuration
2017-12-06 15:04:49 +09:00
Eugen Rochko e20895f251
Add list of lists component to web UI (#5811)
* Add list of lists component to web UI

* Add list adding

* Add list removing

* List editor modal

* Add API account search limited by following=true relation

* Rework list editor modal

* Remove mandatory pagination of GET /api/v1/lists/:id/accounts

* Adjust search input placeholder

* Fix rspec (#5890)

* i18n: (zh-CN) Add missing translations for #5811 (#5891)

* i18n: (zh-CN) yarn manage:translations -- zh-CN

* i18n: (zh-CN) Add missing translations for #5811

* Fix some issues

- Display loading/missing state for list timelines
- Order lists alphabetically in overview
- Fix async list editor reset
- Redirect to /lists after deleting unpinned list
- Redirect to / after pinning a list

* Remove dead list columns when a list is deleted or fetch returns 404
2017-12-05 23:02:27 +01:00
Yamagishi Kazutoshi 74320971e2 Add invite filter (#5862) 2017-12-01 16:40:02 +01:00
Yamagishi Kazutoshi eee3b32b77 Fix invites form path (#5861) 2017-12-01 12:26:57 +01:00
aschmitz eeaec39888 Allow hiding of reblogs from followed users (#5762)
* Allow hiding of reblogs from followed users

This adds a new entry to the account menu to allow users to hide
future reblogs from a user (and then if they've done that, to show
future reblogs instead).

This does not remove or add historical reblogs from/to the user's
timeline; it only affects new statuses.

The API for this operates by sending a "reblogs" key to the follow
endpoint. If this is sent when starting a new follow, it will be
respected from the beginning of the follow relationship (even if
the follow request must be approved by the followee). If this is
sent when a follow relationship already exists, it will simply
update the existing follow relationship. As with the notification
muting, this will now return an object ({reblogs: [true|false]}) or
false for each follow relationship when requesting relationship
information for an account. This should cause few issues due to an
object being truthy in many languages, but some modifications may
need to be made in pickier languages.

Database changes: adds a show_reblogs column (default true,
non-nullable) to the follows and follow_requests tables. Because
these are non-nullable, we use the existing MigrationHelpers to
perform this change without locking those tables, although the
tables are likely to be small anyway.

Tests included.

See also <https://github.com/glitch-soc/mastodon/pull/212>.

* Rubocop fixes

* Code review changes

* Test fixes

This patchset closes #648 and resolves #3271.

* Rubocop fix

* Revert reblogs defaulting in argument, fix tests

It turns out we needed this for the same reason we needed it in muting:
if nil gets passed in somehow (most usually by an API client not passing
any value), we need to detect and handle it.

We could specify a default in the parameter and then also catch nil, but
there's no great reason to duplicate the default value.
2017-11-28 15:00:35 +01:00
nullkal 1b57d4dd3a Fix account migration feature (#5837)
* Make removable account migration

* Fix error during update of account migration setting

* Add notice when update account migration setting
2017-11-28 14:31:23 +01:00
Eugen Rochko 706e534455
Add UI for setting up account migration (#5832) 2017-11-27 22:47:06 +01:00
Eugen Rochko 740f8a95a9
Add consumable invites (#5814)
* Add consumable invites

* Add UI for generating invite codes

* Add tests

* Display max uses and expiration in invites table, delete invite

* Remove unused column and redundant validator

- Default follows not used, probably bad idea
- InviteCodeValidator is redundant because RegistrationsController
  checks invite code validity

* Add admin setting to disable invites

* Add admin UI for invites, configurable role for invite creation

- Admin UI that lists everyone's invites, always available
- Admin setting min_invite_role to control who can invite people
- Non-admin invite UI only visible if users are allowed to

* Do not remove invites from database, expire them instantly
2017-11-27 16:07:59 +01:00
Eugen Rochko e84fecb7e9
Add logging of admin actions (#5757)
* Add logging of admin actions

* Update brakeman whitelist

* Log creates, updates and destroys with history of changes

* i18n: Update Polish translation (#5782)

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

* Split admin navigation into moderation and administration

* Redesign audit log page

* 🇵🇱 (#5795)

* Add color coding to audit log

* Change dismiss->resolve, log all outcomes of report as resolve

* Update terminology (e-mail blacklist) (#5796)

* Update terminology (e-mail blacklist)

imho looks better

* Update en.yml

* Fix code style issues

* i18n-tasks normalize
2017-11-24 02:05:53 +01:00
Yamagishi Kazutoshi bfdcf76a64 Add “staff” badge to admin user pages (#5735)
* Allow staff badge to change from setting to hidden

* i18n: Update Polish translation
2017-11-19 04:13:57 +01:00
Eugen Rochko 24cafd73a2
Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
Clworld e1ebf36352 Fix regression #5467: fixes emoji copy (#5739) 2017-11-17 18:22:31 +01:00
Eugen Rochko 3e90987c8b Fix some rubocop style issues (#5730) 2017-11-17 10:06:26 +09:00
Surinna Curtis 031a5a8f92 Optional notification muting (#5087)
* Add a hide_notifications column to mutes

* Add muting_notifications? and a notifications argument to mute!

* block notifications in notify_service from hard muted accounts

* Add specs for how mute! interacts with muting_notifications?

* specs testing that hide_notifications in mutes actually hides notifications

* Add support for muting notifications in MuteService

* API support for muting notifications (and specs)

* Less gross passing of notifications flag

* Break out a separate mute modal with a hide-notifications checkbox.

* Convert profile header mute to use mute modal

* Satisfy eslint.

* specs for MuteService notifications params

* add trailing newlines to files for Pork :)

* Put the label for the hide notifications checkbox in a label element.

* Add a /api/v1/mutes/details route that just returns the array of mutes.

* Define a serializer for /api/v1/mutes/details

* Add more specs for the /api/v1/mutes/details endpoint

* Expose whether a mute hides notifications in the api/v1/relationships endpoint

* Show whether muted users' notifications are muted in account lists

* Allow modifying the hide_notifications of a mute with the /api/v1/accounts/:id/mute endpoint

* make the hide/unhide notifications buttons work

* satisfy eslint

* In probably dead code, replace a dispatch of muteAccount that was skipping the modal with launching the mute modal.

* fix a missing import

* add an explanatory comment to AccountInteractions

* Refactor handling of default params for muting to make code cleaner

* minor code style fixes oops

* Fixed a typo that was breaking the account mute API endpoint

* Apply white-space: nowrap to account relationships icons

* Fix code style issues

* Remove superfluous blank line

* Rename /api/v1/mutes/details -> /api/v2/mutes

* Don't serialize "account" in MuteSerializer

Doing so is somewhat unnecessary since it's always the current user's account.

* Fix wrong variable name in api/v2/mutes

* Use Toggle in place of checkbox in the mute modal.

* Make the Toggle in the mute modal look better

* Code style changes in specs and removed an extra space

* Code review suggestions from akihikodaki

Also fixed a syntax error in tests for AccountInteractions.

* Make AddHideNotificationsToMute Concurrent

It's not clear how much this will benefit instances in practice, as the
number of mutes tends to be pretty small, but this should prevent any
blocking migrations nonetheless.

* Fix up migration things

* Remove /api/v2/mutes
2017-11-15 03:56:41 +01:00
ThibG 58bca7b1e4 Filter searched toots to be consistent with blocking behaviors (#5383) 2017-11-15 01:53:33 +01:00
Eugen Rochko fbef909c2a
Add option to block direct messages from people you don't follow (#5669)
* Add option to block direct messages from people you don't follow

Fix #5326

* If the DM responds to a toot by recipient, allow it through

* i18n: Update Polish translation (for #5669) (#5673)
2017-11-14 21:12:57 +01:00
Eugen Rochko 7bb8b0b2fc
Add moderator role and add pundit policies for admin actions (#5635)
* Add moderator role and add pundit policies for admin actions

* Add rake task for turning user into mod and revoking it again

* Fix handling of unauthorized exception

* Deliver new report e-mails to staff, not just admins

* Add promote/demote to admin UI, hide some actions conditionally

* Fix unused i18n
2017-11-11 20:23:33 +01:00
Eugen Rochko 1032f3994f
Add ability to disable login and mark accounts as memorial (#5615)
Fix #5597
2017-11-07 19:06:44 +01:00
nullkal b6e2e999bd Show the local couterpart of emoji when it exists in /admin/custom_emojis (#5467)
* Show the local couterpart of emoji when it exists in admin/custom_emojis

* Fix indentation

* Fix error

* Add class table-action-link to Overwrite link

* Make it enable to overwrite emojis

* Make Code Climate happy
2017-11-07 14:49:32 +01:00
nullkal 781105293c Feature: Unlisted custom emojis (#5485) 2017-10-27 16:11:30 +02:00
erin 22da775a85 Fix copying emojos: redirect to the page you were on (#5509) 2017-10-26 23:44:24 +09:00
Nolan Lawson fa0be3f834 Add option to reduce motion (#5393)
* Add option to reduce motion

* Use HOC to wrap all Motion calls

* fix case-sensitive issue

* Avoid updating too frequently

* Get rid of unnecessary change to _simple_status.html.haml
2017-10-16 09:36:15 +02:00
KY 1bdd694a0a Fix #5351 - Redirection with page params after copy emoji (#5387)
* Fix #5351 - Redirection with page params after copy emoji

* Add page params to copy emoji link
2017-10-14 14:40:10 +02:00
Eugen Rochko b8db386e05 Fix UserTrackingConcern firing on every request, optimize some queries (#5368)
- For some reason, :if option on before_action did not work. It got
  executed every time, returned false, and the action run anyway,
  which led to the current_sign_in_at and sign_in_count being
  updated on every request
- Return "do not filter" early in FeedManager#filter_from_home? if
  the status is authored by receiver. Usually this method is not
  called for own statuses at all, but it is called when Feed#get
  uses the database
- Return early if #reload_stale_associations! has nothing to load
  to save a database query with WHERE 1=0
2017-10-13 16:44:29 +02:00
Yamagishi Kazutoshi dc91fd482a Set InstancePresenter to Auth::RegistrationsController#create (#5366) 2017-10-13 09:35:07 +02:00
takayamaki 552d22bec9 sign_in and sign_up views present og meta infos (#5308) 2017-10-11 00:52:25 +02:00
Eugen Rochko 61d3ecc805 Fix custom emoji copy not copying file (#5298) 2017-10-10 15:18:27 +02:00
Akihiko Odaki cc796298c9 Fix pagination in Api::V1::BlocksController (#5285) 2017-10-09 17:30:31 +02:00
nullkal 633426b261 Add moderation note (#5240)
* Add moderation note

* Add frozen_string_literal

* Make rspec pass
2017-10-07 20:26:43 +02:00
Eugen Rochko f486ef2666 Redesign public hashtag pages (#5237) 2017-10-07 20:00:35 +02:00
Eugen Rochko 3a3475450e Encode custom emojis as resolveable objects in ActivityPub (#5243)
* Encode custom emojis as resolveable objects in ActivityPub

* Improve code style
2017-10-07 17:43:42 +02:00
Eugen Rochko 49cc0eb3e7 Improve admin UI for custom emojis, add copy/disable/enable (#5231) 2017-10-05 23:42:05 +02:00
utam0k b3af3f9f8c Implement EmailBlackList (#5109)
* Implement BlacklistedEmailDomain

* Use Faker::Internet.domain_name

* Remove note column

* Add frozen_string_literal comment

* Delete unnecessary codes

* Sort alphabetically

* Change of wording

* Rename BlacklistedEmailDomain to EmailDomainBlock
2017-10-04 15:16:10 +02:00
Yamagishi Kazutoshi 178f718a9b Separate notifications preferences from general preferences (#4447)
* Separate notifications preferences from general preferences

* Refine settings/notifications/show

* remove preferences.notifications
2017-10-04 10:22:52 +02:00
aschmitz 468523f4ad Non-Serial ("Snowflake") IDs (#4801)
* Use non-serial IDs

This change makes a number of nontrivial tweaks to the data model in
Mastodon:

* All IDs are now 8 byte integers (rather than mixed 4- and 8-byte)
* IDs are now assigned as:
  * Top 6 bytes: millisecond-resolution time from epoch
  * Bottom 2 bytes: serial (within the millisecond) sequence number
  * See /lib/tasks/db.rake's `define_timestamp_id` for details, but
    note that the purpose of these changes is to make it difficult to
    determine the number of objects in a table from the ID of any
    object.
* The Redis sorted set used for the feed will have values used to look
  up toots, rather than scores. This is almost always the same as the
  existing behavior, except in the case of boosted toots. This change
  was made because Redis stores scores as double-precision floats,
  which cannot store the new ID format exactly. Note that this doesn't
  cause problems with sorting/pagination, because ZREVRANGEBYSCORE
  sorts lexicographically when scores are tied. (This will still cause
  sorting issues when the ID gains a new significant digit, but that's
  extraordinarily uncommon.)

Note a couple of tradeoffs have been made in this commit:

* lib/tasks/db.rake is used to enforce many/most column constraints,
  because this commit seems likely to take a while to bring upstream.
  Enforcing a post-migrate hook is an easier way to maintain the code
  in the interim.
* Boosted toots will appear in the timeline as many times as they have
  been boosted. This is a tradeoff due to the way the feed is saved in
  Redis at the moment, but will be handled by a future commit.

This would effectively close Mastodon's #1059, as it is a
snowflake-like system of generating IDs. However, given how involved
the changes were simply within Mastodon, it may have unexpected
interactions with some clients, if they store IDs as doubles
(or as 4-byte integers). This was a problem that Twitter ran into with
their "snowflake" transition, particularly in JavaScript clients that
treated IDs as JS integers, rather than strings. It therefore would be
useful to test these changes at least in the web interface and popular
clients before pushing them to all users.

* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Restructure feed pushes/unpushes

This was necessary because the previous behavior used Redis zset scores
to identify statuses, but those are IEEE double-precision floats, so we
can't actually use them to identify all 64-bit IDs. However, it leaves
the code in a much better state for refactoring reblog handling /
coalescing.

Feed-management code has been consolidated in FeedManager, including:

* BatchedRemoveStatusService no longer directly manipulates feed zsets
* RemoveStatusService no longer directly manipulates feed zsets
* PrecomputeFeedService has moved its logic to FeedManager#populate_feed

(PrecomputeFeedService largely made lots of calls to FeedManager, but
didn't follow the normal adding-to-feed process.)

This has the effect of unifying all of the feed push/unpush logic in
FeedManager, making it much more tractable to update it in the future.

Due to some additional checks that must be made during, for example,
batch status removals, some Redis pipelining has been removed. It does
not appear that this should cause significantly increased load, but if
necessary, some optimizations are possible in batch cases. These were
omitted in the pursuit of simplicity, but a batch_push and batch_unpush
would be possible in the future.

Tests were added to verify that pushes happen under expected conditions,
and to verify reblog behavior (both on pushing and unpushing). In the
case of unpushing, this includes testing behavior that currently leads
to confusion such as Mastodon's #2817, but this codifies that the
behavior is currently expected.

* Rubocop fixes

I could swear I made these changes already, but I must have lost them
somewhere along the line.

* Address review comments

This addresses the first two comments from review of this feature:

https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336735
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336931

This adds an optional argument to FeedManager#key, the subtype of feed
key to generate. It also tests to ensure that FeedManager's settings are
such that reblogs won't be tracked forever.

* Hardcode IdToBigints migration columns

This addresses a comment during review:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139337452

This means we'll need to make sure that all _id columns going forward
are bigints, but that should happen automatically in most cases.

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Only implement timestamp IDs for Status IDs

Per discussion in #4801, this is only being merged in for Status IDs at
this point. We do this in a migration, as there is no longer use for
a post-migration hook. We keep the initialization of the timestamp_id
function as a Rake task, as it is also needed after db:schema:load (as
db/schema.rb doesn't store Postgres functions).

* Change internal streaming payloads to stringified IDs as well

This is equivalent to 591a9af356faf2d5c7e66e3ec715502796c875cd from
#5019, with an extra change for the addition to FeedManager#unpush.

* Ensure we have a status_id_seq sequence

Apparently this is not a given when specifying a custom ID function,
so now we ensure it gets created. This uses the generic version of this
function to more easily support adding additional tables with timestamp
IDs in the future, although it would be possible to cut this down to a
less generic version if necessary. It is only run during db:schema:load
or the relevant migration, so the overhead is extraordinarily minimal.

* Transition reblogs to new Redis format

This provides a one-way migration to transition old Redis reblog entries
into the new format, with a separate tracking entry for reblogs.

It is not invertible because doing so could (if timestamp IDs are used)
require a database query for each status in each users' feed, which is
likely to be a significant toll on major instances.

* Address review comments from @akihikodaki

No functional changes.

* Additional review changes

* Heredoc cleanup

* Run db:schema:load hooks for test in development

This matches the behavior in Rails'
ActiveRecord::Tasks::DatabaseTasks.each_current_configuration, which
would otherwise break `rake db:setup` in development.

It also moves some functionality out to a library, which will be a good
place to put additional related functionality in the near future.
2017-10-04 09:56:37 +02:00
Akihiko Odaki 63f0979799 Validate id of ActivityPub representations (#5114)
Additionally, ActivityPub::FetchRemoteStatusService no longer parses
activities.
OStatus::Activity::Creation no longer delegates to ActivityPub because
the provided ActivityPub representations are not signed while OStatus
representations are.
2017-10-04 01:13:48 +02:00
ThibG dfaa219f88 Fix HTTP responses for salmon and ActivityPub inbox processing (#5200)
* Return sensible HTTP status for ActivityPub inbox processing

* Return sensible HTTP status for salmon slap processing

* Return additional information to debug signature verification failures
2017-10-03 23:21:19 +02:00
Eugen Rochko 47ecd652d3 Make Chrome splash screen same color as web UI's background color (#5169) 2017-10-02 01:23:32 +02:00
Akihiko Odaki cdacac8c6c Fix order of paginated accounts in FollowerDomainsController and spec (#3357)
* Fix order of paginated accounts in FollowerDomainsController

Unordered pagination could result in unexpected behavior.

* Cover Settings::FollowerDomainsController more
2017-09-30 23:06:09 +02:00
Eugen Rochko eb605141ff Fix #5104 - GET /api/v1/apps/verify_credentials to confirm app works (#5112) 2017-09-30 22:05:42 +02:00
Eugen Rochko f4ca116ea8 After 7 days of repeated delivery failures, give up on inbox (#5131)
- A successful delivery cancels it out
- An incoming delivery from account of the inbox cancels it out
2017-09-29 03:16:20 +02:00
Eugen Rochko 76f360c625 If HTTP signature is wrong and webfinger cache is stale, retry with resolve (#5129)
If the signature could not be verified and the webfinger of the account
was last retrieved longer than the cache period, try re-resolving the
account and then attempting to verify the signature again
2017-09-28 17:50:14 +02:00
Eugen Rochko 4ec1771165 Add ability to specify alternative text for media attachments (#5123)
* Fix #117 - Add ability to specify alternative text for media attachments

- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input

* Add tests

* Change undo button blend mode to difference
2017-09-28 15:31:31 +02:00
Eugen Rochko 293972f716 New API: GET /api/v1/custom_emojis to get a server's custom emojis (#5051) 2017-09-23 01:57:23 +02:00
nullkal 7d16bb379d Use OrderedCollectionPage to return followers/following list (#4949) 2017-09-19 16:37:06 +02:00
Andrew 0401a24558 Add support for multiple themes (#4959)
* Add support for selecting a theme

* Fix codeclimate issues

* Look up site default style if current user is not available due to e.g. not being logged in

* Remove outdated comment in common.js

* Address requested changes in themes PR

* Fix codeclimate issues

* Explicitly check current_account in application controller and only check theme availability if non-nil

* codeclimate

* explicit precedence with &&

* Fix code style in application_controller according to @nightpool's suggestion, use default style in embedded.html.haml

* codeclimate: indentation + return
2017-09-19 16:36:23 +02:00
Eugen Rochko 41e6c8b151 Fix incomplete account records being read (#4998)
* Fix incomplete account records being read

- Put account processing into redis lock
- Do not save until record is complete

* Fix spaces
2017-09-19 06:53:16 +02:00
Yamagishi Kazutoshi 1664e52cbb Fix custom emojis index (#5006) 2017-09-19 05:06:13 +02:00
Eugen Rochko 09a94b575e Admin interface for listing, adding and removing custom emojis (#5002)
* Admin interface for listing, adding and removing custom emojis

* Only display local ones in the list
2017-09-19 03:52:38 +02:00
Eugen Rochko 54edb4b853 When accessing uncached media attachment, redownload it (#4955)
* When accessing uncached media attachment, redownload it

* Prevent re-download of rejected media
2017-09-16 03:01:45 +02:00
Eugen Rochko 472df24579 When web UI URL used while logged out, redirect to static page (#4954) 2017-09-15 00:57:08 +02:00
Eugen Rochko 9239e4ce4d Uploads for admin site settings (#4913)
* Improve OpenGraph tags for about pages

* Add thumbnail admin setting

* Fix error

* Fix up
2017-09-14 00:04:30 +02:00
nullkal da77f65c46 Add instance search feature (#4925) 2017-09-13 12:30:07 +02:00
Eugen Rochko 0ef9d45d05 Fix error when following locked accounts (#4896) 2017-09-11 23:50:37 +02:00
Eugen Rochko c450ddb613 Fix POST /api/v1/follows error when already following (#4878) 2017-09-10 15:09:06 +02:00
Eugen Rochko 7d7844a47f Default follows for new users (#4871)
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
2017-09-10 09:58:38 +02:00
Eugen Rochko e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko 9b994c4aee Fix #4794 - Fake instant follow in API response when account is believed unlocked (#4799) 2017-09-05 17:48:13 +02:00
nullkal 2293466edd Show pinned statuses only in the top of the profile page (#4803)
* Show pinned statuses only in the top of the profile page

* Refactor AccountsController#show_pinned_statuses?
2017-09-04 12:53:18 +02:00
Eugen Rochko a187dcefa1 Instantly upgrade account to ActivityPub if we receive ActivityPub payload (#4766) 2017-09-03 01:11:23 +02:00
Eugen Rochko 7dc5035031 Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko d1a78eba15 Embed modal (#4748)
* Embed modal

* Proxy OEmbed requests from web UI
2017-08-31 03:38:35 +02:00
Eugen Rochko 2db9ccaf3e Add sharedInbox to actors (#4737) 2017-08-31 00:02:59 +02:00
Eugen Rochko e95bdec7c5 Update status embeds (#4742)
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
2017-08-30 10:23:43 +02:00
masarakki 649a20ab46 authorize-follow-requests-after-unlocking (#4658) 2017-08-26 12:40:03 +02:00
nullkal c2af138113 Allow multiple pinned statuses to be shown and make them be ordered b… (#4690)
* Allow multiple pinned statuses to be shown and make them be ordered by pinned date

* Set timestamps NOT NULL

* Make single-line pinned_statuses

* Spec for pinned_statuses

* Remove redundant empty line
2017-08-25 18:50:52 +02:00
Eugen Rochko 9caa90025f Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
unarist b01a19fe39 Fetch reblogs as Announce activity instead of Note object (#4672)
* Process Create / Announce activity in FetchRemoteStatusService

* Use activity URL in ActivityPub for reblogs

* Redirect to the original status on StatusesController#show
2017-08-24 16:21:42 +02:00
HIKARU KOBORI 2c6a85832c Fix typo in admin/status_controller.rb (#4679) 2017-08-24 04:03:52 +02:00
nullkal 80393a23d0 Use checkboxes for application scope setting (#4671) 2017-08-23 15:16:20 +02:00
Eugen Rochko c1b086a538 Fix up the applications area (#4664)
- Section it into "Development" area
- Improve UI of application form, index, and details
2017-08-23 00:59:35 +02:00
Colin Mitchell 871c0d251a Application prefs section (#2758)
* Add code for creating/managing apps to settings section

* Add specs for app changes

* Fix controller spec

* Fix view file I pasted over by mistake

* Add locale strings. Add 'my apps' to nav

* Add Client ID/Secret to App page. Add some visual separation

* Fix rubocop warnings

* Fix embarrassing typo

I lost an `end` statement while fixing a merge conflict.

* Add code for creating/managing apps to settings section

- Add specs for app changes
- Add locale strings. Add 'my apps' to nav
- Add Client ID/Secret to App page. Add some visual separation
- Fix some bugs/warnings

* Update to match code standards

* Trigger notification

* Add warning about not sharing API secrets

* Tweak spec a bit

* Cleanup fixture creation by using let!

* Remove unused key

* Add foreign key for application<->user
2017-08-22 18:33:57 +02:00
Eugen Rochko 72bb3e03fd Support more variations of ActivityPub keyId in signature (#4630)
- Tries to avoid performing HTTP request if the keyId is an actor URI
- Likewise if the URI is a fragment URI on top of actor URI
- Resolves public key, returns owner if the owner links back to the key
2017-08-21 22:57:34 +02:00
Eugen Rochko 6e9eda5331 ActivityPub migration procedure (#4617)
* ActivityPub migration procedure

Once one account is detected as going from OStatus to ActivityPub,
invalidate WebFinger cache for other accounts from the same domain

* Unsubscribe from PuSH updates once we receive an ActivityPub payload

* Re-subscribe to PuSH unless already unsubscribed, regardless of protocol
2017-08-21 01:14:40 +02:00
Eugen Rochko 74e5078795 Fix #4637 - Re-add missing doorkeeper_authorize for /api/v1/verify_credentials (#4650) 2017-08-21 00:41:08 +02:00
Yamagishi Kazutoshi 2edfdab6e6 Don't send Link header when don't know prev and next links (#4633) 2017-08-18 10:42:59 +02:00
Eugen Rochko ca7ea1aba9 Redesign public profiles (#4608)
* Redesign public profiles

* Responsive design

* Change public profile status filtering defaults and add options

- No longer displays private/direct toots even if you are permitted access
- By default omits replies
- "With replies" option
- "Media only" option

* Redesign account grid cards

* Fix style issues
2017-08-16 17:12:58 +02:00
Clworld f814661fca Make share intent modal to make "signed in as" shown. (#4611)
* Make share intent modal to make "signed in as" shown.

* fix glitch on mobile.
2017-08-16 16:48:44 +02:00
abcang e120d09c98 Fix require_user! behavior when not logged in (#4604) 2017-08-15 14:14:12 +02:00
Eugen Rochko 3c6503038e Add protocol handler. Handle follow intents (#4511)
* Add protocol handler. Handle follow intents

* Add share intent

* Improve code in intents controller

* Adjust share form CSS
2017-08-14 04:53:31 +02:00
Eugen Rochko 6df8bd277b Set correct content-type for ActivityPub JSON (#4592) 2017-08-14 04:16:43 +02:00
Eugen Rochko a2aeacbfee Add alternate links to ActivityPub resources from HTML/HEAD variants (#4586) 2017-08-13 00:45:04 +02:00
Eugen Rochko b7370ac8ba ActivityPub delivery (#4566)
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix #889

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
2017-08-13 00:44:41 +02:00
Eugen Rochko fdea173237 Add Digest header to requests with body, handle acct and URI keyId (#4565) 2017-08-09 23:54:14 +02:00
Eugen Rochko dd7ef0dc41 Add ActivityPub inbox (#4216)
* Add ActivityPub inbox

* Handle ActivityPub deletes

* Handle ActivityPub creates

* Handle ActivityPub announces

* Stubs for handling all activities that need to be handled

* Add ActivityPub actor resolving

* Handle conversation URI passing in ActivityPub

* Handle content language in ActivityPub

* Send accept header when fetching actor, handle JSON parse errors

* Test for ActivityPub::FetchRemoteAccountService

* Handle public key and icon/image when embedded/as array/as resolvable URI

* Implement ActivityPub::FetchRemoteStatusService

* Add stubs for more interactions

* Undo activities implemented

* Handle out of order activities

* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities

* Add fragment IDs to all transient activity serializers

* Add tests and fixes

* Add stubs for missing tests

* Add more tests

* Add more tests
2017-08-08 21:52:15 +02:00
Eugen Rochko df605f0f8b Add "signed in as" header to some pages (#4523) 2017-08-05 04:24:58 +02:00
nullkal dfcd2834f9 Redirect to PasswordController#new when reset_password_token is invalid (#4506) 2017-08-03 17:45:45 +02:00
Eugen Rochko 92cb451da8 Fix #4026 - Accept backup codes for disabling 2FA (#4382) 2017-07-26 23:36:33 +02:00
Akihiko Odaki a6ea7e282f Merge queries in Api::V1::FavouritesController (#4359) 2017-07-25 16:01:03 +02:00
Sorin Davidoi 9075c90c46 feat: Enable push subscription for mobile devices by default (#4321) 2017-07-23 23:27:23 +02:00
Eugen Rochko 4b911fea03 Add button to re-subscribe to all accounts from a domain (#4285) 2017-07-20 23:07:13 +02:00
Eugen Rochko 1fcdaafa6f Fix webfinger retries (#4275)
* Do not raise unretryable exceptions in ResolveRemoteAccountService

* Removed fatal exceptions from ResolveRemoteAccountService

Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-20 01:59:07 +02:00
Yamagishi Kazutoshi 716f4cb11c Add feature to revoke sessions (#4259) 2017-07-19 04:59:04 +02:00
Eugen Rochko a5a07da892 Correct OStatus inflection (Ostatus -> OStatus) (#4255) 2017-07-19 01:37:26 +02:00
Yamagishi Kazutoshi 3267e4a785 Add unfollow modal (optional) (#4246)
* Add unfollow modal

* unfollowing someone

* remove unnecessary prop
2017-07-18 17:14:43 +02:00
Akihiko Odaki 89b988cab5 Introduce Ostatus name space (#4164)
* Wrap methods of ProcessFeedService::ProcessEntry in classes

This is a change same with 425acecfdb, except
that it has the following changes:

* Revert irrelevant change in find_or_create_conversation
* Fix error handling for RemoteActivity

* Introduce Ostatus name space
2017-07-18 16:39:47 +02:00
abcang 4d42a38954 Improve admin page (#4121)
* Improve admin page

* Fix test

* Add spec

* Improve select style
2017-07-18 16:38:22 +02:00
Eugen Rochko a6328fc1b1 Minor ActivityPub JSON fixes (#4214)
- Objects must have attributedTo instead of actor
- The current attribute belongs to CollectionPage, not Collection
2017-07-16 10:28:55 +02:00
Eugen Rochko 8c45cd0e36 Improve ActivityPub representations (#3844)
* Improve webfinger templates and make tests more flexible

* Clean up AS2 representation of actor

* Refactor outbox

* Create activities representation

* Add representations of followers/following collections, do not redirect /users/:username route if format is empty

* Remove unused translations

* ActivityPub endpoint for single statuses, add ActivityPub::TagManager for better
URL/URI generation

* Add ActivityPub::TagManager#to

* Represent all attachments as Document instead of Image/Video specifically
(Because for remote ones we may not know for sure)

Add mentions and hashtags representation to AP notes

* Add AP-resolvable hashtag URIs

* Use ActiveModelSerializers for ActivityPub

* Clean up unused translations

* Separate route for object and activity

* Adjust cc/to matrices

* Add to/cc to activities, ensure announce activity embeds target status and
not the wrapper status, add "id" to all collections
2017-07-15 03:01:39 +02:00
Eugen Rochko cd9b2ab2f7 Fix #2672 - Connect signed PuSH subscription requests to instance domain (#4205)
* Fix #2672 - Connect signed PuSH subscription requests to instance domain

Resolves #2739

* Fix return of locate_subscription

* Fix tests
2017-07-14 23:01:20 +02:00
unarist 1896a154f5 Fix response of unreblog/unfavourite APIs (#4204)
Both APIs process asynchronously, so reblogged/favourited fields in the response should be set to `false` manually.
2017-07-14 20:44:53 +02:00
Eugen Rochko 1618b68bfa HTTP signatures (#4146)
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Eugen Rochko c42092ba7a Add option to opt out of search engines on public profile/status pages (#4199) 2017-07-14 16:41:02 +02:00
Sorin Davidoi 0c7c188c45 Web Push Notifications (#3243)
* feat: Register push subscription

* feat: Notify when mentioned

* feat: Boost, favourite, reply, follow, follow request

* feat: Notification interaction

* feat: Handle change of public key

* feat: Unsubscribe if things go wrong

* feat: Do not send normal notifications if push is enabled

* feat: Focus client if open

* refactor: Move push logic to WebPushSubscription

* feat: Better title and body

* feat: Localize messages

* chore: Fix lint errors

* feat: Settings

* refactor: Lazy load

* fix: Check if push settings exist

* feat: Device-based preferences

* refactor: Simplify logic

* refactor: Pull request feedback

* refactor: Pull request feedback

* refactor: Create /api/web/push_subscriptions endpoint

* feat: Spec PushSubscriptionController

* refactor: WebPushSubscription => Web::PushSubscription

* feat: Spec Web::PushSubscription

* feat: Display first media attachment

* feat: Support direction

* fix: Stuff broken while rebasing

* refactor: Integration with session activations

* refactor: Cleanup

* refactor: Simplify implementation

* feat: Set VAPID keys via environment

* chore: Comments

* fix: Crash when no alerts

* fix: Set VAPID keys in testing environment

* fix: Follow link

* feat: Notification actions

* fix: Delete previous subscription

* chore: Temporary logs

* refactor: Move migration to a later date

* fix: Fetch the correct session activation and misc bugs

* refactor: Move migration to a later date

* fix: Remove follow request (no notifications)

* feat: Send administrator contact to push service

* feat: Set time-to-live

* fix: Do not show sensitive images

* fix: Reducer crash in error handling

* feat: Add badge

* chore: Fix lint error

* fix: Checkbox label overlap

* fix: Check for payload support

* fix: Rename action "type" (crash in latest Chrome)

* feat: Action to expand notification

* fix: Lint errors

* fix: Unescape notification body

* fix: Do not allow boosting if the status is hidden

* feat: Add VAPID keys to the production sample environment

* fix: Strip HTML tags from status

* refactor: Better error messages

* refactor: Handle browser not implementing the VAPID protocol (Samsung Internet)

* fix: Error when target_status is nil

* fix: Handle lack of image

* fix: Delete reference to invalid subscriptions

* feat: Better error handling

* fix: Unescape HTML characters after tags are striped

* refactor: Simpify code

* fix: Modify to work with #4091

* Sort strings alphabetically

* i18n: Updated Polish translation

it annoys me that it's not fully localized :P

* refactor: Use current_session in PushSubscriptionController

* fix: Rebase mistake

* fix: Set cacheName to mastodon

* refactor: Pull request feedback

* refactor: Remove logging statements

* chore(yarn): Fix conflicts with master

* chore(yarn): Copy latest from master

* chore(yarn): Readd offline-plugin

* refactor: Use save! and update!

* refactor: Send notifications async

* fix: Allow retry when push fails

* fix: Save track for failed pushes

* fix: Minify sw.js

* fix: Remove account_id from fabricator
2017-07-13 22:15:32 +02:00
Eugen Rochko 056b5ed72f Improve UI of admin site settings (#4163) 2017-07-12 03:24:04 +02:00
Eugen Rochko e6c81a635b Fix #2848 - Rake task to redownload avatars/headers (#4156)
Can be filtered by a specific domain
Resolves #2292
2017-07-11 17:25:49 +02:00
Eugen Rochko f93de3a516 Fix #3462 - Require authentication for search API (#4155)
This makes it consistent with /api/v1/accounts/search and
previous behaviour has been an oversight.
2017-07-11 17:08:26 +02:00
Eugen Rochko e19eefe219 Redesign the landing page, mount public timeline on it (#4122)
* Redesign the landing page, mount public timeline on it

* Adjust the standalone mounted component to the lacking of router

* Adjust auth layout pages to new design

* Fix tests

* Standalone public timeline polling every 5 seconds

* Remove now obsolete translations

* Add responsive design for new landing page

* Address reviews

* Add floating clouds behind frontpage form

* Use access token from public page when available

* Fix mentions and hashtags links, cursor on status content in standalone mode

* Add footer link to source code

* Fix errors on pages that don't embed the component, use classnames

* Fix tests

* Change anonymous autoPlayGif default to false

* When gif autoplay is disabled, hover to play

* Add option to hide the timeline preview

* Slightly improve alt layout

* Add elephant friend to new frontpage

* Display "back to mastodon" in place of "login" when logged in on frontpage

* Change polling time to 3s
2017-07-11 15:27:59 +02:00
STJrInuyasha 7a889a8e12 Remote following success page (#4129)
* Added a success page to remote following
Includes follow-through links to web (the old redirect target) and back to the remote user's profile

* Use Account.new in spec instead of a fake with only id
(fixes spec)

* Fabricate(:account) over Account.new

* Remove self from the success text
(and all HTML with it)
2017-07-10 18:05:06 +02:00
のら ca45bd0361 Add Japanese translation of terms and flash (#4137) 2017-07-10 14:04:05 +02:00
Yamagishi Kazutoshi 2b9721d1b3 Add setting a always mark media as sensitive (#4136) 2017-07-10 14:00:32 +02:00
Yamagishi Kazutoshi f8212da329 Add attribute for default privacy to verify credentials (#4075)
* Add attribute for default privacy to verify credentials

* add raw_note

* source
2017-07-10 03:29:34 +02:00
Eugen Rochko 864e3f8d9c Replace OEmbed and initial state Rabl templates with serializers (#4110)
* Replace OEmbed Rabl template with serializer

* Replace initial state rabl with serializer
2017-07-08 14:51:05 +02:00
Eugen Rochko 00df69bc89 Fix #4058 - Use a long-lived cookie to keep track of user-level sessions (#4091)
* Fix #4058 - Use a long-lived cookie to keep track of user-level sessions

* Fix tests, smooth migrate from previous session-based identifier
2017-07-07 23:25:15 +02:00
Eugen Rochko 8b2cad5637 Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
Damien Erambert 18d3fa953b Add a setting allowing the use of system's default font in Web UI (#4033)
* add a system_font_ui setting on the server

* Plug the system_font_ui on the front-end

* add EN/FR locales for the new setting

* put Roboto after all other fonts

* remove trailing whitespace so CodeClimate is happy

* fix user_spec.rb

* correctly write user_spect this time

* slightly better way of adding the classes

* add comments to the system-font stack for clarification

* use .system-font for the class instead

* don't use multiple lines for comments

* remove trailing whitespace

* use the classnames module for consistency

* use `mastodon-font-sans-serif` instead of Roboto directly
2017-07-06 22:39:56 +02:00
Yamagishi Kazutoshi 275c5b51ed Customizable privacy policy from admin interface (#4062) 2017-07-04 15:19:24 +02:00
Eugen Rochko 42b8220632 Fix #1624 - Send e-mail notifications to admins about new reports (#3949) 2017-06-27 00:04:00 +02:00
Eugen Rochko 5e8d037e27 Fix #3910 - Require OTP authentication to disable 2FA (#3935)
* Fix #3910 - Require OTP authentication to disable 2FA. Also, remove ability
to generate new OTP backup codes *after* initial backup codes were handed
out during activation

* Restore recovery code re-generation

* Improve display of some 2FA elements
2017-06-25 23:51:46 +02:00
Eugen Rochko ed7dc1704d Bind web UI access tokens to sessions (#3940)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test

* Fix #2347 - Bind web UI access token to session

When you logout, session also destroys the access token, so it's no longer
valid. If access token is destroyed some other way, the session is also
destroyed, requiring a re-login.

Fix #1681 - Add scheduler to remove revoked access tokens and grants

* Fix test
2017-06-25 23:51:32 +02:00
Eugen Rochko f7301bd5b9 Add overview of active sessions (#3929)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test
2017-06-25 16:54:30 +02:00
masarakki ff142eb64d setting-for-account-deletable (#3852) 2017-06-19 15:12:31 +02:00
Eugen Rochko f3be605286 Rename FollowRemoteAccountService to ResolveRemoteAccountService (#3847)
Rename Activitypub to ActivityPub
2017-06-19 01:51:04 +02:00
Eugen Rochko 91c71471ab Fix account delete form not accepting password, update suspended (#3745)
account before removing content for quicker feedback to end-users
2017-06-14 20:30:12 +02:00
Eugen Rochko 4a618908e8 Account deletion (#3728)
* Add form for account deletion

* If avatar or header are gone from source, remove them

* Add option to have SuspendAccountService remove user record, add tests

* Exclude suspended accounts from search
2017-06-14 18:01:27 +02:00
Eugen Rochko dc8a6244fc Fix #2619 - When redis feed is empty, fall back to database (#3721)
* Fix #2619 - When redis feed is empty, fall back to database

* Use redis value to return feed from database only while RegenerationWorker
hasn't finished running

* Fix specs

* Replace usage of reject!
2017-06-14 13:37:03 +02:00
René Klačan dcf0530218 Make sure email is case insensitive on all places (#3688)
When case insensitivity is enabled via devise's `config.case_insensitive_keys` then `.find_for_authentication` method needs to be used instead of `.find_by` because second mentioned returns `nil` when valid email with different cases is passed.

More info https://github.com/plataformatec/devise/wiki/How-To:-Use-case-insensitive-emails
2017-06-11 02:29:08 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) 4919b89ab8 Improve default language decision and spec (#3351)
* Improve default language decision

This change allows to takes account of accepted language determined by
the user agent even if the custom default locale of the instance is
configured.

* Cover Localized more

* Fix code style
2017-06-10 09:44:02 +02:00
Matt Jankowski 2925372ff4 Move create/destroy actions for api/v1/statuses to namespace (#3678)
Each of mute, favourite, reblog has been updated to:

- Have a separate controller with just a create and destroy action
- Preserve historical route names to not break the API
- Mild refactoring to break up long methods
2017-06-10 09:39:26 +02:00
Matt Jankowski 5282ba862a Move reblogged_by and favourited_by actions out of api/v1/statuses and into unique controllers (#3646)
* Add specs for api statuses routes

* Update favourited_by and reblogged_by api routes

* Move methods into new controllers

* Use load_accounts methods to simplify index actions

* Clean up load_accounts methods

* Clean up link header generation

* Check for link headers in specs

* Remove unused actions from api/v1/statuses controller

* Remove specs for moved actions
2017-06-09 14:12:40 -04:00
Eugen Rochko 8902e265b4 Add explit admin actions to (re)subscribe/unsubscribe remote accounts (#3640)
* Add explit admin actions to (re)subscribe/unsubscribe remote accounts
and re-download avatar/header

* Improve how admin NSFW toggle looks
2017-06-08 14:58:22 +02:00
Yamagishi Kazutoshi 0a0b9a271a Improve RuboCop rules (compatibility to Code Climate) (#3636)
08f8de84eb/Gemfile.lock (L38)
Code Climate is using RuboCop v0.46.0.

Change several rules to maintain compatibility.
2017-06-08 13:24:28 +02:00
Matt Jankowski 73540ffe6b Clean up for api/base controller (#3629)
* Move ApiController to Api/BaseController

* API controllers inherit from Api::BaseController

* Add coverage for various error cases in api/base controller
2017-06-07 20:09:25 +02:00
Matt Jankowski f0634ba876 Coverage improvement and concern extraction for rate limit headers in API controller (#3625)
* Coverage for rate limit headers

* Move rate limit headers methods to concern

* Move throttle check to condition on before_action

* Move match_data variable into method

* Move utc timestamp to separate method

* Move header setting into smaller methods

* specs cleanup
2017-06-07 17:23:26 +02:00
Eugen Rochko 722d152082 Fix #3063 - Add dynamic app manifest (#3563)
* Fix #3063 - Add dynamic app manifest

* Added short_name

* Add background_color
2017-06-06 19:29:42 +02:00
Daigo 3 Dango 2985d08951 Redirect to streaming_api_base_url (#3579)
* Redirect to streaming_api_base_url

When Rails receives a request to streaming API, it most likely
means that there is another host which is configured to respond
to it. This is to redirect clients to that host if
`STREAMING_API_BASE_URL` is set as another host.

* Use the new Ruby 1.9 hash syntax
2017-06-05 12:09:29 +02:00
Naoki Kosaka e014bf8ed0 Fix limit_param in favourites_controller.rb (#3553) 2017-06-04 14:52:26 +02:00
takayamaki 3eedad2737 change sidekiq queueing to bulk push (#3536) 2017-06-04 00:11:15 +02:00
ThibG d567a382e3 Ensure well_known controllers use WEB_DOMAIN by including RoutingHelper (#2388)
This fixes #2375.
2017-06-02 22:21:36 +02:00
Akihiko Odaki (@fn_aki@pawoo.net) 10768aa204 Spec response for forgery (#3248)
Remove protect_from_forgery in ApiController, which is disabled by the
following skip_before_action, as well.
2017-06-01 20:56:55 +02:00
Matt Jankowski 5c63523972 Spec coverage and refactor for the api/v1/accounts controllers (#3451) 2017-05-31 21:36:24 +02:00
Matt Jankowski de4681b2be Move admin/pubsubhubbub controller to admin/subscriptions (#3442) 2017-05-31 20:39:35 +02:00
Matt Jankowski a132332b86 Add Instance class to list admin records (#3443) 2017-05-31 20:38:44 +02:00
Matt Jankowski 5236a62861 Improve spec coverage and clean up api/v1/blocks controller (#3464) 2017-05-31 20:34:51 +02:00
Matt Jankowski 0f155829b7 Improve spec coverage and clean up api/v1/follow_requests controller (#3465) 2017-05-31 20:32:11 +02:00
Matt Jankowski 84dda45df9 Improve spec coverage and clean up api/v1/domain_blocks controller (#3466) 2017-05-31 20:31:14 +02:00
Matt Jankowski 9c7505489f Refactor api/v1/notifications controller (#3470) 2017-05-31 20:30:55 +02:00
Matt Jankowski 75cad1d9d6 Improve spec coverage and clean up api/v1/favourites controller (#3472) 2017-05-31 20:30:39 +02:00
Matt Jankowski bf811e4d4a Improve spec coverage and clean up api/v1/mutes controller (#3481) 2017-05-31 20:27:34 +02:00
Matt Jankowski d6774d2ca3 Refactor and spec coverage for api/v1/timelines actions (#3482) 2017-05-31 20:27:17 +02:00
Matt Jankowski 79335e46fd Refactor api/v1/apps controller (#3471) 2017-05-30 21:16:28 -04:00
Matt Jankowski 7c6e02aaf3 Refactor api/v1/reports controller (#3469) 2017-05-30 21:13:31 -04:00
Matt Jankowski 7f55430652 Refactor api/v1/search controller (#3468) 2017-05-30 21:11:54 -04:00
Matt Jankowski 8235623362 Improve spec coverage and clean up api/v1/media controller (#3467) 2017-05-30 21:11:29 -04:00
Matt Jankowski 83435c49ea Clean up api/subscriptions controller (#3448) 2017-05-31 02:15:09 +02:00
Jack Jennings 33f669a5f8 Add status destroy authorization to policy (#3453)
* Add status destroy authorization to policy

* Create explicit unreblog status authorization
2017-05-30 22:56:31 +02:00
Matt Jankowski 3576fa0d59 Improve api oembed controller (#3450)
* Add StreamEntryFinder class to parse URLs

* Use StreamEntryFinder and clean up api/oembed controller
2017-05-30 16:30:06 -04:00
Matt Jankowski 1dcfb90202 Clean up api/salmon controller (#3449) 2017-05-30 16:28:58 -04:00
Matt Jankowski 0ebe7d6d23 Remove exports/base controller in favor of shared concern (#3444) 2017-05-30 19:06:01 +02:00
Jack Jennings 3a2003ba86 Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Akihiko Odaki 922fb74197 Remove methods from ObfuscateFilename and spec (#3347)
* Remove methods from ObfuscateFilename

* Spec ObfuscateFilename
2017-05-29 18:14:16 +02:00
Atsushi Yamamoto 402c19a924 Add preference setting for delete toot modal (#3368)
* Set delete_modal preference to true by default
* Does not show confirmation modal if delete_modal is false
* Add ja translation for preference setting page
2017-05-29 17:56:13 +02:00
unarist ecef03bb15 Fix empty flash message on the settings page (#3345) 2017-05-27 13:04:28 +02:00
Akihiko Odaki 860e257a68 Remove redundant call of recent scope in AccountsController (#3330)
recent is included in paginate_by_max_id.
2017-05-26 16:35:25 +02:00
Akinori MUSHA f6a93fc150 Go to root after login in single user mode (#3289)
In single user mode, visitors are redirected to the single user's
profile page.  So, if you are the owner without a session, you start
from that page, click the login button and authenticate yourself
expecting you'll soon get started with the home page, but in reality
you'll get redirected back to where you started from -- your own
profile page.

This fixes the behavior by redirecting you home after login if you
have started from your own profile page.
2017-05-26 14:14:03 +02:00
unarist cf4fe6cab8 More use of next link header on account (media) timelines (#3311)
This will reduce requests on who have only few statuses.

- Use next link header to detect more items from first request
- Omit next link header if result items are fewer than requested count
(It had omit it only if result was empty before)
2017-05-25 17:09:13 +02:00
Yamagishi Kazutoshi 676ba50601 Show error message to suspended user (#3281) 2017-05-24 16:39:09 +02:00
unarist 3ce9ca4c99 Fix following/followers API to return correct link headers (#3268)
Link headers in following/followers API should include follow_id as max_id/since_id.

However, these API use current_user's account_id instead of follow_id from #3167.
This causes irrelevant result on loading more users.
2017-05-23 23:26:23 +02:00
Eugen Rochko 2ca1f0737a Fix Devise destroy method being available to delete user record (#3266)
(You may think that we need account deletions, but this way would've just orphaned the db records)
2017-05-23 21:32:42 +02:00
alpaca-tc 7ee5fc5d68 Toggle sensitive from admin page (#3261) 2017-05-23 19:45:43 +02:00
Matt Jankowski 4289ed1d13 Refactor of API timeline actions (#3263)
- Increase coverage to exercise all parts of each action
- Move into namespace to share common code
- Misc refactor of each action for smaller methods, simpler code
2017-05-23 18:11:39 +02:00