Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AO3-6895 Reduce the number of badly closed database connections #5041

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/controllers/challenge_signups_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

class ChallengeSignupsController < ApplicationController
include ExportsHelper
Resque::Job.extend(ResqueExecutorWrap)

before_action :users_only, except: [:summary, :display_summary, :requests_summary]
before_action :load_collection, except: [:index]
Expand Down
1 change: 1 addition & 0 deletions app/controllers/troubleshooting_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# A controller used to let admins and tag wranglers perform some
# troubleshooting on tags and works.
class TroubleshootingController < ApplicationController
Resque::Job.extend(ResqueExecutorWrap)
before_action :check_permission_to_wrangle
before_action :load_item
before_action :check_visibility
Expand Down
2 changes: 1 addition & 1 deletion app/jobs/redis_set_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@

def perform(*args, **kwargs)
scan_and_remove(redis, key, batch_size: batch_size) do |batch|
perform_on_batch(batch, *args, **kwargs)
Rails.application.executor.wrap {perform_on_batch(batch, *args, **kwargs)}

Check warning on line 38 in app/jobs/redis_set_job.rb

View workflow job for this annotation

GitHub Actions / Rubocop

[rubocop] reported by reviewdog 🐶 Space missing inside {. Raw Output: app/jobs/redis_set_job.rb:38:40: C: Layout/SpaceInsideBlockBraces: Space missing inside {.

Check warning on line 38 in app/jobs/redis_set_job.rb

View workflow job for this annotation

GitHub Actions / Rubocop

[rubocop] reported by reviewdog 🐶 Space missing inside }. Raw Output: app/jobs/redis_set_job.rb:38:80: C: Layout/SpaceInsideBlockBraces: Space missing inside }.
end
end

Expand Down
2 changes: 2 additions & 0 deletions app/models/bookmark.rb
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ def check_new_external_work
after_create :update_work_stats
after_destroy :update_work_stats, :update_pseud_index

Resque::Job.extend(ResqueExecutorWrap)

def invalidate_bookmark_count
work = Work.where(id: self.bookmarkable_id)
if work.present? && self.bookmarkable_type == 'Work'
Expand Down
1 change: 1 addition & 0 deletions app/models/challenge_assignment.rb
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ def signups_match
}

scope :posted, -> { joins(WORKS_JOIN).where("challenge_assignments.creation_id IS NOT NULL AND works.posted = 1") }
Resque::Job.extend(ResqueExecutorWrap)

# should be faster than unfulfilled scope because no giant left joins
def self.unfulfilled_in_collection(collection)
Expand Down
2 changes: 1 addition & 1 deletion app/models/challenge_signup_summary.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
class ChallengeSignupSummary

Resque::Job.extend(ResqueExecutorWrap)
attr_reader :collection, :challenge

def initialize(collection)
Expand Down
18 changes: 10 additions & 8 deletions app/models/concerns/async_with_resque.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@ def async(method, *args)

# Actually perform the delayed action.
def perform(method, *args)
if method.is_a?(Integer)
# TODO: For backwards compatibility, if the "method" is an integer, we
# treat it like an ID and use perform_on_instance instead. But once all
# of the jobs in the queue have been processed (or deleted), we should
# be able to remove this check.
perform_on_instance(method, *args)
else
send(method, *args)
Rails.application.executor.wrap do
if method.is_a?(Integer)
# TODO: For backwards compatibility, if the "method" is an integer, we
# treat it like an ID and use perform_on_instance instead. But once all
# of the jobs in the queue have been processed (or deleted), we should
# be able to remove this check.
perform_on_instance(method, *args)
else
send(method, *args)
end
end
Comment on lines +25 to 35
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on this TODO, it sounds like we can delete the if and just do the following?

Suggested change
Rails.application.executor.wrap do
if method.is_a?(Integer)
# TODO: For backwards compatibility, if the "method" is an integer, we
# treat it like an ID and use perform_on_instance instead. But once all
# of the jobs in the queue have been processed (or deleted), we should
# be able to remove this check.
perform_on_instance(method, *args)
else
send(method, *args)
end
end
Rails.application.executor.wrap { send(method, *args) }

end

Expand Down
1 change: 1 addition & 0 deletions app/models/concerns/filterable.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
# A module for types that are supposed to have FilterTaggings, calculated based
# on their tags. Includes Taggable.
module Filterable
Resque::Job.extend(ResqueExecutorWrap)
extend ActiveSupport::Concern
include Taggable

Expand Down
1 change: 1 addition & 0 deletions app/models/indexing/index_queue.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
class IndexQueue
Resque::Job.extend(ResqueExecutorWrap)

BATCH_SIZE = 1000
REDIS = REDIS_GENERAL
Expand Down
1 change: 1 addition & 0 deletions app/models/potential_match.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
class PotentialMatch < ApplicationRecord
Resque::Job.extend(ResqueExecutorWrap)
# We use "-1" to represent all the requested items matching
ALL = -1

Expand Down
1 change: 1 addition & 0 deletions app/models/pseud.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ class Pseud < ApplicationRecord
include Searchable
include WorksOwner
include Justifiable
Resque::Job.extend(ResqueExecutorWrap)

has_one_attached :icon do |attachable|
attachable.variant(:standard, resize_to_limit: [100, 100])
Expand Down
1 change: 1 addition & 0 deletions app/models/redis_hit_counter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
class RedisHitCounter
class << self
include RedisScanning
Resque::Job.extend(ResqueExecutorWrap)

# Records a hit for the given IP address on the given work ID. If the IP
# address hasn't visited the work within the current 24 hour block, we
Expand Down
2 changes: 1 addition & 1 deletion app/models/search/async_indexer.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
class AsyncIndexer

Resque::Job.extend(ResqueExecutorWrap)
REDIS = REDIS_GENERAL

####################
Expand Down
1 change: 1 addition & 0 deletions app/models/search/index_sweeper.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
class IndexSweeper
Resque::Job.extend(ResqueExecutorWrap)

REDIS = AsyncIndexer::REDIS

Expand Down
1 change: 1 addition & 0 deletions app/models/search/indexer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ class Indexer
Series: %w[BookmarkedSeriesIndexer],
ExternalWork: %w[BookmarkedExternalWorkIndexer]
}.freeze
Resque::Job.extend(ResqueExecutorWrap)

delegate :klass, :klass_with_includes, :index_name, :document_type, to: :class

Expand Down
1 change: 1 addition & 0 deletions app/models/serial_work.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ class SerialWork < ApplicationRecord
after_destroy :update_series_index, :update_work_index

scope :in_order, -> { order(:position) }
Resque::Job.extend(ResqueExecutorWrap)

# If you add or remove a work from a series, make sure restricted? is still accurate
def adjust_series_visibility
Expand Down
2 changes: 2 additions & 0 deletions app/models/series.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ class Series < ApplicationRecord
maximum: ArchiveConfig.TITLE_MAX,
too_long: ts("must be less than %{max} letters long.", max: ArchiveConfig.TITLE_MAX)

Resque::Job.extend(ResqueExecutorWrap)

# return title.html_safe to overcome escaping done by sanitiser
def title
read_attribute(:title).try(:html_safe)
Expand Down
2 changes: 2 additions & 0 deletions app/models/stat_counter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ class StatCounter < ApplicationRecord

after_commit :enqueue_to_index, on: :update

Resque::Job.extend(ResqueExecutorWrap)

def enqueue_to_index
IndexQueue.enqueue(self, :stats)
end
Expand Down
2 changes: 2 additions & 0 deletions app/models/tag.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ class Tag < ApplicationRecord
include Wrangleable
include Rails.application.routes.url_helpers

Resque::Job.extend(ResqueExecutorWrap)

NAME = "Tag"

# Note: the order of this array is important.
Expand Down
2 changes: 2 additions & 0 deletions app/models/tagging.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ class Tagging < ApplicationRecord
after_create :update_filters
after_destroy :update_filters

Resque::Job.extend(ResqueExecutorWrap)

def update_filters
return unless taggable.is_a?(Filterable)

Expand Down
2 changes: 2 additions & 0 deletions app/models/user.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ class User < ApplicationRecord
# properly
include BackwardsCompatiblePasswordDecryptor

Resque::Job.extend(ResqueExecutorWrap)

# Allows other models to get the current user with User.current_user
cattr_accessor :current_user

Expand Down
2 changes: 2 additions & 0 deletions app/models/work.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ class Work < ApplicationRecord
include WorkChapterCountCaching
include Creatable

Resque::Job.extend(ResqueExecutorWrap)

########################################################################
# ASSOCIATIONS
########################################################################
Expand Down
1 change: 1 addition & 0 deletions lib/bookmarkable.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
module Bookmarkable
Resque::Job.extend(ResqueExecutorWrap)

def self.included(bookmarkable)
bookmarkable.class_eval do
Expand Down
5 changes: 5 additions & 0 deletions lib/resque_executor_wrap.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module ResqueExecutorWrap
def around_perform_wrap_executor(*args)

Check warning on line 2 in lib/resque_executor_wrap.rb

View workflow job for this annotation

GitHub Actions / Rubocop

[rubocop] reported by reviewdog 🐶 Unused method argument - `args`. If it's necessary, use `_` or `_args` as an argument name to indicate that it won't be used. You can also write as `around_perform_wrap_executor(*)` if you want the method to accept any arguments but don't care about them. Raw Output: lib/resque_executor_wrap.rb:2:37: W: Lint/UnusedMethodArgument: Unused method argument - `args`. If it's necessary, use `_` or `_args` as an argument name to indicate that it won't be used. You can also write as `around_perform_wrap_executor(*)` if you want the method to accept any arguments but don't care about them.
Rails.application.executor.wrap { yield }

Check warning on line 3 in lib/resque_executor_wrap.rb

View workflow job for this annotation

GitHub Actions / Rubocop

[rubocop] reported by reviewdog 🐶 Consider using explicit block argument in the surrounding method's signature over `yield`. Raw Output: lib/resque_executor_wrap.rb:3:5: C: Style/ExplicitBlockArgument: Consider using explicit block argument in the surrounding method's signature over `yield`.
end
end
1 change: 1 addition & 0 deletions lib/searchable.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
module Searchable
Resque::Job.extend(ResqueExecutorWrap)

def self.included(searchable)
searchable.class_eval do
Expand Down
Loading