A previous refactor of the `Service::Base::Step` class introduced a
non thread-safe behavior. `#call` mutates instance variables at runtime,
and since a step instance is the same for any given service class, this
can sometimes lead to `context` being the wrong one for the running
service.
This patch makes use of `Concurrent::ThreadLocalVar` to fix the issue.
This patch adds two things:
1. An outcome matcher (`on_lock_not_acquired`), allowing to react when
there was a problem with the lock.
2. Compatibility with the steps inspector, allowing to display properly
the steps of a service containing locks.
There’s currently a bug when using a dedicated class as a policy in
services: if that class delegates its `#call` method (to an underlying
strategy object for example), then an error will be raised saying steps
aren’t allowed to provide default parameters.
This should not happen, and this patch fixes that issue.
Currently, there are two ways (kind of) for accessing `params` inside a
service:
- when there is no contract or it hasn’t been reached yet, `params` is
just the hash that was provided to the service. To access a key, you
have to use the bracket notation `params[:my_key]`.
- when there is a contract and it has been executed successfully,
`params` now references the contract and the attributes are accessible
using methods (`params.my_key`).
This patch unifies how `params` exposes its attributes. Now, even if
there is no contract at all in a service, `params` will expose its
attributes through methods, that way things are more consistent.
This patch also makes sure there is always a `params` object available
even when no `params` key is provided to the service (this allows a
contract to fail because its attributes are blank instead of having the
service raising an error because it doesn’t find `params` in its context).
This patch aims to improve the steps inspector output:
- The service class name is displayed at the top.
- Next to each step is displayed the time it took to run said step.
- Steps that didn’t run are hidden.
- `#inspect` automatically outputs the error when it is present.
In our CI env, sometimes, we see problems regarding getting a connection
from the pool. As it’s currently set to 1, increasing it a bit should
fix that kind of issues.
In some cases in CI env, it seems the AR connection isn’t available and
the `ensure` block is executed. It’s calling `#verify!` on the
connection, so it can fail sometimes. This is probably why
`#clear_active_connections!` was failing too sometimes.
Here, we just check the connection is present before clearing the
connections.
Following a recent commit (0ba0126ade),
the benchmark script wasn’t working anymore (and the related rake task).
This patch fixes it. It also adds some information about Ruby YJIT being
enabled or not.
This patch adds a new step to services named `try`.
It’s useful to rescue exceptions that some steps could raise. That way,
if an exception is caught, the service will stop its execution and can
be inspected like with any other steps.
Just wrap the steps that can raise with a `try` block:
```ruby
try do
step :step_that_can_raise
step :another_step_that_can_raise
end
```
By default, `try` will catch any exception inheriting from
`StandardError`, but we can specify what exceptions to catch:
```ruby
try(ArgumentError, RuntimeError) do
step :will_raise
end
```
An outcome matcher has been added: `on_exceptions`. By default it will
be executed for any exception caught by the `try` step.
Here also, we can specify what exceptions to catch:
```ruby
on_exceptions(ArgumentError, RuntimeError) do |exception|
…
end
```
Finally, an RSpec matcher has been added:
```ruby
it { is_expected.to fail_with_exception }
# or
it { is_expected.to fail_with_exception(ArgumentError) }
```
Currently, if an association is added as a tracked field in
`PostRevisor`, the `PostRevisionSerializer` class will try to serialize
it somehow. This will raise an error as ActiveRecord collection proxies
can't be serialized.
This patch addresses this issue by skipping any association tracked by
the `PostRevisor` class.
We decided to make contracts immutable once their validations have run.
Indeed, it doesn’t make a lot of sense to modify a contract value
outside the contract itself.
If processing is needed, then it should happen inside the contract
itself.
This patch replaces the parameters provided to a service through
`params` by the contract object.
That way, it allows better consistency when accessing input params. For
example, if you have a service without a contract, to access a
parameter, you need to use `params[:my_parameter]`. But with a contract,
you do this through `contract.my_parameter`. Now, with this patch,
you’ll be able to access it through `params.my_parameter` or
`params[:my_parameter]`.
Some methods have been added to the contract object to better mimic a
Hash. That way, when accessing/using `params`, you don’t have to think
too much about it:
- `params.my_key` is also accessible through `params[:my_key]`.
- `params.my_key = value` can also be done through `params[:my_key] =
value`.
- `#slice` and `#merge` are available.
- `#to_hash` has been implemented, so the contract object will be
automatically cast as a hash by Ruby depending on the context. For
example, with an AR model, you can do this: `user.update(**params)`.
Currently in services, we don’t make a distinction between input
parameters, options and dependencies.
This can lead to user input modifying the service behavior, whereas it
was not the developer intention.
This patch addresses the issue by changing how data is provided to
services:
- `params` is now used to hold all data coming from outside (typically
user input from a controller) and a contract will take its values from
`params`.
- `options` is a new key to provide options to a service. This typically
allows changing a service behavior at runtime. It is, of course,
totally optional.
- `dependencies` is actually anything else provided to the service (like
`guardian`) and available directly from the context object.
The `service_params` helper in controllers has been updated to reflect
those changes, so most of the existing services didn’t need specific
changes.
The options block has the same DSL as contracts, as it’s also based on
`ActiveModel`. There aren’t any validations, though. Here’s an example:
```ruby
options do
attribute :allow_changing_hidden, :boolean, default: false
end
```
And here’s an example of how to call a service with the new keys:
```ruby
MyService.call(params: { key1: value1, … }, options: { my_option: true }, guardian:, …)
```
Currently, when calling a service with its block form, a `#result`
method is automatically created on the caller object. Even if it never
clashed so far, this could happen.
This patch removes that method, and instead use a more classical way of
doing things: the result object is now provided as an argument to the
main block. This means if we need to access the result object in an
outcome block, it will be done like this from now on:
```ruby
MyService.call(params) do |result|
on_success do
# do something with the result object
do_something(result)
end
end
```
In the same vein, this patch introduces the ability to match keys from
the result object in the outcome blocks, like we already do with step
definitions in a service. For example:
```ruby
on_success do |model:, contract:|
do_something(model, contract)
end
```
Instead of
```ruby
on_success do
do_something(result.model, result.contract)
end
```
Creating or updating flags generates global side effects. Sometimes it
seems the state can leak from the flag specs.
This is probably related to the use of `fab!`. This patch replaces those
calls with standard `let`s. While the overall performances of these
tests will be a little less good, their state should not leak anymore.
This patch improves the custom `array` type available in contracts.
It’s now able to split strings on `|` on top of `,`, and to be more
consistent, it also tries to cast the resulting items to integers.
This is extracted from https://github.com/discourse/discourse/pull/29129.
In some chat specs, we provide an array as a value for group lists like
`chat_allowed_groups`, which is wrong. This results in a value like
`"1|2|[3]"` instead of `"1|2|3"`.
Currently, `ChatSDK.create` restricts what parameters can be provided to
the underlying service. This prevents the `discourse-ai` plugin from
using it in one of its specs.
This patch allows extra parameters to be provided.
Currently, when the MessageFormat compiler fails on some translations,
we just have the raw output from the compiler in the logs and that’s not
always very helpful.
Now, when there is an error, we iterate over the translation keys and
try to compile them one by one. When we detect one that is failing, it’s
added to a list that is now outputted in the logs. That way, it’s easier
to know which keys are not properly translated, and the problems can be
addressed quicker.
---
The previous implementation of this patch had a bug: it wasn’t handling
locales with country/region code properly. So instead of iterating over
the problematic keys, it was raising an error.
While using `OpenStruct` is nice, it’s generally not a very good idea as
it usually leads to performance problems.
The `OpenStruct` source code even says basically to avoid it.
Since the context object is crucial in our services, this patch replaces
`OpenStruct` with a custom implementation instead.
Currently, when the MessageFormat compiler fails on some translations,
we just have the raw output from the compiler in the logs and that’s not
always very helpful.
Now, when there is an error, we iterate over the translation keys and
try to compile them one by one. When we detect one that is failing, it’s
added to a list that is now outputted in the logs. That way, it’s easier
to know which keys are not properly translated, and the problems can be
addressed quicker.
Currently in services, the `contract` step is only used to define where
the contract will be called in the execution flow. Then, a `Contract`
class has to be defined with validations in it.
This patch allows the `contract` step to take a block containing
validations, attributes, etc. directly. No need to then open a
`Contract` class later in the service.
It also has a nice side effect, as it’s now easy to define multiples
contracts inside the same service. Before, we had the `class_name:`
option, but it wasn’t really useful as you had to redefine a complete
new contract class.
Now, when using a name for the contract other than `default`, a new
contract will be created automatically using the provided name.
Example:
```ruby
contract(:user) do
attribute :user_id, :integer
validates :user_id, presence: true
end
```
This will create a `UserContract` class and use it, also putting the
resulting contract in `context[:user_contract]`.
With the current implementation, a service step can be written as:
```ruby
def my_step(a_default_value: 2)
…
end
```
That’s a pattern we want to avoid as default values (if needed) should
be probably defined in a contract.
This patch makes a service raise an exception if a default value is
encountered.
Currently, when certain search terms are provided, this can lead to
`Search.need_segmenting?` raising an error because it makes `URI#path`
to return `nil` instead of a string.
This patch forces a cast to string so it won’t raise anymore.
This will help to enforce a consistent pattern for creating service
actions.
This patch also namespaces actions and policies, making everything
related to a service available directly in
`app/services/<concept-name>`, making things more consistent at that
level too.
- fetch models inside services
- validate `user_id` in contracts
- use policy objects
- extract more logic to actions
- write specs for services and action
This patch removes the `with_service` helper from the code base.
Instead, we can pass a block with actions directly to the `.call` method
of a service.
This simplifies how to use services:
- use `.call` without a block to run the service and get its result
object.
- use `.call` with a block of actions to run the service and execute
arbitrary code depending on the service outcome.
It also means a service is now “self-contained” and can be used anywhere
without having to include a helper or whatever.
Currently, when the default locale is Japanese, the search for a topic
using its URL, path or ID doesn’t work as expected. It will either
return wrong results or no result at all.
The problem lies with how we process the provided terms in Japanese
mode. For example, if `http://localhost/t/-/55` is provided, currently
this will result in `http localhost t 5 5` to be searched for.
This patch addresses the issue by checking whether the provided term
needs segmenting. If the provided term is a number, or a path or a full
URL, then it doesn’t need segmenting. When that happens we skip the
processing we normally apply for Japanese, making the search return the
expected results.
This is a follow-up of d749227e87bfc5df96a5b2fbace7601a83351633.
This patch checks if the key `not_found` is present on the result object
instead of calling `#blank?` on the model, as it can trigger an
`ActiveRecord` relation.
Currently, `FlagSerializer#short_description` properly provides
`base_path` to its translations, but `FlagSerializer#description` does
not. This breaks the link to guidelines when flagging a post, for
example.
This patch provides `base_path` for `FlagSerializer#description` too.
Now that `ActiveRecord` relations are properly handled in a `model`
step, putting this step back will allow the service to stops its
execution if there are no users to be removed without calling the pretty
big SQL query contained in the `CalculateMembershipsForRemoval` action.
This patch allows using an AR relation as a model in services without
fetching associated records. It will just check if the relation is empty
or not. In the former case, the execution will stop at that point, as
expected.
This patch removes two freedom patches:
- `mail_disable_starttls.rb`: this has been fixed in the 2.8 release of
the mail gem, so we don’t need it anymore.
- `rails4.rb`: those methods have been deprecated for a while now and
should have been dropped with Discourse v3.2.
When upgrading to Rails 7.1, we had some problems because we were using
several tagged loggers at the same time. They were all added to the main
broadcast logger shipped with Rails, but the Rails 7.1 codebase contains
a bug making a request being run as many times as there are tagged loggers.
The fix was to use the code from the Rails 7.2 codebase.
This patch adds a small spec to ensure the behavior will stay the proper
one in the future.
Currently, in the list controller, when encountering an unsafe redirect
error, a 404 is rendered. The problem is that it’s done in a way that it
also logs a fatal error (because a `Discourse::NotFound` exception was
raised inside a `rescue_from` block).
This patch addresses that issue by simply rendering a 404 without
raising any error.
Currently, when a badly named category slug is provided, it can lead to
an infinite redirect.
This patch addresses the issue by properly unescaping `request.fullpath`
so the path is successfully rewritten and the redirect happens as
expected.
Following a recent refactor, some methods from `FlagSettings` have been
renamed (`custom_types` -> `additional_message_types`). The
`PostActionType` model was using `custom_types` but when the renaming
was done, it was renamed to `with_additional_message` instead of
`additional_message_types`, which under the right circumstances will
raise an error.
Currently, descriptions for flag types aren’t interpolated, returning
`%{base_path}` in their string, for example. This breaks the navigation
on the sites.
The behavior changed probably because of an upgrade of Ruby, as two
hashes were passed to `I18n.t` (`vars` and `default`) without using the
splat operator.
Currently, if MF definitions are missing (typically because there’s a
compilation error), `I18n.messageFormat` will try to access
`I18n._mfMessages.hasMessage` resulting in a crash that will in turn
crash Ember.
This patch addresses the issue by using the optional chaining operator
making the `I18n.messageFormat` method return a "Missing Key" message.
MF strings won’t be rendered properly, but the site will stay usable.
The current pluralization rules used by the I18n system in Ruby are
obsolete and don’t follow the official rules available at
unicode.org/cldr/charts/45/supplemental/language_plural_rules.html.
Using https://github.com/ruby-i18n/ruby-cldr, new and updated ones have
been generated.
We have some MF strings that are outputting HTML tags (typically links)
and their attributes are using single quotes. The problem is that with
the current implementation of MessageFormat, single quotes act as an
escaping mechanism for special characters like `{`. This then prevents
from interpolating some variables in the strings.
This patch addresses that issue by using double quotes instead,
restoring the expected behavior.
This patch fixes the `i18n:check` rake task which has been broken by
the `MessageFormat` upgrade.
It also adds a spec to ensure we generate valid MF code for all our
available locales.
Currently, when adding translation overrides, values aren’t validated
for MF strings. This results in being able to add invalid plural keys or
even strings containing invalid syntax.
This patch addresses this issue by compiling the string when saving an
override if the key is detected as an MF one.
If there’s an error from the compiler, it’s added to the model errors,
which in turn is displayed to the user in the admin UI, helping them to
understand what went wrong.
We can get translations with invalid plural keys from Crowdin
or from custom overrides. Currently, this will raise an error and the
locales won’t be outputted at all.
This patch addresses this issue by using the new `strict: false` option
of our `messageformat-wrapper` gem, allowing to generate locales even if
there are invalid plural keys present.
Currently, when a plugin registers a new reviewable type or extends a
list method (through `register_reviewble_type` and `extend_list_method`
respectively), the new array is statically computed and always returns
the same value. It will continue to return the same value even if the
plugin is disabled (it can be a problem in a multisite env too).
To address this issue, this patch changes how `extend_list_method`
works. It’s now using `DiscoursePluginRegistry.define_filtered_register`
to create a register on the fly and store the extra values from various
plugins. It then combines the original values with the ones from the
registry. The registry is already aware of disabled plugins, so when a
plugin is disabled, its registered values won’t be returned.
Before Rails 7.1, the `config.i18n.raise_on_missing_translations` option
was raising only in controllers and views, now it’s anywhere in the app.
It means it raises each time `#description` is called for a setting that
is missing a proper description (and we have a ton of them). Most of the
time it’s fine, as those are usually settings that aren’t shown to the
user.
We can’t just let the code blow up every time there’s a setting with a
missing description, that’s why it’s currently returning an empty
string when the translation is missing.
However, this silently broke our I18n integrity spec that was relying on
the old “Translation missing” message to detect missing translations.
This patch addresses this issue by checking the description isn’t an
empty string. It caught a missing translation by the way.
This patch upgrades the MessageFormat library to version 3.3.0 from
0.1.5.
Our `I18n.messageFormat` method signature is unchanged, and now uses the
new API under the hood.
We don’t need dedicated locale files for handling pluralization rules
anymore as everything is now included by the library itself.
The compilation of the messages now happens through our
`messageformat-wrapper` gem. It then outputs an ES module that includes
all its needed dependencies.
Most of the changes happen in `JsLocaleHelper` and in the `ExtraLocales`
controller.
A new method called `.output_MF` has been introduced in
`JsLocaleHelper`. It handles all the fetching, compiling and
transpiling to generate the proper MF messages in JS. Overrides and
fallbacks are also handled directly in this method.
The other main change is that now the MF translations are served through
the `ExtraLocales` controller instead of being statically compiled in a
JS file, then having to patch the messages using overrides and
fallbacks. Now the MF translations are just another bundle that is
created on the fly and cached by the client.
When bad data is provided in the URI for redirecting to a category,
Rails raises an `ActionController::Redirecting::UnsafeRedirectError`
error, leading to a 500 error.
This patch catches the exception to render a 404 instead.
Currently redirecting to an external URL through a permalink doesn’t
work because Rails raises a
`ActionController::Redirecting::UnsafeRedirectError` error.
This wasn’t the case before we upgraded to Rails 7.0.
This patch fixes the issue by using `allow_other_host: true` on the
redirect.
* DEV: Upgrade Rails to 7.1
* FIX: Remove references to `Rails.logger.chained`
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
* DEV: Make parameters optional to all FakeLogger methods
* FIX: Set `override_level` on Logster loggers (#27519)
A followup to a19c5a0e10
* FIX: Don’t duplicate Rack response
---------
Co-authored-by: Jarek Radosz <[email protected]>
* DEV: Upgrade Rails to 7.1
* FIX: Remove references to `Rails.logger.chained`
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
* DEV: Make parameters optional to all FakeLogger methods
* FIX: Set `override_level` on Logster loggers (#27519)
A followup to a19c5a0e10
* FIX: Don’t duplicate Rack response
---------
Co-authored-by: Jarek Radosz <[email protected]>
`Rails.logger.chained` was provided by Logster before Rails 7.1
introduced their broadcast logger. Now all the loggers are added to
`Rails.logger.broadcasts`.
Some code in our initializers was still using `chained` instead of
`broadcasts`.
Currently when a cache entry is corrupt, we log the event without doing
anything else. It means the cache is still corrupt, and the proper value
isn’t computed again.
Normally, it’s very rare the cache becomes corrupt, but it can happen
when upgrading Rails for example and the cache format changes. This is
normally handled automatically by Rails but since we’re using a custom
cache class, we have to do it ourselves.
This patch takes the same approach the Rails team did, when a cache
entry is corrupt, we treat it as a miss, recomputing the proper value
and caching it in the new format.
The version of Chromium we have in our images (120) is relatively
unstable and our system specs break regularly.
This patch makes sure Chrome is used instead for system specs.
To generate letter avatars, we’re currently using the ImageMagick suite
and we’re using the Helvetica font family. However, that font isn’t
shipped anymore in the latest stable version of Debian (Bookworm).
Instead it seems to have been replaced by the Nimbus font. The rendering
is extremely similar (not to say it’s the same thing) so it shouldn’t be
noticeable.
That change is necessary for us to upgrade our docker images to Debian
Bookworm.
This patch allows running system specs on an aarch64 Linux system
(typically our `discourse_dev` docker image).
As Chrome isn’t available for the aarch64 architecture (yet), we have to
rely on Firefox instead. This has some drawbacks like not being able to
access the browser logs like we do with the Chrome webdriver.
