Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outbox: record logs #1197

Open
wants to merge 154 commits into
base: trunk
Choose a base branch
from
Open

Outbox: record logs #1197

wants to merge 154 commits into from

Conversation

mattwiebe
Copy link
Contributor

We don't explicitly have to merge this, since we're already two levels deep on a feature branch, but this is the idea.

We can build on this with CLI and in the Outbox post listing, including retry functionality, and "copy to clipboard" so users can submit that with bug reports.

@mattwiebe
Copy link
Contributor Author

What do we want to do with the data we log? Is it meant for site owners or us?

A few use-cases for site owners:

  1. Being able to verify that updates are being POSTed to followers. Most of the work our plugin does is invisible to the user, this can make it visible. That's why I'm storing successful logs, not just errors.
  2. Ability to manually retry failed POSTs (would need some UI)
  3. Monitoring: show a "networking health" dashboard, highlight servers with failed POSTs

But also for us, give the ability to, say, export a debug log of failed requests (sanitized of any PII) when getting support would give us a huge advantage over today.

I wonder if it would make sense to integrate with Debug Bar.

That seems like a great idea! I was initially thinking of integrating it into an Outbox post_type listing as a column.

mattwiebe and others added 3 commits January 22, 2025 16:23
* Outbox: Fix Query

If `author` is null, it will return every activity for the blog and application Actor, but we only want to have the Actors activities.

* Update includes/rest/class-outbox-controller.php

Co-authored-by: Konstantin Obenland <[email protected]>

* fix user check!

* fix phpcs

* use constants

* Add tests for new functionality

* Update tests/includes/rest/class-test-outbox-controller.php

---------

Co-authored-by: Konstantin Obenland <[email protected]>
@@ -104,8 +104,24 @@ private static function send_activity_to_followers( $activity, $actor_id, $outbo

$json = $activity->to_json();

// We will store the json as generated by the transformer, even though it 's also in plaintext in $outbox_item->post_content
// This will also allow us to keep the logs below leaner.
\add_post_meta( $outbox_item->ID, 'activitypub_sent_json', $json );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this information?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coming back to it, I was just thinking of all of the places I would want to possibly error_log while debugging this code. And this was one. But since we store it in the outbox content anyway, it's definitely superfluous and I'll remove it.

// This will also allow us to keep the logs below leaner.
\add_post_meta( $outbox_item->ID, 'activitypub_sent_json', $json );
// This will allow error checking later, that all inboxes have been sent.
\add_post_meta( $outbox_item->ID, 'activitypub_sent_inboxes', $inboxes, true );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we simply keep a list of inboxes that bounce? this is a lot of data we store per Outbox Activity!?!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted a fully debuggable picture of what happens, given that the list of inboxes POSTed to for any given outbox activity will be somewhat dynamically generated. At the same time I was envisioning using it as a basis for rolling batching, which I did not get to.

But since we're not doing batching yet either, and all of the inboxes are stored in the logs below, we don't need it right now.

obenland and others added 2 commits January 23, 2025 08:17
* Outbox: Tests and fixes for Post Scheduler

* Move extrafields post to Actor class

* Add test coverage for remaining Actor methods

* Move cap-manipulating tests to the end

* Account for inheritance in Transformer Factory

Reverts changes to `add_to_outbox()`

* Remove outbox items after every test.

* Use fully qualified class name

* Split extra fields tests to not hit time limit

* Adjust bog modes

* debug

* debug

* debug

* debug

* Use Profile_update action.

`wp_update_user` wasn't introduced until WP 6.3.

* revert changes to add_to_outbox
* rename dispatcher

* Simple dispatcher based on the Outbox-Collection

This is a simple rewrite of the current dispatcher system, to use the Outbox instead of the Scheduler.

This is a first draft and will be improved over time, to better handle:

* Re-tries
* Errors
* Logging
* Batch processing

* update changelog

* mark post as `publish` after federation id done

* show only published activities

* fix missing rename

* use pending instead of draft

* do not check for post_status

* fix tests

props @obenland

* Send `Update`s to Blog Actor in dual mode

* Update includes/class-dispatcher.php

Co-authored-by: Konstantin Obenland <[email protected]>

* Update includes/class-dispatcher.php

Co-authored-by: Konstantin Obenland <[email protected]>

* Update includes/rest/class-outbox-controller.php

Co-authored-by: Konstantin Obenland <[email protected]>

* Check if Activity should be sent to followers

* the unique check will be done `send_activity_to_followers`

* fix tests

* fix PHPCS

* move scheduler behind action

* Add `private` visibility

* Add Announce activity

* Announce the full object!

* fix indent

* Update includes/transformer/class-base.php

Co-authored-by: Konstantin Obenland <[email protected]>

* add doc-block

* only boost content not profile updates

* Also handle `Delete` when bundling Blog Actor inboxes

* Update docs

* Avoid activitypub_actor_mode bleeding into other tests

* Fix comments tests

* Account for inheritance in Activity objects

* Move hook to the right place

* fix typo!

* trigger scheduler

* Fix tests

---------

Co-authored-by: Matt Wiebe <[email protected]>
Co-authored-by: Konstantin Obenland <[email protected]>
Base automatically changed from update/dispatcher to add/outbox-collection January 23, 2025 14:43
Base automatically changed from add/outbox-collection to trunk January 31, 2025 06:52
@github-actions github-actions bot added the [Focus] Compatibility Ensuring the plugin plays well with other plugins label Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Feature] Collections [Focus] Compatibility Ensuring the plugin plays well with other plugins Skip Changelog Disables the "Changelog Updated" action for PRs where changelog entries are not necessary.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants