Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data Liberation] Add XML API, Stream API, WXR URL Rewriter API #1952

Merged
merged 36 commits into from
Oct 28, 2024
Merged
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
1ef710f
Data liberation: Kickoff the project
adamziel Oct 11, 2024
234a8bf
Port the URL rewriters from adamziel/site-transfer-protocol
adamziel Oct 13, 2024
819febd
Port WP_HTML_Processor et al. from WordPress
adamziel Oct 13, 2024
0a6167b
Move WordPress core files
adamziel Oct 13, 2024
826fe75
Outline the next steps
adamziel Oct 13, 2024
0633e6f
Add PHPCS and CBF
adamziel Oct 14, 2024
4406fcf
Update HTML API, fix unit tests
adamziel Oct 15, 2024
0cfd334
Merge branch 'trunk' into data-liberation-bring-in-php-parsers
adamziel Oct 15, 2024
b90a9d6
Bump CI PHP version to 8.1
adamziel Oct 15, 2024
081535b
Adjust the CI setup for PHP
adamziel Oct 15, 2024
aca88fe
Run npm instlal insteaf of installing just nx
adamziel Oct 15, 2024
897af50
Use the correct nx project name
adamziel Oct 15, 2024
f7679b0
Remove the network functions and only lint the src directory
adamziel Oct 15, 2024
5b9ec7d
Remove special casing for direct matching pathname prefixes
adamziel Oct 15, 2024
97fed71
Fix linting errors
adamziel Oct 15, 2024
96c1ce4
Move the additional functions to pbpcbf.php
adamziel Oct 15, 2024
e15408a
Replace iterate_urls with url_matches
adamziel Oct 15, 2024
b788eea
Lint PHP
adamziel Oct 15, 2024
b83933c
Thoroughly test WP_URL_In_Text_Processor
adamziel Oct 28, 2024
fb0204c
Enable tests for WP_Block_Markup_Processor
adamziel Oct 28, 2024
b1ea8dc
Enable all PHPUnit tests
adamziel Oct 28, 2024
4335044
Enable URLParserWHATWGComplianceTests
adamziel Oct 28, 2024
91863ca
move $is_relative declaration clsoer to where it's used
adamziel Oct 28, 2024
d2aeea4
Add a single tricky test case for wp_rewrite_urls()
adamziel Oct 28, 2024
60db1e1
Preserve urlencoded data in the rewritten path
adamziel Oct 28, 2024
2da0386
Unit test urldecoding UTF-8 data
adamziel Oct 28, 2024
54bea02
Lint
adamziel Oct 28, 2024
54c901d
Remove messing with private WP_HTML_Tag_Processor attributes
adamziel Oct 28, 2024
a62532b
Remove the commented out dead code from WP_URL_In_Text_Processor
adamziel Oct 28, 2024
238decd
Uncomment the public suffix list verification
adamziel Oct 28, 2024
37622ab
PHP 8.1 compat
adamziel Oct 28, 2024
e12190f
PHP 8.1 compliance
adamziel Oct 28, 2024
34cac36
[Data Liberation] Add XML API, Stream API, WXR URL Rewriter API
adamziel Oct 28, 2024
5dadb3e
Adjust how append_bytes() work to fix a failing test
adamziel Oct 28, 2024
06c5503
Lint
adamziel Oct 28, 2024
2af08a4
Merge branch 'trunk' into data-liberation-xml-parsers
adamziel Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Adjust how append_bytes() work to fix a failing test
adamziel committed Oct 28, 2024
commit 5dadb3e825afbfe81dbe24c7fb2af86a748d1f04
Original file line number Diff line number Diff line change
@@ -54,6 +54,8 @@ static public function stream($node_visitor_callback)
{
$xml_processor = new WP_XML_Processor('', [], WP_XML_Processor::IN_PROLOG_CONTEXT);
return new ProcessorByteStream($xml_processor, function($state) use($xml_processor, $node_visitor_callback) {
$buffer = $xml_processor->flush_processed_xml();

$new_bytes = $state->consume_input_bytes();
if (null !== $new_bytes) {
$xml_processor->append_bytes($new_bytes);
@@ -64,9 +66,8 @@ static public function stream($node_visitor_callback)
$node_visitor_callback($xml_processor);
}

$buffer = '';
if ($tokens_found > 0) {
$buffer .= $xml_processor->get_processed_xml();
$buffer .= $xml_processor->flush_processed_xml();
} else if (
$tokens_found === 0 &&
!$xml_processor->is_paused_at_incomplete_input() &&
@@ -87,6 +88,7 @@ static public function stream($node_visitor_callback)
});
}


public function pause() {
return array(
'xml' => $this->xml,
@@ -114,19 +116,27 @@ public function resume($paused) {
*/
public function append_bytes( string $next_chunk )
{
$this->xml .= $next_chunk;
}

public function flush_processed_xml() {
$this->get_updated_xml();

$new_xml = $this->get_unprocessed_xml() . $next_chunk;
$breadcrumbs = $this->get_breadcrumbs();
$processed_xml = $this->get_processed_xml();
$unprocessed_xml = $this->get_unprocessed_xml();

$breadcrumbs = $this->get_breadcrumbs();
$parser_context = $this->get_parser_context();

$this->reset_state();

$this->xml = $new_xml;
$this->xml = $unprocessed_xml;
$this->stack_of_open_elements = $breadcrumbs;
$this->parser_context = $parser_context;
$this->had_previous_chunks = true;
}

return $processed_xml;
}

/**
* Constructor.
Original file line number Diff line number Diff line change
@@ -1898,7 +1898,6 @@ public function has_bookmark( $bookmark_name ) {
return array_key_exists( $bookmark_name, $this->bookmarks );
}

protected $first_time = false;
public function get_processed_xml() {
// Flush updates
$this->get_updated_xml();
Original file line number Diff line number Diff line change
@@ -92,5 +92,4 @@ https://playground.internal/path-not-taken was the second best choice.
</wp:postmeta>
</item>
</channel>
</rss>

</rss>