Breakdown during filter assembly RU AdList

[dimisa-RUAdList]

Prerequisites

• I verified that this is not a filter issue (MUST be reported at filter issue tracker)
• This is not a support issue or a question
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue
• The issue is not present after wholly disabling uBlock Origin ("uBO") in the browser
• I checked the documentation to understand that the issue I report is not a normal behavior

I tried to reproduce the issue when...

• uBO is the only extension
• uBO with default lists/settings
• using a new, unmodified browser profile

Description

Today I discovered that the RU AdList filter: https://easylist-downloads.adblockplus.org/advblock+cssfixes.txt stopped building properly. When building on the adblockplus.org domain, the js-fixes-experimental.txt file with rules specific to uBlock Origin was no longer pulled into it: https://github.com/easylist/ruadlist/blob/master/advblock%2Bcssfixes.txt

For a while I thought it was some kind of bug on the adblockplus.org server, but then I got a response from them that it was not a bug. Eyeo has launched a new system for delivering filter lists, while those that are not recommended for Adblock Plus are not pulled into builds.

As the css-fixes-experimental.txt file continued to be pulled into the build, I figured it was because it was originally intended to be used in Adblock Plus (in the future) and has syntax compatible with it #?#. Therefore, I remade js-fixes-experimental.txt similarly, in the hope that this will be enough: easylist/ruadlist@4c512db

But none of this happened, the file still did not pull up. Then I tried to add the !#include directive to the css-fixes-experimental.txt file, both in short form: easylist/ruadlist@e269436 and with the full path: easylist/ruadlist@4762478. In both cases, there was no success.

I am currently looking for a way to return the js-fixes-experimental.txt file to the RU AdList filter for uBlock Origin

As far as I understand, a similar issue has already been discussed here: #2191

A specific URL where the issue occurs

mail.ru

Steps to Reproduce

unknown

Expected behavior

unknown

Actual behavior

unknown

uBlock Origin version

uBlock Origin development build 1.43.1b10

Browser name and version

Google Chrome 103.0.5060.134

Operating System and version

Windows 7

[dimisa-RUAdList]

@gorhill

[gothic-bum]

Per:
https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#include-file-name

The sub-list must be in the same directory as the main one, i.e. it is not allowed to load a sub-list which is located outside where the current one resides.

[dimisa-RUAdList]

I can create a new RU AdList file on github, like this:

[Adblock Plus 2.0]
! Title: RU AdList for uBO only
! Last modified: %timestamp%
! Expires: 2 days (фильтр обновляется автоматически каждые 2 дня)
! Homepage: https://forums.lanik.us/viewforum.php?f=102
!
! Комбинированная версия списков фильтров RU AdList против рекламы
! и экспериментальных списков RU AdList CSS Fixes и RU AdList JS Fixes
! На данный момент нормально работает только в uBlock Origin
!
! --- RU AdList Classic ---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/adservers.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/first_level.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/general_block.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/general_hide.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/popup.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_antisocial.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_block.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_hide.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_special.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/thirdparty.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/whitelist.txt
!
! --- RU AdList CSS Fixes---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/css-fixes-experimental.txt
!
! --- RU AdList JS Fixes---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/js-fixes-experimental.txt

But there are two problems. I need to add AWRL there: https://github.com/easylist/antiadblockfilters/tree/master/antiadblockfilters

And the address of this new file must be added to uBlock Origin.

[dimisa-RUAdList]

I can put this file myself here: https://github.com/easylist/ruadlist

Is this how it will work? >

! Title: RU AdList for uBO only
! Last modified: %timestamp%
! Expires: 2 days (фильтр обновляется автоматически каждые 2 дня)
! Homepage: https://forums.lanik.us/viewforum.php?f=102
!
! Комбинированная версия списков фильтров RU AdList против рекламы
! и экспериментальных списков RU AdList CSS Fixes и RU AdList JS Fixes
! На данный момент нормально работает только в uBlock Origin
!
! --- RU AdList Classic ---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/adservers.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/first_level.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/general_block.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/general_hide.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/popup.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_antisocial.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_block.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_hide.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/specific_special.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/thirdparty.txt
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/advblock/whitelist.txt
!
! --- RU AdList CSS Fixes---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/css-fixes-experimental.txt
!
! --- RU AdList JS Fixes---
!#include https://raw.githubusercontent.com/easylist/ruadlist/master/js-fixes-experimental.txt
!
! --- Adblock Warning Removal List---
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_arabic.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_chinese.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_czech.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_dutch.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_english.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_finnish.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_french.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_german.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_hebrew.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_indonesian.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_italian.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_latvian.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_polish.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_romanian.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_russian.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_slovak.txt
!#include https://raw.githubusercontent.com/easylist/antiadblockfilters/master/antiadblockfilters/antiadblock_spanish.txt

[gorhill]

I am thinking about you create a GitHub worker to stitch the lists and import at the same time https://github.com/easylist/antiadblockfilters/tree/master/antiadblockfilters, something like I did to copy content of uAssets to uAssetsCDN (example) -- it's just mostly shell script commands in the end so stitching should be straightforward.

[gorhill]

Is this how it will work?

This won't work, some of the sublists are outside https://github.com/easylist/ruadlist.

[dimisa-RUAdList]

To be honest, I've never done anything like this. But if I have clear step-by-step instructions, then I'm ready.

[gorhill]

Ok, I will try to gather all the steps, not sure I can do that today though. I will list the steps here and add more steps as I identify them:

[dimisa-RUAdList]

Maybe I should place the file without AWRL for now, but so that users have scriptlets? Without them, a lot of ads are unblocked, especially on problem sites.

Everything is in one place: https://raw.githubusercontent.com/easylist/ruadlist/*

[gorhill]

What would be the URL of the list for now?

[dimisa-RUAdList]

https://github.com/easylist/ruadlist/RuAdList-uBO.txt
Or:
https://raw.githubusercontent.com/easylist/ruadlist/master/RuAdList-uBO.txt

[gorhill]

I get 404: Not Found with https://raw.githubusercontent.com/easylist/ruadlist/master/RuAdList-uBO.txt.

[dimisa-RUAdList]

I didn't know it would be needed right now. Uploaded: easylist/ruadlist@9dabf30

[gorhill]

You need to use relative URLs:

! --- RU AdList Classic ---
!#include advblock/adservers.txt
!#include advblock/first_level.txt
!#include advblock/general_block.txt
!#include advblock/general_hide.txt
!#include advblock/popup.txt
!#include advblock/specific_antisocial.txt
!#include advblock/specific_block.txt
!#include advblock/specific_hide.txt
!#include advblock/specific_special.txt
!#include advblock/thirdparty.txt
!#include advblock/whitelist.txt
!
! --- RU AdList CSS Fixes---
!#include css-fixes-experimental.txt
!
! --- RU AdList JS Fixes---
!#include js-fixes-experimental.txt

[dimisa-RUAdList]

Ok: easylist/ruadlist@227d015

Will the %include easylist combinations:antiadblockfilters.txt% directive not work here by any chance?

[gorhill]

assets.json updates every ~13 days, so it will take a while before everybody is set to the new location.

Tomorrow I will try to detail the whole setup to have the list stitched and published automatically each time you change any of the sublists.

[dimisa-RUAdList]

So that users do not suffer, I have added an unsynchronized AWRL for now.
easylist/ruadlist@a3efea9
easylist/ruadlist@5889e6b

[krystian3w]

As far as I understand, a similar issue has already been discussed here: #2191

IMO not similar due fanboy social at:

Https://easylist-downloads.adblockplus.org/fanboy-social.txt has don't touched #!include (but maybe outdated real supplement file or deleted form server yesterday) #1574 (maybe more about space between !#).

[gorhill]

Ok here is what I have, but since I can't test these steps, I can only hope I didn't overlook anything and that this will work as intended. I am assuming you are updating your repo through command line using linux.

In easylist/ruadlist repo...

Create a new branch, named gh-pages (this branch will be empty, this is not a fork of master):

• git switch --orphan gh-pages
• touch RuAdList-uBO.txt
• git add RuAdList-uBO.txt
• git commit -m "Initial commit on gh-pages branch"
• git push origin gh-pages

A GitHub action in master will update the list in the gh-pages branch each time something changes in master

• git checkout master
• mkdir -p .github/workflows
• touch .github/workflows/main.yml
• Paste content of https://gist.github.com/gorhill/fc3b83502b7ac759af53be7d631510ff into .github/workflows/main.yml
• git add .github/workflows/main.yml
• git commit -m "Add a github action to publish official RuAdList-uBO.txt on gh-pages branch"
• git push origin master

This should cause the following URL to become the official RUADList after the the first time the GitHub action executes: https://raw.githubusercontent.com/easylist/ruadlist/gh-pages/RuAdList-uBO.txt.

You can see the history of GitHub action executions at https://github.com/easylist/ruadlist/actions.

When all is confirmed working well, I will update the official URL in assets.json.

One limitation of this workflow is that it won't trigger when something change in https://github.com/easylist/antiadblockfilters, which content is pulled in only at list stitching time, which occurs only when something changes in https://github.com/easylist/ruadlist.

[krystian3w]

FiltersHeroes "Actions" can watch on easylist/antiadblockfilters on every day to found diffrences: https://github.com/FiltersHeroes/ScriptsPlayground/blob/master/Readme_SFLB.md (based on cron in CI actions)

Translation to Python 3.8+ should unlock use local on Windows/macOS (to compare with old VICHS https://github.com/FiltersHeroes/ScriptsPlayground/commits/9ffd0926b6ea409e07ecd383919e3ad7587628b9/Readme_VICHS.md?browsing_rename_history=true&new_path=Readme_SFLB.md&original_branch=master)

Also this is hawkeye handwritten ecosystem - so he know better how fix bugs if fail "install" in repo.

[dimisa-RUAdList]

@gorhill
My thanks for creating the guide. I have a busy schedule right now and I can't seem to find a free interval to follow your instructions. I'm going to do this as soon as I can.

But I would like to solve one problem right now. In uBlock Origin Legacy, a new filter location is not pulled up even after multiple filter updates. I think for him this location should be specified separately. Have a look, please.

[lsh1215225]

https://github.com/easylist/ruadlist/RuAdList-uBO.txt Or: https://raw.githubusercontent.com/easylist/ruadlist/master/RuAdList-uBO.txt

awesome, https://raw.githubusercontent.com/easylist/ruadlist/master/RuAdList-uBO.txt is useful

[gorhill]

Issue is obsolete, RUAdList now supports differential updates at https://github.com/dimisa-RUAdList/RUAdListCDN