eventsource: alternative to wildcard to filter suffix?

[Pierre-Alexandre35]

Hello,

We currently have a Minio event-source listens to minio bucket. Our bucket used to be like this that have 2 differents sensors wich launch 2 differents pipelines (1 for the reports one for the invoices):
bucket/uploads/reports/ --> path where .csv files are uploaded
bucket/uploads/invoices/ --> path where .csv files are uploaded

This was working as expected with that eventsource file:

{{- $root := .Values -}}
{{- range $value := .Values.eventsource }}
---
apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: "{{$value.name}}"
spec:
  minio:
    humssource:
      bucket:
        name: {{ $root.minio.bucket }}
      endpoint: {{ $root.minio.endpoint }}
      events:
        - s3:ObjectCreated:Put
      filter:
        prefix: {{$value.filter.prefix}}
      insecure: {{$value.insecure}}
      accessKey:
        key: {{ $root.minio.accessKey.key }}
        name: {{ $root.minio.accessKey.name }}
      secretKey:
        key: {{ $root.minio.secretKey.key }}
        name: {{ $root.minio.secretKey.name }}
{{- end }}

values.yaml

  - name: "eventsource-minio-invoices"
    filter:
        prefix: "uploads/invoices"
    insecure : true

  - name: "eventsource-minio-reports"
    filter:
        prefix: "uploads/reports"
    insecure : true

This was creating 2 eventsources correctly which launched 2 differents pipelines. Howver, the requirements changed and now the minio structure changed to this:

bucket/uploads/{COUNTRY}/{CITY}/invoices
bucket/uploads/{COUNTRY}/{CITY}/reports. 

it looks like this:

my-bucket
└── uploads
    ├── england
    │   ├── leeds
    │   │   └── invoices
    │   │       └── reports
    │   ├── london
    │   │   ├── invoices
    │   │   └── reports
    │   └── manchester
    │       ├── invoices
    │       └── reports
    └── france
        ├── nice
        │   ├── invoices
        │   └── reports
        ├── rennes
        │   ├── invoices
        │   │   └── 2024-02-01-548745.csv
        │   └── reports
        └── toulouse
            ├── invoices
            └── reports

There are so many differents countries and cities. I will also need to extract the country and city name on the pipeline. My issue here is I can't do this:

  - name: "eventsource-minio-invoices"
    filter:
        prefix: "uploads/*/*/invoices"
    insecure : true

  - name: "eventsource-minio-reports"
    filter:
        prefix: "uploads/*/*/reports"
    insecure : true

I have also tried to do this:

  - name: "eventsource-minio-invoices"
    filter:
        suffix: "uploads"
        prefix: "invoices"
    insecure : true

  - name: "eventsource-minio-reports"
    filter:
        suffix: "uploads"
        prefix: "reports"
    insecure : true

this is not working as well since the suffix need the be the last part, in this case the file name. Do you have any suggestion for me?

[Joibel]

This is the wrong repository for this, please ask it in https://github.com/argoproj/argo-events/discussions

[agilgur5]

OP opened argoproj/argo-events#3325