Adding report functionality to ml feed

[jay-dhanwant-yral]

To create a report functionality that feeds negative signal to ml feed for serving the negatively engaged videos less

[jay-dhanwant-yral]

Proposed architecture

graph TB
    subgraph "Batch Processing Layer (Cloud Composer)"
        A[Video Embeddings] --> B[K-means Clustering]
        B --> C[Cluster Assignment]
        D[User Interactions] --> E[User Preference Calculator]
        
        C --> F[(Video Clusters Table)]
        E --> G[(User Preferences Table)]
        B --> H[(Cluster Metadata Table)]
    end

    subgraph "Real-time Recommendation Layer (Fly.io)"
        I[User Request] --> J[Load User Profile]
        J --> K[Candidate Generation]
        
        subgraph "Parallel Candidate Generation"
            K --> L1[Personalized Candidates]
            K --> L2[Similarity-based Candidates]
            K --> L3[Diverse Candidates]
            
            L1 --> M[Candidate Merger]
            L2 --> M
            L3 --> M
        end
        
        M --> N[Re-ranking]
        N --> O[Final Feed]
    end

    subgraph "Data Storage (BigQuery)"
        P[(Video Embeddings)]
        Q[(Watch History)]
        R[(User Interactions)]
        
        P --> A
        R --> D
        Q --> J
    end

    subgraph "Caching Layer"
        S[Offchain?]
        S -- Cache --> J
        S -- Cache --> K
    end

    %% Data flow connections
    F --> K
    G --> J
    H --> K

Loading

[siyara-m-yral]

• Should take a week for implementation. But need to connect with Komal for backend work

[jay-dhanwant-yral]

• Create report endpoint in ml feed server
• Create corresponding tables and dags
• Refactor the feed to incorporate the user level block list