Skip to content
New issue

Backend Data Fetching, Caching, Tracking

No due date 0% complete

This is to integrate any logics for data fetching/caching/tracking to implement a hybrid approach for optimized performance and cost effective.

Data Fetching

Define a common data fetcher interface to implement different fetching strategies, Athena, s3, database.
The initial efforts will focus on Athena to handle any request for data fetching.

Caching

  • Imp…

This is to integrate any logics for data fetching/caching/tracking to implement a hybrid approach for optimized performance and cost effective.

Data Fetching

Define a common data fetcher interface to implement different fetching strategies, Athena, s3, database.
The initial efforts will focus on Athena to handle any request for data fetching.

Caching

  • Implement different caching tools, and analyze queries for pre-computed, pre-aggregated results.
  • Simple request: Fetch from database, S3, or Redis directly.
  • Complex request: Use Athena queries through the WTKLedClient.

Tracking

Connects the backend server with Postgres DB (Type: Postgres, DB Name: tap_api)
Primarily the database will be used to track/log queries as well as key results from the queries. These results will be visualized on a map for research purposes and presented to DOE.

Loading