-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Auto-generated Pull Request for refactor/uniprot-fetch #433
Commits on Aug 23, 2024
-
refactor(uniprot.go): rename redis import alias for clarity and updat…
…e environment variable keys The import alias for the Redis package is changed from 'r' to 'rds' to enhance code readability and avoid confusion with other variables or packages. Additionally, the environment variable keys for Redis service host and port are simplified from 'REDIS_MASTER_SERVICE_HOST' and 'REDIS_MASTER_SERVICE_PORT' to 'REDIS_SERVICE_HOST' and 'REDIS_SERVICE_PORT' respectively, to make them more intuitive and consistent with common naming conventions.
Configuration menu - View commit details
-
Copy full SHA for b6fb1f3 - Browse repository at this point
Copy the full SHA b6fb1f3View commit details -
refactor(mapping.go): update Uniprot API URL and parameters for clari…
…ty and efficiency The Uniprot API URL and its parameters in `mapping.go` have been updated to use the new REST API endpoint. This change simplifies the URL construction and aligns with the updated API standards. The parameters are now more explicitly defined, improving the readability and maintainability of the code. The response format has been changed from tab-separated values to JSON, and the number of results returned is explicitly set to 500, enhancing data handling and parsing capabilities.
Configuration menu - View commit details
-
Copy full SHA for e9e09bd - Browse repository at this point
Copy the full SHA e9e09bdView commit details -
feat(uniprot.go): add UniProt API response structs to parse JSON data
Adding new structs `UniProtResponse`, `UniProtEntry`, `UniProtCrossReference`, and `UniProtCrossRefProperty` enables the application to deserialize JSON responses from the UniProt API effectively.
Configuration menu - View commit details
-
Copy full SHA for 1995dd9 - Browse repository at this point
Copy the full SHA 1995dd9View commit details -
feat: upgrade Redis client from v7 to v9 for enhanced features and pe…
…rformance This update transitions the Redis client library from version 7 to version 9 across the project. The upgrade includes changes in the `go.mod` file to reflect the new version dependency, and updates in the source code to use the new `github.com/redis/go-redis/v9` import path.
Configuration menu - View commit details
-
Copy full SHA for 741e72f - Browse repository at this point
Copy the full SHA 741e72fView commit details -
feat(uniprot.go): add extractNextPageURL function to parse pagination…
… URLs The new function `extractNextPageURL` is introduced to handle the extraction of the 'next' page URL from the Link header in API responses.
Configuration menu - View commit details
-
Copy full SHA for f4bb789 - Browse repository at this point
Copy the full SHA f4bb789View commit details -
refactor(uniprot.go): simplify gene name handling and extract cross-r…
…eference info The function `handleGeneNames` is replaced with `extractCrossReferenceInfo` to streamline the process of extracting gene names and dictyBase IDs from UniProt entries. This change simplifies the code by removing redundant error handling and Redis operations, focusing instead on directly extracting and returning the necessary information.
Configuration menu - View commit details
-
Copy full SHA for 8447bf9 - Browse repository at this point
Copy the full SHA 8447bf9View commit details -
refactor(uniprot.go): replace handleGeneIDs with extractUniprotMaps f…
…or better abstraction The function `handleGeneIDs` was replaced with `extractUniprotMaps` to improve the abstraction level and maintainability of the code. The new function directly constructs a list of `UniprotMap` structures from the `UniProtResponse`, which simplifies the handling of UniProt data by separating the concerns of data extraction and data storage.
Configuration menu - View commit details
-
Copy full SHA for 6520e26 - Browse repository at this point
Copy the full SHA 6520e26View commit details -
refactor(uniprot.go): replace readLine function with decodeUniprotRes…
…ponse to streamline data processing The readLine function was removed and replaced with decodeUniprotResponse to handle the response from the UniProt API more efficiently. This change allows the application to directly decode the gzip-compressed JSON response into a structured format, improving both the clarity and performance of the data processing workflow.
Configuration menu - View commit details
-
Copy full SHA for 2162191 - Browse repository at this point
Copy the full SHA 2162191View commit details -
refactor(uniprot.go): replace handleIsoforms with loadUniprotMapsToRe…
…dis for enhanced data handling The function `handleIsoforms` is replaced by `loadUniprotMapsToRedis` to improve the efficiency and clarity of data handling in Redis. The new function uses a pipeline for batch processing Redis commands, which reduces the number of round trips to the server.
Configuration menu - View commit details
-
Copy full SHA for d643998 - Browse repository at this point
Copy the full SHA d643998View commit details -
refactor(uniprot.go): streamline Uniprot data processing and enhance …
…security This change introduces several improvements to the Uniprot data processing in the Go application: 1. **Security Enhancements**: The URL validation ensures that only HTTPS requests are made to the expected domain, enhancing the security of data transfers. 2. **Code Simplification**: The removal of the `Count` struct and related logic simplifies the codebase, making it easier to maintain and understand. 3. **Efficiency Improvements**: By replacing the line-by-line parsing with JSON decoding, the process becomes more efficient and less error-prone. 4. **Pagination Support**: The addition of pagination handling allows the application to process large datasets that span multiple pages, ensuring complete data retrieval. 5. **Error Handling**: Improved error handling provides clearer error messages, making it easier to troubleshoot issues during data fetching and processing.
Configuration menu - View commit details
-
Copy full SHA for f7b3084 - Browse repository at this point
Copy the full SHA f7b3084View commit details -
refactor(uniprot.go): improve error formatting in Redis ping error me…
…ssage Adding context support to the Redis ping method allows for better control over timeouts and cancellations of database operations, which is crucial for maintaining the responsiveness and stability of the application. The error message formatting has been improved by using `%w` to wrap the error, which aids in error handling by allowing the error to be unwrapped in higher-level code.
Configuration menu - View commit details
-
Copy full SHA for 79490cc - Browse repository at this point
Copy the full SHA 79490ccView commit details
Commits on Aug 24, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5822c8d - Browse repository at this point
Copy the full SHA 5822c8dView commit details -
refactor: standardize ID naming across multiple files for consistency
Standardizing the naming convention for 'ID' related variables and functions across various files enhances code readability and maintainability.
Configuration menu - View commit details
-
Copy full SHA for ec00c9e - Browse repository at this point
Copy the full SHA ec00c9eView commit details -
feat(.golangci.yml): increase linter timeout from 5m to 15m to accomm…
…odate larger projects Increasing the timeout for the linter from 5 minutes to 15 minutes allows for more comprehensive linting processes, especially beneficial for larger projects that may require more time to analyze fully.
Configuration menu - View commit details
-
Copy full SHA for 89fa71f - Browse repository at this point
Copy the full SHA 89fa71fView commit details -
chore(.gitignore): add 'importer' directory to .gitignore
Adding the 'importer' directory to the .gitignore file ensures that temporary or sensitive files within this directory are not tracked or uploaded to the version control system, maintaining the cleanliness and security of the repository.
Configuration menu - View commit details
-
Copy full SHA for 834677c - Browse repository at this point
Copy the full SHA 834677cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8251c10 - Browse repository at this point
Copy the full SHA 8251c10View commit details
Commits on Aug 26, 2024
-
feat(cli/flag.go): add new flag.go file to define CLI flags for unipr…
…ot module Introduces a new file to handle command-line interface flags specifically for the uniprot module, laying the groundwork for future CLI enhancements and functionality.
Configuration menu - View commit details
-
Copy full SHA for f63e6bd - Browse repository at this point
Copy the full SHA f63e6bdView commit details -
feat(client.go): add Redis client setup in new client package
Introduces a new client package for managing Redis connections within the application. This setup includes a function `SetRedisClient` that initializes a Redis client using configuration from the application context and verifies the connection by pinging the Redis server.
Configuration menu - View commit details
-
Copy full SHA for cab6451 - Browse repository at this point
Copy the full SHA cab6451View commit details -
feat(cli/flag.go): add UniprotFlags function and default uniprotURL
This change introduces a new function `UniprotFlags` to the CLI package, which provides flags for configuring the Uniprot URL and Redis service connection details. The `uniprotURL` is pre-configured to fetch specific data from the Uniprot database, enhancing the ease of use for the end-users by providing a ready-to-use URL. The Redis service host and port can now be configured via environment variables or command-line flags, improving the flexibility and configurability of the application.
Configuration menu - View commit details
-
Copy full SHA for 355acec - Browse repository at this point
Copy the full SHA 355acecView commit details -
feat(internal/uniprot/cli): add action.go to handle UniProt data fetc…
…hing and Redis storage Introduces a new module in the `internal/uniprot/cli` package for handling the fetching of UniProt data and storing it in Redis. This module includes functionality to validate URLs, make HTTP requests to the UniProt API, parse the JSON response, and store the mappings between UniProt IDs and gene names/IDs in Redis.
Configuration menu - View commit details
-
Copy full SHA for c71e9d0 - Browse repository at this point
Copy the full SHA c71e9d0View commit details -
refactor(internal/uniprot/cli/action.go): streamline Redis client set…
…up and error handling The removal of the `client.SetRedisClient` function and the direct use of `registry.GetRedisClient` simplifies the Redis client setup process. This change ensures that the Redis client is retrieved directly from the registry without additional setup overhead, enhancing code maintainability. Additionally, replacing generic error returns with `cli.Exit` provides clearer error messages and standardized exit codes, improving the command-line interface's usability and error management.
Configuration menu - View commit details
-
Copy full SHA for f3687b7 - Browse repository at this point
Copy the full SHA f3687b7View commit details -
feat(main.go): add Uniprot mappings loading functionality to loader CLI
The loader command line application now supports loading Uniprot mappings, enhancing its functionality to handle more diverse data types. This update includes the necessary CLI flags and setup for Uniprot data, allowing users to manage and load Uniprot mappings directly through the command line interface.
Configuration menu - View commit details
-
Copy full SHA for f4de768 - Browse repository at this point
Copy the full SHA f4de768View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9e0d8e9 - Browse repository at this point
Copy the full SHA 9e0d8e9View commit details -
chore(.gitignore): add 'loader' to .gitignore to prevent tracking of …
…temporary loader files Adding 'loader' to the .gitignore file ensures that temporary files created by the loader process are not tracked by git, keeping the repository clean from unnecessary files.
Configuration menu - View commit details
-
Copy full SHA for 84e86bb - Browse repository at this point
Copy the full SHA 84e86bbView commit details -
feat(cli/action.go): enhance logging in LoadUniprotMappings function
Added detailed logging to the LoadUniprotMappings function to improve traceability and debugging capabilities. This includes logging at the start of processing each Uniprot page, after loading entries to Redis, and for each individual Uniprot entry loaded. Additionally, a summary log entry is added at the end to indicate the total number of entries processed.
Configuration menu - View commit details
-
Copy full SHA for ae59173 - Browse repository at this point
Copy the full SHA ae59173View commit details -
refactor(internal/uniprot/cli/action.go): encapsulate Redis loading l…
…ogic into RedisUniprotLoader class This change introduces a new `RedisUniprotLoader` class that implements the `UniprotLoader` interface, encapsulating the Redis data loading logic. This refactoring improves the code structure by separating concerns, making the `LoadUniprotMappings` function cleaner and more focused on its primary responsibility. The use of an interface for loading also makes the system more flexible and easier to extend or modify in the future, such as adding different storage mechanisms.
Configuration menu - View commit details
-
Copy full SHA for 881af72 - Browse repository at this point
Copy the full SHA 881af72View commit details -
refactor(uniprot/cli): centralize Redis related code into a separate …
…module This change improves the code organization by moving all Redis-related functionalities into a dedicated module under `internal/uniprot/redis`. This separation of concerns makes the codebase easier to maintain and enhances the modularity of the application.
Configuration menu - View commit details
-
Copy full SHA for efee885 - Browse repository at this point
Copy the full SHA efee885View commit details -
feat(go.mod): add new dependencies for JSON handling and in-memory Redis
Added `github.com/alicebob/gopher-json` for improved JSON processing capabilities, enhancing the application's ability to handle JSON data more efficiently. Also, `github.com/alicebob/miniredis/v2` is included to facilitate in-memory Redis testing, which allows for better testing environments without the need for a live Redis setup.
Configuration menu - View commit details
-
Copy full SHA for b8dda8f - Browse repository at this point
Copy the full SHA b8dda8fView commit details -
test(internal/uniprot/redis): add unit tests for RedisUniprotLoader f…
…unctionality Introduces comprehensive unit tests for the RedisUniprotLoader to ensure its reliability in various scenarios, including handling single and multiple UniprotMap entries, error conditions, empty inputs, and duplicate entries.
Configuration menu - View commit details
-
Copy full SHA for e1be29f - Browse repository at this point
Copy the full SHA e1be29fView commit details
Commits on Aug 27, 2024
-
fix(redis_test.go): update test logic to check existence rather than …
…fetching directly Refactoring the test code in `redis_test.go` improves readability by breaking long function calls into multiple lines, making the code easier to read and maintain. Additionally, updating the test logic to use `HExists` instead of `HGet` for checking the existence of keys in Redis is more semantically correct and efficient, as it directly checks for the presence of the key without retrieving its value.
Configuration menu - View commit details
-
Copy full SHA for 1ef59f5 - Browse repository at this point
Copy the full SHA 1ef59f5View commit details -
test(redis_test.go): update unit tests to reflect changes in handling…
… duplicate entries The unit tests in `redis_test.go` are updated to accommodate the new logic for handling duplicate entries in the Redis cache. The tests now verify that the last entry correctly overwrites the previous ones for different keys, ensuring that the cache behaves as expected when duplicate UniprotIDs or GeneIDs are encountered.
Configuration menu - View commit details
-
Copy full SHA for a42f51f - Browse repository at this point
Copy the full SHA a42f51fView commit details -
fix(Dockerfile): rename output binary from 'content' to 'dataloader' …
…for clarity The output binary name is changed from 'content' to 'dataloader' to better reflect its functionality and purpose, enhancing clarity and maintainability of the Dockerfile.
Configuration menu - View commit details
-
Copy full SHA for 08f514f - Browse repository at this point
Copy the full SHA 08f514fView commit details