Chunk CRUD

[NolanTrem]

Introduces a number of improvements over chunk management.

---

Important

Introduces CRUD operations for chunks, including new API endpoints, database updates, and integration tests.

• Behavior:
  • Adds CRUD operations for chunks, including ingestion, updating, and deletion.
  • New API endpoints in ingestion_router.py for ingest_chunks, update_chunk, and delete.
  • Integration tests added in runner_sdk.py for chunk operations.
• Database:
  • Updates PostgresVectorHandler in vector.py to support chunk retrieval and updates.
  • Adds get_chunk() and modifies get_document_chunks() to order by chunk_order.
• Services:
  • Implements update_chunk_ingress() in ingestion_service.py for updating chunk content and metadata.
  • Adds parse_update_chunk_input() in IngestionServiceAdapter.
• Orchestration:
  • Adds workflows for chunk operations in ingestion_workflow.py and simple/ingestion_workflow.py.
• SDK:
  • Adds updateChunk() method in r2rClient.ts and update_chunks() in ingestion.py for chunk updates.
• Misc:
  • Updates action.yml to include tests for chunk operations.

^{This description was created by for 578971a. It will automatically update as commits are pushed.}

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to 578971a in 40 seconds

More details

• Looked at 893 lines of code in 14 files
• Skipped 0 files when reviewing.
• Skipped posting 3 drafted comments based on config settings.

1. py/tests/integration/runner_sdk.py:1801

• Draft comment:
  Consider adding assertions or checks to verify the success of the chunk ingestion operation. This will help ensure that the operation performs as expected and any issues are caught early.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The test functions for chunk operations (ingest, update, delete) do not include assertions or checks to verify the success of the operations. This could lead to silent failures if the operations do not perform as expected.

2. py/tests/integration/runner_sdk.py:1855

• Draft comment:
  Consider adding assertions or checks to verify the success of the chunk update operation. This will help ensure that the operation performs as expected and any issues are caught early.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The test functions for chunk operations (ingest, update, delete) do not include assertions or checks to verify the success of the operations. This could lead to silent failures if the operations do not perform as expected.

3. py/tests/integration/runner_sdk.py:1902

• Draft comment:
  Consider adding assertions or checks to verify the success of the chunk deletion operation. This will help ensure that the operation performs as expected and any issues are caught early.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The test functions for chunk operations (ingest, update, delete) do not include assertions or checks to verify the success of the operations. This could lead to silent failures if the operations do not perform as expected.

Workflow ID: wflow_0lQQqxPBp5zoVLcV

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
yc_demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 24, 2024 11:48pm
yc-demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 24, 2024 11:48pm

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
recommendation_platform	⬜️ Ignored (Inspect)			Oct 24, 2024 11:48pm