Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EPIC: Export - Create ExportContentletProcessor reusing existing export logic #30557

Open
6 tasks
Tracked by #29474 ...
jgambarios opened this issue Nov 1, 2024 · 2 comments
Open
6 tasks
Tracked by #29474 ...

Comments

@jgambarios
Copy link
Contributor

jgambarios commented Nov 1, 2024

Parent Issue

#29575

Task

We need to create a new ExportContentletProcessor that reuses the existing export logic from com.dotmarketing.portlets.contentlet.action.EditContentletAction while maintaining the current struts export functionality. This requires careful refactoring to ensure both the new job-based export and the existing export action work consistently. The processor should handle file storage, generate download links, and notify users upon completion.

Objectives

  • Extract and centralize export logic from EditContentletAction
  • Create new ExportContentletProcessor implementing job-based export
  • Maintain backward compatibility with existing struct export functionality
  • Ensure consistent export behavior across both implementations
  • Implement secure file storage and download mechanism
  • Integrate with dotCMS notification system

Technical Requirements

Code Refactoring

  • Extract export logic from EditContentletAction into a shared service/utility
  • Ensure the extracted logic handles all current export scenarios
  • Maintain all existing export configurations and options

New Processor Implementation

  • Create ExportContentletProcessor class
  • Integrate shared export logic
  • Add proper error handling and validation
  • Implement progress tracking
  • Add cancellation support
  • Implement file storage mechanism
  • Generate secure download links
  • Integrate with notification system

Proposed Objective

Core Features

Proposed Priority

Priority 2 - Important

Tasks

Preview Give feedback
  1. Team : Scout Type : Task
  2. Team : Scout Type : Task
  3. Team : Scout Type : Task
  4. Team : Scout Type : Task
  5. Team : Scout Type : Task
  6. Team : Scout Type : Task
@fmontes
Copy link
Member

fmontes commented Nov 4, 2024

I have a few questions and points for clarification regarding the ExportContentletProcessor task:

  1. Data Integrity Checks: Are we considering data integrity checks during the file handling process?

  2. Export Volume Estimate: What is our estimate for how much content can be exported within a 20-minute timeframe? Are we targeting 10K, 100K items, or another range?

  3. Error Management Strategy: Since we aim for a fail-continue option, what is our plan for error handling? What does look like for this version?

@jgambarios
Copy link
Contributor Author

@fmontes

  1. Data Integrity Checks: I see logic in the current export that validates for binary fields if the file exist, if does not exist a "NOT_FOUND" will be used, so, it won't fail and kind of let the user about the not found file, we should improve this by adding this error in the job final report.

  2. Export Volume Estimate: I don't have this data yet, but for sure I can get this numbers before to start the task in order to know how it behaves the current implementation VS how it will behave when we finish this first stage, and with those numbers we can start discussing about possible performance improvements, if needed.

  3. Error Management Strategy: Analysing the current implementation it seems we are "ignoring" failures in order to continue the export process on error avoiding the export process to stop, I think we should, for this ticket, improve the error handling, in order to properly handling the "fail-continue" option and to properly report in the job final report, when the "continue" is enabled, what contentles fail and why.

@nollymar nollymar added Epic and removed Type : Task labels Nov 27, 2024
@fmontes fmontes changed the title Create ExportContentletProcessor reusing existing export logic EPIC: Impport/Export - Create ExportContentletProcessor reusing existing export logic Jan 10, 2025
@fmontes fmontes changed the title EPIC: Impport/Export - Create ExportContentletProcessor reusing existing export logic EPIC: Import/Export - Create ExportContentletProcessor reusing existing export logic Jan 10, 2025
@fmontes fmontes changed the title EPIC: Import/Export - Create ExportContentletProcessor reusing existing export logic EPIC: Export - Create ExportContentletProcessor reusing existing export logic Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Next 1-3 Sprints
Development

No branches or pull requests

3 participants