Proforma: reuse existing files on import, if possible

[kkoehn]

Use xml_id_path to recreate proforma structure with tests and model_solutions containing multiple files

also sets xml_id_path for exported files, if it is not set. (similiar to uuid)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.61%. Comparing base (d99a715) to head (ebef98c).
Report is 13 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2538      +/-   ##
==========================================
+ Coverage   69.52%   69.61%   +0.09%     
==========================================
  Files         202      202              
  Lines        6360     6379      +19     
==========================================
+ Hits         4422     4441      +19     
  Misses       1938     1938              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MrSerth]

I've done a first round of my review, during which I found a few questions. Let's check these first and I'll continue with a second round.

While reviewing, I also got one extra question: Given the changes in CodeOcean to the XML Path ID and the updated metadata, would you recommend to reexport all exercises to CodeHarbor after merging this PR?

[kkoehn]

While reviewing, I also got one extra question: Given the changes in CodeOcean to the XML Path ID and the updated metadata, would you recommend to reexport all exercises to CodeHarbor after merging this PR?

Yes that would absolutely make sense!

[MrSerth]

Nicely done, thank you! The current code changes look good and I've understand them.

Unfortunately, I still had issues verifying the code changes. Once again, I was using the already well-known file changes from here:

• task_845_new.zip
• task_845_old.zip

First of all, when the new version was in CodeHarbor and I tried uploading the old version, I got an exception there (the other direction from old to new works). Then, even when resolving this in CodeHarbor, I still couldn't import the old version in CodeOcean if the new version was stored already.

The error I got was: Validation failed: Files feedback message must be blank. The problematic file is:

#<CodeOcean::File id: 67823922, content: "# actual tests here (in functions)\r\nfrom z_helpers...", context_id: 1101, context_type: "Exercise", file_id: nil, file_type_id: 8, hidden: true, name: "p51test", read_only: true, created_at: "2024-09-25 19:28:50.331358000 +0000", updated_at: "2024-09-25 19:29:34.363568000 +0000", native_file: nil, role: "regular_file", hashed_content: "70721d58fcbac5c9570e63633e778d9c", feedback_message: "Feedback", weight: nil, path: nil, file_template_id: nil, hidden_feedback: false, xml_id_path: ["63332861"]>

---

Despite the listed issue, I also found an issue with the proglang method. Below is the fixed entry that also works if no execution_environment has been selected:

    def proglang
      regex = %r{^openhpi/co_execenv_(?<language>[^:]*):(?<version>[^-]*)(?>-.*)?$}
      match = regex.match @exercise&.execution_environment&.docker_image
      match ? {name: match[:language], version: match[:version]} : nil
    end

Could you include that, too, please (ideally with a test)?

[kkoehn]

please note that import_spec.rb:129 is flaky right now - will look into it tomorrow

[MrSerth]

I started reviewing tonight, but noticed another issue: Some files can be left unreferenced.

Specifically, consider an update form the "old" to the "new" version of task 845 (ZIP files attached above). As part of the update, I see the following query in CodeOcean:

  CodeOcean::File Update All (1.0ms)  UPDATE "files" SET "context_id" = $1, "context_type" = $2 WHERE ("files"."id") IN (SELECT "files"."id" FROM "files" WHERE "files"."context_id" = $3 AND "files"."context_type" = $4 AND ("files"."id" = $5 OR "files"."id" = $6) ORDER BY "files"."path" ASC, "files"."name" ASC)  [["context_id", nil], ["context_type", nil], ["context_id", 1101], ["context_type", "Exercise"], ["id", 67824363], ["id", 67824368]]

And indeed, as expected after this query, some files were updated to have a context_id and context_type of NULL. This shouldn't be the case, I think, and files no longer referenced should be deleted completely.

Otherwise, the functionality seems to work fine. I'll continue my review once the functionality is fixed.