fix: truncate long lines in PDF conversion #439
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PDF processing will fail with "! Dimension too large." if a line of text is way too long. Implement a simple processing helper method to call fold (from coreutils) to fold long lines for all code files.
We shouldn't modify student submissions, instead we use the temp file when rendering them to PDF. Cleanup will be performed after rendering is complete.
Run fold on provided files and compare the output with diff. If the file doesn't contain any lines that are over the configured threshold, it will be identical to the original. In this case we replace the temp file with a symlink for easy identification in the template.
Redirect stdout to /dev/null since we don't need the diff output
Set a hard limit of 1000 characters, and truncate everything in the same line after the limit is reached. Otherwise we could get PDF files with hundreds of pages which is completely unreadable.
macite
requested changes
May 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes the PDF processing issue when there are super long lines in the submitted code files by using
cutandfoldfrom coreutils to wrap lines. Additionally, introduce https://github.com/ruby/shellwords for proper shell escape.There are two limits configured, the hard limit is 1000 characters, and the soft limit is 160 characters. Any lines over 1000 characters will be truncated, and then line breaks will be applied for any lines over 160 characters long. If the rendered file has been modified, a warning will be added into the PDF document to indicate the rendered file differs from the original submission. Unit tests and test files have been added for these changes.
Inkscape has been replaced with
librsvgfor SVG to PDF conversion in jupynotex. Inkscape pulls in a lot of unnecessary dependencies (dbus, gtk, etc.), and it emits annoying irrelevant errors that has to be redirected to/dev/null.rsvg-convertis much more lightweight in comparison, performs equally well and has none of the downsides of Inkscape: https://gitlab.gnome.org/GNOME/librsvgGenerated result (excluding Jupyter Notebook):
Jupyter Notebook:
