Migrate to lutaml-model

[andrew2net]

No description provided.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[ronaldtse]

@andrew2net is this in progress?

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[andrew2net]

@ronaldtse yes, this is in progress. I can finish XML serialization in a week. But there is the blocking issue metanorma/metanorma-standoc#952

[ronaldtse]

lutaml-model doesn't allow any additional siblings element with map_all content

Can you please file an issue and link to it? However, can’t you model down a level so that map_all occurs on the “root” and then the “eref” is handled by a separate model?

[andrew2net]

Can you please file an issue and link to it?

@ronaldtse do you mean to file a lutaml-model issue? I can, but I think it's quite tricky to select all nodes (including text nodes) what aren't any of other defined siblings and are placed exact position. Anyway I will file an issue. Let's see what Hassan will say.

However, can’t you model down a level so that map_all occurs on the “root” and then the “eref” is handled by a separate model?

That's I want to do, but it needs to change the grammar.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[andrew2net]

@ronaldtse we have the note option for the to_xml method https://github.com/relaton/relaton-bib#adding-notes
I have no idea how to implement the equivalent with lutaml-model. Do we still need the option?

[ronaldtse]

@ronaldtse we have the note option for the to_xml method https://github.com/relaton/relaton-bib#adding-notes I have no idea how to implement the equivalent with lutaml-model. Do we still need the option?

The proper way is to move this note from to_xml into an attribute. Then you can set it before calling any serialization.

[ronaldtse]

FYI @opoudjis if Metanorma uses this.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[ronaldtse]

Lutaml::Model supports equivalence checks of attributes

• https://github.com/lutaml/lutaml-model#comparison

[ronaldtse]

As above.

[ronaldtse]

@andrew2net for the migrated models, why not convert all the model elements like docidentifier to lutaml-model directly instead of using the model ... keyword to maintain separate serialization and data models? They can be defined in the same model unless there is something very special that is done...

[andrew2net]

@ronaldtse there are some reasons to have separate models

• RelatonXML has 2 variants, bibitem and bidata. With data separated from mappings we can use Bibitem.to_xml(item) or Bibdata.to_xml(item) we can have output what we need.
• Except RelatonXML we require rendering BibXML. So we can use different renderers for same data BibXML.to_xml(item) or Bibitem.to_xml(item)
• We also have asciibib and bibtex formats that lutaml-model doesn't support. I'm certain we shouldn't have all that in one class.
• Many elements have extra functionality. For example, Docidentifier has remove_part, remove_date, all_parts. I think it's better to separate Relaton data model and renderers.
• In some cases, Relaton uses collections with extra functionalities instead of Arrays. For example, TitleCollection and RelationCollection.

[ronaldtse]

Thank you @andrew2net for the insights. These are all valid concerns and use cases that lutaml-model does not handle well currently.

For items 1 and 2, they are about "alternative serializer definitions". I'm hoping to address it by an extension of this task, which is about a proposed Transform class:

• Support model-to-model (lutaml-model to lutaml-model) transformation lutaml/lutaml-model#165

For item 3, it is about additional serialization formats that lutaml-model does not support (asciibib, bibtex) and likely will not support. We need to find a way to allow users to define their own serializer formats in lutaml-model.

For item 4, it is about methods that are only about the "data model" (attribute and type definitions), and are irrelevant to "serializers" (the xml do... etc blocks). Right now, the arrangement of lutaml-model mixing "data model" (attributes) and "serializer" (xml do..) is not ideal. I still think that item 4 can be implemented in lutaml-model models today.

For item 5, it is about handling collections that are more complex than arrays, e.g. they need additional methods and have serialization formats. We have this task:

• Support collections using custom classes lutaml/lutaml-model#215

I will file the other issues in lutaml-model as enhancements. Thank you for the details.

[andrew2net]

@ronaldtse
Since bibitem and bibdata are just a subsets of whole Relaton data model, and have same classes relations, for 1 we can create different root mappings using current lutam-model functionality. These mappings just exclude some attributes and have different root element. For example bibitem doesn't have ext flavor data, and bibdata doesn't have id attribute.

For 2 it needs to remap classes relationships. Since 2 is XML, the model-to-model transformation could work. Is that functionality expected soon?

For 3 we can use decorator pattern to implement serialization that lutaml-model doean't support.

Concernig 4, we can implement all the extra methods in the mapping classes. It's against single responsibility principle, but it's doeable.

For 5 (collections) we need to implement lutaml/lutaml-model#215. When is it expected?

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml

[ronaldtse]

@andrew2net wow I really like the latest changes! As it was well said before — a developer’s worth is characterized by not how much code is added, but how much removed! 👍👍

[andrew2net]

@ronaldtse now it needs to add functionality that we had in separated models. I think this is a good time to review the functions and decide which need to be kept.

[andrew2net]

@ronaldtse until the lutaml-model to lutaml-model transformation is ready I don't see another way but to use an external model for Bibitem and Bibdata mappings.

[hound]

Some files could not be reviewed due to errors:

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unre...

Error: unrecognized cop Performance/CaseWhenSplat found in .rubocop.yml, unrecognized cop Performance/Count found in .rubocop.yml, unrecognized cop Performance/Detect found in .rubocop.yml, unrecognized cop Performance/FlatMap found in .rubocop.yml, unrecognized cop Performance/ReverseEach found in .rubocop.yml, unrecognized cop Performance/Size found in .rubocop.yml, unrecognized cop Performance/StringReplacement found in .rubocop.yml