Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat/use dynamic harvest field #2762

Merged
merged 38 commits into from
Nov 10, 2022
Merged

Feat/use dynamic harvest field #2762

merged 38 commits into from
Nov 10, 2022

Commits on May 23, 2022

  1. Test all kind of extras formats

    @maudetes
    maudetes committed May 23, 2022
    Configuration menu
    Copy the full SHA
    8a2e0e1 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2022

  1. Add protected extras hateoas in api fields

    @maudetes
    maudetes committed Jun 17, 2022
    Configuration menu
    Copy the full SHA
    f6068a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2022

  1. Add protected extras for harvest as embedded documents

    @maudetes
    maudetes committed Jun 22, 2022
    Configuration menu
    Copy the full SHA
    18e90ee View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2022

  1. [WIP] trying some things out

    @maudetes
    maudetes committed Jun 28, 2022
    Configuration menu
    Copy the full SHA
    c1dd281 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. [WIP] add harvest as new dataset field

    @maudetes
    maudetes committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    576aadf View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. Merge branch 'master' into feat/add-protected-extras

    @maudetes
    maudetes committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    23acdfb View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2022

  1. Update and fix tests

    @maudetes
    maudetes committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    4770077 View commit details
    Browse the repository at this point in the history

  2. Remove dataset in resource extras endpoint

    @maudetes
    maudetes committed Jul 25, 2022
    Configuration menu
    Copy the full SHA
    a02d431 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2022

  1. Fix tests

    @maudetes
    maudetes committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    ecee6a6 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2022

  1. Store harvest dates as date/datetime types 

    We could also store it as isoformat (it was previously the case for last_update).
However, we'll need to manipulate dates a bit more with additional created_at and
last_modified fields.
    @maudetes
    maudetes committed Jul 27, 2022
    Configuration menu
    Copy the full SHA
    8893be3 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2022

  1. WIP use dynamic field for harvest metadata

    @maudetes
    maudetes committed Jul 28, 2022
    Configuration menu
    Copy the full SHA
    6c4e686 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2022

  1. Iterate on harvest dynamic field

    @maudetes
    maudetes committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    67dfdda View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2022

  1. Use modified_at in udata harvest metadata

    @maudetes
    maudetes committed Aug 29, 2022
    Configuration menu
    Copy the full SHA
    389c5e3 View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2022

  1. Remove protected extras, out of scope for this branch

    @maudetes
    maudetes committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    e938c24 View commit details
    Browse the repository at this point in the history

  2. Remove extras modification, out of scope for this branch

    @maudetes
    maudetes committed Oct 3, 2022
    Configuration menu
    Copy the full SHA
    48e5ee0 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2022

  1. Remove dates modification

    @maudetes
    maudetes committed Oct 4, 2022
    Configuration menu
    Copy the full SHA
    aaa79c4 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2022

  1. WIP add tests

    @maudetes
    maudetes committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    7182970 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2022

  1. Add test on dynamic field + resource

    @maudetes
    maudetes committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    c2f8273 View commit details
    Browse the repository at this point in the history

  2. Fix api fields being updated before app is created 

    Fixture order did not work as I expected, being not-deterministic
    @maudetes
    maudetes committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    7591162 View commit details
    Browse the repository at this point in the history

  3. Expose harvest in apiv2 and serialization

    @maudetes
    maudetes committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    77f7b3e View commit details
    Browse the repository at this point in the history

  4. Fixt test

    @maudetes
    maudetes committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    6e1364d View commit details
    Browse the repository at this point in the history

  5. Fix indentation

    @maudetes
    maudetes committed Oct 7, 2022
    Configuration menu
    Copy the full SHA
    0a93082 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2022

  1. Add migration

    @maudetes
    maudetes committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    1f15f38 View commit details
    Browse the repository at this point in the history

  2. Use uri identifier as remote_url if validated

    @maudetes
    maudetes committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    e752d3f View commit details
    Browse the repository at this point in the history

  3. Add more tests on harvest metadata

    @maudetes
    maudetes committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    57a434b View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into feat/use-dynamic-harvest-field

    @maudetes
    maudetes committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    9bdaa15 View commit details
    Browse the repository at this point in the history

  5. Update changelog

    @maudetes
    maudetes committed Oct 10, 2022
    Configuration menu
    Copy the full SHA
    31e3dde View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2022

  1. Add resilience on migration if dataset validation fails

    @maudetes
    maudetes committed Oct 11, 2022
    Configuration menu
    Copy the full SHA
    6f2ac8f View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2022

  1. Merge branch 'master' into feat/use-dynamic-harvest-field

    @maudetes
    maudetes committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    7cafdbe View commit details
    Browse the repository at this point in the history

  2. Update last_update to take harvest dates into account

    @maudetes
    maudetes committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    f8d8e1a View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2022

  1. Remove dct_identifier on resource harvest metadata

    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    43ad0e1 View commit details
    Browse the repository at this point in the history

  2. Make code simpler following review comments

    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    0a53ed2 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into feat/use-dynamic-harvest-field

    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    a526ae7 View commit details
    Browse the repository at this point in the history

  4. Raise an error if DCT.identifier is missing 

    The entire job now fails at initlization time if any of the dataset has a DCT.identifier missing
    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    ca603cc View commit details
    Browse the repository at this point in the history

  5. Raise only when identifier is None, not 0 :p

    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    ada6d54 View commit details
    Browse the repository at this point in the history

  6. Clean additionnal harvest extras in migration

    @maudetes
    maudetes committed Nov 4, 2022
    Configuration menu
    Copy the full SHA
    4581dec View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2022

  1. Fix harvest migration

    @maudetes
    maudetes committed Nov 8, 2022
    Configuration menu
    Copy the full SHA
    4788809 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2022

  1. Update changelog

    @maudetes
    maudetes committed Nov 9, 2022
    Configuration menu
    Copy the full SHA
    7a579a2 View commit details
    Browse the repository at this point in the history