Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New descriptor for OLCI classification and quality; BUFRCREX_CodeFlag_en_40.csv #210

Open
marijanacrepulja opened this issue Oct 15, 2024 · 11 comments · May be fixed by #218
Open

New descriptor for OLCI classification and quality; BUFRCREX_CodeFlag_en_40.csv #210

marijanacrepulja opened this issue Oct 15, 2024 · 11 comments · May be fixed by #218
Assignees
@marijanacrepulja
Labels
ocean satellite
Milestone
FT2025-1

Comments

@marijanacrepulja
Copy link
Collaborator

marijanacrepulja commented Oct 15, 2024 •
edited
Loading

Initial request

Proposal for the new BUFR descriptor to represent OLCI classification and quality. This is to support assimilation of the OLCI data in NWP models.

https://sentinels.copernicus.eu/documents/247904/4812102/S3IPF%20PDS%20004.1%20-%20i2r5%20-%20Product%20Data%20Format%20Specification%20-%20OLCI%20Level%201.pdf/0f4a2cab-171b-e240-8b7e-d4d6410db762

Amendment details

1. Add a new element in table B 0-40-081 for OLCI classification and quality

F X Y ELEMENT NAME UNIT SCALE REFERENCE VALUE DATA WITH (bits)
0-40-081 OLCI Level 1 classification and quality Flag table 0 0 31

2. Add a new CODE/FLAG TABLE 0-40-081 for OLCI Level 1 classification and quality

Bit No OLCI Level 1 classification and quality
1 - 17 Reserved
18 Land
19 Coastline
20 Fresh inland water
21 Tidal regions
22 Preliminary Cloud
23 Stray light risk
24 Invalid
25 Cosmetic
26 Duplicated
27 Sun glint risk
28 Dubious
29 Saturated
30 Partly saturated
All 31 Missing

3. Add to abbreviation table
OLCI - Ocean and Land Colour Instrument

Comments

No response

Requestor(s)

Enter list of requestors.

Stakeholder(s)

Copernicus EU
Copernicus Atmosphere Monitoring Service (CAMS)

Publication(s)

Example: Manual on Codes (WMO-No. 306), Volume I.2, BUFR, BUFRCREX_CodeFlag_en_40.csv

Expected impact of change

MEDIUM

Collaborators

No response

References

No response

Validation

No response
@marijanacrepulja marijanacrepulja added this to the FT2025-1 milestone Oct 15, 2024
@amilan17 amilan17 moved this from Submitted to In discussion in BUFR4 Amendments Oct 15, 2024
@amilan17
Copy link
Member

amilan17 commented Oct 15, 2024 •
edited
Loading

https://github.com/wmo-im/tt-tdcf/wiki/2024.10.15.tt.tdcf notes:

  • add link to documentation in issue
  • add OLCI to abbreviation table
  • Jeff recommends using bits 16-30 to optimize compression
  • change units to flag table from numeric (typo)
  • create sample for validation

@marijanacrepulja marijanacrepulja self-assigned this Nov 12, 2024
@amilan17
Copy link
Member

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.13.tt.tdcf notes:

  • Marijana updated the proposal based on feedback from last meeting and added samples
  • @marijanacrepulja will update the branch
  • @antoinemerle will validate

marijanacrepulja added a commit that referenced this issue Dec 4, 2024
@marijanacrepulja
#210
96d6da0
@marijanacrepulja
Copy link
Collaborator Author

Hi @antoinemerle I have updated branch and samples. Your help with validation is greatly appreciated :)

marijanacrepulja added a commit that referenced this issue Dec 4, 2024
@marijanacrepulja
#210
9822667
@amilan17
Copy link
Member

amilan17 commented Dec 4, 2024

https://github.com/wmo-im/et-data/wiki/2024.12.04.et.data notes:
branch is udpated: ready for validation

@amilan17 amilan17 moved this from In progress to In validation / review in BUFR4 Amendments Dec 4, 2024
@amilan17 amilan17 linked a pull request Dec 5, 2024 that will close this issue
Open
@amilan17 amilan17 linked a pull request Dec 5, 2024 that will close this issue
210 new descriptor for olci classification and quality bufrcrex codeflag en 40csv #218
Open
@marijanacrepulja
Copy link
Collaborator Author

Sample BUFR file and decoded txt file: olci.zip

@antoinemerle could you please help with validation of the sample BUFR file attached.
Many thanks

@sergioh-pessoal
Copy link

sergioh-pessoal commented Jan 14, 2025 •
edited
Loading

@marijanacrepulja I've decoded the sample and get the same values. see file
olci.bufr.decoded.inpe.txt.gz

@antoinemerle
Copy link
Contributor

Dear @marijanacrepulja ,

I have decoded the data and succeed to read it.

Two comments for anyone that wants to follow the procedure:

  • I had to create the tables for 44 in my SW
  • I had to create the local tables
  • only comment is on the value of the OlciLevelOneClassificationAndQuality is this expected to be 8704 ? I see you have the same in your decoding output but I don't understand how this value can be valid.

Thanks ! and sorry for the delay

Here is the output

{ "messages" : [
  [

    {
      "key" : "edition",
      "value" : 4
    },
    {
      "key" : "masterTableNumber",
      "value" : 0
    },
    {
      "key" : "bufrHeaderCentre",
      "value" : 98
    },
    {
      "key" : "bufrHeaderSubCentre",
      "value" : 0
    },
    {
      "key" : "updateSequenceNumber",
      "value" : 1
    },
    {
      "key" : "dataCategory",
      "value" : 3
    },
    {
      "key" : "internationalDataSubCategory",
      "value" : 255
    },
    {
      "key" : "dataSubCategory",
      "value" : 0
    },
    {
      "key" : "masterTablesVersionNumber",
      "value" : 44
    },
    {
      "key" : "localTablesVersionNumber",
      "value" : 0
    },
    {
      "key" : "typicalYear",
      "value" : 2024
    },
    {
      "key" : "typicalMonth",
      "value" : 11
    },
    {
      "key" : "typicalDay",
      "value" : 11
    },
    {
      "key" : "typicalHour",
      "value" : 12
    },
    {
      "key" : "typicalMinute",
      "value" : 20
    },
    {
      "key" : "typicalSecond",
      "value" : 0
    },
    {
      "key" : "typicalDate",
      "value" : "20241111"
    },
    {
      "key" : "typicalTime",
      "value" : "122000"
    },
    {
      "key" : "numberOfSubsets",
      "value" : 1
    },
    {
      "key" : "observedData",
      "value" : 1
    },
    {
      "key" : "compressedData",
      "value" : 0
    },
    {
      "key" : "unexpandedDescriptors",
      "value" :
      [
        4001, 4002, 4003, 4004, 5001, 6001, 40081
      ]
    },
    [
      [

        {
          "key" : "subsetNumber",
          "value" : 1
        },
        {
          "key" : "year",
          "value" : 2024,
          "units" : "a"
        },
        [

          {
            "key" : "month",
            "value" : 11,
            "units" : "mon"
          },
          [

            {
              "key" : "day",
              "value" : 11,
              "units" : "d"
            },
            [

              {
                "key" : "hour",
                "value" : 12,
                "units" : "h"
              },
              [

                {
                  "key" : "latitude",
                  "value" : 45.6,
                  "units" : "deg"
                },
                [

                  {
                    "key" : "longitude",
                    "value" : 16.2,
                    "units" : "deg"
                  },
                  {
                    "key" : "OlciLevelOneClassificationAndQuality",
                    "value" : 8704,
                    "units" : "Flag table"
                  }
                ]
              ]
            ]
          ]
        ]
      ]
    ]
]

]}

@sergioh-pessoal
Copy link

@marijanacrepulja and @antoinemerle. I understand that the value 8704 = 0000000000000000010001000000000 (in binary). This corresponds to the composition of 2 informations: bit 18 = 1 (land) and bit 22 = 1. (Preliminary Cloud).

@antoinemerle
Copy link
Contributor

@sergioh-pessoal THanks a million for the explaination

@sergioh-pessoal
Copy link

@marijanacrepulja I rechecked 0-40-081 and it is ok. Please disregard my previous comment about missing value in this table. Thanks

@amilan17
Copy link
Member

https://github.com/wmo-im/et-data/wiki/2025.01.14.et.data notes:
passed validation, ready for FT preparation

@amilan17 amilan17 moved this from In validation / review to Validated in BUFR4 Amendments Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@marijanacrepulja marijanacrepulja

Labels
ocean satellite
Projects
BUFR4 Amendments
Status: Validated
Milestone
FT2025-1
Development

Successfully merging a pull request may close this issue.

210 new descriptor for olci classification and quality bufrcrex codeflag en 40csv wmo-im/BUFR4
4 participants
@amilan17 @antoinemerle @sergioh-pessoal @marijanacrepulja