Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gstat: Print detailed blob statistics and table size #8394

Merged
merged 2 commits into from
Jan 20, 2025
Merged

gstat: Print detailed blob statistics and table size #8394

merged 2 commits into from
Jan 20, 2025

Conversation

Noremos
Copy link
Contributor

@Noremos Noremos commented Jan 16, 2025

The new blob statistics looks as follow:

BLOBS (140)
    Primary pointer page: 753, Index root page: 754
    Total formats: 1, used formats: 1
    Average record length: 16.00, total records: 10
    Average version length: 0.00, total versions: 0, max versions: 0
    Average fragment length: 0.00, total fragments: 0, max fragments: 0
    Average unpacked length: 16.00, compression ratio: 1.00
    Pointer pages: 1, data page slots: 2
    Data pages: 2, average fill: 9%
    Primary pages: 1, secondary pages: 1, swept pages: 0
    Empty pages: 0, full pages: 0
    Blobs: 10, total length: 101451015, blob pages: 12439
        Level 0: 3, total length: 15, blob pages: 0
        Level 1: 6, total length: 1451000, blob pages: 182
        Level 2: 1, total length: 100000000, blob pages: 12257
    Table size: 101924864 bytes
    Fill distribution:
         0 - 19% = 2
        20 - 39% = 0
        40 - 59% = 0
        60 - 79% = 0
        80 - 99% = 0

The old one:

BLOBS (139)
    Primary pointer page: 319, Index root page: 320
    Total formats: 1, used formats: 1
    Average record length: 16.00, total records: 10
    Average version length: 0.00, total versions: 0, max versions: 0
    Average fragment length: 0.00, total fragments: 0, max fragments: 0
    Average unpacked length: 16.00, compression ratio: 1.00
    Pointer pages: 1, data page slots: 2
    Data pages: 2, average fill: 6%
    Primary pages: 1, secondary pages: 1, swept pages: 0
    Empty pages: 0, full pages: 0
    Blobs: 10, total length: 101451015, blob pages: 12257
        Level 0: 3, Level 1: 6, Level 2: 1
    Fill distribution:
         0 - 19% = 2
        20 - 39% = 0
        40 - 59% = 0
        60 - 79% = 0
        80 - 99% = 0

Gstat completion time Thu Jan 16 11:02:42 2025

Print detailed blob statistics and table size
60d2b99 
The new blob statistics looks as follow:

```
...
Empty pages: 0, full pages: 0
Blobs: 9, total length: 1451015, blob pages: 182
    Level 0: 3, total length: 15, blob pages: 0
    Level 1: 6, total length: 1451000, blob pages: 182
Table size: 1515520 bytes
Fill distribution:
...
```
hvlad
hvlad requested changes Jan 18, 2025
View reviewed changes
src/include/firebird/impl/msg/gstat.h Outdated Show resolved Hide resolved
src/utilities/gstat/dba.epp Show resolved Hide resolved
@dyemanov
Copy link
Member

Do we want it being backported into v5?

@dyemanov dyemanov merged commit b09e0aa into FirebirdSQL:master Jan 20, 2025
24 checks passed
@hvlad
Copy link
Member

hvlad commented Jan 20, 2025

Do we want it being backported into v5?

No objection

dyemanov pushed a commit that referenced this pull request Jan 20, 2025
@Noremos @dyemanov
gstat: Print detailed blob statistics and table size (#8394)
10df7bd 
* Print detailed blob statistics and table size

The new blob statistics looks as follow:

```
...
Empty pages: 0, full pages: 0
Blobs: 9, total length: 1451015, blob pages: 182
    Level 0: 3, total length: 15, blob pages: 0
    Level 1: 6, total length: 1451000, blob pages: 182
Table size: 1515520 bytes
Fill distribution:
...
```

* Add new message instead of modifying existing one

---------

Co-authored-by: Artyom Abakumov <[email protected]>
@sim1984
Copy link

sim1984 commented Jan 21, 2025

You've messed something up with the formatting of the blob output by levels.

gstat -z
gstat version WI-V5.0.2.1597 Firebird 5.0 4a63159

gstat horses -u SYSDBA -p ***** -r -t HORSE

Analyzing database pages ...
HORSE (179)
    Primary pointer page: 709, Index root page: 710
    Total formats: 2, used formats: 1
    Average record length: 237.77, total records: 545113
    Average version length: 0.00, total versions: 0, max versions: 0
    Average fragment length: 0.00, total fragments: 0, max fragments: 0
    Average unpacked length: 2100.00, compression ratio: 8.83
    Pointer pages: 4, data page slots: 10176
    Data pages: 10176, average fill: 90%
    Primary pages: 9545, secondary pages: 631, swept pages: 0
    Empty pages: 7, full pages: 10165
    Blobs: 73761, total length: 7620692, blob pages: 0
        Level 0: 0, Level 1: 73761, Level 2: 7620692 <--- Wtf?
    Table size: 166789120 bytes
    Fill distribution:
         0 - 19% = 8
        20 - 39% = 0
        40 - 59% = 1
        60 - 79% = 2
        80 - 99% = 10165

dyemanov added a commit that referenced this pull request Jan 21, 2025
@dyemanov
Postfix for PR #8394: gstat: Print detailed blob statistics and table…
7e96b33 
… size
dyemanov added a commit that referenced this pull request Jan 21, 2025
@dyemanov
Postfix for PR #8394: gstat: Print detailed blob statistics and table…
b4c5b07 
… size
@Noremos
Copy link
Contributor Author

Noremos commented Jan 21, 2025

You've messed something up with the formatting of the blob output by levels.

gstat -z
gstat version WI-V5.0.2.1597 Firebird 5.0 4a63159

gstat horses -u SYSDBA -p ***** -r -t HORSE

Analyzing database pages ...
HORSE (179)
    Primary pointer page: 709, Index root page: 710
    Total formats: 2, used formats: 1
    Average record length: 237.77, total records: 545113
    Average version length: 0.00, total versions: 0, max versions: 0
    Average fragment length: 0.00, total fragments: 0, max fragments: 0
    Average unpacked length: 2100.00, compression ratio: 8.83
    Pointer pages: 4, data page slots: 10176
    Data pages: 10176, average fill: 90%
    Primary pages: 9545, secondary pages: 631, swept pages: 0
    Empty pages: 7, full pages: 10165
    Blobs: 73761, total length: 7620692, blob pages: 0
        Level 0: 0, Level 1: 73761, Level 2: 7620692 <--- Wtf?
    Table size: 166789120 bytes
    Fill distribution:
         0 - 19% = 8
        20 - 39% = 0
        40 - 59% = 1
        60 - 79% = 2
        80 - 99% = 10165

Sorry, forgot to change the message number

@Noremos
Copy link
Contributor Author

Noremos commented Jan 21, 2025

There is one non-obvious behavior that i forgot to mention. Previously, blobs statistics were only displayed if they were present. I followed the same logic, so statistics for each level are displayed only if blobs of that level are present. Is this ok?

@sim1984
Copy link

sim1984 commented Jan 21, 2025

There is one non-obvious behavior that i forgot to mention. Previously, blobs statistics were only displayed if they were present. I followed the same logic, so statistics for each level are displayed only if blobs of that level are present. Is this ok?

I think so. This may create a problem for tools interpreting statistics, but such problems always arise when changing the output format and they are quite easy to fix.

P.S. Maybe it is worth thinking about an alternative extensible format that does not break parsing when adding new data, for example JSON/XML. But this is for the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hvlad hvlad hvlad approved these changes

Assignees

@Noremos Noremos

Labels
component: gstat fix-version: 6.0 Alpha 1 qa: done successfully type: improvement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Noremos @dyemanov @hvlad @sim1984 @pavel-zotov