2021-08-17 weekly planning

[kgodey]

Discussion topics

• Retrospective: Summary of last week's work.
• Active Work: Work planned for the next week.
• Updates: General updates everyone should be aware of.
• Questions: Questions for the team to answer.

About

Each Tuesday, we start a discussion about Mathesar's progress. Everyone is welcome to add a new topic to discuss or to comment on existing topics.

[kgodey]

Retrospective

This post lists issue updates from 2021-08-10 until now by milestone.

03. Working with Schemas

• Issues started, not yet finished:
  • [@pavish] Schema list page interactivity #518

04. Tables from File Import

• Issues closed:
  • [@kgodey] Updating import_verified in tables patch endpoint results in a 400 #565
  • [@kgodey] import_verified field missing in table resource #546
  • [@pavish] Import status indicators #296
• Issues started, not yet finished:
  • [@pavish] File import preview page #295
• New issues created:
  • Use suggested types in import preview step #568, created by @kgodey
  • Updating import_verified in tables patch endpoint results in a 400 #565, created by @pavish
  • Tables PATCH endpoint does not update column properties #562, created by @pavish
  • import_verified field missing in table resource #546, created by @kgodey

05. Editable Tables

• Issues closed:
  • [@pavish] Change in url by browser back/forward buttons do not change current db and schema #548
  • [@pavish] Switch open tables when schema is changed #524
• Issues started, not yet finished:
  • [@pavish] Implement record editing #195
• New issues created:
  • Change in url by browser back/forward buttons do not change current db and schema #548, created by @pavish

2021-08 improvements

• Issues closed:
  • [@andyharley-cci] Staging deploys failing at host verification step #534
• Issues started, not yet finished:
  • [@andyharley-cci] Handle non-unicode encoded datafiles #421
  • [@mr-gabe49] Select component keyboard accessibility #483
  • [@kgodey] Serve built static files in production #358
  • [@andyharley-cci] Set up staging environment for Mathesar #225
  • [@Dhruvi16] Create button component in the component library. #322
• New issues created:
  • Double quotes in file headers break CSV import #572, created by @eito-fis
  • Patch on a table, with same name results in a 500 #566, created by @pavish
  • Call to columns patch endpoint with both name and type results in 500 #564, created by @pavish
  • Previews endpoint casts a column and returns 200, while column patch endpoint fails with 500 #563, created by @pavish
  • Define common error structure #560, created by @pavish
  • Duplicate tables are present with the same oid in mathesar_table #559, created by @pavish
  • Staging deploys failing at host verification step #534, created by @andyharley-cci

06. Working with Tables

• Issues closed:
  • [@eito-fis] Extend column API to support duplicating columns #452
  • Allow columns to have a default value. #391
• Issues started, not yet finished:
  • [@eito-fis] Add filter to show records with duplicate values in a set of columns #501
  • [@eito-fis] DB and API work for adding a default value to columns #494
  • [@pegnott] Implement adding a new column #495
  • [@eito-fis] DB function to get rows with duplicate values #515

07. Initial Data Types

• Issues closed:
  • [@kgodey] Update filtering issues to cover storing filter/group information in the backend. #557
  • [@mathemancer] Backend work for custom MATHESAR_TYPES.PERCENTAGE type. #434
  • [@mathemancer] Backend work for Mathesar email custom type. #409
  • [@mathemancer] Handle DOUBLE PRECISION data type in the backend. #383
  • [@mathemancer] Handle REAL data type in the backend. #382
  • [@mathemancer] Handle DECIMAL data type in the backend. #381
  • [@mathemancer] Handle BIGINT data type in the backend. #380
  • [@mathemancer] Handle INTEGER data type in the backend. #379
  • [@mathemancer] Handle SMALLINT data type in the backend. #378
  • [@kgodey] Ensure that the API only accepts DB types as input everywhere we use types. #369
  • [@kgodey] Extend the v0/databases/'s types list to show both Mathesar and DB type information. #368
  • [@kgodey] Store friendly to DB type mapping in the backend #367
  • [@kgodey] Mapping DB data types to Mathesar data types #269
• Issues started, not yet finished:
  • [@powellc] Handle BOOLEAN data type in the backend #387
  • [@mathemancer] Backend work for Mathesar money custom type  #417
  • [@ghislaineguerin] Design for Number data types #238
  • [@ghislaineguerin] Design for data types that are not "Mathesar types" #537
  • [@ghislaineguerin] Design for boolean data type #237
  • [@ghislaineguerin] Design for date & time data types #240
  • [@ghislaineguerin] Design for email data type #235
  • [@ghislaineguerin] Design for URI data type #236
  • [@ghislaineguerin] Design for duration data type #241
  • [@ghislaineguerin] Design for money data type #239
• New issues created:
  • Update filtering issues to cover storing filter/group information in the backend. #557, created by @kgodey
  • Update implementation issues for Initial Data Types milestone #552, created by @kgodey
  • Design for data types that are not "Mathesar types" #537, created by @kgodey

08. Working with Views

• Issues started, not yet finished:
  • [@ghislaineguerin] Design for usage of Foreign Key constraints #243

09. Data Modeling

• New issues created:
  • Create design and backend issues for Data Modeling milestone #538, created by @kgodey

10. Multiple Database Support

• Issues started, not yet finished:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395
• New issues created:
  • Set up user defined types for Postgres types that aren't reflected in SQL Alchemy #558, created by @kgodey

2021-09 improvements

• New issues created:
  • Issues with working on a Windows machine #570, created by @asharonbaltazar
  • Column API should support creating columns without names  #555, created by @eito-fis

[kgodey]

Active Work

This post lists active work based on the Active GitHub project. It also includes a current summary of milestones involved in active work.

[2021-08-18] 03. Working with Schemas

• 13 issues already closed.
• Issues currently in progress:
  • [@pavish] Schema list page interactivity #518

[2021-08-20] 04. Tables from File Import

• 30 issues already closed.
• Issues currently in progress:
  • [@pavish] File import preview page #295
• Issues not yet started:
  • [@kgodey] Tables PATCH endpoint does not update column properties #562
  • [@pavish] Use suggested types in import preview step #568

[2021-08-25] 05. Editable Tables

• 8 issues already closed.
• Issues currently in progress:
  • [@pavish] Implement record editing #195
• Issues not yet started:
  • [@pavish] Implement adding new records #197

[2021-08-31] 2021-08 improvements

• 6 issues already closed.
• Issues currently in progress:
  • [@andyharley-cci] Handle non-unicode encoded datafiles #421
  • [@mr-gabe49] Select component keyboard accessibility #483
  • [@kgodey] Serve built static files in production #358
  • [@andyharley-cci] Set up staging environment for Mathesar #225
  • [@Dhruvi16] Create button component in the component library. #322
• Issues not yet started:
  • [@kgodey] Call to columns patch endpoint with both name and type results in 500 #564
  • [@kgodey] Previews endpoint casts a column and returns 200, while column patch endpoint fails with 500 #563
  • [@kgodey] Patch on a table, with same name results in a 500 #566
  • [@arvind-1] Remove database keys API #313
• Issues remaining in backlog:
  • 5 backend and database issues
  • 10 frontend issues
  • 6 design issues
  • 2 infrastructure issues

[2021-09-06] 06. Working with Tables

• 16 issues already closed.
• Issues currently in progress:
  • [@eito-fis] Add filter to show records with duplicate values in a set of columns #501
  • [@eito-fis] DB and API work for adding a default value to columns #494
  • [@pegnott] Implement adding a new column #495
  • [@eito-fis] DB function to get rows with duplicate values #515
• Issues not yet started:
  • [@asharonbaltazar] Implement showing and changing a column's type #498
• Issues remaining in backlog:
  • 13 frontend issues
  • 1 other issues

[2021-09-13] 07. Initial Data Types

• 17 issues already closed.
• Issues currently in progress:
  • [@powellc] Handle BOOLEAN data type in the backend #387
  • [@mathemancer] Backend work for Mathesar money custom type  #417
  • [@ghislaineguerin] Design for Number data types #238
  • [@ghislaineguerin] Design for data types that are not "Mathesar types" #537
  • [@ghislaineguerin] Design for boolean data type #237
  • [@ghislaineguerin] Design for date & time data types #240
  • [@ghislaineguerin] Design for email data type #235
  • [@ghislaineguerin] Design for URI data type #236
  • [@ghislaineguerin] Design for duration data type #241
  • [@ghislaineguerin] Design for money data type #239
• Issues not yet started:
  • [@powellc] Implement filtering options for Boolean types #388
  • [@powellc] Implementation for boolean data type #247
  • [@eito-fis] Spec for implementing display options for Columns #392
  • [@eito-fis] Handle TIMESTAMP data type in the backend #424
  • [@eito-fis] Handle DATE data type in the backend #425
  • [@eito-fis] Handle TIME data type in the backend #426
  • [@eito-fis] Implement filtering options for date & time types #427
  • [@eito-fis] Implement grouping options for date & time types #429
  • [@rootxnet] Handle VARCHAR data type in the backend. #404
  • [@rootxnet] Handle TEXT data type in the backend. #405
  • [@rootxnet] Handle CHAR data type in the backend. #407
  • [@rootxnet] Implement filtering options for Text types. #406
  • [@rootxnet] Implement grouping for Text types. #408
  • [@kgodey] Implement filtering options for the email type #410
  • [@kgodey] Implement grouping options for the email type #411
  • [@powellc] Implement range based grouping for numeric types. #384
  • [@powellc] Implement filtering options for Number types. #385
  • [@kgodey] Update implementation issues for Initial Data Types milestone #552
• Issues remaining in backlog:
  • 20 backend and database issues
  • 8 frontend issues

[2021-09-16] 08. Working with Views

• 4 issues already closed.
• Issues currently in progress:
  • [@ghislaineguerin] Design for usage of Foreign Key constraints #243
• Issues not yet started:
  • [@ghislaineguerin] Support for deleting a database view #454
  • [@ghislaineguerin] Support for listing database views #455
  • [@ghislaineguerin] Support for editing records within a view #456
  • [@ghislaineguerin] Support for opening and interacting with a View #466
• Issues remaining in backlog:
  • 8 backend and database issues
  • 1 frontend issues
  • 1 design issues
  • 1 other issues

[2021-09-20] 09. Data Modeling

• 3 issues already closed.
• Issues not yet started:
  • [@kgodey] Create design and backend issues for Data Modeling milestone #538
• Issues remaining in backlog:
  • 4 backend and database issues
  • 3 frontend issues
  • 2 design issues

[2021-09-22] 10. Multiple Database Support

• 7 issues already closed.
• Issues currently in progress:
  • [@aaditkamat] Backend support for tables without single-column primary keys #395
• Issues remaining in backlog:
  • 4 backend and database issues
  • 3 frontend issues
  • 2 design issues

[kgodey]

Updates

Team

• @mathemancer is on parental leave for a couple of weeks starting today.
• This is @eito-fis's last week as an intern, but he'll be staying on as a member of the core team and contributing when he has time.

Roadmap

• Our milestone deadlines have been adjusted again. We still have 04. Tables from File Import and 03. Working with Schemas due this week.
• We're going to pause frontend feature work after milestone 5 (editing records) to focus on tests and stability, as part of our August improvements milestone. Issues for this will be created by @pavish soon.

[kgodey]

Questions

@eito-fis, @ghislaineguerin, @pavish: Your responses are required. Others are also welcome to respond.

Retrospective

• What went well this week?
• What could have gone better this week?
• Are there any improvements to our process you'd like to suggest?

Active Work

• Does your workload for this week look okay?
• We have two milestones due in the next week. Are there any blockers to meeting our upcoming deadlines?

Discussion

• Anything else you'd like to discuss?

[ghislaineguerin]

What went well this week?

I'm not sure, overall things got done, but I can't make much progress due to the number of pending reviews I have open.

What could have gone better this week?

I should have locked a day to only focus on closing all reviews.

Are there any improvements to our process you'd like to suggest?

Maybe I need to try a shorter period for review, from a week to a half week.

Does your workload for this week look okay?

Yes, it does.

We have two milestones due in the next week. Are there any blockers to meeting our upcoming deadlines?

None that I can think of.

Anything else you'd like to discuss?

Nothing comes to mind right now. Thanks

[kgodey]

What went well this week?

I closed a bunch of issues, including the type mapping meta-issue which sets us up for consistent use of types in the future.

What could have gone better this week?

I didn't make any progress on speccing out new milestones. It's not blocking anything yet, but it will block design if I don't get to it this week.

Are there any improvements to our process you'd like to suggest?

None.

Does your workload for this week look okay?

Yes, but I'm not 100% sure I can get it all done.

We have two milestones due in the next week. Are there any blockers to meeting our upcoming deadlines?

It depends on how work on #562 goes. It is not straightforward to implement because it involves a bunch of alter statements in a transaction. We will need to move out the deadline for 04. Tables from File Import if I can't get the PR done by tomorrow.

[pavish]

What went well this week?

The layout/structural changes needed for pages to be schema centric is complete. The base changes for file import preview page and indicators was finally done. I also had to focus more on interviews, PR reviews, new issues, during the last week.

What could have gone better this week?

I could not focus much on all the design discussions, since there were a lot of them open. It takes considerable time to go through the design discussions and think it over. I did not get time to balance this.

Are there any improvements to our process you'd like to suggest?

This was discussed on Matrix. I will reiterate them here:

• I would like all new frontend issues to be updated with the apis needed for them. (I think it might be worth it to also update existing issues, since I had no idea of all the apis that were present. eg., type_suggestions).
• Before the start of each milestone, I would like to spec out frontend issues, go over the associated apis, and test them out. This will help us identify if the issues are ready for development.

Does your workload for this week look okay?

Yes, the workload for this week feels more comfortable.

We have two milestones due in the next week. Are there any blockers to meeting our upcoming deadlines?

The preview page #295 is blocked on the PATCH endpoint #562, so I'll start focusing on milestone 5, once I complete all the other issues in m4.

Anything else you'd like to discuss?

None, at the moment.