Update to latest Timely #31158
Update to latest Timely #31158
Conversation
antiguru
force-pushed
the
update_td
branch
3 times, most recently
from
840cf18 to
13b616c
Compare
|
Rebasing this over #31204 should fix the upgrade test panic. |
There was a problem hiding this comment.
LGTM, though I'd like to better understand why the multi-producer EventLink doesn't work.
And an independent observation: It feels bad that so many crates depend on Timely/DD even though they are not running any dataflows. Seems like Timely/DD internals could be better encapsulated in the cluster layer.
| }) | ||
| .or_else(|| time_any.downcast_ref::<(Timestamp, Subtime)>().map(|t| t.0)) | ||
| /// Helper trait to extract a timestamp from various types of timestamp used in rendering. | ||
| trait ExtractTimestamp: Clone + 'static { |
There was a problem hiding this comment.
Nice!
I thought we had an issue somewhere about not reporting reachability for WMR dataflows, that maybe we could consider fixed now (assuming we are happy with only reporting the top-level timestamp). But I can't find it now.
There was a problem hiding this comment.
Maybe https://github.com/MaterializeInc/database-issues/issues/5271? I tested the example and it shows reachability data for the recursive scope (and it surfaced a bug in the reachability query, which I fixed in the second commit!):
CREATE TABLE t (a int);
INSERT INTO t VALUES (1);
CREATE MATERIALIZED VIEW mv AS
WITH MUTUALLY RECURSIVE
numbers (n int) as (
SELECT a FROM t
UNION ALL
(
WITH rebound AS (SELECT * FROM numbers)
SELECT distinct t1.n + t2.n AS n
FROM rebound AS t1, rebound AS t2
WHERE t1.n <= 256 AND t2.n <= 256
)
)
SELECT count(*) FROM numbers;
Point Materialize at latest Timely. We need to incorporate some changes around reachability logging, which is now typed, and event iterators that return cow'ed data. Some of the complexity stems from the fact that event links are single-writer, so we need separate event links for each reachability log variant. Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
Point Materialize at latest Timely. We need to incorporate some changes
around reachability logging, which is now typed, and event iterators that
return cow'ed data.
Some of the complexity stems from the fact that event links are
single-writer, so we need separate event links for each reachability log
variant.
Signed-off-by: Moritz Hoffmann [email protected]
Motivation
Tips for reviewer
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.