forked from PrefectHQ/prefect
-
Notifications
You must be signed in to change notification settings - Fork 0
/
RELEASE-NOTES.md
8073 lines (5590 loc) · 475 KB
/
RELEASE-NOTES.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Prefect Release Notes
## Release 3.0.0rc1
We're excited to announce the release candidate of Prefect 3.0. It's the most flexible, powerful, fastest version of Prefect yet. Prefect 3.0 includes several exciting new features. Install it by running `pip install prefect==3.0.0rc1` and check out the docs [here](https://docs-3.prefect.io/3.0rc/getting-started/index).
### Run tasks independently of flows
You can now run and serve tasks outside of flows and inside of other tasks.
```python
from prefect import task
@task
def my_background_task(name: str):
print(f"Hello, {name}!")
if __name__ == "__main__":
my_background_task.delay("ford")
```
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13695
- https://github.com/PrefectHQ/prefect/pull/13692
- https://github.com/PrefectHQ/prefect/pull/13681
- https://github.com/PrefectHQ/prefect/pull/13662
- https://github.com/PrefectHQ/prefect/pull/13653
- https://github.com/PrefectHQ/prefect/pull/13643
- https://github.com/PrefectHQ/prefect/pull/13589
- https://github.com/PrefectHQ/prefect/pull/13684
- https://github.com/PrefectHQ/prefect/pull/13676
- https://github.com/PrefectHQ/prefect/pull/13276
- https://github.com/PrefectHQ/prefect/pull/13611
- https://github.com/PrefectHQ/prefect/pull/13547
- https://github.com/PrefectHQ/prefect/pull/13706
### Transactional semantics
Use rollback and commit hooks to facilitate idempotent python code.
```python
from prefect import flow, task
from prefect.transactions import transaction
@task
def first_task():
print('first')
@first_task.on_rollback
def roll(txn):
print('rolling back')
@task
def second_task():
raise RuntimeError("oopsie")
@flow
def txn_flow():
with transaction():
first_task()
second_task()
if __name__ == "__main__":
txn_flow()
```
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13559
- https://github.com/PrefectHQ/prefect/pull/13534
- https://github.com/PrefectHQ/prefect/pull/13535
- https://github.com/PrefectHQ/prefect/pull/13480
- https://github.com/PrefectHQ/prefect/pull/13452
- https://github.com/PrefectHQ/prefect/pull/13450
- https://github.com/PrefectHQ/prefect/pull/13484
- https://github.com/PrefectHQ/prefect/pull/13454
- https://github.com/PrefectHQ/prefect/pull/13477
- https://github.com/PrefectHQ/prefect/pull/13431
- https://github.com/PrefectHQ/prefect/pull/13264
- https://github.com/PrefectHQ/prefect/pull/13337
- https://github.com/PrefectHQ/prefect/pull/13456
- https://github.com/PrefectHQ/prefect/pull/13572
- https://github.com/PrefectHQ/prefect/pull/13582
- https://github.com/PrefectHQ/prefect/pull/13627
- https://github.com/PrefectHQ/prefect/pull/13568
- https://github.com/PrefectHQ/prefect/pull/13438
- https://github.com/PrefectHQ/prefect/pull/13573
- https://github.com/PrefectHQ/prefect/pull/13578
- https://github.com/PrefectHQ/prefect/pull/13414
### Open source Events and Automations
Trigger actions, such as sending notifications, pausing schedules, starting flow runs and more in response to Prefect events.
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13293
- https://github.com/PrefectHQ/prefect/pull/13521
- https://github.com/PrefectHQ/prefect/pull/13335
### More flexible variables and new artifact types
Variables can now be any JSON compatible type including dicts, lists, and integers. Progress and Image artifacts make it easy to add visual annotations to your flow run graph.
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13500
- https://github.com/PrefectHQ/prefect/pull/13520
- https://github.com/PrefectHQ/prefect/pull/13469
- https://github.com/PrefectHQ/prefect/pull/13641
- https://github.com/PrefectHQ/prefect/pull/13605
### Faster and richer CLI
Improved CLI speed and several added commands and conveniences.
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13292
- https://github.com/PrefectHQ/prefect/pull/13596
- https://github.com/PrefectHQ/prefect/pull/13606
- https://github.com/PrefectHQ/prefect/pull/13533
### Updated navigation, styling, and interaction design
The new Runs page displays both flow and task run information, and an improved sidebar and switcher makes navigating Prefect simpler than ever.
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13395
- https://github.com/PrefectHQ/prefect/pull/13280
- https://github.com/PrefectHQ/prefect/pull/13696
- https://github.com/PrefectHQ/prefect/pull/13668
- https://github.com/PrefectHQ/prefect/pull/13670
- https://github.com/PrefectHQ/prefect/pull/13723
### Enhancements
- Create artifact for unsuccessful dbt task runs — https://github.com/PrefectHQ/prefect/pull/13348
- Add filter on `task_run.expected_start_time` — https://github.com/PrefectHQ/prefect/pull/13491
- Add utilities to serialize context to a dictionary and hydrate context from a dictionary — https://github.com/PrefectHQ/prefect/pull/13529
- Add API endpoints for deployment count and next flow run — https://github.com/PrefectHQ/prefect/pull/13544
- Allow flow parameter schema generation when dependencies are missing — https://github.com/PrefectHQ/prefect/pull/13315
- Change the default value for `enforce_parameter_schema` from `False` to `True` — https://github.com/PrefectHQ/prefect/pull/13594
- Migrate schemas to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13574
- Removes block auto-instrumentation — https://github.com/PrefectHQ/prefect/pull/13407
- Migrate all uses of the banned characters validation to a self-validator — https://github.com/PrefectHQ/prefect/pull/13370
- Ignore and warn on unrecognized settings - https://github.com/PrefectHQ/prefect/pull/13624
### Fixes
- Remove unnecessary flow run infrastructure override access checks — https://github.com/PrefectHQ/prefect/pull/13401
- Enforce False case when flow run id is null — https://github.com/PrefectHQ/prefect/pull/13464
- Fix workspace variable hydration to allow for JSON data — https://github.com/PrefectHQ/prefect/pull/13548
- Remove unused settings/experimental work pool flags: `PREFECT_EXPERIMENTAL_ENABLE_WORK_POOLS` and `PREFECT_EXPERIMENTAL_WARN_WORK_POOLS` — https://github.com/PrefectHQ/prefect/pull/13144
- Pin `pydantic>=2.7` for `Secret` — https://github.com/PrefectHQ/prefect/pull/13613
- Skip on cancellation hooks if runner can't load flow — https://github.com/PrefectHQ/prefect/pull/13660
- Refactor lazy imports to avoid accidental eager imports — https://github.com/PrefectHQ/prefect/pull/13296
- Allow block registration to use client schemas for server model creation — https://github.com/PrefectHQ/prefect/pull/13602
- Replace our customized `Duration` types with plain `timedelta`s — https://github.com/PrefectHQ/prefect/pull/13603
### Experimental
- Add `prefect.yaml` and cli support for new schedule fields — https://github.com/PrefectHQ/prefect/pull/13318
### Documentation
- Transition documentation hosting from Netlify to Mintlify — https://github.com/PrefectHQ/prefect/pull/13634
- Add Python 3.12 to list of Docker images — https://github.com/PrefectHQ/prefect/pull/13321
- Update `index.md` — https://github.com/PrefectHQ/prefect/pull/13353
- Improve tutorial section — https://github.com/PrefectHQ/prefect/pull/13297
- Fix jinja template in automations doc — https://github.com/PrefectHQ/prefect/pull/13422
- Update development section docs — https://github.com/PrefectHQ/prefect/pull/13247
- Update Ray integration docs — https://github.com/PrefectHQ/prefect/pull/13467
- Update Variables docs to include JSON types — https://github.com/PrefectHQ/prefect/pull/13493
- Update quickstart guide for usability — https://github.com/PrefectHQ/prefect/pull/13562
- Remove `deployments-block-based` concept page and refs for 3.0 — https://github.com/PrefectHQ/prefect/pull/13626
- Remove `infrastructure` concept page and refs for 3.0 — https://github.com/PrefectHQ/prefect/pull/13629
- Update docs image paths and remove outdated images — https://github.com/PrefectHQ/prefect/pull/13666
- Remove references to `prefect.software` from docs — https://github.com/PrefectHQ/prefect/pull/13382
- Update `host.md` — https://github.com/PrefectHQ/prefect/pull/13351
- Simplify rate limits page — https://github.com/PrefectHQ/prefect/pull/13689
- Removing references to deprecated block types and add disclaimer — https://github.com/PrefectHQ/prefect/pull/13651
- Update guides — https://github.com/PrefectHQ/prefect/pull/13253
- Remove `storage` concept page and refs - https://github.com/PrefectHQ/prefect/pull/13630
### Integrations
- Migrate `prefect-dbt` to pydantic v2 - https://github.com/PrefectHQ/prefect/pull/13718
- Migrate `prefect-email` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13654
- Migrate `prefect-slack` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13673
- Migrate `prefect-shell` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13675
- Migrate `prefect-gcp` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13650
- Migrate `prefect-github` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13655
- Migrate `prefect-gitlab` to pydantic v2 — https://github.com/PrefectHQ/prefect/pull/13656
- Migrate `prefect-docker` to pydantic v2 - https://github.com/PrefectHQ/prefect/pull/13697
- Migrate `prefect-sqlalchemy` to pydantic v2 - https://github.com/PrefectHQ/prefect/pull/13700
- Add `PrefectDistributedClient` to `prefect-dask` — https://github.com/PrefectHQ/prefect/pull/13537
- Update `RayTaskRunner` for compatibility with new engine — https://github.com/PrefectHQ/prefect/pull/13575
- Update `DaskTaskRunner` for compatibility with the updated engine — https://github.com/PrefectHQ/prefect/pull/13555
- prefect-dbt artifact consolidation and markdown fixes — https://github.com/PrefectHQ/prefect/pull/13379
- prefect-dbt - Cause unsuccessful dbt tasks to fail — https://github.com/PrefectHQ/prefect/pull/13405
- DBT Tasks extra_command_args Fix — https://github.com/PrefectHQ/prefect/pull/13308
- Update dbt-core dependency — https://github.com/PrefectHQ/prefect/pull/13394
### Breaking Changes
- Remove `prefect deployment build` CLI from `main` — https://github.com/PrefectHQ/prefect/pull/13366
- Remove `prefect agent` CLI from `main` — https://github.com/PrefectHQ/prefect/pull/13365
- Remove `prefect deployment apply` CLI from `main` — https://github.com/PrefectHQ/prefect/pull/13367
- Remove `PrefectAgent` class — https://github.com/PrefectHQ/prefect/pull/13374
- Remove `prefect.software` — https://github.com/PrefectHQ/prefect/pull/13375
- Remove `deployments` module — https://github.com/PrefectHQ/prefect/pull/13373
- Remove `EcsTask` from `main` — https://github.com/PrefectHQ/prefect/pull/13417
- Remove `AzureContainerInstanceJob` from `main` — https://github.com/PrefectHQ/prefect/pull/13418
- Remove `VertexAICustomTrainingJob` from `main` — https://github.com/PrefectHQ/prefect/pull/13419
- Remove `CloudRunJob` from `main` — https://github.com/PrefectHQ/prefect/pull/13420
- Remove infrastructure blocks from `main` — https://github.com/PrefectHQ/prefect/pull/13424
- Remove `Infrastructure`, `BlockWorker` from `main` — https://github.com/PrefectHQ/prefect/pull/13430
- Remove deprecated storage blocks from `main` — https://github.com/PrefectHQ/prefect/pull/13410
- Remove `prefect-agent` as a possible work pool type — https://github.com/PrefectHQ/prefect/pull/13444
- Remove old engine — https://github.com/PrefectHQ/prefect/pull/13542
- Remove Python 3.8 support — https://github.com/PrefectHQ/prefect/pull/13331
- Remove `deprecated` module and its references — https://github.com/PrefectHQ/prefect/pull/13345
- Remove old task runners and futures modules — https://github.com/PrefectHQ/prefect/pull/13593
- Remove `is_state` — https://github.com/PrefectHQ/prefect/pull/13569
- Remove deprecated options from `prefect work-queue` and refs to agents - https://github.com/PrefectHQ/prefect/pull/13638
### Contributors
- @bsignoret
* @jaraics made their first contribution in https://github.com/PrefectHQ/prefect/pull/13144
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.19.0...3.0rc1
## Release 2.19.3
### New method for generating parameter schemas without dependencies
`prefect deploy` now works even when dependencies are missing from the current environment. This can speed up deployment via CI by removing the need to install dependencies before deploying your flows.
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13620
- https://github.com/PrefectHQ/prefect/pull/13315
### Enhancements
- Provide URL in CLI output upon work pool creation — https://github.com/PrefectHQ/prefect/pull/13597
### Fixes
- Ensure graceful cancellation of flow runs corresponding to deleted deployments — https://github.com/PrefectHQ/prefect/pull/13669
### Integrations
- Add loading state to concurrency limits table in the Prefect UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2483
- Remove old schema properties from deployments in the Prefect UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2482
- Add handling for multi-word dbt CLI commands — https://github.com/PrefectHQ/prefect/pull/13616
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.19.2...2.19.3
## Release 2.19.0
### Support for major infrastructure and distributed task integrations
As `prefect-dask` and other integrations have been added to the `prefect` codebase, this release adds these integrations as `extra` requirements of the `prefect` package, making it easier to install support for everything in your Prefect stack.
```bash
pip install prefect[dask]
```
We loved this community contribution so much, we did it for all our first-party integrations.
```bash
pip install prefect[aws,kubernetes,dask,dbt,sqlalchemy,slack]
```
You can see the full list of Prefect's `extra` requirements in [our `setup.py`](https://github.com/PrefectHQ/prefect/blob/main/setup.py#L43).
See the following pull requests for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13289
- https://github.com/PrefectHQ/prefect/pull/13310
- https://github.com/PrefectHQ/prefect/pull/13320
### Support for timeout seconds in global concurrency context manager
You may want to fail immediately if a global concurrency slot is unavailable. Rather than block and wait, you can now specify a `timeout_seconds` argument in the global concurrency context manager and catch a `TimeoutError` if a slot is not available within the specified time.
```python
@flow
def fail_immediately_flow():
try:
with concurrency("there-can-be-only-one", occupy=1, timeout_seconds=0.1):
do_something_resource_intensive()
except TimeoutError:
return Cancelled(message="Another flow run is already running")
```
See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13262
### Manage global concurrency limits via the CLI
Global concurrency limits let you control how many operations can run simultaneously-- now you can create, read, edit, and delete global concurrency limits via the Prefect CLI!
To create a new concurrency limit, use the `prefect gcl create` command. You must specify a `--limit` argument, and can optionally specify a `--slot-decay-per-second` and `--disable` argument.
```bash
prefect gcl create my-concurrency-limit --limit 5 --slot-decay-per-second 1.0
```
You can inspect the details of a concurrency limit using the `prefect gcl inspect` command:
```bash
prefect gcl inspect my-concurrency-limit
```
To update a concurrency limit, use the `prefect gcl update` command. You can update the `--limit`, `--slot-decay-per-second`, `--enable`, and `--disable` arguments:
```bash
prefect gcl update my-concurrency-limit --limit 10
```
See all available commands and options by running `prefect gcl --help` or read our [docs](/docs/guides/global-concurrency-limits.md#managing-global-concurrency-limits-and-rate-limits).
For implementation details, see the following pull requests:
- https://github.com/PrefectHQ/prefect/pull/13194
- https://github.com/PrefectHQ/prefect/pull/13196
- https://github.com/PrefectHQ/prefect/pull/13214
- https://github.com/PrefectHQ/prefect/pull/13218
- https://github.com/PrefectHQ/prefect/pull/13233
- https://github.com/PrefectHQ/prefect/pull/13238
### Enhancements
- Remove registry conflict warning — https://github.com/PrefectHQ/prefect/pull/13155
- Remove top-level Artifacts tab from Prefect UI:
- https://github.com/PrefectHQ/prefect/pull/13226
- https://github.com/PrefectHQ/prefect/pull/13261
### Fixes
- Fix work pool base job template generation for `ECSTask` block — https://github.com/PrefectHQ/prefect/pull/13256
- Fix selecting correct files when using ignore file in `GcsBucket`'s `put_directory` — https://github.com/PrefectHQ/prefect/pull/13290
- Add `Resuming` flow runs to `BypassCancellingFlowRunsWithNoInfra` orchestration policy — https://github.com/PrefectHQ/prefect/pull/13299
- Fix `apprise 1.8.0` imports — https://github.com/PrefectHQ/prefect/pull/13311
- Remove `dataclass` from custom constrained types - https://github.com/PrefectHQ/prefect/pull/13257
### Experimental
#### Engine
- Add crash detection for flow runs — https://github.com/PrefectHQ/prefect/pull/13266
- Consolidate run creation logic on Task — https://github.com/PrefectHQ/prefect/pull/13271
- Skip timeout context if not needed — https://github.com/PrefectHQ/prefect/pull/13306
- Add parent task tracking — https://github.com/PrefectHQ/prefect/pull/12915
- Syncify task engine — https://github.com/PrefectHQ/prefect/pull/13234
- Syncify flow engine — https://github.com/PrefectHQ/prefect/pull/13246
- Use Prefect-specific `TestClient` for sync calls — https://github.com/PrefectHQ/prefect/pull/13265
- Add new sync compatibility setting — https://github.com/PrefectHQ/prefect/pull/13224
#### Deployment Schedule Behavior
- Add new fields to `DeploymentSchedule` schemas — https://github.com/PrefectHQ/prefect/pull/13204
- Allow both `active` and `schedule` parameters in `update_deployment_schedule` method — https://github.com/PrefectHQ/prefect/pull/13259
- Update JSON schema validation for job varariables — https://github.com/PrefectHQ/prefect/pull/13182
### Documentation
- Update block concept page to reflect product updates — https://github.com/PrefectHQ/prefect/pull/13193
- Update example repo links to `prefecthq` repos — https://github.com/PrefectHQ/prefect/pull/13258
- Update storage guide — https://github.com/PrefectHQ/prefect/pull/13294
- Update integration libraries — https://github.com/PrefectHQ/prefect/pull/13277
- Update `Hosting a Prefect server instance` page — https://github.com/PrefectHQ/prefect/pull/13225
- Simplify `prefect-aws` and `prefect-dbt` docs index pages — https://github.com/PrefectHQ/prefect/pull/13232
- Expand discussion of resolution order for cloud-provider service auth — https://github.com/PrefectHQ/prefect/pull/13239
- Fix repo url typo in storage guide — https://github.com/PrefectHQ/prefect/pull/13304
### Integrations
- Add pre-built Prefect DBT tasks — https://github.com/PrefectHQ/prefect/pull/12964
### Contributors
- @Andrew-S-Rosen
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.18.3...2.19.0
## Release 2.18.3
### Experimental
#### Engine
- Wire up new engine to deployment runs — https://github.com/PrefectHQ/prefect/pull/12914
### Fixes
- Fix parameters becoming unresponsive and disappearing in Prefect UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2355
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.18.2...2.18.3
## Release 2.18.2
### Providing a deployment name to `flow.serve` is now optional
When running `flow.serve`, you can now omit the deployment name. If you do not provide a deployment name, the deployment name will default to the name of the flow. This change makes it easier to run flows without needing to specify a deployment name each time:
```python
@flow
def etl_flow():
pass
if __name__ == "__main__":
etl_flow.serve()
```
results in:
```bash
Your flow 'etl-flow' is being served and polling for scheduled runs!
To trigger a run for this flow, use the following command:
$ prefect deployment run 'etl-flow/etl-flow'
```
See the following PR for implementation details:
- https://github.com/PrefectHQ/prefect/pull/13069
### Enhancements
- Add `PREFECT_SERVER_CSRF_PROTECTION_ENABLED` setting to UI settings — https://github.com/PrefectHQ/prefect/pull/13168
- Allow case-insensitive state and state type handling when listing flow runs via CLI — https://github.com/PrefectHQ/prefect/pull/13152
### Fixes
- Fix deployment parameter defaults on Deployments page in the UI - https://github.com/PrefectHQ/prefect-ui-library/pull/2344
- Sync value between form and JSON when entering flow parameters on the Deployments page in the UI - https://github.com/PrefectHQ/prefect-ui-library/pull/2342
- Revert console setup changes to fix interactivity — https://github.com/PrefectHQ/prefect/pull/13145
- Warn when work queues paused when starting a worker or agent — https://github.com/PrefectHQ/prefect/pull/13159
- Standardize work pool type as `Process` — https://github.com/PrefectHQ/prefect/pull/13176
- Raise a clearer error when deleting and inspecting blocks — https://github.com/PrefectHQ/prefect/pull/13136
- Fix csrf race condition that caused some pages to not render content when refreshing — https://github.com/PrefectHQ/prefect/pull/13172
### Experimental
#### Events and Automations
- Add work queue status events — https://github.com/PrefectHQ/prefect/pull/12900
- Add work pool status events — https://github.com/PrefectHQ/prefect/pull/13158
- Add support for negative label values in `ResourceSpecification` and filters — https://github.com/PrefectHQ/prefect/pull/13192
- Add automations SDK methods — https://github.com/PrefectHQ/prefect/pull/12830
- Add a retention policy for events — https://github.com/PrefectHQ/prefect/pull/13160
- Allow streaming OSS events via `prefect event stream` — https://github.com/PrefectHQ/prefect/pull/13161
- Update `prefect automation inspect` to handle automations with same name — https://github.com/PrefectHQ/prefect/pull/12904
- Update `automation pause` and `automation resume` to handle automations with same name — https://github.com/PrefectHQ/prefect/pull/13131
- Rename `prefect.work-pool.not_ready` to `prefect.work-pool.not-ready` — https://github.com/PrefectHQ/prefect/pull/13202
- Correct an issue that would cause the `work-queue.ready` event to overfire — https://github.com/PrefectHQ/prefect/pull/13117
#### Engine
- Add dedicated synchronous function handling — https://github.com/PrefectHQ/prefect/pull/12889
- Add async `task.submit` support with new task engine — https://github.com/PrefectHQ/prefect/pull/13153
- Fix subflow handling in new engine — https://github.com/PrefectHQ/prefect/pull/12913
- Handle *args / **kwargs correctly — https://github.com/PrefectHQ/prefect/pull/13142
#### Deployment schedule behavior
- Add columns to ORM `DeploymentSchedule` and add migrations — https://github.com/PrefectHQ/prefect/pull/13186
- Add server default for non-nullable deployment schedule column - https://github.com/PrefectHQ/prefect/pull/13206
### Integrations
- Add `keep_container_group` to ACI worker — https://github.com/PrefectHQ/prefect/pull/13143
- Improve Vertex AI worker performance — https://github.com/PrefectHQ/prefect/pull/13139
- Migrate `prefect-ray` to core — https://github.com/PrefectHQ/prefect/pull/12869
- Log full output of databricks job — https://github.com/PrefectHQ/prefect/pull/13151
- Update Snowflake Connector example in UI — https://github.com/PrefectHQ/prefect/pull/12903
- Fix pydantic v1 prefect-databricks — https://github.com/PrefectHQ/prefect/pull/13166
- Fix inclusion of commas in tag scrubbing — https://github.com/PrefectHQ/prefect/pull/13190
- Handle empty `service_account_info` for cached Vertex client — https://github.com/PrefectHQ/prefect/pull/13175
- Add `dlt-prefect` recipe — https://github.com/PrefectHQ/prefect/pull/13203
### Documentation
- Add third-party secrets guide — https://github.com/PrefectHQ/prefect/pull/13173
- Update documentation on nested / autonomous tasks — https://github.com/PrefectHQ/prefect/pull/13154
- Update Prefect Snowflake docs — https://github.com/PrefectHQ/prefect/pull/13171
- Update prefect-dbt index page — https://github.com/PrefectHQ/prefect/pull/13187
- Fix `az acr create` command in ACI worker guide — https://github.com/PrefectHQ/prefect/pull/12909
- Update prefect-dbt index page - https://github.com/PrefectHQ/prefect/pull/13187
### Contributors
- @h2oa made their first contribution in https://github.com/PrefectHQ/prefect/pull/13157
- @ConstantinoSchillebeeckx
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.18.1...2.18.2
## Release 2.18.1
### Fixes
- Fix improper context access for nested async task outside of flow — https://github.com/PrefectHQ/prefect/pull/12810
- Fix using default interval schedule in `prefect deploy` — https://github.com/PrefectHQ/prefect/pull/12833
- Handle case in `validationUpdate` schema where definitions are falsy — https://github.com/PrefectHQ/prefect/pull/12880
- Allow `prefect cloud login` to override current workspace — https://github.com/PrefectHQ/prefect/pull/12867
- Remove extra quotes in `prefect deployment run --watch` — https://github.com/PrefectHQ/prefect/pull/12894
### Experimental
#### Events and Automations
- Support filtering by automation name:
- https://github.com/PrefectHQ/prefect/pull/12850
- https://github.com/PrefectHQ/prefect/pull/12884
- https://github.com/PrefectHQ/prefect/pull/12887
- Add support for using the "normal" Trigger classes for `flow.serve` and `.deploy` — https://github.com/PrefectHQ/prefect/pull/12789
- Add an account-level event subscriber — https://github.com/PrefectHQ/prefect/pull/12808
- Emit flow run state change events — https://github.com/PrefectHQ/prefect/pull/12825
- Emit deployment status persistence and events — https://github.com/PrefectHQ/prefect/pull/12853
- Enable event streaming from `PrefectCloudEventSubscriber` via CLI — https://github.com/PrefectHQ/prefect/pull/12796
- Update the `prefect automation delete` CLI — https://github.com/PrefectHQ/prefect/pull/12876
#### Engine
- Add new experimental engine for tasks and flows with improved readability and extensibility — https://github.com/PrefectHQ/prefect/pull/12856
### Documentation
- Improve installation instructions — https://github.com/PrefectHQ/prefect/pull/12783
- Improve quickstart — https://github.com/PrefectHQ/prefect/pull/12798
- Migrate `prefect-azure` docs to Integrations section of the Prefect docs — https://github.com/PrefectHQ/prefect/pull/12794
- Update storage guide credentials blocks — https://github.com/PrefectHQ/prefect/pull/12819
- Remove `server` import recommendations — https://github.com/PrefectHQ/prefect/pull/12823
- Remove link to removed API page — https://github.com/PrefectHQ/prefect/pull/12824
- Add Azure Container Instances worker guide — https://github.com/PrefectHQ/prefect/pull/12846
- Improve wording on integrations index page — https://github.com/PrefectHQ/prefect/pull/12852
#### Prefect UI Library
- Add `FormattedDate` component to display accessible, long-form timestamps consistently
- Update modal buttons and add auto-close to the parameters and job variable modals — https://github.com/PrefectHQ/prefect-ui-library/pull/2320
- Add flow run list information density — https://github.com/PrefectHQ/prefect-ui-library/pull/2321
- Fix "Run a deployment" action not populating the default parameters from the deployment — https://github.com/PrefectHQ/prefect-ui-library/pull/2322
- Fix schema form properties with no default value from defaulting to `null` (`None`) — https://github.com/PrefectHQ/prefect-ui-library/pull/2323
- Update date-fns and date-fns-tz — https://github.com/PrefectHQ/prefect-ui-library/pull/2319
- Use correct icon colors for non-destructive actions in the UI — https://github.com/PrefectHQ/prefect-ui-library/pull/2328
### Integrations
#### Prefect CGP
- Remove API ref to nonexistent Google Cloud Run V2 page — https://github.com/PrefectHQ/prefect-gcp/pull/260
- Fix VPC access for Cloud v2 worker — https://github.com/PrefectHQ/prefect-gcp/pull/266
- Handle case where `vpc` isn't in job template — https://github.com/PrefectHQ/prefect-gcp/pull/267
## New Contributors
* @keizobabybear made their first contribution in https://github.com/PrefectHQ/prefect/pull/12852
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.18.0...2.18.1
## Release 2.18.0
### Breaking Changes
- Deployment configuration update: The `prefect deploy` command now only supports the `prefect.yaml` file. The `deployment.yaml` file is no longer supported
following its deprecation last June. Users should update their deployment configurations to use `prefect.yaml` instead. Running `prefect deploy` on a version prior to 2.18.0 will migrate your `deployment.yaml` file to a `prefect.yaml` file. - https://github.com/PrefectHQ/prefect/pull/12731
- `prefect deploy` options update: The `-f/--flow` option has been removed from `prefect deploy` following its deprecation last June. Please deploy
using the flow entrypoint instead. - https://github.com/PrefectHQ/prefect/pull/12732
- `prefect project` removal: The `projects` command group has been removed following its deprecation last June. For instance, instead of using `prefect project init`, use `prefect init` instead. — https://github.com/PrefectHQ/prefect/pull/12737
- `--ci` option removal: The `--ci` option in `prefect deploy` has been removed to unify the deployment experience across different environments. This removal follows its scheduled deprecation. Please use the `--no-prompt` option instead, e.g. `prefect --no-prompt deploy`. — https://github.com/PrefectHQ/prefect/pull/12740
### Enhancements
- Improve account selection in `prefect cloud login` and `workspace set` — https://github.com/PrefectHQ/prefect/pull/12717
### Fixes
- Raise clearer flow validation error — https://github.com/PrefectHQ/prefect/pull/12715
- Exclude job_variables when exclude=None — https://github.com/PrefectHQ/prefect/pull/12712
- Remove experimental flags on infrastructure overrides — https://github.com/PrefectHQ/prefect/pull/12742
### Experimental
#### Pydantic V2 Compatibility
- Introduce self-validating types — https://github.com/PrefectHQ/prefect/pull/12707
- Refactor `field_validator` and `model_validator` to map Pydantic kwargs between versions — https://github.com/PrefectHQ/prefect/pull/12676
- Fix type-hinting for self-validating fields — https://github.com/PrefectHQ/prefect/pull/12710
- Fix types NonNegativeDuration / PositiveDuration — https://github.com/PrefectHQ/prefect/pull/12711
#### Events and Automations
- Implement the `run-deployment` automation action — https://github.com/PrefectHQ/prefect/pull/12677
- Implement the `send-notification` action — https://github.com/PrefectHQ/prefect/pull/12693
- Make `TriggeredAction.firing` required — https://github.com/PrefectHQ/prefect/pull/12697
- Add an Actions service — https://github.com/PrefectHQ/prefect/pull/12699
- Implement the `call-webhook` action and adds all Action client-side schemata — https://github.com/PrefectHQ/prefect/pull/12728
- Implement `change-flow-run-state`, `cancel-flow-run`, and `suspend-flow-run` — https://github.com/PrefectHQ/prefect/pull/12730
- Add functions for querying and counting events — https://github.com/PrefectHQ/prefect/pull/12696
- Implement the `pause-deployment` and `resume-deployment` actions — https://github.com/PrefectHQ/prefect/pull/12733
- Add `/events/filter` and `/events/count-by` route trees — https://github.com/PrefectHQ/prefect/pull/12736
- Allow for creating automations via deployments when experimental events is on — https://github.com/PrefectHQ/prefect/pull/12701
- Add ability to stream out events via websocket — https://github.com/PrefectHQ/prefect/pull/12744
- Implement the `pause-automation` and `resume-automation` actions — https://github.com/PrefectHQ/prefect/pull/12738
- Add automations CLI — https://github.com/PrefectHQ/prefect/pull/12754
- Rename `prefect-cloud.*` events and labels to `prefect.*` — https://github.com/PrefectHQ/prefect/pull/12755
- Add ability to emit events to an ephemeral Prefect server — https://github.com/PrefectHQ/prefect/pull/12762
- Disable `events` and `automations` API routes when experimental events setting is not enabled — https://github.com/PrefectHQ/prefect/pull/12777
- Add compatibility tests for client and server triggers and actions — https://github.com/PrefectHQ/prefect/pull/12778
- Disable the automations integration flows for Prefect Cloud — https://github.com/PrefectHQ/prefect/pull/12784
- Add pause and resume the work pool and work queue actions — https://github.com/PrefectHQ/prefect/pull/12735
- Add helper functions for creating an events client or subscriber — https://github.com/PrefectHQ/prefect/pull/12759
- Add default posture to `EventTrigger` schema — https://github.com/PrefectHQ/prefect/pull/12764
- Fix writing events for SQLite + SQLAlchemy<2 — https://github.com/PrefectHQ/prefect/pull/12679
### Documentation
- Update `prefect.yaml` example in work pools concepts page — https://github.com/PrefectHQ/prefect/pull/12695
- Fix typo in Quickstart — https://github.com/PrefectHQ/prefect/pull/12729
- Simplify quickstart — https://github.com/PrefectHQ/prefect/pull/12725
- Add `.serve`, `.deploy`, and composite trigger examples to deployment triggers docs — https://github.com/PrefectHQ/prefect/pull/12743
- Update automations images — https://github.com/PrefectHQ/prefect/pull/12752
- Simplify tutorial — https://github.com/PrefectHQ/prefect/pull/12765
- Remove disclaimer for Python 3.12 experimental support — https://github.com/PrefectHQ/prefect/pull/12771
- Clarify deployment trigger examples — https://github.com/PrefectHQ/prefect/pull/12782
- Remove Prefect-managed integration libraries to be archived from the integrations catalog — https://github.com/PrefectHQ/prefect/pull/12781
- Fix broken link to push work pool guide — https://github.com/PrefectHQ/prefect/pull/12748
- Fix minor restructure to improve legibility of work pools tutorial — https://github.com/PrefectHQ/prefect/pull/12747
- Fix `typing` import and typos in tasks tutorial — https://github.com/PrefectHQ/prefect/pull/12746
- Simplify installation — https://github.com/PrefectHQ/prefect/pull/12772
- Fix import syntax in `variables.Variable` example — https://github.com/PrefectHQ/prefect/pull/12727
- Fix typo in How-to Guide document — https://github.com/PrefectHQ/prefect/pull/12761
## New Contributors
* @hboehmer-IW made their first contribution in https://github.com/PrefectHQ/prefect/pull/12721
* @avriiil made their first contribution in https://github.com/PrefectHQ/prefect/pull/12748
* @takashimakazuki made their first contribution in https://github.com/PrefectHQ/prefect/pull/12761
### Integrations
- Add support for a capacity provider — https://github.com/PrefectHQ/prefect-aws/pull/407
- Improve error handling for task creation — https://github.com/PrefectHQ/prefect-aws/pull/406
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.17.1...2.18.0
## Release 2.17.1
### Fixes
- Fix events storage import — https://github.com/PrefectHQ/prefect/pull/12681
- Remove `opentelemetry` import — https://github.com/PrefectHQ/prefect/pull/12684
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.17.0...2.17.1
## Release 2.17.0
### Manage Prefect variables via the Python SDK
Prefect variables are useful for storing and reusing data and configuration between and across workflows; and previously you could only create and update variables via the Prefect UI. With this release, you can now get and set Prefect variables directly in your Python code with the new `Variable.set` and `Variable.get` methods!
For an example of reading and writing variable values in Python see the following example:
```python
from prefect.variables import Variable
# set a variable
variable = Variable.set(name="the_answer", value="42")
# get a variable
answer = Variable.get('the_answer')
print(answer.value)
# 42
# get a variable with a default value
answer = Variable.get('not_the_answer', default='42')
print(answer.value)
# 42
# update a variable
answer = Variable.set(name="the_answer", value="43", overwrite=True)
print(answer.value)
#43
```
Refer to the [docs](https://docs.prefect.io/latest/guides/variables/#accessing-variables) for more information and see the PR for implementation details: https://github.com/PrefectHQ/prefect/pull/12596
### Enhancements
- Allow flows inside tasks
— https://github.com/PrefectHQ/prefect/pull/12559
— https://github.com/PrefectHQ/prefect/pull/12607
- Add `User-Agent` header containing the running Prefect version — https://github.com/PrefectHQ/prefect/pull/12601
- Adds deployment version to the flow run object — https://github.com/PrefectHQ/prefect/pull/12591
### Fixes
- Transition flow runs without active infrastructure directly to cancelled — https://github.com/PrefectHQ/prefect/pull/12582
- Remove duplicate CLI output when reauthorizing with `prefect cloud login` — https://github.com/PrefectHQ/prefect/pull/12664
- Add `blob_storage` extra as requirement for Azure `prefect.yaml` recipes — https://github.com/PrefectHQ/prefect/pull/12333
- Exclude Typer 0.12.2 from solver — https://github.com/PrefectHQ/prefect/pull/12618
- Correct `schedules`/`is_schedule_active` deprecation windows — https://github.com/PrefectHQ/prefect/pull/12616
### Experimental / In-Flight Features
#### Pydantic V2 Compatibility
- Add `pydantic` V2 compatible `field_validator` — https://github.com/PrefectHQ/prefect/pull/12576
- Add `pydantic` V2 `model_validator` — https://github.com/PrefectHQ/prefect/pull/12635
- Expose `field_validator` in `pydantic` compatibility layer — https://github.com/PrefectHQ/prefect/pull/12608
- Add `ConfigDict` to `pydantic` compatibility layer — https://github.com/PrefectHQ/prefect/pull/12629
- Add `model_fields_set` to `pydantic` compatibility layer — https://github.com/PrefectHQ/prefect/pull/12654
- Map `copy_on_model_validation` to `revalidate_instances` in `pydantic` compatibility layer — https://github.com/PrefectHQ/prefect/pull/12644
#### Events and Automations
- Enable `EventsWorker` to emit events to Prefect servers — https://github.com/PrefectHQ/prefect/pull/12637
- Add ORM models and database migrations for events storage — https://github.com/PrefectHQ/prefect/pull/12651
- Add automations API — https://github.com/PrefectHQ/prefect/pull/12620
- Add reactive and composite triggers — https://github.com/PrefectHQ/prefect/pull/12650
- Add proactive triggers — https://github.com/PrefectHQ/prefect/pull/12660
- Add `EventPersister` service to store received events - https://github.com/PrefectHQ/prefect/pull/12662
### Deprecations
- Remove expired deprecations from `prefect/__init__.py` — https://github.com/PrefectHQ/prefect/pull/12613
### Documentation
- Update references to deployment schedules — https://github.com/PrefectHQ/prefect/pull/12595
- Add missing navigation items for `prefect shell` CLI command — https://github.com/PrefectHQ/prefect/pull/12598
- Update formatting for `prefect shell` CLI command — https://github.com/PrefectHQ/prefect/pull/12606
- Add comment to blocks concept page when using `SecretStr` with `pydantic` V2 — https://github.com/PrefectHQ/prefect/pull/12632
- Fix name format in `run_deployment` docstring — https://github.com/PrefectHQ/prefect/pull/12628
- Add documentation for flow run job variables — https://github.com/PrefectHQ/prefect/pull/12490
- Add example of retrieving an artifact in Python code — https://github.com/PrefectHQ/prefect/pull/12666
### Contributors
- @hainenber
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.16.9...2.17.0
## Release 2.16.9
### `prefect deploy` with `-jv/--job-variable` option
In a prior release, we added a `-jv/--job-variable` option for providing job variables when running a deployment using `prefect deployment run`. We want to be consistent in our CLI by allowing you to use this option while creating deployments during `prefect deploy`! Thus, we have added a `-jv/--job-variable` option to `prefect deploy` to replace the `-v/--variables` option, which we have now deprecated.
See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/12410
### Enhancements
- Remove nested task constraint that prevented tasks called from other tasks — https://github.com/PrefectHQ/prefect/pull/12548
- Stop creating artifacts for unpersisted results - https://github.com/PrefectHQ/prefect/pull/12454
- Allow for deletion of work pool workers via API — https://github.com/PrefectHQ/prefect/pull/12330
- Raise more informative error on `prefect worker start -t bad-type` - https://github.com/PrefectHQ/prefect/pull/12586
- Add tooltip and increase width to support better displaying long Prefect variable names in the UI https://github.com/PrefectHQ/prefect-ui-library/pull/2275
### Fixes
- Raise lower bound on `typer` dependency — https://github.com/PrefectHQ/prefect/pull/12512
- Skip flow run cancellation if no associated deployment — https://github.com/PrefectHQ/prefect/pull/12001
- Handle referenced blocks in base templates during `job_variable` validation — https://github.com/PrefectHQ/prefect/pull/12329
- Select correct `AsyncWaiter` for successively awaited flow and task calls — https://github.com/PrefectHQ/prefect/pull/12510
- Handle flow run creation for runner-managed deployments — https://github.com/PrefectHQ/prefect/pull/12319
- Expose `ignore_warnings` in `Flow.deploy` — https://github.com/PrefectHQ/prefect/pull/12569
- Allow `prefect cloud login` re-authentication in non-interactive mode — https://github.com/PrefectHQ/prefect/pull/12575
- Update ECS provisioner IAM policy to include `ecs:TagResource` permission — https://github.com/PrefectHQ/prefect/pull/12551
- Correctly populate custom default parameters in the flow submission form in the UI - https://github.com/PrefectHQ/prefect-ui-library/pull/2280
### Experimental / In-Flight Features
#### Flow Run Infrastructure Overrides
- Add support for adding job variables to trigger definitions via CLI - https://github.com/PrefectHQ/prefect/pull/12276
#### Pydantic V2 Compatibility
- Add dynamic importing of Pydantic modules
- https://github.com/PrefectHQ/prefect/pull/12498
- https://github.com/PrefectHQ/prefect/pull/12503
- Refactor Pydantic V2 compatibility layer into submodules — https://github.com/PrefectHQ/prefect/pull/12522
- Enable support for `mode="json"` in `model_dump` function by default — https://github.com/PrefectHQ/prefect/pull/12540
#### Events and Automations
- Add message publisher and consumer abstractions, with in-memory implementation — https://github.com/PrefectHQ/prefect/pull/12485
- Add events HTTP and websocket endpoints — https://github.com/PrefectHQ/prefect/pull/12499
- Add a diagnostic service which consumes events and prints a summary of them — https://github.com/PrefectHQ/prefect/pull/12501
- Add internal events client for publishing events from other server-side areas — https://github.com/PrefectHQ/prefect/pull/12520
- Add an internal orchestration API client for use in events — https://github.com/PrefectHQ/prefect/pull/12534
- Add server-side automations schema models — https://github.com/PrefectHQ/prefect/pull/12549
- Add ORM classes and model modules for automations and its state tables — https://github.com/PrefectHQ/prefect/pull/12581
### Integrations - Prefect AWS
- Fix `S3Bucket.copy_object` target path resolution — https://github.com/PrefectHQ/prefect-aws/pull/385
- Add Python 3.12 support and remove 3.7 support — https://github.com/PrefectHQ/prefect-aws/pull/405
- Change logging prefix to avoid unnecessary task definition registrations — https://github.com/PrefectHQ/prefect-aws/pull/400
### Deprecations
- Deprecate `KubernetesCusterConfig` block — https://github.com/PrefectHQ/prefect/pull/12571
- Remove use of PartialModel — <https://github.com/PrefectHQ/prefect/pull/12574>
### Documentation
- Add `prefect shell` commands to guides index — https://github.com/PrefectHQ/prefect/pull/12494
- Update Prefect Cloud plan information — https://github.com/PrefectHQ/prefect/pull/12505
- Add timeout information to flows concept page — https://github.com/PrefectHQ/prefect/pull/12550
- Remove outdated doc warning on calling tasks within tasks — https://github.com/PrefectHQ/prefect/pull/12580
- Remove broken link from FAQ page - https://github.com/PrefectHQ/prefect/pull/12590
- Fix typo in FAQ page — https://github.com/PrefectHQ/prefect/pull/12584
### Contributors
* @hainenber
* @jwijffels made their first contribution in https://github.com/PrefectHQ/prefect/pull/12575
* @ShaoyiZhang made their first contribution in https://github.com/PrefectHQ/prefect/pull/12584
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.16.8...2.16.9
## Release 2.16.8
### Fixes
- Disable CSRF protection by default - https://github.com/PrefectHQ/prefect/pull/12479
- Fix issue causing UI not to be built when creating docker images - https://github.com/PrefectHQ/prefect/pull/12481
## Release 2.16.7
### Introducing `prefect shell` 💻 for observing CLI commands
You can now observe CLI commands as a Prefect flow. For example, take the command:
```console
» curl http://wttr.in/Chicago\?format\=3
Chicago: ⛅️ +50°F
```
To run this as a Prefect flow, you can use the following CLI command:
```python
» prefect shell watch "curl http://wttr.in/Chicago?format=3"
17:32:39.562 | INFO | prefect.engine - Created flow run 'powerful-mushroom' for flow 'Shell Command'
17:32:40.171 | INFO | Flow run 'powerful-mushroom' - Chicago: ⛅️ +50°F
17:32:40.315 | INFO | Flow run 'powerful-mushroom' - Finished in state Completed()
```
See these [docs](https://docs.prefect.io/latest/guides/cli-shell) to learn how to:
- run a shell command as a Prefect flow on-demand with `watch`
- schedule a shell command as a recurring Prefect flow using `serve`
See the PR for implementation details: https://github.com/PrefectHQ/prefect/pull/11998
### Enhancements
- Integrate composite triggers with the `DeploymentTrigger` YAML representation — https://github.com/PrefectHQ/prefect/pull/12413
- Add JSON Artifacts — https://github.com/PrefectHQ/prefect/pull/12295
- Add auto-provisioning option for Cloud Run V2 push work pools — https://github.com/PrefectHQ/prefect/pull/12422
- Increase late runs after seconds setting default — https://github.com/PrefectHQ/prefect/pull/12457
### Fixes
- Properly display falsy `concurrency_limit` value in CLI — https://github.com/PrefectHQ/prefect/pull/12358
- Correct wrong date in `prefect deploy` deprecation warning for `schedule` — https://github.com/PrefectHQ/prefect/pull/12399
- Prompt user confirmation for pausing work queue in default work pool — https://github.com/PrefectHQ/prefect/pull/12334
- Correct type for `slot_decay_per_second` in client SDK — https://github.com/PrefectHQ/prefect/pull/12401
- Sync SDK upgrades with UI upgrades — https://github.com/PrefectHQ/prefect/pull/12429
- Pin uvicorn to < 0.29 — https://github.com/PrefectHQ/prefect/pull/12463
### Experimental
- More robust error handling in `TaskWorker` — https://github.com/PrefectHQ/prefect/pull/12382
- Add `model_validate_json` to Pydantic compat layer — https://github.com/PrefectHQ/prefect/pull/12412
- Add `model_dump_json` to Pydantic compat layer — https://github.com/PrefectHQ/prefect/pull/12406
- Add hybrid `BaseModel` and public `pydantic` module — https://github.com/PrefectHQ/prefect/pull/12424
- Add Pydantic `TypeAdapter` backport — https://github.com/PrefectHQ/prefect/pull/12445
- Add `model_copy` to Pydantic compat layer — https://github.com/PrefectHQ/prefect/pull/12418
### Documentation
- Add `prefect shell` CLI documentation — https://github.com/PrefectHQ/prefect/pull/12474
- Add links to serverless and push serverless work pool guides for dependency management — https://github.com/PrefectHQ/prefect/pull/12392
- Add example of transitioning all running flows to `CANCELLED` via Prefect client — https://github.com/PrefectHQ/prefect/pull/12390
- Temporarily remove social cards — https://github.com/PrefectHQ/prefect/pull/12465
### Contributors
- @hainenber
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.16.6...2.16.7
## Release 2.16.6
### Fix new behavior in `typer 0.10.0` that broke the `prefect` CLI
See the following pull request for implementation details: https://github.com/PrefectHQ/prefect/pull/12398
### Enhancements
- Improve nested schema hydration for templating — https://github.com/PrefectHQ/prefect/pull/12384
- Improve interactive workspace selection in the CLI — https://github.com/PrefectHQ/prefect/pull/12387
- Use Pydantic V2 for schema generation where possible — https://github.com/PrefectHQ/prefect/pull/12210
- Enable CSRF protection by default — https://github.com/PrefectHQ/prefect/pull/12377
### Fixes
- Handle new `typer.Option` behavior — https://github.com/PrefectHQ/prefect/pull/12398
### Experimental
- Add experimental `model_validate` function for Pydantic V2 compatibility — https://github.com/PrefectHQ/prefect/pull/12370
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.16.5...2.16.6
## Release 2.16.5
### Multi-select deletion of flow runs
It is now easier to bulk select and delete flow runs through the UI. Listings of filterable and selectable flow runs (e.g. on the flow runs, flow, and deployment pages) now include a top-level checkbox for (de)selecting all currently filtered flow runs for bulk deletion.
See the following pull request for implementation details:
- https://github.com/PrefectHQ/prefect/pull/12356
- https://github.com/PrefectHQ/prefect-ui-library/pull/2227
- https://github.com/PrefectHQ/prefect/pull/12285
### Visualize state changes and artifacts in the UI
Additionally, the flow run graph UI enhancements for visualizing state changes and artifacts added in 2.16.4 are now enabled by default. See [the release notes in 2.16.14 for more details](https://github.com/PrefectHQ/prefect/blob/main/RELEASE-NOTES.md#release-2164)!
### Enhancements
- Keep artifacts file in prefect-client — https://github.com/PrefectHQ/prefect/pull/12316
- remove feature flagging around enhanced-deployment-experiment — https://github.com/PrefectHQ/prefect/pull/12360
- Feature : #11773 UI: Add checkboxes for runs for an individual flow to allow multi-selection/-deletion — https://github.com/PrefectHQ/prefect/pull/12285
- Add a capability to verify ssl certificates in Prefect CLI — https://github.com/PrefectHQ/prefect/pull/11771
- Add `prefect task-run` command group to CLI — https://github.com/PrefectHQ/prefect/pull/12307
- Correct emit background task state change events — https://github.com/PrefectHQ/prefect/pull/12352
- Update `CsrfTokenApi` to retry failed requests due to invalid tokens — https://github.com/PrefectHQ/prefect/pull/12373
### Fixes
- Refactor logic to set `task_key` for background tasks — https://github.com/PrefectHQ/prefect/pull/12337
- Correct a memory leak with the outbound task run websockets — https://github.com/PrefectHQ/prefect/pull/12346
- Correctly type hint in flow run state change hooks — https://github.com/PrefectHQ/prefect/pull/12231
### Experimental
- Create `CsrfToken` model and utilities — https://github.com/PrefectHQ/prefect/pull/12289
- Create `csrf_token` endpoint to generate tokens for clients — https://github.com/PrefectHQ/prefect/pull/12297
- Integrate `CsrfMiddleware` into API server — https://github.com/PrefectHQ/prefect/pull/12303
- Add CSRF support to client — https://github.com/PrefectHQ/prefect/pull/12314
- Return 422 when CSRF is disabled and delete expired tokens — https://github.com/PrefectHQ/prefect/pull/12342
- Add `model_dump` definition for Pydantic v2 compatibility layer — https://github.com/PrefectHQ/prefect/pull/12345
- Add experimental `model_json_schema` definition for Pydantic V2 compatibility - https://github.com/PrefectHQ/prefect/pull/12362
- Implement CSRF support in the UI — https://github.com/PrefectHQ/prefect/pull/12354
### Documentation
- Add upstream dependencies guide to docs — https://github.com/PrefectHQ/prefect/pull/12351
- Update documentation on event and metric automation triggers — https://github.com/PrefectHQ/prefect/pull/12366
- Add documentation on compound and sequence automation triggers — https://github.com/PrefectHQ/prefect/pull/12374
- Add CSRF settings to common settings section in docs — https://github.com/PrefectHQ/prefect/pull/12376
### Uncategorized
- Pin `BuildKit` to 0.12.5 to fix issue with test image build — https://github.com/PrefectHQ/prefect/pull/12343
- Backporting the Prefect Cloud composite trigger schemas — https://github.com/PrefectHQ/prefect/pull/12378
### Contributors
* @hainenber
* @JiginJayaprakash made their first contribution in https://github.com/PrefectHQ/prefect/pull/12307
* @baisystems made their first contribution in https://github.com/PrefectHQ/prefect/pull/11771
**All changes**: https://github.com/PrefectHQ/prefect/compare/2.16.4...2.16.5
## Release 2.16.4
### Flow Run Graph updates
The Flow Run Graph has been updated to display additional layers of information! Interactive and real-time state changes and artifacts are now visible in context on the graph.
<img width="892" alt="The Prefect flow run graph" src="https://github.com/PrefectHQ/prefect/assets/6776415/422a878e-f8bc-46b5-ae76-830f24861aaf">
These new layers are available for opt-in usage via the `PREFECT_EXPERIMENTAL_ENABLE_ARTIFACTS_ON_FLOW_RUN_GRAPH` and `PREFECT_EXPERIMENTAL_ENABLE_STATES_ON_FLOW_RUN_GRAPH` settings.
### Agents
A year ago, we released workers as a replacement for agents. Workers significantly enhance the experience of deploying flows and simplify the specification of each flow's infrastructure and runtime environment.
With this release we are adding a six month (September 14) deprecation warning to agents and related concepts. Please note that:
- Deprecation will not impact or break any work running with agents and agent-related concepts - although we encourage users to upgrade to workers because they provide a better deployment experience, you can continue to use existing agents and related concepts after deprecation
- After September 14, Prefect Cloud users will not be able to create new agent work pools or infrastructure blocks
- After September 14, new minor versions of the Prefect Python package will not include agents
Like agents, workers support creating deployments through the Prefect CLI and through Python, but require different syntax.
For more information, please refer to the [Upgrade from Agents to Workers](https://docs.prefect.io/latest/guides/upgrade-guide-agents-to-workers/) guide.
### Enhancements
* Give better client-side feedback on websocket authT/authZ issues - https://github.com/PrefectHQ/prefect/pull/12221
* Allow table artifact cells to render markdown content - [`#2190`](https://github.com/PrefectHQ/prefect-ui-library/pull/2190)
* Add an 'AzureBlobStorageContainer' block - [`#139`](https://github.com/PrefectHQ/prefect-azure/pull/139)
* API for task run counts by state - https://github.com/PrefectHQ/prefect/pull/12244
* Improved UI handling of custom flow run states. Badges for a state with a custom name will now more closely resemble their underlying state - https://github.com/PrefectHQ/prefect-ui-library/pull/2210 and https://github.com/PrefectHQ/prefect-ui-library/pull/2208
### Fixes
* Fix support for legacy schedule in `build_from_flow` - https://github.com/PrefectHQ/prefect/pull/12257
* Remove extra field from create endpoints - https://github.com/PrefectHQ/prefect/pull/12240
* Prevent invalid link problem for `prefect deployment run` - https://github.com/PrefectHQ/prefect/pull/12267
* Hydrate run input when resuming a flow run - https://github.com/PrefectHQ/prefect/pull/12259
* Fix state select in notifications [`#2216`](https://github.com/PrefectHQ/prefect-ui-library/pull/2216)
* Adds porting of network configuration to generated base job templates [`#392`](https://github.com/PrefectHQ/prefect-aws/pull/392)
### Experimental
* Expose `serve` method on tasks - https://github.com/PrefectHQ/prefect/pull/12239
* Avoid relative module path problem with subscription keys - https://github.com/PrefectHQ/prefect/pull/12227
### Deprecations
* Deprecate `PrefectAgent` and `prefect agent` command group - https://github.com/PrefectHQ/prefect/pull/12273
* Deprecate Infrastructrue blocks - https://github.com/PrefectHQ/prefect/pull/12282
* Deprecate `Deployment` class and deployment `build` and `apply` commands - https://github.com/PrefectHQ/prefect/pull/12283
* Deprecate the `packaging` module - https://github.com/PrefectHQ/prefect/pull/12291
* Deprecate storage blocks with implementations in Prefect integration packages - https://github.com/PrefectHQ/prefect/pull/12299
* Do not create `default-agent-pool` in empty state server - https://github.com/PrefectHQ/prefect/pull/12214
### Typing
* Refactor: Inject Client - https://github.com/PrefectHQ/prefect/pull/12258
* Fix: Variables methods type-hinting - https://github.com/PrefectHQ/prefect/pull/12278
### Documentation
* Fix formatting in concept -> work-pools.md - https://github.com/PrefectHQ/prefect/pull/12230
* Add `get_run_logger` to imports in interactive workflow examples - https://github.com/PrefectHQ/prefect/pull/12284
* Add `on_running` state change hook to State -> Concepts docs - https://github.com/PrefectHQ/prefect/pull/12293
* Fix some argument names in deploy guide - https://github.com/PrefectHQ/prefect/pull/12301
## New Contributors
- @hainenber made their first contribution in https://github.com/PrefectHQ/prefect/pull/12232
**Full Changelog**: https://github.com/PrefectHQ/prefect/compare/2.16.3...2.16.4
## Release 2.16.3
### Enhanced deployment parameters in the Prefect UI
We’ve refined the deployment parameters UI to significantly enhance default value handling and list management. This release introduces support for tuple-type parameters and a revamped list UI, freeing users from the constraints of JSON for list inputs. You now have the flexibility to utilize JSON or Prefect variables for any parameter value, enhancing the versatility of deployment configurations. Moreover, we’ve synchronized validation across the UI and deployment schemas, ensuring that user inputs are consistently checked against the defined parameter requirements, which simplifies the deployment process and minimizes configuration errors.
These improvements are aimed at providing a more efficient and user-friendly interface for managing deployment parameters. Check it out for yourself!
<img width="791" alt="paramsui" src="https://github.com/PrefectHQ/prefect/assets/42048900/d95c854f-1b95-46fb-8214-9132c923214f">
See the following PRs for implementation details:
- https://github.com/PrefectHQ/prefect/pull/12168
- https://github.com/PrefectHQ/prefect/pull/12179
- https://github.com/PrefectHQ/prefect/pull/12186
- https://github.com/PrefectHQ/prefect/pull/12187
- https://github.com/PrefectHQ/prefect/pull/12182
- https://github.com/PrefectHQ/prefect/pull/12219
### Enhancements
- Adds `on_running` state change hook — https://github.com/PrefectHQ/prefect/pull/12153
- Add flow run state data to flow run graph API — https://github.com/PrefectHQ/prefect/pull/12130
- Add schedules shorthand support to `Deployment.build_from_flow` — https://github.com/PrefectHQ/prefect/pull/12181
- Send flow run and task run retry logs to API — https://github.com/PrefectHQ/prefect/pull/12211
- Add the flow run graph states layer and handle selection — https://github.com/PrefectHQ/prefect/pull/12166 - asked Brandon
- Add a paused deployment filter and update `is_schedule_active` filter — https://github.com/PrefectHQ/prefect/pull/12202
- Updating client-side schemas for automation triggers to reflect updates in Prefect Cloud — https://github.com/PrefectHQ/prefect/pull/12205
### Fixes
- Address two memory leak in concurrency management — https://github.com/PrefectHQ/prefect/pull/12141
### Experimental
- Add Job Variables tab to the Flow Runs page — https://github.com/PrefectHQ/prefect/pull/12206
- Add support for calling `.map` for autonomous task runs — https://github.com/PrefectHQ/prefect/pull/12171
- Simplify already authenticated logic for `prefect cloud login` — https://github.com/PrefectHQ/prefect/pull/12209
- Add better parity with Prefect Cloud for task scheduling protocol - https://github.com/PrefectHQ/prefect/pull/12212
- Allow creating autonomous task runs via `__call__` — https://github.com/PrefectHQ/prefect/pull/12158
### Integrations
- Fix VPC Connector usage in Cloud Run Worker v2 - https://github.com/PrefectHQ/prefect-gcp/pull/252