Skip to content

Releases: dragonflyoss/Dragonfly2

v2.1.2

15 Aug 15:19
5fb1ef9
Compare
Choose a tag to compare

Changelog

v2.1.1

15 Aug 09:56
93009c4
Compare
Choose a tag to compare

Changelog

  • 761f5ef chore(deps): bump github.com/aws/aws-sdk-go from 1.44.312 to 1.44.317 (#2606)
  • 9446250 chore(deps): bump github.com/go-playground/validator/v10 from 10.14.1 to 10.15.0 (#2608)
  • 47523d7 chore(deps): bump github.com/shirou/gopsutil/v3 from 3.23.6 to 3.23.7 (#2628)
  • 49130df chore(deps): bump go.uber.org/zap from 1.24.0 to 1.25.0 (#2629)
  • ae0b2a9 chore(deps): bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 (#2605)
  • c5e8f8d chore(deps): bump google.golang.org/api from 0.134.0 to 0.136.0 (#2626)
  • 47807bd chore(deps): bump gorm.io/gorm from 1.25.2 to 1.25.3 (#2630)
  • 342438e chore: optimize unhandled file close error (#2599)
  • 93009c4 chore: update console version to v1.0.2 (#2635)
  • e621fce chore: update d7y.io/api version to v2.0.18 (#2616)
  • dcea13a chore: use subtle compare to verify proxy auth (#2601)
  • da7e305 feat: add NewSafeDialer and fix ssrf in manager preheat api (#2611)
  • 77c91cb feat: add lock to dag vertex and replace rand.Seed with rand.New (#2614)
  • 7f67bc5 feat: add tls client config for preheat in manager (#2612)
  • 732819b feat: case insensitive string comparison in evaluater package of the manager (#2632)
  • 25d91ec feat: change cache size in manager (#2623)
  • 25d12b2 feat: change cache size in manager (#2633)
  • c55148b feat: change default cluster name to cluster-1 (#2604)
  • a5b0f74 feat: download tiny file with https scheme (#2617)
  • 9e1bb52 feat: net.JoinHostPort replace fmt.Sprintf (#2622)
  • 05a6680 fix: directories created via os.MkdirAll are not checked for permissions (#2613)
  • 1b9d2b0 fix: improper use strings.TrimLeft (#2603)
  • afa665c fix: incorrect log message in the scheduler (#2618)
  • 9cc9fa2 fix: invalid error handling (#2610)
  • 931cd3f fix: list personal access token with query string (#2624)
  • d9e8b28 fix: manager generates mTLS certificates for arbitrary IP addresses (#2615)
  • a64140f fix: potential nil panic (#2602)
  • 094b79d fix: usage of architecture-dependent int type in the scheduler (#2619)
  • bd140f1 refactor: file close error

v2.1.0

04 Aug 07:07
8f6f20b
Compare
Choose a tag to compare

Dragonfly v2.1.0 is released! 🎉🎉🎉 Thanks to the Xinxin Zhao for helping to refactor the console and the manager provides a new console for users to operate Dragonfly. Welcome to visit d7y.io website.

Features

  • Console v1.0.0 is released and it provides a new console for users to operate Dragonfly.
  • Add network topology feature and it can probe the network latency between peers, providing better scheduling capabilities.
  • Provides the ability to control the features of the scheduler in the manager. If the scheduler preheat feature is not in feature flags, then it will stop providing the preheating in the scheduler.
  • dfstore adds GetObjectMetadatas and CopyObject to supports using Dragonfly as the JuiceFS backend.
  • Add personal access tokens feature in the manager and personal access token contains your security credentials for the restful open api.
  • Add TLS config to manager rest server.
  • Fix dfdaemon fails to start when there is no available scheduler address.
  • Add cluster in the manager and the cluster contains a scheduler cluster and a seed peer cluster.
  • Fix object downloads failed by dfstore when dfdaemon enabled concurrent.
  • Scheduler adds database field in config and moves the redis config to database field.
  • Replace net.Dial with grpc health check in dfdaemon.
  • Fix filtering and evaluation in scheduling. Since the final length of the filter is the candidateParentLimit used, the parents after the filter is wrong.
  • Fix storage can not write records to file when bufferSize is zero.
  • Hiding sensitive information in logs, such as the token in the header.
  • Use unscoped delete when destroying the manager's resources.
  • Add uk_scheduler index and uk_seed_peer index in the table of the database.
  • Remove security domain feature and security feature in the manager.
  • Add advertise port config to manager and scheduler.
  • Fix fsm changes state failed when register task.

Break Change

  • The M:N relationship model between the scheduler cluster and the seed peer cluster is no longer supported. In the future, a P2P cluster will be a cluster in the manager, and a cluster will only include a scheduler cluster and a seed peer cluster.

Console

signin

You can see Manager Console for more details.

AI Infrastructure

Maintainers

The community has added four new Maintainers, hoping to help more contributors participate in community.

  • Yiyang Huang: He works for Volcano Engine and will focus on the engineering work for Dragonfly.
  • Manxiang Wen: He works for Baidu and will focus on the engineering work for Dragonfly.
  • Mohammed Farooq: He works for Intel and will focus on the engineering work for Dragonfly.
  • Zhou Xu: He is a PhD student at Dalian University of Technology and will focus on the intelligent scheduling algorithms.

Others

You can see CHANGELOG for more details.

Links

v2.1.0-rc.0

03 Aug 14:50
c82d2f7
Compare
Choose a tag to compare

Changelog

  • c82d2f7 chore: update submodule version (#2596)
  • db82dd9 feat: change seed peer state to active in UpdateSeedPeer api (#2592)
  • a151be1 feat: create seed peer with active state in manager (#2593)
  • 733b938 feat: implement DeleteSeedPeer api in manager (#2591)
  • a966c04 feat: use unscoped delete for resource in manager (#2595)

v2.1.0-beta.4

01 Aug 12:17
b723181
Compare
Choose a tag to compare

Changelog

  • 58f486a chore(deps): bump github.com/aliyun/aliyun-oss-go-sdk from 2.2.7+incompatible to 2.2.8+incompatible (#2588)
  • 0f25876 chore(deps): bump github.com/aws/aws-sdk-go from 1.44.306 to 1.44.312 (#2584)
  • e12523a chore(deps): bump github.com/casbin/casbin/v2 from 2.72.1 to 2.73.0 (#2585)
  • 310a717 chore(deps): bump github.com/onsi/gomega from 1.27.8 to 1.27.10 (#2586)
  • a478be6 chore(deps): bump google.golang.org/api from 0.132.0 to 0.134.0 (#2587)
  • 3044b86 chore: clean temporary file when backsource error (#2575)
  • b723181 feat: add personal access token middleware to open api (#2590)
  • d894aff feat: add personal access tokens api to rest server (#2583)
  • f1fbfde feat: provide support for JuiceFS objectStorage implementation (#2578)
  • 75ddb8b feat: support tls in manager rest server (#2580)
  • cb41c0e feat: update api version (#2577)
  • 0299830 refactor: support for JuiceFS objectStorage implementation (#2579)

v2.1.0-beta.3

26 Jul 12:34
55cb752
Compare
Choose a tag to compare

Changelog

v2.1.0-beta.2

25 Jul 13:54
96ec90e
Compare
Choose a tag to compare

Changelog

  • 9da77c8 chore(deps): bump github.com/MysteriousPotato/go-lockable from 0.1.0 to 0.2.0 (#2504)
  • 93f17f5 chore(deps): bump github.com/MysteriousPotato/go-lockable from 0.2.0 to 1.0.0 (#2548)
  • 700c299 chore(deps): bump github.com/aws/aws-sdk-go from 1.44.258 to 1.44.306 (#2565)
  • e5ebce9 chore(deps): bump github.com/bits-and-blooms/bitset from 1.7.0 to 1.8.0 (#2551)
  • 3586704 chore(deps): bump github.com/casbin/casbin/v2 from 2.68.0 to 2.71.1 (#2501)
  • c7d02aa chore(deps): bump github.com/casbin/casbin/v2 from 2.71.1 to 2.72.1 (#2561)
  • 28f162a chore(deps): bump github.com/shirou/gopsutil/v3 from 3.23.5 to 3.23.6 (#2535)
  • e2b334a chore(deps): bump golang.org/x/oauth2 from 0.9.0 to 0.10.0 (#2532)
  • a35e83b chore(deps): bump google.golang.org/api from 0.128.0 to 0.129.0 (#2503)
  • 9ed62a2 chore(deps): bump google.golang.org/api from 0.129.0 to 0.130.0 (#2533)
  • de5bc37 chore(deps): bump google.golang.org/api from 0.130.0 to 0.131.0 (#2549)
  • 0f52e2e chore(deps): bump google.golang.org/api from 0.131.0 to 0.132.0 (#2564)
  • 52b36cb chore(deps): bump google.golang.org/protobuf from 1.30.0 to 1.31.0 (#2502)
  • 6fb518c chore(deps): bump gorm.io/driver/postgres from 1.5.0 to 1.5.2 (#2534)
  • fbd23d1 chore(deps): bump gorm.io/gorm from 1.25.1 to 1.25.2 (#2505)
  • 25ff199 chore(deps): bump helm/kind-action from 1.7.0 to 1.8.0 (#2553)
  • a7ef2c3 chore(deps): bump k8s.io/component-base from 0.27.2 to 0.27.4 (#2562)
  • bfb6440 chore: add fcgxz2003 to maintainer (#2522)
  • 864943f chore: change tainer address port from 9000 to 9090 in scheduler (#2571)
  • 3e6a5fc chore: change trainer expose port from 8002 to 9090 in Dockerfile (#2569)
  • a7f3c7c feat: add GNNModelIDV1 and MLPModelIDV1 in idgen (#2540)
  • be6d808 feat: add finished piece count element in types (#2557)
  • b58879a feat: add inference client in grpc (#2493)
  • 778b90c feat: add optional to build information (#2567)
  • aa95dfe feat: add pieces element in download record (#2531)
  • c7e43be feat: add training service (#2543)
  • 355d9dd feat: add update model rest api (#2530)
  • 1da3550 feat: implement FindProbedHosts and add LoadRandomHosts to host manager (#2519)
  • 9d1e07c feat: implement Train grpc api in trainer (#2541)
  • 6301d22 feat: manager adds createModel function (#2521)
  • 0736617 feat: manager adds trainer config (#2494)
  • a45fcd9 feat: remove ProbedAt function in network topology (#2529)
  • 1d1e696 feat: replace keys with scan in redis client (#2507)
  • 9a36cc5 feat: set scan count for redis client (#2508)
  • cede829 feat: update api verison to v1.9.7 and compatible with tiny task response (#2547)
  • bd3d76a feat: update api version to 2.0.8 (#2566)
  • b4a390d feat: update api version to v2.0.7 (#2560)
  • b3f23f7 feat: update dy7.io/api to v2 (#2558)
  • 512ad52 fix: also add ca to RootCAs (#2516)
  • 01287f4 fix: change model state in the same scheduler id (#2537)
  • 0490a7b fix: if condition judgment of clearing file in trainer service (#2544)
  • 96ec90e fix: remove archives.rlcp in .goreleaser.yaml refer to https://gorele… (#2573)
  • 22e7d62 fix: response of cluster rest api (#2572)
  • 4b2759c fix: scheduler.template.yaml comments (#2526)
  • 650cc67 fix: storage and announcer unit tests (#2542)
  • a8f7c56 refactor: create model grpc api in manager (#2528)
  • 2cb0acc refactor: store pieceRecords in download record (#2539)

v2.1.0-beta.1

28 Jun 12:10
91a9d6f
Compare
Choose a tag to compare

Changelog

Read more

v2.1.0-beta.0

15 Jun 15:03
b3a5168
Compare
Choose a tag to compare

Changelog

  • aaec21c chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#2419)
  • ebe1a68 chore(deps): bump github.com/mdlayher/vsock from 1.2.0 to 1.2.1 (#2405)
  • 1867095 chore(deps): bump github.com/montanaflynn/stats from 0.7.0 to 0.7.1 (#2407)
  • 5a1a4fb chore(deps): bump github.com/shirou/gopsutil/v3 from 3.23.3 to 3.23.5 (#2434)
  • 73998b2 chore(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#2408)
  • dd8f18e chore(deps): bump go.uber.org/atomic from 1.10.0 to 1.11.0 (#2404)
  • b66d098 chore(deps): bump google.golang.org/grpc from 1.56.0-dev to 1.57.0-dev (#2433)
  • 5901576 chore(deps): bump k8s.io/component-base from 0.26.0 to 0.27.2 (#2432)
  • 86ce24c chore: update dfget recursive log (#2459)
  • 09570c3 chore: update grpc api definition to v1.9.0 (#2444)
  • 0da92ef chore: update grpc proto version (#2463)
  • b3a5168 feat: add Access-Control-Expose-Headers to headers (#2467)
  • 5ffa7bb feat: add ProbedAt to network topology (#2413)
  • 37d43ad feat: add ip to uk_scheduler index and uk_seed_peer index in manager (#2426)
  • 499ad60 feat: add storage to trainer (#2431)
  • 6c3116c feat: change DefaultProbeInterval to 20 minute (#2440)
  • c233730 feat: change Dequeue to private func (#2420)
  • 812f37a feat: enable configuration of some directory modes for dfdaemon (#2340)
  • 2b5b96b feat: implement Enqueue and AverageRTT in probes.go (#2393)
  • 47e5427 feat: implement SyncProbes api in scheduler grpc service (#2449)
  • 32eac81 feat: optimize announcer in scheduler and client (#2445)
  • f53d966 feat: optimize dfpath format (#2460)
  • af95813 feat: optimize network topology comment (#2415)
  • 739fee6 feat: remove dirty file
  • 40fb466 feat: remove redis Pipelined in network topology (#2416)
  • b24f1df feat: remove useless fields in network topology (#2439)
  • 579798f feat: specify the version of golangci-lint as v1.52.2 (#2421)
  • f673583 feat: support breakpoint resume for running tasks (#2457)
  • ec85ae5 feat: support to collect and snapshot in network topology (#2429)
  • 284d437 fix: Interval in SyncProbesResponse (#2466)
  • 64062c1 fix: announcer in scheduler (#2451)
  • 3b2a926 fix: delete host in network topology (#2417)
  • 0bbd437 fix: e2e test dfget recursive (#2458)
  • 73e11e0 refactor: network topology package (#2412)

v2.1.0-alpha.9

26 May 07:35
e33804e
Compare
Choose a tag to compare

Changelog