Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate and potentially add support for spark connect #284

Open
razvan opened this issue Sep 18, 2023 · 1 comment
Open

Investigate and potentially add support for spark connect #284

razvan opened this issue Sep 18, 2023 · 1 comment
Assignees
@razvan
Labels
customer-request type/feature-new

Comments

@razvan
Copy link
Member

razvan commented Sep 18, 2023

Spark Connect

Spark 3.5 introduces a new client called Spark Connect.

The use case seems to be thin clients that connect to a running spark driver.

This probably means that the operator needs to be able to start spark connect servers without spark applications and publish a service for "connect" clients.
@razvan razvan changed the title Investigate and potentially ad support for spark connect Investigate and potentially add support for spark connect Sep 18, 2023
@adwk67 adwk67 self-assigned this Dec 20, 2023
@adwk67 adwk67 removed their assignment Aug 30, 2024
@sbernauer sbernauer mentioned this issue Feb 17, 2025
Closed
@timrobertson100
Copy link

We have started exploring Spark Connect at GBIF.org. Our primary use case is to explore having a long running spark cluster hold an in memory cached table, for apps to do filtered data egress with minimal startup cost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@razvan razvan

Labels
customer-request type/feature-new
Projects
Stackable End-to-End Coordination
Status: Proposed
Milestone
No milestone
Development

No branches or pull requests
4 participants
@razvan @timrobertson100 @adwk67 @sbernauer