Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support to specify client kerberosAuthType as fromTicketCache #5961

Closed
wants to merge 4 commits into from

Conversation

turboFei
Copy link
Member

@turboFei turboFei commented Jan 10, 2024

🔍 Description

The use case:

    1. the user code running in flink cluster and wrapped by transparent ugi.doAs
    1. the current UserGroupInfomation is the proxy user not login/real user
    1. user specify url with kyuubiClientTicketCache but does not work
    1. the proxy UserGroupInfomation is used and then throw GSS issue.
Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207)
    ... 22 more

The root cause is that, for this case, the result of isHadoopUserGroupInformationDoAs is true.

So, isFromSubjectAuthMode is true.

In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check isHadoopUserGroupInformationDoAs.

After this pr, customer can specify kerberosAuthType=fromTicketCache to leverage ticket cache prefer than ugi.doAs.

Issue References 🔗

This pull request fixes #

Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Types of changes 🔖

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request ⚰️

Behavior With This Pull Request 🎉

Related Unit Tests


Checklist 📝

Be nice. Be informative.

@turboFei turboFei changed the title Warning if Kyuubi client ticket cache does not exists Warning if Kyuubi client ticket cache does not exist Jan 10, 2024
@turboFei turboFei changed the title Warning if Kyuubi client ticket cache does not exist Support to specify client kerberosAuthType as fromTicketCache Jan 10, 2024
@turboFei turboFei requested a review from pan3793 January 10, 2024 20:46
@turboFei turboFei self-assigned this Jan 10, 2024
@turboFei turboFei added this to the v1.8.1 milestone Jan 10, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 12, 2024

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (4855ffe) 61.27% compared to head (c01a99e) 61.17%.
Report is 3 commits behind head on master.

❗ Current head c01a99e differs from pull request most recent head 6e06500. Consider uploading reports for the commit 6e06500 to get more accurate results

Files Patch % Lines
.../org/apache/kyuubi/jdbc/hive/KyuubiConnection.java 27.27% 13 Missing and 3 partials ⚠️
.../kyuubi/jdbc/hive/auth/KerberosAuthentication.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #5961      +/-   ##
============================================
- Coverage     61.27%   61.17%   -0.11%     
  Complexity       23       23              
============================================
  Files           622      622              
  Lines         36882    36896      +14     
  Branches       5014     5014              
============================================
- Hits          22599    22570      -29     
- Misses        11854    11885      +31     
- Partials       2429     2441      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pan3793 pan3793 closed this in 54086b0 Jan 15, 2024
pan3793 pushed a commit that referenced this pull request Jan 15, 2024
…etCache

# 🔍 Description

The use case:
- 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs`
- 2. the current UserGroupInfomation is the proxy user not login/real user
- 3. user specify url with kyuubiClientTicketCache but does not work
- 4. the proxy UserGroupInfomation is used and then throw GSS issue.
```
Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207)
    ... 22 more
```

The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true.

So, `isFromSubjectAuthMode` is true.

In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`.

After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`.

## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #5961 from turboFei/warn_ticket_cache.

Closes #5961

6e06500 [Fei Wang] exception
c01a99e [Fei Wang] refine
098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi
042fa22 [Fei Wang] warn

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 54086b0)
Signed-off-by: Cheng Pan <[email protected]>
pan3793 pushed a commit that referenced this pull request Jan 16, 2024
# 🔍 Description
Followup #5961

```
scala> Files.exists(Paths.get(null))
<console>:14: error: ambiguous reference to overloaded definition,
both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path
and  method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path
match argument types (Null) and expected result type java.nio.file.Path
       Files.exists(Paths.get(null))
                          ^

scala> Files.exists(Paths.get(""))
res0: Boolean = true

scala>

```
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes :bookmark:

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request :coffin:

#### Behavior With This Pull Request :tada:

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #5980 from turboFei/fix_npe.

Closes #5961

99d4a16 [Fei Wang] fix npe

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
pan3793 pushed a commit that referenced this pull request Jan 16, 2024
# 🔍 Description
Followup #5961

```
scala> Files.exists(Paths.get(null))
<console>:14: error: ambiguous reference to overloaded definition,
both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path
and  method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path
match argument types (Null) and expected result type java.nio.file.Path
       Files.exists(Paths.get(null))
                          ^

scala> Files.exists(Paths.get(""))
res0: Boolean = true

scala>

```
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes :bookmark:

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request :coffin:

#### Behavior With This Pull Request :tada:

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #5980 from turboFei/fix_npe.

Closes #5961

99d4a16 [Fei Wang] fix npe

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit cbe177b)
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Feb 5, 2024
…omTicketCache

# 🔍 Description

The use case:
- 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs`
- 2. the current UserGroupInfomation is the proxy user not login/real user
- 3. user specify url with kyuubiClientTicketCache but does not work
- 4. the proxy UserGroupInfomation is used and then throw GSS issue.
```
Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207)
    ... 22 more
```

The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true.

So, `isFromSubjectAuthMode` is true.

In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`.

After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`.

## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#5961 from turboFei/warn_ticket_cache.

Closes apache#5961

6e06500 [Fei Wang] exception
c01a99e [Fei Wang] refine
098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi
042fa22 [Fei Wang] warn

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Feb 5, 2024
… exists

# 🔍 Description
Followup apache#5961

```
scala> Files.exists(Paths.get(null))
<console>:14: error: ambiguous reference to overloaded definition,
both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path
and  method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path
match argument types (Null) and expected result type java.nio.file.Path
       Files.exists(Paths.get(null))
                          ^

scala> Files.exists(Paths.get(""))
res0: Boolean = true

scala>

```
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes :bookmark:

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request :coffin:

#### Behavior With This Pull Request :tada:

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#5980 from turboFei/fix_npe.

Closes apache#5961

99d4a16 [Fei Wang] fix npe

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Mar 21, 2024
…omTicketCache

# 🔍 Description

The use case:
- 1. the user code running in flink cluster and wrapped by transparent `ugi.doAs`
- 2. the current UserGroupInfomation is the proxy user not login/real user
- 3. user specify url with kyuubiClientTicketCache but does not work
- 4. the proxy UserGroupInfomation is used and then throw GSS issue.
```
Caused by: org.apache.kyuubi.shade.org.apache.thrift.transport.TTransportException: GSS initiate failed
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
    at org.apache.kyuubi.shade.org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.lambda$open$0(TSubjectTransport.java:47)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.kyuubi.jdbc.hive.auth.TSubjectTransport.open(TSubjectTransport.java:42)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.openTransport(KyuubiConnection.java:458)
    at org.apache.kyuubi.jdbc.hive.KyuubiConnection.<init>(KyuubiConnection.java:207)
    ... 22 more
```

The root cause is that, for this case, the result of `isHadoopUserGroupInformationDoAs` is true.

So, `isFromSubjectAuthMode` is true.

In this pr, I want to specify the kerberosAuthType to fromTicketCache, and do not check `isHadoopUserGroupInformationDoAs`.

After this pr, customer can specify `kerberosAuthType=fromTicketCache` to leverage ticket cache prefer than `ugi.doAs`.

## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️

#### Behavior With This Pull Request 🎉

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#5961 from turboFei/warn_ticket_cache.

Closes apache#5961

6e06500 [Fei Wang] exception
c01a99e [Fei Wang] refine
098a37b [Fei Wang] do not infer fromSubject if fromTicketCache && ugi
042fa22 [Fei Wang] warn

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Mar 21, 2024
… exists

# 🔍 Description
Followup apache#5961

```
scala> Files.exists(Paths.get(null))
<console>:14: error: ambiguous reference to overloaded definition,
both method get in class Paths of type (x$1: java.net.URI)java.nio.file.Path
and  method get in class Paths of type (x$1: String, x$2: String*)java.nio.file.Path
match argument types (Null) and expected result type java.nio.file.Path
       Files.exists(Paths.get(null))
                          ^

scala> Files.exists(Paths.get(""))
res0: Boolean = true

scala>

```
## Issue References 🔗

This pull request fixes #

## Describe Your Solution 🔧

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

## Types of changes :bookmark:

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request :coffin:

#### Behavior With This Pull Request :tada:

#### Related Unit Tests

---

# Checklist 📝

- [ ] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#5980 from turboFei/fix_npe.

Closes apache#5961

99d4a16 [Fei Wang] fix npe

Authored-by: Fei Wang <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants