Skip to content

Commit

Permalink
remove tikv-importer
Browse files Browse the repository at this point in the history
  • Loading branch information
csuzhangxc committed Dec 31, 2024
1 parent c90644a commit b3fd589
Show file tree
Hide file tree
Showing 6 changed files with 6 additions and 209 deletions.
104 changes: 1 addition & 103 deletions en/enable-tls-between-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ To enable TLS between TiDB components, perform the following steps:

1. Generate certificates for each component of the TiDB cluster to be created:

- A set of server-side certificates for the PD/TiKV/TiDB/TiFlash/TiProxy/TiKV Importer/TiDB Lightning component, saved as the Kubernetes Secret objects: `${cluster_name}-${component_name}-cluster-secret`.
- A set of server-side certificates for the PD/TiKV/TiDB/TiFlash/TiProxy/TiDB Lightning component, saved as the Kubernetes Secret objects: `${cluster_name}-${component_name}-cluster-secret`.
- A set of shared client-side certificates for the various clients of each component, saved as the Kubernetes Secret objects: `${cluster_name}-cluster-client-secret`.

> **Note:**
Expand Down Expand Up @@ -400,47 +400,6 @@ This section describes how to issue certificates using two methods: `cfssl` and
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=internal tiflash-server.json | cfssljson -bare tiflash-server
```
- TiKV Importer
If you need to [restore data using TiDB Lightning](restore-data-using-tidb-lightning.md), you need to generate a server-side certificate for the TiKV Importer component.
1. Generate the default `importer-server.json` file:
{{< copyable "shell-regular" >}}
```shell
cfssl print-defaults csr > importer-server.json
```
2. Edit this file to change the `CN` and `hosts` attributes:
```json
...
"CN": "TiDB",
"hosts": [
"127.0.0.1",
"::1",
"${cluster_name}-importer",
"${cluster_name}-importer.${namespace}",
"${cluster_name}-importer.${namespace}.svc"
"${cluster_name}-importer.${namespace}.svc",
"*.${cluster_name}-importer",
"*.${cluster_name}-importer.${namespace}",
"*.${cluster_name}-importer.${namespace}.svc"
],
...
```
`${cluster_name}` is the name of the cluster. `${namespace}` is the namespace in which the TiDB cluster is deployed. You can also add your customized `hosts`.
3. Generate the TiKV Importer server-side certificate:
{{< copyable "shell-regular" >}}
``` shell
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=internal importer-server.json | cfssljson -bare importer-server
```
- TiDB Lightning
If you need to [restore data using TiDB Lightning](restore-data-using-tidb-lightning.md), you need to generate a server-side certificate for the TiDB Lightning component.
Expand Down Expand Up @@ -549,12 +508,6 @@ This section describes how to issue certificates using two methods: `cfssl` and
kubectl create secret generic ${cluster_name}-tiflash-cluster-secret --namespace=${namespace} --from-file=tls.crt=tiflash-server.pem --from-file=tls.key=tiflash-server-key.pem --from-file=ca.crt=ca.pem
```
- The TiKV Importer cluster certificate Secret:
``` shell
kubectl create secret generic ${cluster_name}-importer-cluster-secret --namespace=${namespace} --from-file=tls.crt=importer-server.pem --from-file=tls.key=importer-server-key.pem --from-file=ca.crt=ca.pem
```
- The TiDB Lightning cluster certificate Secret:
{{< copyable "shell-regular" >}}
Expand Down Expand Up @@ -963,61 +916,6 @@ This section describes how to issue certificates using two methods: `cfssl` and
After the object is created, `cert-manager` generates a `${cluster_name}-tiflash-cluster-secret` Secret object to be used by the TiFlash component of the TiDB server.
- TiKV Importer
If you need to [restore data using TiDB Lightning](restore-data-using-tidb-lightning.md), you need to generate a server-side certificate for the TiKV Importer component.
```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ${cluster_name}-importer-cluster-secret
namespace: ${namespace}
spec:
secretName: ${cluster_name}-importer-cluster-secret
duration: 8760h # 365d
renewBefore: 360h # 15d
subject:
organizations:
- PingCAP
commonName: "TiDB"
usages:
- server auth
- client auth
dnsNames:
- "${cluster_name}-importer"
- "${cluster_name}-importer.${namespace}"
- "${cluster_name}-importer.${namespace}.svc"
- "*.${cluster_name}-importer"
- "*.${cluster_name}-importer.${namespace}"
- "*.${cluster_name}-importer.${namespace}.svc"
ipAddresses:
- 127.0.0.1
- ::1
issuerRef:
name: ${cluster_name}-tidb-issuer
kind: Issuer
group: cert-manager.io
```
In the file, `${cluster_name}` is the name of the cluster:
- Set `spec.secretName` to `${cluster_name}-importer-cluster-secret`.
- Add `server auth` and `client auth` in `usages`.
- Add the following DNSs in `dnsNames`. You can also add other DNSs according to your needs:
- `${cluster_name}-importer`
- `${cluster_name}-importer.${namespace}`
- `${cluster_name}-importer.${namespace}.svc`
- Add the following 2 IP addresses in `ipAddresses`. You can also add other IP addresses according to your needs:
- `127.0.0.1`
- `::1`
- Add the Issuer created above in `issuerRef`.
- For other attributes, refer to [cert-manager API](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.CertificateSpec).
After the object is created, `cert-manager` generates a `${cluster_name}-importer-cluster-secret` Secret object to be used by the TiKV Importer component of the TiDB server.
- TiDB Lightning
If you need to [restore data using TiDB Lightning](restore-data-using-tidb-lightning.md), you need to generate a server-side certificate for the TiDB Lightning component.
Expand Down
2 changes: 1 addition & 1 deletion en/restore-from-gcs.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This document describes how to restore the TiDB cluster data backed up using TiD

The restore method described in this document is implemented based on CustomResourceDefinition (CRD) in TiDB Operator v1.1 or later versions. For the underlying implementation, [TiDB Lightning TiDB-backend](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends#tidb-lightning-tidb-backend) is used to perform the restore.

TiDB Lightning is a tool used for fast full import of large amounts of data into a TiDB cluster. It reads data from local disks, Google Cloud Storage (GCS) or Amazon S3. TiDB Lightning supports three backends: `Importer-backend`, `Local-backend`, and `TiDB-backend`. In this document, `TiDB-backend` is used. For the differences of these backends and how to choose backends, see [TiDB Lightning Backends](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends). To import data using `Importer-backend` or `Local-backend`, see [Import Data](restore-data-using-tidb-lightning.md).
TiDB Lightning is a tool used for fast full import of large amounts of data into a TiDB cluster. It reads data from local disks, Google Cloud Storage (GCS) or Amazon S3. TiDB Lightning supports two backends: `Local-backend` and `TiDB-backend`. In this document, `TiDB-backend` is used. For the differences of these backends and how to choose backends, see [TiDB Lightning Backends](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends). To import data using `Local-backend`, see [Import Data](restore-data-using-tidb-lightning.md).

This document shows an example in which the backup data stored in the specified path on [GCS](https://cloud.google.com/storage/docs/) is restored to the TiDB cluster.

Expand Down
2 changes: 1 addition & 1 deletion en/restore-from-s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This document describes how to restore the TiDB cluster data backed up using TiD

The restore method described in this document is implemented based on CustomResourceDefinition (CRD) in TiDB Operator v1.1 or later versions. For the underlying implementation, [TiDB Lightning TiDB-backend](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends#tidb-lightning-tidb-backend) is used to perform the restore.

TiDB Lightning is a tool used for fast full import of large amounts of data into a TiDB cluster. It reads data from local disks, Google Cloud Storage (GCS) or Amazon S3. TiDB Lightning supports three backends: `Importer-backend`, `Local-backend`, and `TiDB-backend`. In this document, `TiDB-backend` is used. For the differences of these backends and how to choose backends, see [TiDB Lightning Backends](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends). To import data using `Importer-backend` or `Local-backend`, see [Import Data](restore-data-using-tidb-lightning.md).
TiDB Lightning is a tool used for fast full import of large amounts of data into a TiDB cluster. It reads data from local disks, Google Cloud Storage (GCS) or Amazon S3. TiDB Lightning supports two backends: `Local-backend` and `TiDB-backend`. In this document, `TiDB-backend` is used. For the differences of these backends and how to choose backends, see [TiDB Lightning Backends](https://docs.pingcap.com/tidb/stable/tidb-lightning-backends). To import data using `Local-backend`, see [Import Data](restore-data-using-tidb-lightning.md).

This document shows an example in which the backup data stored in the specified path on the S3-compatible storage is restored to the TiDB cluster.

Expand Down
103 changes: 1 addition & 102 deletions zh/enable-tls-between-components.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/enable-tls-between-components/']
本文主要描述了在 Kubernetes 上如何为 TiDB 集群组件间开启 TLS。TiDB Operator 从 v1.1 开始已经支持为 Kubernetes 上 TiDB 集群组件间开启 TLS。开启步骤为:

1. 为即将被创建的 TiDB 集群的每个组件生成证书:
- 为 PD/TiKV/TiDB/TiFlash/TiProxy/TiKV Importer/TiDB Lightning 组件分别创建一套 Server 端证书,保存为 Kubernetes Secret 对象:`${cluster_name}-${component_name}-cluster-secret`
- 为 PD/TiKV/TiDB/TiFlash/TiProxy/TiDB Lightning 组件分别创建一套 Server 端证书,保存为 Kubernetes Secret 对象:`${cluster_name}-${component_name}-cluster-secret`
- 为它们的各种客户端创建一套共用的 Client 端证书,保存为 Kubernetes Secret 对象:`${cluster_name}-cluster-client-secret`

> **注意:**
Expand Down Expand Up @@ -396,46 +396,6 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/enable-tls-between-components/']
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=internal tiflash-server.json | cfssljson -bare tiflash-server
```
- TiKV Importer Server 端证书
如需要[使用 TiDB Lightning 恢复 Kubernetes 上的集群数据](restore-data-using-tidb-lightning.md),则需要为其中的 TiKV Importer 组件生成如下的 Server 端证书。
首先生成默认的 `importer-server.json` 文件:
{{< copyable "shell-regular" >}}
```shell
cfssl print-defaults csr > importer-server.json
```
然后编辑这个文件,修改 `CN``hosts` 属性:
```json
...
"CN": "TiDB",
"hosts": [
"127.0.0.1",
"::1",
"${cluster_name}-importer",
"${cluster_name}-importer.${namespace}",
"${cluster_name}-importer.${namespace}.svc",
"*.${cluster_name}-importer",
"*.${cluster_name}-importer.${namespace}",
"*.${cluster_name}-importer.${namespace}.svc"
],
...
```
其中 `${cluster_name}` 为集群的名字,`${namespace}` 为 TiDB 集群部署的命名空间,用户也可以添加自定义 `hosts`
最后生成 TiKV Importer Server 端证书:
{{< copyable "shell-regular" >}}
``` shell
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=internal importer-server.json | cfssljson -bare importer-server
```
- TiDB Lightning Server 端证书
如需要[使用 TiDB Lightning 恢复 Kubernetes 上的集群数据](restore-data-using-tidb-lightning.md),则需要为其中的 TiDB Lightning 组件生成如下的 Server 端证书。
Expand Down Expand Up @@ -548,14 +508,6 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/enable-tls-between-components/']
kubectl create secret generic ${cluster_name}-tiflash-cluster-secret --namespace=${namespace} --from-file=tls.crt=tiflash-server.pem --from-file=tls.key=tiflash-server-key.pem --from-file=ca.crt=ca.pem
```
TiKV Importer 集群证书 Secret:
{{< copyable "shell-regular" >}}
``` shell
kubectl create secret generic ${cluster_name}-importer-cluster-secret --namespace=${namespace} --from-file=tls.crt=importer-server.pem --from-file=tls.key=importer-server-key.pem --from-file=ca.crt=ca.pem
```
TiDB Lightning 集群证书 Secret:
{{< copyable "shell-regular" >}}
Expand Down Expand Up @@ -951,59 +903,6 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/enable-tls-between-components/']
创建这个对象以后,`cert-manager` 会生成一个名字为 `${cluster_name}-tiflash-cluster-secret` 的 Secret 对象供 TiDB 集群的 TiFlash 组件使用。
- TiKV Importer 组件的 Server 端证书。
如需要[使用 TiDB Lightning 恢复 Kubernetes 上的集群数据](restore-data-using-tidb-lightning.md),则需要为其中的 TiKV Importer 组件生成如下的 Server 端证书。
```yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: ${cluster_name}-importer-cluster-secret
namespace: ${namespace}
spec:
secretName: ${cluster_name}-importer-cluster-secret
duration: 8760h # 365d
renewBefore: 360h # 15d
subject:
organizations:
- PingCAP
commonName: "TiDB"
usages:
- server auth
- client auth
dnsNames:
- "${cluster_name}-importer"
- "${cluster_name}-importer.${namespace}"
- "${cluster_name}-importer.${namespace}.svc"
- "*.${cluster_name}-importer"
- "*.${cluster_name}-importer.${namespace}"
- "*.${cluster_name}-importer.${namespace}.svc"
ipAddresses:
- 127.0.0.1
- ::1
issuerRef:
name: ${cluster_name}-tidb-issuer
kind: Issuer
group: cert-manager.io
```
其中 `${cluster_name}` 为集群的名字:
- `spec.secretName` 请设置为 `${cluster_name}-importer-cluster-secret`
- `usages` 请添加上 `server auth``client auth`
- `dnsNames` 需要填写这些 DNS,根据需要可以填写其他 DNS:
- `${cluster_name}-importer`
- `${cluster_name}-importer.${namespace}`
- `${cluster_name}-importer.${namespace}.svc`
- `ipAddresses` 需要填写这两个 IP ,根据需要可以填写其他 IP:
- `127.0.0.1`
- `::1`
- `issuerRef` 请填写上面创建的 Issuer;
- 其他属性请参考 [cert-manager API](https://cert-manager.io/docs/reference/api-docs/#cert-manager.io/v1.CertificateSpec)。
创建这个对象以后,`cert-manager` 会生成一个名字为 `${cluster_name}-importer-cluster-secret` 的 Secret 对象供 TiDB 集群的 TiKV Importer 组件使用。
- TiDB Lightning 组件的 Server 端证书。
如需要[使用 TiDB Lightning 恢复 Kubernetes 上的集群数据](restore-data-using-tidb-lightning.md),则需要为其中的 TiDB Lightning 组件生成如下的 Server 端证书。
Expand Down
2 changes: 1 addition & 1 deletion zh/restore-from-gcs.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/restore-from-gcs/']

本文使用的恢复方式基于 TiDB Operator v1.1 及以上的 CustomResourceDefinition (CRD) 实现,底层通过使用 [TiDB Lightning TiDB-backend](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends#tidb-lightning-tidb-backend) 来恢复数据。

TiDB Lightning 是一款将全量数据高速导入到 TiDB 集群的工具,可用于从本地盘、Google Cloud Storage (GCS) 或 Amazon S3 云盘读取数据。目前,TiDB Lightning 支持三种后端:`Importer-backend``Local-backend``TiDB-backend`。本文介绍的方法使用 `TiDB-backend`。关于这三种后端的区别和选择,请参阅 [TiDB Lightning 文档](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends)。如果要使用 `Importer-backend` 或者 `Local-backend` 导入数据,请参阅[使用 TiDB Lightning 导入集群数据](restore-data-using-tidb-lightning.md)
TiDB Lightning 是一款将全量数据高速导入到 TiDB 集群的工具,可用于从本地盘、Google Cloud Storage (GCS) 或 Amazon S3 云盘读取数据。目前,TiDB Lightning 支持两种后端:`Local-backend``TiDB-backend`。本文介绍的方法使用 `TiDB-backend`。关于这三种后端的区别和选择,请参阅 [TiDB Lightning 文档](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends)。如果要使用 `Local-backend` 导入数据,请参阅[使用 TiDB Lightning 导入集群数据](restore-data-using-tidb-lightning.md)

以下示例将存储在 [GCS](https://cloud.google.com/storage/docs/) 上指定路径上的集群备份数据恢复到 TiDB 集群。

Expand Down
2 changes: 1 addition & 1 deletion zh/restore-from-s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ aliases: ['/docs-cn/tidb-in-kubernetes/dev/restore-from-s3/']

本文使用的恢复方式基于 TiDB Operator v1.1 及以上的 CustomResourceDefinition (CRD) 实现,底层通过使用 [TiDB Lightning TiDB-backend](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends#tidb-lightning-tidb-backend) 来恢复数据。

TiDB Lightning 是一款将全量数据高速导入到 TiDB 集群的工具,可用于从本地盘、Google Cloud Storage (GCS) 或 Amazon S3 云盘读取数据。目前,TiDB Lightning 支持三种后端:`Importer-backend``Local-backend``TiDB-backend`。本文介绍的方法使用 `TiDB-backend`。关于这三种后端的区别和选择,请参阅 [TiDB Lightning 文档](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends)。如果要使用 `Importer-backend` 或者 `Local-backend` 导入数据,请参阅[使用 TiDB Lightning 导入集群数据](restore-data-using-tidb-lightning.md)
TiDB Lightning 是一款将全量数据高速导入到 TiDB 集群的工具,可用于从本地盘、Google Cloud Storage (GCS) 或 Amazon S3 云盘读取数据。目前,TiDB Lightning 支持两种后端:`Local-backend``TiDB-backend`。本文介绍的方法使用 `TiDB-backend`。关于这三种后端的区别和选择,请参阅 [TiDB Lightning 文档](https://docs.pingcap.com/zh/tidb/stable/tidb-lightning-backends)。如果要使用 `Local-backend` 导入数据,请参阅[使用 TiDB Lightning 导入集群数据](restore-data-using-tidb-lightning.md)

以下示例将兼容 S3 的存储(指定路径)上的备份数据恢复到 TiDB 集群。

Expand Down

0 comments on commit b3fd589

Please sign in to comment.