Skip to content

Commit

Permalink
More verbose source installation instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
edenhill committed Apr 12, 2022
1 parent 0b2e874 commit e7e0661
Show file tree
Hide file tree
Showing 2 changed files with 126 additions and 26 deletions.
121 changes: 121 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# confluent-kafka-python installation instructions

## Install pre-built wheels (recommended)

Confluent provides pre-built Python wheels of confluent-kafka-python with
all dependencies included.

To install, simply do:

```bash
python3 -m pip install confluent-kafka
```

If you get a build error or require Kerberos/GSSAPI support please read the next section: *Install from source*


## Install from source

It is sometimes necessary to install confluent-kafka from source, rather
than from prebuilt binary wheels, such as when:
- You need GSSAPI/Kerberos authentication.
- You're on a Python version we do not provide prebuilt wheels for.
- You're on an architecture or platform we do not provide prebuilt wheels for.
- You want to build confluent-kafka-python from the master branch.


### Install from source on RedHat, CentOS, Fedora, etc

```bash
#
# Perform these steps as the root user (e.g., in a 'sudo bash' shell)
#

# Install build tools and Kerberos support.

yum install -y python3 python3-pip python3-devel gcc make cyrus-sasl-gssapi krb5-workstation

# Install the latest version of librdkafka:

rpm --import https://packages.confluent.io/rpm/7.0/archive.key

echo '
[Confluent-Clients]
name=Confluent Clients repository
baseurl=https://packages.confluent.io/clients/rpm/centos/$releasever/$basearch
gpgcheck=1
gpgkey=https://packages.confluent.io/clients/rpm/archive.key
enabled=1' > /etc/yum.repos.d/confluent.repo

yum install -y librdkafka-devel


#
# Now build and install confluent-kafka-python as your standard user
# (e.g., exit the root shell first).
#

python3 -m pip install --no-binary confluent-kafka confluent-kafka


# Verify that confluent_kafka is installed:

python3 -c 'import confluent_kafka; print(confluent_kafka.version())'
```

### Install from source on Debian or Ubuntu

```bash
#
# Perform these steps as the root user (e.g., in a 'sudo bash' shell)
#

# Install build tools and Kerberos support.

apt install -y wget software-properties-common lsb-release gcc make python3 python3-pip python3-dev libsasl2-modules-gssapi-mit krb5-user


# Install the latest version of librdkafka:

wget -qO - https://packages.confluent.io/deb/7.0/archive.key | apt-key add -

add-apt-repository "deb https://packages.confluent.io/clients/deb $(lsb_release -cs) main"

apt update

apt install -y librdkafka-dev


#
# Now build and install confluent-kafka-python as your standard user
# (e.g., exit the root shell first).
#

python3 -m pip install --no-binary confluent-kafka confluent-kafka


# Verify that confluent_kafka is installed:

python3 -c 'import confluent_kafka; print(confluent_kafka.version())'
```


### Install from source on Mac OS X

```bash

# Install librdkafka from homebrew

brew install librdkafka


# Build and install confluent-kafka-python

python3 -m pip install --no-binary confluent-kafka confluent-kafka


# Verify that confluent_kafka is installed:

python3 -c 'import confluent_kafka; print(confluent_kafka.version())'

```
31 changes: 5 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,20 +222,16 @@ Install
**NOTE:** The pre-built Linux wheels do NOT contain SASL Kerberos/GSSAPI support.
If you need SASL Kerberos/GSSAPI support you must install librdkafka and
its dependencies using the repositories below and then build
confluent-kafka using the command in the "Install from
source from PyPi" section below.
confluent-kafka using the instructions in the
"Install from source" section below.

**Install AvroProducer and AvroConsumer**

$ pip install "confluent-kafka[avro]"

**Install from source from PyPi**
*(requires librdkafka + dependencies to be installed separately)*:
**Install from source**

$ pip install --no-binary :all: confluent-kafka


For source install, see *Prerequisites* below.
For source install, see the *Install from source* section in [INSTALL.md](INSTALL.md).


Broker Compatibility
Expand Down Expand Up @@ -274,30 +270,13 @@ Linux distribution's `ca-certificates` package which needs to be installed
through `apt`, `yum`, et.al.

If your system stores CA certificates in another location you will need to
configure the client with `'ssl.ca.location': '/path/to/cacert.pem'`.
configure the client with `'ssl.ca.location': '/path/to/cacert.pem'`.

Alternatively, the CA certificates can be provided by the [certifi](https://pypi.org/project/certifi/)
Python package. To use certifi, add an `import certifi` line and configure the
client's CA location with `'ssl.ca.location': certifi.where()`.


Prerequisites
=============

* Python >= 2.7 or Python 3.x
* [librdkafka](https://github.com/edenhill/librdkafka) >= 1.6.0 (latest release is embedded in wheels)

librdkafka is embedded in the macosx manylinux wheels, for other platforms, SASL Kerberos/GSSAPI support or
when a specific version of librdkafka is desired, following these guidelines:

* For **Debian/Ubuntu** based systems, add this APT repo and then do `sudo apt-get install librdkafka-dev python-dev`:
http://docs.confluent.io/current/installation.html#installation-apt

* For **RedHat** and **RPM**-based distros, add this YUM repo and then do `sudo yum install librdkafka-devel python-devel`:
http://docs.confluent.io/current/installation.html#rpm-packages-via-yum

* On **OSX**, use **homebrew** and do `brew install librdkafka`


License
=======
Expand Down

0 comments on commit e7e0661

Please sign in to comment.