Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vec-246: supporting asdb 6.4 for prism image search application. #71

Merged
merged 5 commits into from
Nov 13, 2024

Conversation

rahul-aerospike
Copy link
Contributor

  • support asdb 6.4 for prism image search application.

@rahul-aerospike rahul-aerospike requested review from hev, arrowplum and dwelch-spike and removed request for hev November 5, 2024 23:09
@hev
Copy link
Contributor

hev commented Nov 6, 2024

@rahul-aerospike I am not sure why, but I seem to be having problems with this at the application layer. I get the following error:

2024-11-06 09:38:17 /usr/local/lib/python3.10/dist-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
2024-11-06 09:38:17   return self.fget.__get__(instance, owner)()
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at types.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at vector-db.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at index.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at auth.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at transact.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at user-admin.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 Traceback (most recent call last):
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 167, in _check_server_version
2024-11-06 09:38:21     response = stub.Get(about_request, credentials=self._token)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/grpc/_channel.py", line 1181, in __call__
2024-11-06 09:38:21     return _end_unary_response_blocking(state, call, False, None)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
2024-11-06 09:38:21     raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
2024-11-06 09:38:21 grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
2024-11-06 09:38:21     status = StatusCode.UNAVAILABLE
2024-11-06 09:38:21     details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"
2024-11-06 09:38:21     debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-11-06T16:38:21.422969173+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"}"
2024-11-06 09:38:21 >
2024-11-06 09:38:21 
2024-11-06 09:38:21 During handling of the above exception, another exception occurred:
2024-11-06 09:38:21 
2024-11-06 09:38:21 Traceback (most recent call last):
2024-11-06 09:38:21   File "/usr/local/bin/waitress-serve", line 8, in <module>
2024-11-06 09:38:21     sys.exit(run())
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/waitress/runner.py", line 283, in run
2024-11-06 09:38:21     app = resolve(module, obj_name)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/waitress/runner.py", line 218, in resolve
2024-11-06 09:38:21     obj = __import__(module_name, fromlist=segments[:1])
2024-11-06 09:38:21   File "/prism/prism.py", line 19, in <module>
2024-11-06 09:38:21     import indexer
2024-11-06 09:38:21   File "/prism/indexer.py", line 14, in <module>
2024-11-06 09:38:21     from avs_client import avs_admin_client, avs_client
2024-11-06 09:38:21   File "/prism/avs_client.py", line 10, in <module>
2024-11-06 09:38:21     avs_client = Client(
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/client.py", line 71, in __init__
2024-11-06 09:38:21     self._channel_provider = channel_provider.ChannelProvider(
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 62, in __init__
2024-11-06 09:38:21     self._check_server_version()
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 171, in _check_server_version
2024-11-06 09:38:21     raise types.AVSServerError(rpc_error=e)
2024-11-06 09:38:21 aerospike_vector_search.types.AVSServerError: AVSServerError(rpc_error=<_InactiveRpcError of RPC that terminated with:
2024-11-06 09:38:21     status = StatusCode.UNAVAILABLE
2024-11-06 09:38:21     details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"
2024-11-06 09:38:21     debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-11-06T16:38:21.422969173+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"}"
2024-11-06 09:38:21 >)

@rahul-aerospike
Copy link
Contributor Author

@rahul-aerospike I am not sure why, but I seem to be having problems with this at the application layer. I get the following error:

2024-11-06 09:38:17 /usr/local/lib/python3.10/dist-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
2024-11-06 09:38:17   return self.fget.__get__(instance, owner)()
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at types.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at vector-db.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at index.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at auth.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at transact.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 /usr/local/lib/python3.10/dist-packages/google/protobuf/runtime_version.py:112: UserWarning: Protobuf gencode version 5.27.2 is older than the runtime version 5.28.2 at user-admin.proto. Please avoid checked-in Protobuf gencode that can be obsolete.
2024-11-06 09:38:21   warnings.warn(
2024-11-06 09:38:21 Traceback (most recent call last):
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 167, in _check_server_version
2024-11-06 09:38:21     response = stub.Get(about_request, credentials=self._token)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/grpc/_channel.py", line 1181, in __call__
2024-11-06 09:38:21     return _end_unary_response_blocking(state, call, False, None)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
2024-11-06 09:38:21     raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
2024-11-06 09:38:21 grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
2024-11-06 09:38:21     status = StatusCode.UNAVAILABLE
2024-11-06 09:38:21     details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"
2024-11-06 09:38:21     debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-11-06T16:38:21.422969173+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"}"
2024-11-06 09:38:21 >
2024-11-06 09:38:21 
2024-11-06 09:38:21 During handling of the above exception, another exception occurred:
2024-11-06 09:38:21 
2024-11-06 09:38:21 Traceback (most recent call last):
2024-11-06 09:38:21   File "/usr/local/bin/waitress-serve", line 8, in <module>
2024-11-06 09:38:21     sys.exit(run())
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/waitress/runner.py", line 283, in run
2024-11-06 09:38:21     app = resolve(module, obj_name)
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/waitress/runner.py", line 218, in resolve
2024-11-06 09:38:21     obj = __import__(module_name, fromlist=segments[:1])
2024-11-06 09:38:21   File "/prism/prism.py", line 19, in <module>
2024-11-06 09:38:21     import indexer
2024-11-06 09:38:21   File "/prism/indexer.py", line 14, in <module>
2024-11-06 09:38:21     from avs_client import avs_admin_client, avs_client
2024-11-06 09:38:21   File "/prism/avs_client.py", line 10, in <module>
2024-11-06 09:38:21     avs_client = Client(
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/client.py", line 71, in __init__
2024-11-06 09:38:21     self._channel_provider = channel_provider.ChannelProvider(
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 62, in __init__
2024-11-06 09:38:21     self._check_server_version()
2024-11-06 09:38:21   File "/usr/local/lib/python3.10/dist-packages/aerospike_vector_search/internal/channel_provider.py", line 171, in _check_server_version
2024-11-06 09:38:21     raise types.AVSServerError(rpc_error=e)
2024-11-06 09:38:21 aerospike_vector_search.types.AVSServerError: AVSServerError(rpc_error=<_InactiveRpcError of RPC that terminated with:
2024-11-06 09:38:21     status = StatusCode.UNAVAILABLE
2024-11-06 09:38:21     details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"
2024-11-06 09:38:21     debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2024-11-06T16:38:21.422969173+00:00", grpc_status:14, grpc_message:"failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.128.3:10000: Failed to connect to remote host: connect: Connection refused (111)"}"
2024-11-06 09:38:21 >)

@hev Can you check if your avs is also running on 10000 port. If not, modify the avs port in aerospike-vector-search.yml

Default port should be 5000
Copy link
Contributor

@hev hev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This worked for me, but let's do it across these also

  • /docker
  • /quote-search

Copy link
Contributor

@arrowplum arrowplum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved but see suggestion

@rahul-aerospike
Copy link
Contributor Author

This worked for me, but let's do it across these also

  • /docker
  • /quote-search

@hev Added support in docker and quote-search as well.

Copy link
Contributor

@hev hev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dwelch-spike dwelch-spike merged commit ff68c61 into main Nov 13, 2024
1 check passed
@dwelch-spike dwelch-spike deleted the asdb-6.4-support-vec-426 branch November 13, 2024 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants