tons of links

_includes/dynamic-index-async-req.mdx

@@ -1,3 +1,3 @@
-:::info Dynamic index requires `ASYNC_INDEXING`
-To use the `dynamic` vector index type, your Weaviate instance must have the `ASYNC_INDEXING` [environment variable](/developers/weaviate/config-refs/env-vars#general) enabled. This can be done by setting the `ASYNC_INDEXING` environment variable to `true`. For Weaviate Cloud users, this can be enabled through the Weaviate Cloud dashboard.
+:::info Dynamic indexes require asynchronous indexing
+The `dynamic` vector index type requires asynchronous indexing. To enable asynchronous indexing, set the `ASYNC_INDEXING` [environment variable](/developers/weaviate/config-refs/env-vars#general) to `true`. Weaviate Cloud users can enable asynchronous indexing in the Weaviate Cloud dashboard.
 :::

_includes/indexes/dynamic-intro.mdx

@@ -1 +1 @@
-[Dynamic index](/developers/weaviate/concepts/vector-index#dynamic-index) offer a flexible approach to indexing. A dynamic index begins as a flat index, and converts automatically to an HNSW index when a [collection](/developers/weaviate/concepts/data#collections) or [tenant](/developers/weaviate/concepts/data#multi-tenancy) reaches a threshold size.
+[Dynamic index](/developers/weaviate/concepts/indexing/vector-indexes/dynamic-indexes) offer a flexible approach to indexing. A dynamic index begins as a flat index, and converts automatically to an HNSW index when a [collection](/developers/weaviate/concepts/data#collections) or [tenant](/developers/weaviate/concepts/data#multi-tenancy) reaches a threshold size.

_includes/indexes/flat-intro.mdx

@@ -1 +1 @@
-[Flat indexes](/developers/weaviate/concepts/vector-index#flat-index) are memory-efficient. They are disk based indexes that perform brute-force vector searches. Flat indexes work best with small collections, less than 10,000 objects. Vector search times increase linearly with the number of objects.
+[Flat indexes](/developers/weaviate/concepts/indexing/vector-indexes/flat-indexes) are memory-efficient. They are disk based indexes that perform brute-force vector searches. Flat indexes work best with small collections, less than 10,000 objects. Vector search times increase linearly with the number of objects.

_includes/indexes/hnsw-intro.mdx

@@ -1 +1 @@
-[HNSW indexes](/developers/weaviate/concepts/vector-index#hierarchical-navigable-small-world-hnsw-index) are high-performance, in-memory indexes. HNSW indexes scale well, meaning that vector searches remain fast even for very large data sets.
+[HNSW indexes](/developers/weaviate/concepts/indexing/vector-indexes/hnsw-indexes) are high-performance, in-memory indexes. HNSW indexes scale well, meaning that vector searches remain fast even for very large data sets.

_includes/pq-compression/overview-text.mdx

@@ -1 +1,3 @@
-[Product quantization (PQ)](/developers/weaviate/concepts/vector-index#hnsw-with-product-quantizationpq) is a form of data compression for vectors. PQ reduces the memory footprint of a vector index, so enabling PQ for HNSW lets you work with larger datasets. For a discussion of how PQ saves memory, see [this page](/developers/weaviate/concepts/vector-quantization#product-quantization).
+Product quantization (PQ) is a form of data compression for vectors. PQ reduces the memory requirements for vector indexes so you can use larger datasets.
+
+For a discussion of how PQ saves memory, see [Product quantization](/developers/weaviate/concepts/vector-quantization#product-quantization).

blog/2022-07-04-weaviate-1-14-release/index.mdx

@@ -193,7 +193,7 @@ The REST API CRUD operations now require you to use both an **object ID** and th
 This ensures that the operations are performed on the correct objects.
 
 ### Background
-One of Weaviate's features is full CRUD support. CRUD operations enable the mutability of data objects and their vectors, which is a key difference between a vector database and an ANN library. In Weaviate, every data object has an ID (UUID). This ID is stored with the data object in a key-value store. IDs don't have to be globally unique, because in Weaviate [classes](/developers/weaviate/manage-data/collections) act as namespaces. While each class has a different [HNSW index](/developers/weaviate/concepts/vector-index#hnsw), including the store around it, which is isolated on disk.
+One of Weaviate's features is full CRUD support. CRUD operations enable the mutability of data objects and their vectors, which is a key difference between a vector database and an ANN library. In Weaviate, every data object has an ID (UUID). This ID is stored with the data object in a key-value store. IDs don't have to be globally unique, because in Weaviate [classes](/developers/weaviate/manage-data/collections) act as namespaces. While each class has a different [HNSW index](/developers/weaviate/concepts/indexing/hnsw-indexes), including the store around it, which is isolated on disk.
 
 There was however one point in the API where reusing IDs between classes was causing serious issues. Most noticeable this was for the [v1/objects/{id}](/developers/weaviate/api/rest#tag/objects) REST endpoints.
 If you wanted to retrieve, modify or delete a data object by its ID, you would just need to specify the ID, without specifying the class name. So if the same ID exists for objects in multiple classes (which is fine because of the namespaces per class), Weaviate would not know which object to address and would address all objects with that ID instead. I.e. if you tried to delete an object by ID, this would result in the deletion of all objects with that ID.

blog/2022-08-09-using-cross-encoders-as-reranker-in-multistage-vector-search/index.mdx

@@ -25,7 +25,7 @@ Vector databases like [Weaviate](/developers/weaviate/) use Bi-Encoder machine l
 
 The models that compute **dense vector embeddings for data** (which later can be used for **search queries**) are so-called *Bi-Encoder models*. Data vectors and query vectors can be compared by computing the similarity (for example cosine similarity) between vectors. All data items in the database can be represented by a vector embedding using a Bi-Encoder model.
 
-> Note. Vector databases **compute** dense vector embeddings **during import**. Calculating vectors up-front coupled with [ANN algorithms](/developers/weaviate/concepts/vector-index) (for faster retrieval) makes working with Bi-Encoder highly efficient.
+> Note. Vector databases **compute** dense vector embeddings **during import**. Calculating vectors up-front coupled with [ANN algorithms](/developers/weaviate/concepts/indexing/vector-indexes) (for faster retrieval) makes working with Bi-Encoder highly efficient.
 
 ![Vector Database](./img/vector-database.png)
 

blog/2022-09-13-why-is-vector-search-so-fast/index.mdx

@@ -98,7 +98,7 @@ Which algorithm works best depends on your project. Performance can be measured
 So, while ANN is not some magic method that will always find the true k nearest neighbors in a dataset, it can find a pretty good approximation of the true k neighbors. And it can do this in a fraction of the time!
 
 ## HNSW in Weaviate
-[Weaviate](/developers/weaviate/) is a great example of a vector database that uses ANN algorithms to offer ultra-fast queries. The first ANN algorithm introduced to Weaviate is a custom implementation of [Hierarchical Navigable Small World graphs (HNSW)](/developers/weaviate/concepts/vector-index#hnsw).
+[Weaviate](/developers/weaviate/) is a great example of a vector database that uses ANN algorithms to offer ultra-fast queries. The first ANN algorithm introduced to Weaviate is a custom implementation of [Hierarchical Navigable Small World graphs (HNSW)](/developers/weaviate/concepts/indexing/hnsw-indexes).
 
 <img
   src={require('./img/ann-hnsw.png').default}

blog/2022-10-11-ann-algorithms-vamana-vs-hnsw/index.mdx

@@ -40,7 +40,7 @@ Weaviate does not look for the exact closest vectors in the store. Instead, it l
 
 Weaviate incrementally builds up an index (graph representation of the vectors and their closest neighbors) with each incoming object. Then when a query arrives, Weaviate traverses the index to obtain a good approximated answer to the query in a fraction of the time that a brute-force approach would take.
 
-[HNSW](/developers/weaviate/concepts/vector-index#hnsw) is the first production-ready indexing algorithm we implemented in Weaviate. It is a robust and fast algorithm that builds a hierarchical representation of the index **in memory** that could be quickly traversed to find the k nearest neighbors of a query vector.
+[HNSW](/developers/weaviate/concepts/indexing/hnsw-indexes) is the first production-ready indexing algorithm we implemented in Weaviate. It is a robust and fast algorithm that builds a hierarchical representation of the index **in memory** that could be quickly traversed to find the k nearest neighbors of a query vector.
 
 ## Need for disk solutions
 There are other challenges to overcome. Databases have grown so fast that even the above-described algorithms will not be enough. We no longer talk about thousands of vectors but **hundred of millions** or **even billions**! Keeping all the vectors in memory and adding a graph representation of vector connections **requires a lot of RAM**. This sparked the emergence of a new set of algorithms that allow vectors to reside on disk instead of in memory whilst retaining high performance.

blog/2022-10-18-how-to-build-an-image-search-application-with-weaviate/index.mdx

@@ -230,7 +230,7 @@ def weaviate_img_search(img_str):
    return weaviate_results["data"]["Get"]["Dog"]
 ```
 
-How cool is that? We can find visually similar dogs in just a simple query to Weaviate. We could even scale this to millions of images running in milliseconds with [Weaviate's ANN index](/developers/weaviate/concepts/vector-index).
+How cool is that? We can find visually similar dogs in just a simple query to Weaviate. We could even scale this to millions of images running in milliseconds with [Weaviate's ANN index](/developers/weaviate/concepts/indexing/hnsw-indexes).
 
 Once we've created the app, we need to define the pages that will be on the website. The homepage will have the ten images of the dogs in our dataset. If you add images, it will also populate on the homepage!
 

blog/2022-12-01-vector-library-vs-vector-database/index.mdx

@@ -56,7 +56,7 @@ Vector libraries are commonly used for applications that do not have changing da
 One of the core features that set vector databases apart from libraries is the ability to store and update your data. Vector databases have full **CRUD** (create, read, update, and delete) support that solves the limitations of a vector library. Additionally, databases are more focused on enterprise-level [production deployments](https://www.youtube.com/watch?v=gXPuhyM11_k).
 
 ### Store Vectors and Objects
-Databases can store both the data objects and vectors. Since both are stored, you can combine vector search with structured filters. Filters allow you to make sure the nearest neighbors match the filter from the metadata. [Here](https://towardsdatascience.com/effects-of-filtered-hnsw-searches-on-recall-and-latency-434becf8041c) is an article on the effects of filtered [Hierarchical Navigable Small World](/developers/weaviate/concepts/vector-index#hnsw) (HNSW) searches on recall and latency.
+Databases can store both the data objects and vectors. Since both are stored, you can combine vector search with structured filters. Filters allow you to make sure the nearest neighbors match the filter from the metadata. [Here](https://towardsdatascience.com/effects-of-filtered-hnsw-searches-on-recall-and-latency-434becf8041c) is an article on the effects of filtered [Hierarchical Navigable Small World (HNSW)](/developers/weaviate/concepts/indexing/hnsw-indexes) searches on recall and latency.
 
 ### CRUD Support
 Vector databases solve a few limitations that vector libraries have. One example is being able to add, remove, or update entries in your index after it has been created. This is especially useful when working with data that is continuously changing.

blog/2023-02-14-what-to-expect-from-weaviate-in-2023/index.mdx

@@ -44,7 +44,7 @@ Speaking of Cloud, arguably the easiest way to spin up a new use case with Weavi
 ### New Vector Indexes
 ![vector indexes](./img/vector-indexes.png)
 
-Last year we gave you a sneak peek into our [Vector Indexing Research](/blog/ann-algorithms-vamana-vs-hnsw), and this year you will be able to try out new vector indexes for yourself. Since the beginning, Weaviate has supported vector indexing with [HNSW](/developers/weaviate/concepts/vector-index), which leads to [best-in-class query times](/developers/weaviate/benchmarks/ann). But not every use case requires single-digit millisecond latencies. Instead, some prefer cost-effectiveness. Due to its relatively high memory footprint, HNSW is only cost-efficient in high-throughput scenarios. However, HNSW is inherently optimized for in-memory access. Simply storing the index or vectors on disk or memory-mapping the index kills performance.
+Last year we gave you a sneak peek into our [Vector Indexing Research](/blog/ann-algorithms-vamana-vs-hnsw), and this year you will be able to try out new vector indexes for yourself. Since the beginning, Weaviate has supported vector indexing with [HNSW](/developers/weaviate/concepts/indexing/vector-indexes), which leads to [best-in-class query times](/developers/weaviate/benchmarks/ann). But not every use case requires single-digit millisecond latencies. Instead, some prefer cost-effectiveness. Due to its relatively high memory footprint, HNSW is only cost-efficient in high-throughput scenarios. However, HNSW is inherently optimized for in-memory access. Simply storing the index or vectors on disk or memory-mapping the index kills performance.
 
 This is why we will offer you not just one but two memory-saving options to index your vectors without sacrificing latency and throughput. In early 2023, you will be able to use Product Quantization, a vector compression algorithm, in Weaviate for the first time. We are already working on a fully disk-based solution which we will release in late 2023.
 

blog/2023-03-07-weaviate-1-18-release/_core-1-18-include.mdx

@@ -59,7 +59,7 @@ More specifically, in `1.18` we implemented the solution proposed in [DiskANN](h
 
 To understand how PQ compression works, imagine every vector you want to store is a unique house address. A unique house address allows you to precisely locate where someone lives including country, state, neighborhood, street number, and even the house number details. The price you pay for this pin-point accuracy is that each address takes up more storage space. Now, imagine instead of storing a unique address for each house we store just the neighborhood name the house is located in. With this representation, you can no longer precisely differentiate between houses that are all in the same neighborhood but you know approximately where the houses are. The advantage however is that you require less memory to store the neighborhood data - this is why it’s a lossy algorithm, we lose information as a result of the compression. You trade accuracy for storage space. Want to save more space? Zoom out further and represent each house instead with the state it’s located in. Require more recall? Zoom in and represent the address on a more accurate scale.
 
-Similarly in PQ compression instead of storing the exact vectors, we replace them in memory with a code that represents the general region in which the vector can be found. Now imagine you conduct this compression for millions or even billions of vectors and the memory savings become quite significant. Using these PQ compressed vectors stored in RAM we now start our search from our root graph quickly conducting broad searches over PQ compressed vector representations to filter out the best nearest neighbor candidates in memory and then drill down by exploring these best candidates when needed. Read more about HNSW+PQ in the documentation [here](/developers/weaviate/concepts/vector-index#hnsw-with-product-quantizationpq). If you want to learn how to configure Weaviate to use PQ refer to the docs [here](/developers/weaviate/config-refs/schema/vector-index#how-to-configure-hnsw).
+Similarly in PQ compression instead of storing the exact vectors, we replace them in memory with a code that represents the general region in which the vector can be found. Now imagine you conduct this compression for millions or even billions of vectors and the memory savings become quite significant. Using these PQ compressed vectors stored in RAM we now start our search from our root graph quickly conducting broad searches over PQ compressed vector representations to filter out the best nearest neighbor candidates in memory and then drill down by exploring these best candidates when needed. Read more about HNSW+PQ in the documentation [here](/developers/weaviate/concepts/indexing/hnsw-indexes). If you want to learn how to configure Weaviate to use PQ refer to the docs [here](/developers/weaviate/config-refs/schema/vector-index#how-to-configure-hnsw).
 
 ## Improvements to BM25 and Hybrid Search
 

blog/2023-03-14-ann-algorithms-hnsw-pq/index.mdx

@@ -84,7 +84,7 @@ A straightforward encoding/compression function uses KMeans to generate the cent
 
 Putting all of this together, the final algorithm would work as follows: Given a set of N vectors, we segment each of them producing smaller dimensional vectors, then apply KMeans clustering per segment over the complete data and find 256 centroids that will be used as predefined centers. When compressing a vector we find the closest centroid per segment and return an array of bytes with the indexes of the centroids per segment. When decompressing a vector, we concatenate the centroids encoded by each byte on the array.
 
-The explanation above is a simplification of the complete algorithm, as we also need to be concerned about performance for which we need to address duplication of calculations, synchronization in multithreading and so on. However what we have covered above should be sufficient to understand what you can accomplish using the PQ feature released in v1.18. Let's see some of the results HNSW implemented with PQ in Weaviate can accomplish next! If you're interested in learning more about PQ you can refer to the documentation [here](/developers/weaviate/concepts/vector-index#hnsw-with-product-quantizationpq). If you want to learn how to configure Weaviate to use PQ refer to the docs [here](/developers/weaviate/config-refs/schema/vector-index#how-to-configure-hnsw).
+The explanation above is a simplification of the complete algorithm, as we also need to be concerned about performance for which we need to address duplication of calculations, synchronization in multithreading and so on. However what we have covered above should be sufficient to understand what you can accomplish using the PQ feature released in v1.18. Let's see some of the results HNSW implemented with PQ in Weaviate can accomplish next! If you're interested in learning more about PQ you can refer to the [documentation](/developers/weaviate/concepts/indexing/hnsw-indexes). If you want to learn how to configure Weaviate to use PQ refer to the docs [here](/developers/weaviate/config-refs/schema/vector-index#how-to-configure-hnsw).
 
 ## KMeans encoding results
 

blog/2023-07-11-weaviate-1-20-release/_core-1-20-include.mdx

@@ -112,7 +112,7 @@ As can be clearly seen above, previously there was a cost to be paid for compres
 </details>
 
 :::tip Read more:
-- [Concepts: Vector Indexing # Product Quantization](/developers/weaviate/concepts/vector-index#hnsw-with-product-quantizationpq)
+- [Concepts: Vector Indexing # Product Quantization](/developers/weaviate/concepts/indexing/hnsw-indexes)
 - [How-to configure: Indexes # Vector index (see `pq`)](/developers/weaviate/config-refs/schema/vector-index#weaviates-vector-index)
 :::