[Docs] Check for broken links (nuclio#3316)

.github/styles/Nuclio/ignore.txt

@@ -23,7 +23,6 @@ HTTPie
 http
 instantiatable
 jessie
-Katacoda
 Kerberos
 Kinesis
 kube

.github/workflows/docs-language-linter.yml

@@ -18,6 +18,9 @@ on:
     paths:
       - "docs/**"
       - '**.md'
+      - ".github/workflows/docs-language-linter.yaml"
+      - ".vale.ini"
+      - ".github/styles/**"
 
 jobs:
   vale:
@@ -29,3 +32,17 @@ jobs:
         with:
           reporter: github-pr-check
 
+  docs_linkcheck:
+    name: Check for broken links
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v4
+      - name: Install dependencies
+        run: |
+          pip install -r docs/requirements.txt
+      - name: Check for broken links
+        env:
+          FORCE_COLOR: 1  # Force color output
+        run: make linkcheck

Makefile

@@ -628,6 +628,15 @@ lint: modules ensure-test-files-annotated
 	$(GOPATH)/bin/golangci-lint run -v
 	@echo Done.
 
+.PHONY: lint-docs
+lint-docs:
+	vale docs
+	@sort .github/styles/Nuclio/ignore.txt -o .github/styles/Nuclio/ignore.txt
+
+.PHONY: linkcheck
+linkcheck:
+	make -C docs/ linkcheck
+
 .PHONY: ensure-test-files-annotated
 ensure-test-files-annotated:
 	$(eval test_files_missing_build_annotations=$(strip $(shell find . -type f -name '*_test.go' -exec bash -c "grep -m 1 -L '//go:build ' {} | grep go" \;)))

README.md

@@ -26,7 +26,7 @@ Translations:
 
 Nuclio is a high-performance "serverless" framework focused on data, I/O, and compute intensive workloads. It is well integrated with popular data science tools, such as [Jupyter](https://jupyter.org/) and [Kubeflow](https://www.kubeflow.org/); supports a variety of data and streaming sources; and supports execution over CPUs and GPUs. The Nuclio project began in 2017 and is constantly and rapidly evolving; many start-ups and enterprises are now using Nuclio in production.
 
-You can use Nuclio as a standalone Docker container or on top of an existing [Kubernetes](https://kubernetes.io) cluster; see the deployment instructions in the Nuclio [documentation](https://docs.nuclio.io/en/stable). You can also use Nuclio through a fully managed application service (in the cloud or on-prem) in the [Iguazio Data Science Platform](https://www.iguazio.com/), which you can [try for free](https://go.iguazio.com/start-your-free-trial).
+You can use Nuclio as a standalone Docker container or on top of an existing [Kubernetes](https://kubernetes.io) cluster; see the deployment instructions in the Nuclio [documentation](https://docs.nuclio.io/en/stable). You can also use Nuclio through a fully managed application service (in the cloud or on-prem) in the [Iguazio Data Science Platform](https://www.iguazio.com/).
 
 If you wish to create and manage Nuclio functions through code - for example, from Jupyter Notebook - see the [Nuclio Jupyter project](https://github.com/nuclio/nuclio-jupyter), which features a Python package and SDK for creating and deploying Nuclio functions from Jupyter Notebook.
 Nuclio is also an integral part of the new open-source [MLRun](https://github.com/mlrun/mlrun) library for data science automation and tracking and of the open-source [Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/) framework for building and deploying portable, scalable ML workflows.
@@ -76,7 +76,6 @@ For a complete step-by-step guide to using Nuclio over Kubernetes, either with t
 - [Getting Started with Nuclio on Kubernetes](/docs/setup/k8s/getting-started-k8s.md)
 - [Getting Started with Nuclio on Google Kubernetes Engine (GKE)](/docs/setup/gke/getting-started-gke.md)
 - [Getting started with Nuclio on Azure Container Services (AKS)](/docs/setup/aks/getting-started-aks.md)
-- [Hands-on live Kubernetes sandbox and guiding instructions for Nuclio, free on Katacoda](https://katacoda.com/javajon/courses/kubernetes-serverless/nuclio)
 
 ## How it works
 
@@ -164,8 +163,6 @@ More examples can be found in the **[Examples page](docs/examples/README.md)**.
   - [Runtime - .NET Core 7.0](/docs/reference/runtimes/dotnetcore/writing-a-dotnetcore-function.md)
   - [Runtime - Shell](/docs/reference/runtimes/shell/shell-reference.md)
 - [Examples](docs/examples/README.md)
-- Sandbox
-  - [Install Nuclio and run functions. Explore and experiment on a free Kubernetes cluster.](https://katacoda.com/javajon/courses/kubernetes-serverless/nuclio)
 - Contributing
   - [Code conventions](/docs/devel/coding-conventions.md)
   - [Contributing to Nuclio](/docs/devel/contributing.md)

README_zh.md

@@ -26,7 +26,7 @@
 
 Nuclio 是一个高性能的 "serverless" 框架，专注于数据、I/O和计算密集型的工作负载. 它与流行的数据科学工具集成地很好，例如 [Jupyter](https://jupyter.org/) 和 [Kubeflow](https://www.kubeflow.org/); 支持多种类型的数据和流式数据源; 并且支持在CPU和GPU上执行任务。Nuclio 项目于 2017 年启动，并处于持续不断的快速发展中；现如今，许多初创企业已将 Nuclio 应用于生产。
 
-你可以将 Nuclio 以一个独立的 Docker 容器运行或者运行在一个已有的 [Kubernetes](https://kubernetes.io) 集群中; 在 Nuclio 文档中查看具体的部署指南。 也可以通过 [Iguazio 数据科学平台](https://www.iguazio.com/) 中的全权托管应用服务平台（云上或者本地）来使用 Nuclio，它提供[免费的试用](https://go.iguazio.com/start-your-free-trial).
+你可以将 Nuclio 以一个独立的 Docker 容器运行或者运行在一个已有的 [Kubernetes](https://kubernetes.io) 集群中; 在 Nuclio 文档中查看具体的部署指南。 也可以通过 [Iguazio 数据科学平台](https://www.iguazio.com/) 中的全权托管应用服务平台（云上或者本地）来使用 Nuclio.
 
 如果想通过编码的方式创建或者管理 Nuclio 函数（functions）- 例如, 使用 Jupyter Notebook - 请参阅 [Nuclio Jupyter 项目](https://github.com/nuclio/nuclio-jupyter), 它包含一个完整的 Python 包和软件开发工具包（SDK）用于通过 Jupyter Notebook 创建和部署 Nuclio 函数。 Nuclio 作为新开源项目 [MLRun](https://github.com/mlrun/mlrun) library，以及开源项目 [Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/) 中不可或缺的部分，分别提供数据科学自动化及追踪能力，以及构建和部署弹性可迁移机器学习（ML）工作流的能力。 
 
@@ -77,7 +77,6 @@ curl -X POST \
 - [在 Kubernetes 中使用 Nuclio](/docs/setup/k8s/getting-started-k8s.md)
 - [在 GKE（Google Kubernetes Engine） 中使用 Nuclio](/docs/setup/gke/getting-started-gke.md)
 - [在 AKS（Azure Container Services） 中使用 Nuclio](/docs/setup/aks/getting-started-aks.md)
-- [在 Katacoda 中使用免费的 Kubernetes 沙箱环境中，按步骤运行 Nuclio](https://katacoda.com/javajon/courses/kubernetes-serverless/nuclio)
 
 ## 架构设计
 
@@ -169,8 +168,6 @@ def handler(context, event):
   - [运行时 - .NET Core 7.0](/docs/reference/runtimes/dotnetcore/writing-a-dotnetcore-function.md)
   - [运行时 - Shell](/docs/reference/runtimes/shell/shell-reference.md)
 - [示例](docs/examples/README.md)
-- 沙箱环境
-  - [在免费的 Kubernetes 集群中安装 Nuclio 并运行函数，以进行各种探索和试验](https://katacoda.com/javajon/courses/kubernetes-serverless/nuclio)
 - 贡献指南
   - [代码规范](/docs/devel/coding-conventions.md)
   - [向 Nuclio 做贡献](/docs/devel/contributing.md)

docs/Makefile

@@ -14,7 +14,12 @@ help:
 
 .PHONY: help Makefile
 
+# Link check target: checks for broken links in the documentation.
+linkcheck:
+	make html
+	@$(SPHINXBUILD) -b linkcheck "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/concepts/best-practices-and-common-pitfalls.md

@@ -47,7 +47,7 @@ Because each Nuclio worker receives its own context across all runtimes, `init_c
 <a id="http-clients-for-testing"></a>
 ## Use HTTP clients over browsers for HTTP(s) tests
 
-When testing functions over HTTP(s), prefer an HTTP client (such as [curl](https://curl.haxx.se/), [HTTPie](https://httpie.org/), or [Postman](https://www.getpostman.com/)) over a browser (such as Google Chrome). 
+When testing functions over HTTP(s), prefer an HTTP client (such as [curl](https://curl.se/), [HTTPie](https://httpie.org/), or [Postman](https://www.postman.com/)) over a browser (such as Google Chrome).
 Browsers tend to create requests for **favicon.ico** and other sneaky things before you even press `ENTER`. This might cause confusion while debugging functions, as you can't really control when your function is invoked.
 
 <a id="tweak-worker-cfg-to-resolve-http-503-errors"></a>

docs/concepts/k8s/function-ingress.md

@@ -21,7 +21,7 @@ However, without an ingress controller running on your cluster, this will have n
 
 ## Setting up an ingress controller
 
-In this guide, you'll set up a [Træfik](https://docs.traefik.io/) controller, but any type of Kubernetes ingress controller should work. You can read [Træfik's excellent documentation](https://docs.traefik.io/user-guides/crd-acme/), but for the purposes of this guide you can simply run the following commands to set up the controller by using either of the following alternative methods:
+In this guide, you'll set up a [Træfik](https://docs.traefik.io/) controller, but any type of Kubernetes ingress controller should work. You can read [Træfik's excellent documentation](https://doc.traefik.io/traefik/user-guides/crd-acme/), but for the purposes of this guide you can simply run the following commands to set up the controller by using either of the following alternative methods:
 
 - Using `kubectl` to apply the resource YAML files. Note that this installs v1.7, and that the YAML files were removed from newer versions:
   ```sh

docs/conf.py

@@ -28,6 +28,16 @@
 templates_path = ["_templates"]
 exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
+linkcheck_ignore = {
+    r'https:\/\/github\.com\/.*\/.*#L\d+-L\d+',
+    # linkcheck doesn't work well with relative paths which contain anchor, so ignore them
+    r'^.*\.html#.*$',
+    r'^\./[^/]+\.html#.*$',
+    r'^\.\./[^/]+\.html#.*$',
+
+}
+linkcheck_anchors = True
+
 language = "go"
 
 # https://sphinx-copybutton.readthedocs.io/en/latest/use.html#strip-and-configure-input-prompts-for-code-cells
@@ -75,6 +85,13 @@
 
 def setup(app):
     app.connect('source-read', process_tables)
+    app.connect('source-read', replace_md_links)
+
+import re
+import markdown
+from sphinx.util import logging
+
+logger = logging.getLogger(__name__)
 
 
 def process_tables(app, docname, source):
@@ -87,8 +104,6 @@ def process_tables(app, docname, source):
     This function is called by sphinx for each document. `source` is a 1-item list. To update the document, replace
     element 0 in `source`.
     """
-    import markdown
-    import re
     md = markdown.Markdown(extensions=['markdown.extensions.tables'])
     table_processor = markdown.extensions.tables.TableProcessor(md.parser, {})
 
@@ -104,3 +119,16 @@ def process_tables(app, docname, source):
     # re-assemble into markdown-with-tables-replaced
     # must replace element 0 for changes to persist
     source[0] = ''.join(blocks)
+
+
+def replace_md_links(app, docname, source):
+    """Replace .md#section links with .html#section links in Markdown files."""
+
+    # Regex pattern to match Markdown links with .md files and anchors
+    md_link_pattern = re.compile(r'\[([^]]+)]\(([^)]+)\.md(#.*?)\)')
+    new_source = md_link_pattern.sub(r'[\1](\2.html\3)', source[0])
+
+    if source[0] != new_source:
+        logger.warn(f'Updated markdown links in {docname}')
+
+    source[0] = new_source

docs/examples/README.md

@@ -11,7 +11,7 @@ Note: all function examples have the explicit field `disableDefaultHttpTrigger:
 - [Image Resize and Convert](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/image) (`image`): A function that demonstrates how to pass a binary-large object (blob) in an HTTP request body and response. The function defines an HTTP request that accepts a binary image or URL as input, converts the input to the target format and size, and returns the converted image in the HTTP response.
 - [HTTP Ingress](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/ingress) (`ingress`): A simple function with an HTTP ingress configuration (using embedded YAML code) that routes specific URL paths to the function.
 - [RabbitMQ](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/rabbitmq) (`rabbitmq`): A multi-trigger function with a configuration that connects to RabbitMQ to read messages and write them to local ephemeral storage. If triggered with an HTTP `GET` request, the function returns the messages that it read from RabbitMQ.
-- [Azure Event Hub](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/eventhub) (`eventhub`): An Azure Event Hub triggered function with a configuration that connects to an Azure Event Hub. The function reads messages from two partitions, process the messages, invokes another function, and sends the processed payload to another Azure Event Hub. You can find a full demo scenario [here](https://github.com/nuclio/demos/tree/master/fleet-alarm-detection-azure).
+- [Azure Event Hub](https://github.com/nuclio/nuclio-templates/tree/master/eventhub) (`eventhub`): An Azure Event Hub triggered function with a configuration that connects to an Azure Event Hub. The function reads messages from two partitions, process the messages, invokes another function, and sends the processed payload to another Azure Event Hub. You can find a full demo scenario [here](https://github.com/nuclio/demos/tree/master/fleet-alarm-detection-azure).
 - [Call Function](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/callfunction) (`callfunction`): A set of two functions that demonstrates the `CallFunction` feature:
 
     - [`fibonacci`](https://github.com/nuclio/nuclio/tree/development/hack/examples/golang/callfunction/fibonacci/fibonacci.go) - For input parameter `n`, returns the `n`-th number in the Fibonacci series (`fib(n)`).

docs/index.md

@@ -10,7 +10,7 @@ For support and additional product information, join the active [Nuclio Slack](h
 
 Nuclio is a high-performance "serverless" framework focused on data, I/O, and compute intensive workloads. It is well integrated with popular data science tools, such as [Jupyter](https://jupyter.org/) and [Kubeflow](https://www.kubeflow.org/); supports a variety of data and streaming sources; and supports execution over CPUs and GPUs. The Nuclio project began in 2017 and is constantly and rapidly evolving; many start-ups and enterprises are now using Nuclio in production.
 
-You can use Nuclio as a standalone Docker container or on top of an existing [Kubernetes](https://kubernetes.io) cluster; see the deployment instructions in the Nuclio documentation. You can also use Nuclio through a fully managed application service (in the cloud or on-prem) in the [Iguazio Data Science Platform](https://www.iguazio.com/), which you can [try for free](https://go.iguazio.com/start-your-free-trial).
+You can use Nuclio as a standalone Docker container or on top of an existing [Kubernetes](https://kubernetes.io) cluster; see the deployment instructions in the Nuclio documentation. You can also use Nuclio through a fully managed application service (in the cloud or on-prem) in the [Iguazio Data Science Platform](https://www.iguazio.com/).
 
 If you wish to create and manage Nuclio functions through code - for example, from Jupyter Notebook - see the [Nuclio Jupyter project](https://github.com/nuclio/nuclio-jupyter), which features a Python package and SDK for creating and deploying Nuclio functions from Jupyter Notebook.
 Nuclio is also an integral part of the new open-source [MLRun](https://github.com/mlrun/mlrun) library for data science automation and tracking and of the open-source [Kubeflow Pipelines](https://www.kubeflow.org/docs/components/pipelines/) framework for building and deploying portable, scalable ML workflows.

docs/reference/api-gateway/nuctl.md

@@ -20,7 +20,7 @@ $ nuctl create apigateway <api-gateway-name> \
 			--namespace <namespace>
 ```
 
-For invoking the function using the api gateway, see [invoking API Gateways](./http.html#invoke-api-gateways).
+For invoking the function using the api gateway, see [invoking API Gateways](./http.md#invoke-api-gateways).
 
 <a id="basic-auth"></a>
 ## Basic authentication
@@ -39,7 +39,7 @@ $ nuctl create apigateway <api-gateway-name> \
 			--namespace <namespace>
 ```
 
-To invoke the function using the API gateway, see [invoking API Gateways with basic authentication](./http.html#invoke-api-gateways-with-basic-authentication).
+To invoke the function using the API gateway, see [invoking API Gateways with basic authentication](./http.md#invoke-api-gateways-with-basic-authentication).
 
 <a id="delete"></a>
 ## Delete an API Gateway

docs/reference/api-gateway/ui.md

@@ -24,11 +24,11 @@ There, you can create an API Gateway with the following parameters:
 
 ![api-gateway](../../../docs/assets/images/api-gateway-ui-none.png)
 
-To invoke the function using the api gateway, see [invoking API Gateways](./http.html#invoke-api-gateways).
+To invoke the function using the api gateway, see [invoking API Gateways](./http.md#invoke-api-gateways).
 
 <a id="basic-auth"></a>
 ### With Basic Authentication
 
 ![api-gateway-basic-auth](../../../docs/assets/images/api-gateway-ui-basic-auth.png)
 
-To invoke the function using the api gateway, see [invoking API Gateways with basic authentication](./http.html#invoke-api-gateways-with-basic-authentication).
+To invoke the function using the api gateway, see [invoking API Gateways with basic authentication](./http.md#invoke-api-gateways-with-basic-authentication).