[BUG] Routing Regression in 1.10.13(ZIO-HTTP)

[lmlynik]

Tapir version: 1.10.13(1.10.12 works correctly)
Scala version: 2.13.14
ZIO-HTTP version: 3.0.0-RC9 and other compatible

Describe the bug

After upgrading Tapir, routes in the application stopped matching(returning 404).
I've isolated the issue to specific Tapir version.

How to reproduce?

//> using scala "2.13.14"
//> using lib "dev.zio::zio:2.1.5"
//> using lib "dev.zio::zio-http:3.0.0-RC9"
//> using lib "com.softwaremill.sttp.tapir::tapir-core:1.10.13"
//> using lib "com.softwaremill.sttp.tapir::tapir-zio:1.10.13"
//> using lib "com.softwaremill.sttp.tapir::tapir-zio-http-server:1.10.13"
//> using lib "com.softwaremill.sttp.tapir::tapir-openapi-docs:1.10.13"
//> using lib "com.softwaremill.sttp.tapir::tapir-json-circe:1.10.13"
//> using lib "com.softwaremill.sttp.tapir::tapir-swagger-ui-bundle:1.10.13"

import sttp.tapir.server.ziohttp.ZioHttpInterpreter
import sttp.tapir.ztapir._
import zio._
import zio.http.Server
import io.circe.generic.auto._
import sttp.tapir.generic.auto._
import sttp.tapir.json.circe._

object Main extends ZIOAppDefault {

  case class Response(message: String)

  val routes = ZioHttpInterpreter().toHttp(
    endpoint.get
      .in("myroute")
      .out(jsonBody[Response])
      .zServerLogic(i => ZIO.succeed(Response(s"""<div>myroute $i</div>""")))
  ) ++ ZioHttpInterpreter().toHttp(
    endpoint.get
      .in("myroute" / path[String]("resource"))
      .out(jsonBody[Response])
      .zServerLogic(i => ZIO.succeed(Response(s"""<div>myroute $i</div>""")))
  ) ++ ZioHttpInterpreter().toHttp(
    endpoint.post
      .in("myroute" / "process")
      .out(jsonBody[Response])
      .zServerLogic(i => ZIO.succeed(Response(s"""<div>myroute/process $i</div>""")))
  ) ++ ZioHttpInterpreter().toHttp(
    endpoint.patch
      .in("myroute" / path[String]("resource") / "subroute" / path[String]("sub"))
      .out(jsonBody[Response])
      .zServerLogic(i => ZIO.succeed(Response(s"""<div>myroute/subroute $i</div>""")))
  ) ++ ZioHttpInterpreter().toHttp(
    endpoint.get
      .in("myroute" / path[String]("resource") / "subroute" / "process")
      .out(jsonBody[Response])
      .zServerLogic(i => ZIO.succeed(Response(s"""<div>myroute/subroute/process $i</div>""")))
  )

  val run =
    Server.serve(routes).provide(Server.default)
}

### GET request to myroute
GET http://localhost:8080/myroute

### GET request to myroute with a path parameter - doesn't match on 1.10.13
GET http://localhost:8080/myroute/sample

### POST request to myroute/process
POST http://localhost:8080/myroute/process

### PATCH request to myroute/subroute with path parameters  - doesn't match on 1.10.13
PATCH http://localhost:8080/myroute/sample1/subroute/sample2

### GET request to myroute/subroute/process with a path parameter  - doesn't match on 1.10.13
GET http://localhost:8080/myroute/sample/subroute/process

Compare route behaviour between 1.10.13 and 1.10.12

[adamw]

Ah yes, you're right. But I think this must be documented as a limitation of the zio-http interpreter: any endpoints that have a common path prefix, must be interpreted together.

So in your example, you'd call ZioHttpInterpreter().toHttp once, giving all endpoints as a parameter.

That's because zio-http became less flexible; it's not possible anymore to dynamically determine if a request "matches" an endpoint. That's why I resorted to matching by path prefix:

tapir/server/zio-http-server/src/main/scala/sttp/tapir/server/ziohttp/ZioHttpInterpreter.scala

Line 102 in 424e2d2

base / PathCodec.trailing

. Or at least, I don't know how to implement the dynamic routing.

I don't think interpreting all endpoints in a single call has any downsides?

[mikoloay]

I don't think interpreting all endpoints in a single call has any downsides?

Hi Adam, I have the same issue as OP and it doesn't seem to work even when interpreting all endpoints at once.

Here's a minimal script that works on 1.10.12 and doesn't on newer versions:

package com.softwaremill

import sttp.tapir.server.ziohttp.{ZioHttpInterpreter, ZioHttpServerOptions}
import zio.{Console, LogLevel, Scope, ZIO, ZIOAppArgs, ZIOAppDefault, ZLayer}
import zio.http.{Response, Routes, Server}
import zio.logging.LogFormat
import zio.logging.backend.SLF4J
import sttp.tapir.*
import sttp.tapir.json.zio.*
import sttp.tapir.ztapir.ZServerEndpoint
import zio.ZIO

object Endpoints:
  val helloEndpoint: PublicEndpoint[Unit, Unit, String, Any] = endpoint.get
    .in("hello")
    .out(stringBody)
  val helloServerEndpoint: ZServerEndpoint[Any, Any] = helloEndpoint.serverLogicSuccess(_ => ZIO.succeed(s"Hello there!"))

  val helloWithNameEndpoint: PublicEndpoint[String, Unit, String, Any] = endpoint.get
      .in("hello")
      .in(path[String]("name"))
      .out(stringBody)
  val helloWithNameServerEndpoint: ZServerEndpoint[Any, Any] = helloWithNameEndpoint.serverLogicSuccess(name => ZIO.succeed(s"Hello there $name!"))

  val all: List[ZServerEndpoint[Any, Any]] = List(helloServerEndpoint, helloWithNameServerEndpoint)


object Main extends ZIOAppDefault:
  override val bootstrap: ZLayer[ZIOAppArgs, Any, Any] = SLF4J.slf4j(LogLevel.Debug, LogFormat.default)

  override def run: ZIO[Any & ZIOAppArgs & Scope, Any, Any] =
    val serverOptions: ZioHttpServerOptions[Any] =
      ZioHttpServerOptions.customiseInterceptors.options

    val app: Routes[Any, Response] = ZioHttpInterpreter(serverOptions).toHttp(Endpoints.all)

    val port = sys.env.get("HTTP_PORT").flatMap(_.toIntOption).getOrElse(8080)

    (for {
      actualPort <- Server.install(app) // or .serve if you don't need the port and want to keep it running without manual readLine
      _ <- Console.printLine(s"Server started at http://localhost:${actualPort}. Press ENTER key to exit.")
      _ <- Console.readLine
    } yield ())
      .provide(
        ZLayer.succeed(Server.Config.default.port(port)),
        Server.live
      )
      .exitCode

[mikoloay]

I think I might have found the source of the issue.
I was working with the code I pasted in the previous comment and after interpreting the endpoints to ZIO Routes I printed them out:

val app: Routes[Any, Response] = ZioHttpInterpreter(serverOptions).toHttp(Endpoints.all)

// [...]

for {
  _ <- ZIO.attempt(app.routes.foreach(println))
} yield ()

which printed out the following:
Route.Handled(* /hello/..., )
Route.Handled(* /hello/{name}/..., )

The ordering caught my attention so I created an app with the routes in reversed order:

val app2: Routes[Any, Response] = Routes(app.routes.reverse)

which resulted in the following ordering:
Route.Handled(* /hello/{name}/..., )
Route.Handled(* /hello/..., )

And then both endpoints worked! So it seems to me that routes returned by the toHttp method should be in the order from the most specific to less specific, or they should at least preserve the order from user's input - then we could only blame ourselves :D but of course auto-ordering would be smoother.

[mikoloay]

I've prepared a modification in the ZioHttpInterpreter.scala class which seems to resolve this issue. I will send a pull request with my proposed solution soon.

[adamw]

I'll be interested to see @mikolololoay's solution, I also worked on this, and I think I might be able to generate an exact translation of tapir's inputs to zio-http's path patterns, what I'm missing though is a way to create a zio.http.codec.PathCodec which would match (and discard) a trailing slash.

[adamw]

Here's my latest take, passes the tests I have so far, though of course I might not have enough tests: #3949

[adamw]

1.10.15 should be out soon, please test :)

[mikoloay]

1.10.15 should be out soon, please test :)

It works in 1.10.15! Thank you 🎉