ZIO + Http4s: a simple API client

Discussing with a brazilian friend about the situation in our country, we realised how difficult it is to find information about public spending, and when available, how difficult it can be to reason about it. Joining our forces, we decided to explore some data exposed by the Brazilian government, aiming to provide an easier way to visualise and understand how the public resources has been used.

The starting point would be: finding some data to analyse, that is relatively easy (at least from a developer’s perspective) to collect. A good candidate is the Portal da Transparência (in literal translation, Transparency Portal), an initiative to make public data available via APIs or downloading CSV files.

Is there a better way to learn about an API than writing a client for it? So let’s do it with ZIO + http4s client!

Why ZIO?

After my talk in Scala UA, someone asked me what has called my attention in the Scala ecossystem recently. I believe ZIO can be a game changer, because it is not “just for functional programmers”. Even though it is strongly based in functional principles, it doesn’t assume the users already understand functional concepts (this is just a Monad!), which can be scary for new joiners.

Among all the powerful features ZIO provides, it’s designed to be easy to use and adopt, what is from my perspective, by far, its best feature. #ScalaThankYou ZIO Team!

Time to code, let’s start defining a ZIO module.

The HttpClient module

The API supports only GET requests, what makes the trait definition very simple:

package pdt.http

import io.circe.Decoder
import org.http4s.client.Client
import zio._

object HttpClient {
  type HttpClient = Has[Service]

  trait Service {
    protected val rootUrl = "http://www.transparencia.gov.br/api-de-dados/"

    def get[T](uri: String, parameters: Map[String, String])
              (implicit d: Decoder[T]): Task[T]
  }

  def http4s: ZLayer[Has[Client[Task]], Nothing, HttpClient] = ???
}

Service has only one method get[T] with arguments resource: String and parameters: Map[String, String], which will become part of the url in the format "resource?key=value". It takes an implicit io.circe.Decoder[T] as well, used to decode the json result into T.

get[T] returns a zio.Task[T], a type alias for ZIO[Any, Throwable, T], which represents an effect that has no requirements, and may fail with a Throwable value, or succeed with a T.

Following the module recipe, we have:

type HttpClient = Has[Service]

In simple terms, Has allows us to use our Service as a dependency. The next line makes it easier to understand:

def http4s: ZLayer[Has[Client[Task]], Nothing, HttpClient] = ???

The http4s method will create a ZLayer, which is very similar to ZIO data type; it requires a Has[Client[Task]] to be built, won’t produce any errors (that’s what that Nothing means) and will return an implementation of our Service: HttpClient, the one we defined using Has.

We should use type aliases to make ZLayer more expressive as well. Knowing our layer can’t fail, we can use URLayer:

def http4s: URLayer[Has[Client[Task]], HttpClient] = ???

What will http4s actually return? In order to answer this question, we need to implement HttpClient.Service first.

The Http4s implementation

Implementing the get request is straightforward:

package pdt.http

import io.circe.Decoder
import org.http4s.Uri
import org.http4s.circe.CirceEntityCodec.circeEntityDecoder
import org.http4s.client.Client
import org.http4s.client.dsl.Http4sClientDsl
import zio._
import zio.interop.catz._

private[http] final case class Http4s(client: Client[Task])
  extends HttpClient.Service with Http4sClientDsl[Task] {

  def get[T](resource: String, parameters: Map[String, String])
            (implicit d: Decoder[T]): Task[T] = {
    val uri = Uri(path = rootUrl + resource).withQueryParams(parameters)

    client.expect[T](uri.toString())
  }

Maybe you are scratching your head due to that import zio.interop.catz._. http4s is built on top of the Cats Effect stack, therefore we need the interop-catz module for interoperability.

An instance of this class can’t be created outside the http package; the instance will be provided through our ZLayer. Let’s go back to HttpClient.http4s, it’s time to implement it!

Providing an HttpClient.Service through ZLayer

Having a service definition, ZLayer.fromService seems appropriate:

object HttpClient {

  def http4s: URLayer[Has[Client[Task]], HttpClient] =
    ZLayer.fromService[Client[Task], Service] { http4sClient =>
      Http4s(http4sClient)
    }
}

Okay, this layer makes our HttpClient available. How can we access it? Let’s start defining something that uses the client, a concrete example always makes learning easier :)

A couple of useful helpers

The first resource, Acordos de Leniência is a good candidate:

• GET /acordos-leniencia/{id} returns an object;
• GET /acordos-leniencia (with filters as query params) returns a list of objects;

The rest of the API exposes basically the same for other resources, just with more filters. Knowing that, we can define two helpers, one for each case:

object HttpClient {
  // ...

  def get[T](resource: String, id: Long)
            (implicit d: Decoder[T]): RIO[HttpClient, T] =
    RIO.accessM[HttpClient](_.get.get[T](s"$resource/$id", Map()))

  def get[T](resource: String, parameters: Map[String, String] = Map())
            (implicit d: Decoder[T]): RIO[HttpClient, List[T]] =
    RIO.accessM[HttpClient](_.get.get[List[T]](resource, parameters))
}

RIO.accessM[HttpClient] effectfully accesses the environment of our effect, giving us Has[HttpClient.Service], so we call the first get to access the effect wrapped by Has - our Service - while the second get is the actual get request.

To make it clear, if we had a post method, the code would be:

RIO.accessM[HttpClient](_.get.post[T](resource, parameters))

Alright, let’s make the whole thing work!

A concrete HttpClient… client (?!?!?)

Then again, Acordos de Leniência is our resource. This is a case class for its possible filters (brazilian api, names in portuguese):

case class AcordoLenienciaRequest(
              cnpjSancionado: Option[String] = None,
              nomeSancionado: Option[String] = None,
              situacao: Option[String] = None,
              dataInicialSancao: Option[LocalDate] = None,
              dataFinalSancao: Option[LocalDate] = None,
              pagina: Int = 1)

And the response:

case class AcordoLeniencia(
              id: Long,
              nomeEmpresa: String,
              dataInicioAcordo: LocalDate,
              dataFimAcordo: LocalDate,
              orgaoResponsavel: String,
              cnpj: String,
              razaoSocial: String,
              nomeFantasia: String,
              ufEmpresa: String,
              situacaoAcordo: String,
              quantidade: Int)

AcordosLenienciaClient couldn’t be simpler:

import io.circe.generic.auto._
import pdt.client.decoders.localDateDecoder
import pdt.http.HttpClient.{HttpClient, get}
import pdt.domain.{AcordoLeniencia, AcordoLenienciaRequest => ALRequest}
import pdt.http.implicits.HttpRequestOps
import zio._

object AcordosLenienciaClient {

  def by(id: Long): RIO[HttpClient, AcordoLeniencia] =
    get[AcordoLeniencia]("acordos-leniencia", id)

  def by(request: ALRequest): RIO[HttpClient, List[AcordoLeniencia]] =
    get[AcordoLeniencia]("acordos-leniencia", request.parameters)
}

The implicit method HttpRequestOps.parameters transforms any request into a Map[String, String]. Check it out how I used shapeless to do so.

Now we just need to put all the pieces together, sort out dependencies, this kind of thing. That happens at the end of the world… also known as Main.

ZIO Modules, assemble!

Here is a program that makes a request to get a list of AcordoLeniencia:

val program = for {
  result <- AcordosLeniencia.by(AcordoLenienciaRequest())
  _ <- putStrLn(result.toString())
} yield ()

It requires a ZLayer that produces an HttpClient, which has Client[Task] as its own dependency. Let’s create the Client[Task] as a managed resource first:

private def makeHttpClient: UIO[TaskManaged[Client[Task]]] =
  ZIO.runtime[Any].map { implicit rts =>
      BlazeClientBuilder
        .apply[Task](Implicits.global)
        .resource
        .toManaged
    }

Now we can sort out the layers:

val httpClientLayer = makeHttpClient.toLayer.orDie
val http4sClientLayer = httpClientLayer >>> HttpClient.http4s

and finally, provide our program with the required layer:

program.provideSomeLayer[ZEnv](http4sClientLayer)

Ready to go:

program.foldM(
  e => putStrLn(s"Execution failed with: ${e.printStackTrace()}") *> ZIO.succeed(1),
  _ => ZIO.succeed(0)
)

And this is how I built it. Some code is different here from the original, for learning purposes. You can find the code on Github.

Before we jump to the conclusion, let’s consolidate what we’ve learnt adding a new dependency, a logger that prints in the console the url requested, and the error message if it fails.

Adding a new dependency, step by step

The Logger module definition:

object Logger {
  type Logger = Has[Service]

  trait Service {
    def info(message: => String): UIO[Unit]
    def error(t: Throwable)(message: => String): UIO[Unit]
  }
}

The implementation, printing to the console:

import zio.console.{Console => ConsoleZIO}

case class Console(console: ConsoleZIO.Service)
  extends Logger.Service {

  def info(message: => String): UIO[Unit] =
    console.putStrLn(message)

  def error(t: Throwable)(message: => String): UIO[Unit] =
    for {
      _ <- console.putStrLn(message)
      _ <- console.putStrLn(t.stackTrace)
    } yield ()
}

Logger makes the implementation available via ZLayer:

object Logger {

  def console: URLayer[ConsoleZIO, Logger] =
    ZLayer.fromService[ConsoleZIO.Service, Service] { console =>
      Console(console)
    }
}

Http4s can now receive and use a logger instance:

private[http] final case class Http4s(logger: Logger.Service, client: Client[Task])
  extends HttpClient.Service with Http4sClientDsl[Task] {

  def get[T](resource: String, parameters: Map[String, String])
            (implicit d: Decoder[T]): Task[T] = {
    val uri = Uri(path = rootUrl + resource).withQueryParams(parameters)

    logger.info(s"GET REQUEST: $uri") *>
      client
        .expect[T](uri.toString())
        .foldM(
          e => logger.error(e)("Request failed") *> IO.fail(e),
          ZIO.succeed(_))
  }
}

The http4s layer needs to adapt:

object HttpClient {

  def http4s: URLayer[Logger with Has[Client[Task]], HttpClient] =
    ZLayer.fromServices[Logger.Service, Client[Task], Service] {
      (logger, http4sClient) =>
        Http4s(logger, http4sClient)
    }
}

Let’s feed our program with the new dependency. The change is in the layer provided:

val http4sClientLayer = (loggerLayer ++ httpClientLayer) >>> HttpClient.http4s

Done!

Summary

My first experience with ZIO has been very pleasant. In order to solve dependencies, the compiler plays on our side; every time something is missing, we have an error in compile time, with a clear indication of what is missing. Besides, ZLayer makes dependency resolution extremely simple and extensible (think about adding a FileLogger for example) without magic.

Any suggestions to improve that code? Please share!

References

• ZIO - Using modules and Layers
• Solving the Dependency Injection Problem with ZIO
• From idea to product with ZLayer
• ZIO, Http4s, Auth, Codecs and zio-test
• Http4s Client