Tern

Building web apps

This guide builds HTTP services with the http, json, and sqlite modules, from the smallest server to a JSON API backed by SQLite. It covers the shape of a handler, routing, query parameters and headers, JSON in and out, status codes, shared state (read-only and changing), the database, what happens when a handler fails, the server's limits, deploying the single binary, and testing, both by calling handlers directly and with curl.

The smallest server

Compiles (not run here)
import http

fn handle(req: http.Request) http.Response:
    return http.text("Hello, World!")

fn main():
    http.serve(8080, handle)
tern run hello.tn                      # prints: tern: serving http://localhost:8080 on 14 threads
curl localhost:8080/anything           # Hello, World!

http.serve never returns. It runs an HTTP/1.1 server with an event loop on every core, keep-alive, and pipelining, and calls your handler once per request. The handler is an ordinary function: it gets a Request value and returns a Response value.

struct Request:
    method: str                 # "GET", "POST", ...
    path: str                   # "/users/42" (without the query string)
    raw_query: str              # "page=2&sort=name" (read values with req.query(name))
    head: str                   # header lines as received (read them with req.header(name))
    body: str
    ...                         # route parameters, middleware values, the connection

struct Response:
    status: int
    content_type: str
    body: str
    headers: str                # extra header lines (add them with res.set_header)
    upgrade: int

http lists every field and function.

The http module has helpers that build common responses: http.text(s), http.html(s), http.json(s) (all status 200), http.status(code, s) (plain text with any status), http.not_found(), and http.redirect(url) (a 302). You can also build a Response yourself, or change the one a helper returns.

Handler shapes

A handler takes one of three forms, and the form decides how requests share data:

HandlerServe withRequests run
fn handle(req: http.Request) http.Responsehttp.serve(port, handle)in parallel
fn handle(req: http.Request, app: App) http.Responsehttp.serve(port, handle, app)in parallel; app is read-only
fn handle(req: http.Request, app: mut App) http.Responsehttp.serve(port, handle, &app)one at a time

App is any type you like: a struct of settings, a map, a sqlite.Db. The last two forms are covered in App state below.

Test handlers without a server

A handler is a plain function, so you can call it with a request you build yourself with http.new_request(method, target, body), where target is the path with its query string. This is the quickest way to test routing and responses: no port, no curl, and the output is exact.

import http

fn handle(req: http.Request) http.Response:
    if req.path == "/":
        return http.html("<h1>Home</h1>")
    if req.path == "/health":
        return http.text("ok")
    return http.not_found()

fn main():
    for path in ["/", "/health", "/nope"]:
        res = handle(http.new_request("GET", path, ""))
        print(res.status, res.content_type, res.body)
Output
200 text/html; charset=utf-8 <h1>Home</h1>
200 text/plain; charset=utf-8 ok
404 text/plain; charset=utf-8 not found

The rest of this guide uses the same trick: every tested example below calls its handler directly.

Routing

There is no router object. A handler is a series of if checks on req.method and req.path, usually handing each route to its own function. For paths with parameters, http.match(path, pattern, &params) compares a path with a pattern such as "/users/{id}/posts/{post}". On a match it returns true and fills params with the (URL-decoded) values; otherwise it returns false and leaves params alone.

import http

fn show_post(user: str, post: str) http.Response:
    return http.text(f"post {post} by user {user}")

fn handle(req: http.Request) http.Response:
    params: {str: str} = {}
    if req.method == "GET" and req.path == "/users":
        return http.json("[]")
    if http.match(req.path, "/users/{id}", &params):
        if req.method != "GET":
            return http.status(405, "method not allowed")
        id = params["id"].to_int(-1)
        if id < 1:
            return http.status(400, "id must be a positive number")
        return http.text(f"user {id}")
    if http.match(req.path, "/users/{user}/posts/{post}", &params):
        return show_post(params["user"], params["post"])
    return http.not_found()

fn main():
    for p in ["/users", "/users/42", "/users/abc", "/users/7/posts/hello%20world", "/users/7/likes"]:
        res = handle(http.new_request("GET", p, ""))
        print(res.status, res.body)
    print(handle(http.new_request("DELETE", "/users/42", "")).status)
Output
200 []
200 user 42
400 id must be a positive number
200 post hello world by user 7
404 not found
405

http.match compares whole segments: /users/{id} doesn't match /users/7/likes, and a trailing slash makes a different path. Parameters are always strings; convert them with to_int and reject bad values with a 400.

Query parameters and headers

req.query(name) returns the decoded value of one query parameter, or "" when it is missing. req.header(name) looks up a request header, ignoring case, and also returns "" when it is missing; req.cookie(name) reads one cookie. To add a response header, call res.set_header(name, value); res.get_header(name) reads one back.

import http

fn search(req: http.Request) http.Response:
    q = req.query("q")
    if q == "":
        return http.status(400, "missing ?q=")
    page = req.query("page").to_int(1)
    lang = req.header("accept-language")
    res = http.text(f"searching for [{q}], page {page}, language [{lang}]")
    res.set_header("Cache-Control", "no-store")
    return res

fn main():
    req = http.new_request("GET", "/search?q=tern+lang%21&page=3", "")
    req.add_header("Accept-Language", "en-US")
    res = search(req)
    print(res.status, res.body, res.get_header("cache-control"))
    print(search(http.new_request("GET", "/search", "")).body)
    old = http.redirect("/new-home")
    print(old.status, old.get_header("location"))
Output
200 searching for [tern lang!], page 3, language [en-US] no-store
missing ?q=
302 /new-home

+ and %21 in the query are decoded. The raw query string is in req.raw_query if you need to handle repeated parameters yourself, and req.queries() and req.headers() return them all as maps.

JSON in and out

json.encode turns any value into JSON text, and json.decode(text, &value) fills a value from JSON. Decoding is all-or-nothing: if the body isn't valid JSON of the right shape, it returns false and changes nothing, which maps directly to a 400. Keys missing from the body keep the value you started with, so start from sensible defaults, and validate required fields yourself.

import http
import json

struct Signup:
    email: str
    name: str
    newsletter: bool

struct Created:
    id: int
    email: str

fn signup(req: http.Request) http.Response:
    if req.method != "POST":
        return http.status(405, "use POST")
    input = Signup("", "", false)
    if not json.decode(req.body, &input):
        return http.status(400, `body must be JSON like {"email": "...", "name": "..."}`)
    if not input.email.contains("@"):
        return http.status(422, "email is required")
    res = http.json(json.encode(Created(1001, input.email)))
    res.status = 201
    res.set_header("Location", "/users/1001")
    return res

fn main():
    for body in [`{"email": "ada@example.com", "name": "Ada"}`, `{"name": "no email"}`, "email=ada"]:
        res = signup(http.new_request("POST", "/signup", body))
        print(res.status, res.content_type, res.body)
Output
201 application/json {"id":1001,"email":"ada@example.com"}
422 text/plain; charset=utf-8 email is required
400 text/plain; charset=utf-8 body must be JSON like {"email": "...", "name": "..."}

Unknown keys in the body are ignored, and a JSON null keeps the default. For output, encode a struct or an array of structs; field names become the JSON keys. If you need a key that isn't a valid field name, build the text by hand with json_quote(s), which returns s as a quoted, escaped JSON string (see examples/web/api.tn).

Status codes

Use http.status(code, message) for any status with a plain-text body, or set res.status on a response built by another helper. The conventions the examples here follow:

StatusWhen
200success (http.text, http.html, http.json)
201created something; add a Location header
204success with no body: http.status(204, "")
302http.redirect(url)
400the request is malformed (bad JSON, a non-numeric id)
404http.not_found(), or http.status(404, "no note 7")
405the path exists but not with this method
422well-formed but invalid (a required field is empty)
500sent for you when a handler hits a runtime error

Any number works as a status. The server writes the standard reason phrase ("Not Found") for the common codes above and the generic word Status for the rest (for example HTTP/1.1 429 Status); clients go by the number.

App state: read-only or mut

Most services need data that outlives one request: configuration, a cache, a database handle. Pass it as the handler's second parameter.

Read-only state (app: App) is shared by every server thread at once. It can't be changed after the server starts, so requests never wait on each other:

Compiles (not run here)
import http

struct App:
    greeting: str
    motd: {str: str}

fn handle(req: http.Request, app: App) http.Response:
    lang = req.query("lang")
    return http.text(app.greeting + " " + app.motd.get(lang, "(no message)"))

fn main():
    app = App("hello!", {"en": "Ship it.", "fr": "Expediez-le."})
    http.serve(8080, handle, app)

Changing state (app: mut App, served with &app) runs requests one at a time: the server holds a lock around each call to the handler. That is the whole concurrency story for shared state: no data races are possible, and there is no lock to forget.

import http
import json

struct App:
    visits: {str: int}
    total: int

fn handle(req: http.Request, app: mut App) http.Response:
    if req.method == "POST" and req.path.starts_with("/visit/"):
        page = req.path[len("/visit/")..]
        app.visits[page] = app.visits.get(page, 0) + 1
        app.total += 1
        return http.status(204, "")
    if req.path == "/stats":
        return http.text(f"{app.total} visits: {json.encode(app.visits)}")
    return http.not_found()

fn main():
    app = App({}, 0)
    for page in ["home", "docs", "home"]:
        handle(http.new_request("POST", "/visit/" + page, ""), &app)
    print(handle(http.new_request("GET", "/stats", ""), &app).body)
    # a real server: http.serve(8080, handle, &app)
Output
3 visits: {"home":2,"docs":1}

Why the two shapes? Tern values are never shared by reference, and a read-only parameter can't change, so any number of threads can read the same App safely. A mut parameter can change, so the server must never run two such calls at once. The type of the handler tells the server which case it is in.

The trade-off: with mut state, a slow request holds up every other request. Keep the changing part small. Put read-mostly data in a read-only App, and put data that many requests write into the database, which handles concurrent writers itself.

SQLite

import sqlite links the system SQLite. sqlite.open(path) opens (or creates) a database file. Then:

  • d.exec(sql, params) int runs a statement and returns the last inserted row id.
  • d.query(sql, params) [{str: str}] returns rows as maps from column name to text.
  • d.query_into(sql, params, &out) int reads rows into your structs, matching columns to fields by name, and returns the number of rows. out can be an array of structs (it is replaced by the rows) or one struct (filled from the first row, if there is one).

Parameters are always a [str] and are bound with ? placeholders, never pasted into the SQL, so user input can't inject SQL. SQLite converts them to the column's type, so [str(id)] works for an INTEGER column.

import sqlite
import os

struct Note:
    id: int
    title: str
    pinned: bool

fn main():
    d = sqlite.open(os.env("DBPATH", "notes.db"))
    d.exec("CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, title TEXT NOT NULL, pinned INTEGER NOT NULL)", [])
    first = d.exec("INSERT INTO notes (title, pinned) VALUES (?, ?)", ["groceries", "0"])
    d.exec("INSERT INTO notes (title, pinned) VALUES (?, ?)", ["it's a trap'; DROP TABLE notes; --", "1"])
    print("first id", first)

    notes: [Note] = []
    n = d.query_into("SELECT id, title, pinned FROM notes ORDER BY id", [], &notes)
    print(n, notes)

    one = Note(0, "", false)
    if d.query_into("SELECT id, title, pinned FROM notes WHERE id = ?", [str(first)], &one) > 0:
        print("found", one.title)
    missing = Note(0, "none", false)
    print(d.query_into("SELECT id, title, pinned FROM notes WHERE id = ?", ["999"], &missing), missing.title)

    rows = d.query("SELECT pinned, count(*) AS n FROM notes GROUP BY pinned", [])
    print(rows)
Output
first id 1
2 [Note{id: 1, title: "groceries", pinned: false}, Note{id: 2, title: "it's a trap'; DROP TABLE notes; --", pinned: true}]
found groceries
0 none
[{"pinned": "0", "n": "1"}, {"pinned": "1", "n": "1"}]

Prefer query_into with a struct: the fields are typed, and the struct encodes straight to JSON. query is handy for ad-hoc results like the count(*) above; every value in it is text.

(The test suite runs this with DBPATH=:memory:, an in-memory database. Don't do that for a server; see below.)

The database in a server

A sqlite.Db is a small handle, so put it in your app state (or pass it as the app itself). A read-only d: sqlite.Db parameter can still run INSERTs: the data lives in the database file, not in a Tern value, and "read-only" is about Tern values. So the usual shape is a read-only app, with requests running in parallel and SQLite coordinating the writes.

What the sqlite package does for you:

  • Each statement runs on a pooled connection with its own prepared-statement cache, so requests query in parallel. The file is put in WAL mode, which lets readers and a writer work at the same time. A statement waits up to 5 seconds for a lock (only its task waits).
  • Group commit. A single write per request joins the writes other requests make at the same moment: one transaction, one commit, and each write still returns only once it is saved.
  • Use a file, not :memory:. Each connection to :memory: is a different, empty database, so a server's requests wouldn't see each other's data.

A complete API

Here is a notes API with everything above: routing with parameters, JSON in and out, status codes, and SQLite. It reads its port and database path from the environment.

Compiles (not run here)
import http
import sqlite
import json
import os

struct Note:
    id: int
    title: str
    body: str

struct NewNote:
    title: str
    body: str

fn list_notes(d: sqlite.Db) http.Response:
    notes: [Note] = []
    d.query_into("SELECT id, title, body FROM notes ORDER BY id", [], &notes)
    return http.json(json.encode(notes))

fn show_note(d: sqlite.Db, id: str) http.Response:
    n = Note(0, "", "")
    if d.query_into("SELECT id, title, body FROM notes WHERE id = ?", [id], &n) == 0:
        return http.status(404, f"no note {id}")
    return http.json(json.encode(n))

fn create_note(d: sqlite.Db, req: http.Request) http.Response:
    input = NewNote("", "")
    if not json.decode(req.body, &input):
        return http.status(400, `body must be JSON: {"title": ..., "body": ...}`)
    if input.title.trim() == "":
        return http.status(422, "title is required")
    id = d.exec("INSERT INTO notes (title, body) VALUES (?, ?)", [input.title, input.body])
    res = http.json(json.encode(Note(id, input.title, input.body)))
    res.status = 201
    res.set_header("Location", f"/notes/{id}")
    return res

fn handle(req: http.Request, d: sqlite.Db) http.Response:
    params: {str: str} = {}
    if req.path == "/notes":
        if req.method == "GET":
            return list_notes(d)
        if req.method == "POST":
            return create_note(d, req)
        return http.status(405, "method not allowed")
    if http.match(req.path, "/notes/{id}", &params):
        if req.method == "DELETE":
            d.exec("DELETE FROM notes WHERE id = ?", [params["id"]])
            return http.status(204, "")
        return show_note(d, params["id"])
    if req.path == "/search":
        q = req.query("q")
        rows = d.query("SELECT id, title FROM notes WHERE title LIKE ?", ["%" + q + "%"])
        return http.json(json.encode(rows))
    if req.path == "/broken":
        rows = d.query("SELECT no_such_column FROM notes", [])
        return http.text(str(len(rows)))
    return http.not_found()

fn main():
    d = sqlite.open(os.env("DBPATH", "notes.db"))
    d.exec("CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, title TEXT NOT NULL, body TEXT NOT NULL)", [])
    port = os.env("PORT", "8080").to_int(8080)
    http.serve(port, handle, d)

Testing with curl

Build it, start it on a free port, and try each route. This is a real session with the program above (-i shows the status line and headers):

$ tern build notes.tn -o notes
$ PORT=8090 ./notes &
tern: serving http://localhost:8090 on 14 threads

$ curl -i -X POST localhost:8090/notes -d '{"title": "groceries", "body": "eggs, milk"}'
HTTP/1.1 201 Created
Content-Type: application/json
Location: /notes/1
Content-Length: 48

{"id":1,"title":"groceries","body":"eggs, milk"}

$ curl -X POST localhost:8090/notes -d '{"title": "ideas", "body": "write docs"}'
{"id":2,"title":"ideas","body":"write docs"}

$ curl localhost:8090/notes
[{"id":1,"title":"groceries","body":"eggs, milk"},{"id":2,"title":"ideas","body":"write docs"}]

$ curl -i localhost:8090/notes/99
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
Content-Length: 10

no note 99

$ curl -i -X POST localhost:8090/notes -d 'not json'
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Content-Length: 46

body must be JSON: {"title": ..., "body": ...}

$ curl 'localhost:8090/search?q=gro'
[{"id":"1","title":"groceries"}]

$ curl -i -X DELETE localhost:8090/notes/1
HTTP/1.1 204 No Content
Content-Type: text/plain; charset=utf-8
Content-Length: 0

$ curl -i localhost:8090/broken
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Content-Length: 21

internal server error

Note that /search returns ids as strings ("id":"1"): d.query returns every value as text. Use query_into when the JSON types matter.

When a handler fails

A runtime error inside a handler (a missing map key, an index out of range, division by zero, a failed SQL statement, even inside a parallel block in the handler) doesn't stop the server. That request gets a 500 with the body internal server error, the error is logged to stderr, and the server keeps serving. The /broken route above logged:

tern: GET /broken failed: runtime error: sqlite: no such column: no_such_column

The failed request's open database snapshot is closed, and with mut state, the lock is released. What the handler changed in a mut app before the error stays changed, so validate input before you start changing state.

Don't rely on 500s for expected problems. Bad input is the client's error: check it and return a 400 or 422 with a message, as the examples do. Use m.get(k, default) or k in m instead of m[k] for keys that come from requests, and to_int(-1) with a range check for numbers.

Timeouts and limits

The server enforces a few limits on its own. Change them with environment variables when starting the binary:

LimitDefaultEnvironment variable
Server threadsone per coreTERN_HTTP_THREADS
Idle keep-alive connections closed after60 secondsTERN_HTTP_IDLE_SEC
A request must arrive completely within10 secondsTERN_HTTP_HEADER_SEC
Largest request body (larger gets 413 Payload Too Large)16 MBTERN_HTTP_MAX_BODY_MB
Largest header block (larger gets 400 Bad Request)64 KB
Headers per request (the rest are ignored)64

There is no time limit on the handler itself. Each server thread runs an event loop for many connections, and a handler runs on that thread until it returns, so a handler that takes two seconds delays the other connections on its thread for two seconds. Keep handlers fast; do slow work with a time budget of your own.

Other gaps in http.serve to know about: no TLS, no HTTP/2, no chunked request bodies (clients must send Content-Length, which curl and browsers do for ordinary requests), and no WebSockets. The engine (http.new_server), and roost on top of it, have HTTPS (with tls), streamed and chunked bodies, WebSockets and server-sent events.

Building and deploying

A Tern web app builds to one native binary:

tern build notes.tn -o notes

It depends only on the C library and, if you import sqlite, the system SQLite (libsqlite3). Copy it to a server built for the same OS and CPU (build on Linux for Linux) and run it. It binds to the port on all network interfaces, and it logs to stderr.

TLS belongs in a reverse proxy in front of the binary. With Caddy, which gets certificates automatically, the whole configuration is:

notes.example.com {
    reverse_proxy localhost:8090
}

With nginx, proxy to http://127.0.0.1:8090 and set a client_max_body_size that matches TERN_HTTP_MAX_BODY_MB. Since the proxy faces the internet, firewall the app's own port so only the proxy can reach it.

To keep it running, use your system's service manager. A systemd unit:

[Unit]
Description=notes API
After=network.target

[Service]
ExecStart=/opt/notes/notes
Environment=PORT=8090 DBPATH=/var/lib/notes/notes.db
Restart=always
User=notes

[Install]
WantedBy=multi-user.target

Back up the database with SQLite's own tools (sqlite3 notes.db ".backup backup.db"), which work while the server is running.

Errors with Result: web, pg and ?

The roost framework and the pg, redis, fs, tls and auth packages also return Result values (the try_ functions, and pg.fetch), so a handler can write the happy path straight down and turn every failure into a response in one place. Errors are strings that say which module failed, what, and why: "pg: query failed: ERROR 42P01: relation \"users\" does not exist".

Compiles (not run here)
import json
import log
import pg
import roost

struct User:
    id: int
    name: str
    email: str

# a helper that returns Result: `?` hands an Err back to the caller as it is
fn find_user(db: pg.Pool, id: int) Result[Option[User], str]:
    return db.fetch_one[User]("SELECT id, name, email FROM users WHERE id = $1", [str(id)])

fn team(db: pg.Pool, name: str) Result[[User], str]:
    users: [User] = db.fetch("SELECT id, name, email FROM users WHERE team = $1 ORDER BY id", [name])?
    return Ok(users)

# the handler maps the outcomes to responses: 404 for "no such user", 500 for a failure
fn show_user(db: pg.Pool, req: roost.Request) roost.Response:
    match find_user(db, req.param_int("id", 0)):
        Ok(found):
            if found is Some(u):
                return roost.json(json.encode(u))
            return roost.status(404, "no such user")
        Err(e):
            log.error(e)                              # the details go to the log...
            return roost.status(500, "internal error")  # ...not to the client

fn main():
    db = pg.pool("postgres://app:secret@localhost/app", 16)
    r = roost.router()
    r.get("/users/{id}", fn(req) show_user(db, req))
    roost.serve(r, 8080)

users: [User] = db.fetch(...)? needs no type argument: ? passes the type the result is assigned to into the call. Where there's nothing to infer from, write it: db.fetch[User](sql, params). The functions without try_ stay as they were (fs.read returns "" and fs.error() says why), for scripts where that's all you need.

Where to look next

  • examples/web/: hello.tn, api.tn (read-only shared data), todo.tn (changing state with mut), and crash.tn (runtime errors becoming 500s).
  • The http, json, and sqlite module pages.
  • Parallel programming, for speeding up one slow request.
Source: docs/guides/web-apps.md. Every example on this page is compiled and run by tern x test docs.