Building web apps
This guide builds HTTP services with the http, json, and sqlite modules, from the smallest server to a JSON API backed by SQLite. It covers the shape of a handler, routing, query parameters and headers, JSON in and out, status codes, shared state (read-only and changing), the database, what happens when a handler fails, the server's limits, deploying the single binary, and testing, both by calling handlers directly and with curl.
The smallest server
import http
fn handle(req: http.Request) http.Response:
return http.text("Hello, World!")
fn main():
http.serve(8080, handle)tern run hello.tn # prints: tern: serving http://localhost:8080 on 14 threads
curl localhost:8080/anything # Hello, World!http.serve never returns. It runs an HTTP/1.1 server with an event loop on every core, keep-alive, and pipelining, and calls your handler once per request. The handler is an ordinary function: it gets a Request value and returns a Response value.
struct Request:
method: str # "GET", "POST", ...
path: str # "/users/42" (without the query string)
raw_query: str # "page=2&sort=name" (read values with req.query(name))
head: str # header lines as received (read them with req.header(name))
body: str
... # route parameters, middleware values, the connection
struct Response:
status: int
content_type: str
body: str
headers: str # extra header lines (add them with res.set_header)
upgrade: inthttp lists every field and function.
The http module has helpers that build common responses: http.text(s), http.html(s), http.json(s) (all status 200), http.status(code, s) (plain text with any status), http.not_found(), and http.redirect(url) (a 302). You can also build a Response yourself, or change the one a helper returns.
Handler shapes
A handler takes one of three forms, and the form decides how requests share data:
| Handler | Serve with | Requests run |
|---|---|---|
fn handle(req: http.Request) http.Response | http.serve(port, handle) | in parallel |
fn handle(req: http.Request, app: App) http.Response | http.serve(port, handle, app) | in parallel; app is read-only |
fn handle(req: http.Request, app: mut App) http.Response | http.serve(port, handle, &app) | one at a time |
App is any type you like: a struct of settings, a map, a sqlite.Db. The last two forms are covered in App state below.
Test handlers without a server
A handler is a plain function, so you can call it with a request you build yourself with http.new_request(method, target, body), where target is the path with its query string. This is the quickest way to test routing and responses: no port, no curl, and the output is exact.
import http
fn handle(req: http.Request) http.Response:
if req.path == "/":
return http.html("<h1>Home</h1>")
if req.path == "/health":
return http.text("ok")
return http.not_found()
fn main():
for path in ["/", "/health", "/nope"]:
res = handle(http.new_request("GET", path, ""))
print(res.status, res.content_type, res.body)200 text/html; charset=utf-8 <h1>Home</h1>
200 text/plain; charset=utf-8 ok
404 text/plain; charset=utf-8 not foundThe rest of this guide uses the same trick: every tested example below calls its handler directly.
Routing
There is no router object. A handler is a series of if checks on req.method and req.path, usually handing each route to its own function. For paths with parameters, http.match(path, pattern, ¶ms) compares a path with a pattern such as "/users/{id}/posts/{post}". On a match it returns true and fills params with the (URL-decoded) values; otherwise it returns false and leaves params alone.
import http
fn show_post(user: str, post: str) http.Response:
return http.text(f"post {post} by user {user}")
fn handle(req: http.Request) http.Response:
params: {str: str} = {}
if req.method == "GET" and req.path == "/users":
return http.json("[]")
if http.match(req.path, "/users/{id}", ¶ms):
if req.method != "GET":
return http.status(405, "method not allowed")
id = params["id"].to_int(-1)
if id < 1:
return http.status(400, "id must be a positive number")
return http.text(f"user {id}")
if http.match(req.path, "/users/{user}/posts/{post}", ¶ms):
return show_post(params["user"], params["post"])
return http.not_found()
fn main():
for p in ["/users", "/users/42", "/users/abc", "/users/7/posts/hello%20world", "/users/7/likes"]:
res = handle(http.new_request("GET", p, ""))
print(res.status, res.body)
print(handle(http.new_request("DELETE", "/users/42", "")).status)200 []
200 user 42
400 id must be a positive number
200 post hello world by user 7
404 not found
405http.match compares whole segments: /users/{id} doesn't match /users/7/likes, and a trailing slash makes a different path. Parameters are always strings; convert them with to_int and reject bad values with a 400.
Query parameters and headers
req.query(name) returns the decoded value of one query parameter, or "" when it is missing. req.header(name) looks up a request header, ignoring case, and also returns "" when it is missing; req.cookie(name) reads one cookie. To add a response header, call res.set_header(name, value); res.get_header(name) reads one back.
import http
fn search(req: http.Request) http.Response:
q = req.query("q")
if q == "":
return http.status(400, "missing ?q=")
page = req.query("page").to_int(1)
lang = req.header("accept-language")
res = http.text(f"searching for [{q}], page {page}, language [{lang}]")
res.set_header("Cache-Control", "no-store")
return res
fn main():
req = http.new_request("GET", "/search?q=tern+lang%21&page=3", "")
req.add_header("Accept-Language", "en-US")
res = search(req)
print(res.status, res.body, res.get_header("cache-control"))
print(search(http.new_request("GET", "/search", "")).body)
old = http.redirect("/new-home")
print(old.status, old.get_header("location"))200 searching for [tern lang!], page 3, language [en-US] no-store
missing ?q=
302 /new-home+ and %21 in the query are decoded. The raw query string is in req.raw_query if you need to handle repeated parameters yourself, and req.queries() and req.headers() return them all as maps.
JSON in and out
json.encode turns any value into JSON text, and json.decode(text, &value) fills a value from JSON. Decoding is all-or-nothing: if the body isn't valid JSON of the right shape, it returns false and changes nothing, which maps directly to a 400. Keys missing from the body keep the value you started with, so start from sensible defaults, and validate required fields yourself.
import http
import json
struct Signup:
email: str
name: str
newsletter: bool
struct Created:
id: int
email: str
fn signup(req: http.Request) http.Response:
if req.method != "POST":
return http.status(405, "use POST")
input = Signup("", "", false)
if not json.decode(req.body, &input):
return http.status(400, `body must be JSON like {"email": "...", "name": "..."}`)
if not input.email.contains("@"):
return http.status(422, "email is required")
res = http.json(json.encode(Created(1001, input.email)))
res.status = 201
res.set_header("Location", "/users/1001")
return res
fn main():
for body in [`{"email": "ada@example.com", "name": "Ada"}`, `{"name": "no email"}`, "email=ada"]:
res = signup(http.new_request("POST", "/signup", body))
print(res.status, res.content_type, res.body)201 application/json {"id":1001,"email":"ada@example.com"}
422 text/plain; charset=utf-8 email is required
400 text/plain; charset=utf-8 body must be JSON like {"email": "...", "name": "..."}Unknown keys in the body are ignored, and a JSON null keeps the default. For output, encode a struct or an array of structs; field names become the JSON keys. If you need a key that isn't a valid field name, build the text by hand with json_quote(s), which returns s as a quoted, escaped JSON string (see examples/web/api.tn).
Status codes
Use http.status(code, message) for any status with a plain-text body, or set res.status on a response built by another helper. The conventions the examples here follow:
| Status | When |
|---|---|
| 200 | success (http.text, http.html, http.json) |
| 201 | created something; add a Location header |
| 204 | success with no body: http.status(204, "") |
| 302 | http.redirect(url) |
| 400 | the request is malformed (bad JSON, a non-numeric id) |
| 404 | http.not_found(), or http.status(404, "no note 7") |
| 405 | the path exists but not with this method |
| 422 | well-formed but invalid (a required field is empty) |
| 500 | sent for you when a handler hits a runtime error |
Any number works as a status. The server writes the standard reason phrase ("Not Found") for the common codes above and the generic word Status for the rest (for example HTTP/1.1 429 Status); clients go by the number.
App state: read-only or mut
Most services need data that outlives one request: configuration, a cache, a database handle. Pass it as the handler's second parameter.
Read-only state (app: App) is shared by every server thread at once. It can't be changed after the server starts, so requests never wait on each other:
import http
struct App:
greeting: str
motd: {str: str}
fn handle(req: http.Request, app: App) http.Response:
lang = req.query("lang")
return http.text(app.greeting + " " + app.motd.get(lang, "(no message)"))
fn main():
app = App("hello!", {"en": "Ship it.", "fr": "Expediez-le."})
http.serve(8080, handle, app)Changing state (app: mut App, served with &app) runs requests one at a time: the server holds a lock around each call to the handler. That is the whole concurrency story for shared state: no data races are possible, and there is no lock to forget.
import http
import json
struct App:
visits: {str: int}
total: int
fn handle(req: http.Request, app: mut App) http.Response:
if req.method == "POST" and req.path.starts_with("/visit/"):
page = req.path[len("/visit/")..]
app.visits[page] = app.visits.get(page, 0) + 1
app.total += 1
return http.status(204, "")
if req.path == "/stats":
return http.text(f"{app.total} visits: {json.encode(app.visits)}")
return http.not_found()
fn main():
app = App({}, 0)
for page in ["home", "docs", "home"]:
handle(http.new_request("POST", "/visit/" + page, ""), &app)
print(handle(http.new_request("GET", "/stats", ""), &app).body)
# a real server: http.serve(8080, handle, &app)3 visits: {"home":2,"docs":1}Why the two shapes? Tern values are never shared by reference, and a read-only parameter can't change, so any number of threads can read the same App safely. A mut parameter can change, so the server must never run two such calls at once. The type of the handler tells the server which case it is in.
The trade-off: with mut state, a slow request holds up every other request. Keep the changing part small. Put read-mostly data in a read-only App, and put data that many requests write into the database, which handles concurrent writers itself.
SQLite
import sqlite links the system SQLite. sqlite.open(path) opens (or creates) a database file. Then:
d.exec(sql, params) intruns a statement and returns the last inserted row id.d.query(sql, params) [{str: str}]returns rows as maps from column name to text.d.query_into(sql, params, &out) intreads rows into your structs, matching columns to fields by name, and returns the number of rows.outcan be an array of structs (it is replaced by the rows) or one struct (filled from the first row, if there is one).
Parameters are always a [str] and are bound with ? placeholders, never pasted into the SQL, so user input can't inject SQL. SQLite converts them to the column's type, so [str(id)] works for an INTEGER column.
import sqlite
import os
struct Note:
id: int
title: str
pinned: bool
fn main():
d = sqlite.open(os.env("DBPATH", "notes.db"))
d.exec("CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, title TEXT NOT NULL, pinned INTEGER NOT NULL)", [])
first = d.exec("INSERT INTO notes (title, pinned) VALUES (?, ?)", ["groceries", "0"])
d.exec("INSERT INTO notes (title, pinned) VALUES (?, ?)", ["it's a trap'; DROP TABLE notes; --", "1"])
print("first id", first)
notes: [Note] = []
n = d.query_into("SELECT id, title, pinned FROM notes ORDER BY id", [], ¬es)
print(n, notes)
one = Note(0, "", false)
if d.query_into("SELECT id, title, pinned FROM notes WHERE id = ?", [str(first)], &one) > 0:
print("found", one.title)
missing = Note(0, "none", false)
print(d.query_into("SELECT id, title, pinned FROM notes WHERE id = ?", ["999"], &missing), missing.title)
rows = d.query("SELECT pinned, count(*) AS n FROM notes GROUP BY pinned", [])
print(rows)first id 1
2 [Note{id: 1, title: "groceries", pinned: false}, Note{id: 2, title: "it's a trap'; DROP TABLE notes; --", pinned: true}]
found groceries
0 none
[{"pinned": "0", "n": "1"}, {"pinned": "1", "n": "1"}]Prefer query_into with a struct: the fields are typed, and the struct encodes straight to JSON. query is handy for ad-hoc results like the count(*) above; every value in it is text.
(The test suite runs this with DBPATH=:memory:, an in-memory database. Don't do that for a server; see below.)
The database in a server
A sqlite.Db is a small handle, so put it in your app state (or pass it as the app itself). A read-only d: sqlite.Db parameter can still run INSERTs: the data lives in the database file, not in a Tern value, and "read-only" is about Tern values. So the usual shape is a read-only app, with requests running in parallel and SQLite coordinating the writes.
What the sqlite package does for you:
- Each statement runs on a pooled connection with its own prepared-statement cache, so requests query in parallel. The file is put in WAL mode, which lets readers and a writer work at the same time. A statement waits up to 5 seconds for a lock (only its task waits).
- Group commit. A single write per request joins the writes other requests make at the same moment: one transaction, one commit, and each write still returns only once it is saved.
- Use a file, not
:memory:. Each connection to:memory:is a different, empty database, so a server's requests wouldn't see each other's data.
A complete API
Here is a notes API with everything above: routing with parameters, JSON in and out, status codes, and SQLite. It reads its port and database path from the environment.
import http
import sqlite
import json
import os
struct Note:
id: int
title: str
body: str
struct NewNote:
title: str
body: str
fn list_notes(d: sqlite.Db) http.Response:
notes: [Note] = []
d.query_into("SELECT id, title, body FROM notes ORDER BY id", [], ¬es)
return http.json(json.encode(notes))
fn show_note(d: sqlite.Db, id: str) http.Response:
n = Note(0, "", "")
if d.query_into("SELECT id, title, body FROM notes WHERE id = ?", [id], &n) == 0:
return http.status(404, f"no note {id}")
return http.json(json.encode(n))
fn create_note(d: sqlite.Db, req: http.Request) http.Response:
input = NewNote("", "")
if not json.decode(req.body, &input):
return http.status(400, `body must be JSON: {"title": ..., "body": ...}`)
if input.title.trim() == "":
return http.status(422, "title is required")
id = d.exec("INSERT INTO notes (title, body) VALUES (?, ?)", [input.title, input.body])
res = http.json(json.encode(Note(id, input.title, input.body)))
res.status = 201
res.set_header("Location", f"/notes/{id}")
return res
fn handle(req: http.Request, d: sqlite.Db) http.Response:
params: {str: str} = {}
if req.path == "/notes":
if req.method == "GET":
return list_notes(d)
if req.method == "POST":
return create_note(d, req)
return http.status(405, "method not allowed")
if http.match(req.path, "/notes/{id}", ¶ms):
if req.method == "DELETE":
d.exec("DELETE FROM notes WHERE id = ?", [params["id"]])
return http.status(204, "")
return show_note(d, params["id"])
if req.path == "/search":
q = req.query("q")
rows = d.query("SELECT id, title FROM notes WHERE title LIKE ?", ["%" + q + "%"])
return http.json(json.encode(rows))
if req.path == "/broken":
rows = d.query("SELECT no_such_column FROM notes", [])
return http.text(str(len(rows)))
return http.not_found()
fn main():
d = sqlite.open(os.env("DBPATH", "notes.db"))
d.exec("CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, title TEXT NOT NULL, body TEXT NOT NULL)", [])
port = os.env("PORT", "8080").to_int(8080)
http.serve(port, handle, d)Testing with curl
Build it, start it on a free port, and try each route. This is a real session with the program above (-i shows the status line and headers):
$ tern build notes.tn -o notes
$ PORT=8090 ./notes &
tern: serving http://localhost:8090 on 14 threads
$ curl -i -X POST localhost:8090/notes -d '{"title": "groceries", "body": "eggs, milk"}'
HTTP/1.1 201 Created
Content-Type: application/json
Location: /notes/1
Content-Length: 48
{"id":1,"title":"groceries","body":"eggs, milk"}
$ curl -X POST localhost:8090/notes -d '{"title": "ideas", "body": "write docs"}'
{"id":2,"title":"ideas","body":"write docs"}
$ curl localhost:8090/notes
[{"id":1,"title":"groceries","body":"eggs, milk"},{"id":2,"title":"ideas","body":"write docs"}]
$ curl -i localhost:8090/notes/99
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
Content-Length: 10
no note 99
$ curl -i -X POST localhost:8090/notes -d 'not json'
HTTP/1.1 400 Bad Request
Content-Type: text/plain; charset=utf-8
Content-Length: 46
body must be JSON: {"title": ..., "body": ...}
$ curl 'localhost:8090/search?q=gro'
[{"id":"1","title":"groceries"}]
$ curl -i -X DELETE localhost:8090/notes/1
HTTP/1.1 204 No Content
Content-Type: text/plain; charset=utf-8
Content-Length: 0
$ curl -i localhost:8090/broken
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
Content-Length: 21
internal server errorNote that /search returns ids as strings ("id":"1"): d.query returns every value as text. Use query_into when the JSON types matter.
When a handler fails
A runtime error inside a handler (a missing map key, an index out of range, division by zero, a failed SQL statement, even inside a parallel block in the handler) doesn't stop the server. That request gets a 500 with the body internal server error, the error is logged to stderr, and the server keeps serving. The /broken route above logged:
tern: GET /broken failed: runtime error: sqlite: no such column: no_such_columnThe failed request's open database snapshot is closed, and with mut state, the lock is released. What the handler changed in a mut app before the error stays changed, so validate input before you start changing state.
Don't rely on 500s for expected problems. Bad input is the client's error: check it and return a 400 or 422 with a message, as the examples do. Use m.get(k, default) or k in m instead of m[k] for keys that come from requests, and to_int(-1) with a range check for numbers.
Timeouts and limits
The server enforces a few limits on its own. Change them with environment variables when starting the binary:
| Limit | Default | Environment variable |
|---|---|---|
| Server threads | one per core | TERN_HTTP_THREADS |
| Idle keep-alive connections closed after | 60 seconds | TERN_HTTP_IDLE_SEC |
| A request must arrive completely within | 10 seconds | TERN_HTTP_HEADER_SEC |
Largest request body (larger gets 413 Payload Too Large) | 16 MB | TERN_HTTP_MAX_BODY_MB |
Largest header block (larger gets 400 Bad Request) | 64 KB | |
| Headers per request (the rest are ignored) | 64 |
There is no time limit on the handler itself. Each server thread runs an event loop for many connections, and a handler runs on that thread until it returns, so a handler that takes two seconds delays the other connections on its thread for two seconds. Keep handlers fast; do slow work with a time budget of your own.
Other gaps in http.serve to know about: no TLS, no HTTP/2, no chunked request bodies (clients must send Content-Length, which curl and browsers do for ordinary requests), and no WebSockets. The engine (http.new_server), and roost on top of it, have HTTPS (with tls), streamed and chunked bodies, WebSockets and server-sent events.
Building and deploying
A Tern web app builds to one native binary:
tern build notes.tn -o notesIt depends only on the C library and, if you import sqlite, the system SQLite (libsqlite3). Copy it to a server built for the same OS and CPU (build on Linux for Linux) and run it. It binds to the port on all network interfaces, and it logs to stderr.
TLS belongs in a reverse proxy in front of the binary. With Caddy, which gets certificates automatically, the whole configuration is:
notes.example.com {
reverse_proxy localhost:8090
}With nginx, proxy to http://127.0.0.1:8090 and set a client_max_body_size that matches TERN_HTTP_MAX_BODY_MB. Since the proxy faces the internet, firewall the app's own port so only the proxy can reach it.
To keep it running, use your system's service manager. A systemd unit:
[Unit]
Description=notes API
After=network.target
[Service]
ExecStart=/opt/notes/notes
Environment=PORT=8090 DBPATH=/var/lib/notes/notes.db
Restart=always
User=notes
[Install]
WantedBy=multi-user.targetBack up the database with SQLite's own tools (sqlite3 notes.db ".backup backup.db"), which work while the server is running.
Errors with Result: web, pg and ?
The roost framework and the pg, redis, fs, tls and auth packages also return Result values (the try_ functions, and pg.fetch), so a handler can write the happy path straight down and turn every failure into a response in one place. Errors are strings that say which module failed, what, and why: "pg: query failed: ERROR 42P01: relation \"users\" does not exist".
import json
import log
import pg
import roost
struct User:
id: int
name: str
email: str
# a helper that returns Result: `?` hands an Err back to the caller as it is
fn find_user(db: pg.Pool, id: int) Result[Option[User], str]:
return db.fetch_one[User]("SELECT id, name, email FROM users WHERE id = $1", [str(id)])
fn team(db: pg.Pool, name: str) Result[[User], str]:
users: [User] = db.fetch("SELECT id, name, email FROM users WHERE team = $1 ORDER BY id", [name])?
return Ok(users)
# the handler maps the outcomes to responses: 404 for "no such user", 500 for a failure
fn show_user(db: pg.Pool, req: roost.Request) roost.Response:
match find_user(db, req.param_int("id", 0)):
Ok(found):
if found is Some(u):
return roost.json(json.encode(u))
return roost.status(404, "no such user")
Err(e):
log.error(e) # the details go to the log...
return roost.status(500, "internal error") # ...not to the client
fn main():
db = pg.pool("postgres://app:secret@localhost/app", 16)
r = roost.router()
r.get("/users/{id}", fn(req) show_user(db, req))
roost.serve(r, 8080)users: [User] = db.fetch(...)? needs no type argument: ? passes the type the result is assigned to into the call. Where there's nothing to infer from, write it: db.fetch[User](sql, params). The functions without try_ stay as they were (fs.read returns "" and fs.error() says why), for scripts where that's all you need.
Where to look next
examples/web/:hello.tn,api.tn(read-only shared data),todo.tn(changing state withmut), andcrash.tn(runtime errors becoming 500s).- The http, json, and sqlite module pages.
- Parallel programming, for speeding up one slow request.