Use cases · Data

Data processing in Tern

Python-like scripts for logs, CSV and JSON that run at native speed, use less memory than Go, and spread across cores without data races.

curl -fsSL https://ternlang.dev/install.sh | sh
tern new myapp && cd myapp

Filter a CSV with a regex

triage.tn
import csv
import re

fn main() Result[int, str]:
    rows = csv.parse("service,level\napi,error\nweb,info\ndb,FATAL\n")?
    bad = re.must("(?i)error|fatal")
    n = 0
    for r in rows[1..]:
        if bad.test(r[1]):
            n += 1
    print(n, "of", len(rows) - 1, "rows need attention")
    return Ok(0)
tern run triage.tn               # 2 of 3 rows need attention

Benchmarks

One threadTernRustGo
CSV, parse every row MB/s928211 csv421
Regex (?i)error|warning|fatal MB/s1,854904 regex26
Regex with capture groups MB/s2,4291,96159
gunzip MB/s2,936963 flate2623
JSON, 300k structs encode + decode172 ms, 107 MB249 ms, 194 MB serde809 ms, 303 MB
Word count hash map of strings122 ms, 40 MB166 ms, 40 MB207 ms, 199 MB

The regex engine is written in Tern and runs in time linear in the input, so a pattern from a user can’t hang your job.

Use every core

fn main():
    sums = [0] * 8
    parallel for i in 0..len(sums):
        sums[i] = i * i           # each iteration writes only its own slot
    print(sums)

Write to a shared variable from every iteration and tern check rejects it, naming the race and how to fix it.

Questions

Is Tern faster than Python for data processing?

Yes, by a wide margin: it compiles to native code and, in our benchmarks, beats Rust and Go at CSV, regex, gzip and JSON.

Does Tern have a dataframe library?

Not yet. It has fast CSV, JSON, regex, compression and hash maps in the standard library, and parallel loops for spreading work across cores.

More about Tern

Tern (ternlang) is an early release: the language will change, and so will these numbers. Benchmarks run on an Apple M4 Pro; the full tables, losses included, are on the home page.