nimler

Nimler is a library for authoring Erlang and Elixir NIFs in the nim programming language. It has mostly complete bindings for the Erlang NIF API and some accessories for making writing NIFs easier, including idiomatic functions for converting between Erlang terms and nim types, and simplifications for using resource objects.

Mostly, Nimler is a minimal, zero-dependency wrapper for Erlang NIF API.

Target	Status
x86_64-linux
arm64-linux

Sample

Positional API sample:

import nimler

using
  env: ptr ErlNifEnv

func add(env; a: int, b: int): (ErlAtom, int) {.xnif.} =
  (AtomOk, a + b)
  
func sub(env; a: int, b: int): (ErlAtom, int) {.xnif.} =
  (AtomOk, a - b)

func mul(env; a: int, b: int): (ErlAtom, int) {.xnif.} =
  (AtomOk, a * b)
  
exportNifs "Elixir.NifMath", [ add, sub, mul ]

Raw API

If porting a NIF from C, it may be easier to start with the raw API, which retains NIF signature:

import nimler

using
  env: ptr ErlNifEnv
  argc: cint
  argv: ptr UncheckedArray[ErlNifTerm]

func add(env, argc, argv): ErlNifTerm {.nif, arity: 2.} =
  # `fromTerm(env, term, to_type)` returns a nim Option.
  let a1 = fromTerm(env, argv[0], int).get(0)
  let a2 = fromTerm(env, argv[1], int).get(0)
  return toTerm(env, a1 + a2)
  
# Specify external NIF name. In this case, the function is exported as "sub"
# rather than "subnums" 
func subnums(env, argc, argv): ErlNifTerm {.nif, arity: 2, name: "sub".} =
  let a1 = fromTerm(env, argv[0], int).get(0)
  let a2 = fromTerm(env, argv[1], int).get(0)
  return toTerm(env, a1 - a2)

# This NIF is tagged with `dirty_cpu`. See the documentation for details on
# using dirty schedulers: https://erlang.org/doc/man/erl_nif.html#functionality
func mul(env, argc, argv): ErlNifTerm {.nif, arity: 2, dirty_cpu.} =
  let a1 = fromTerm(env, argv[0], int).get(0)
  let a2 = fromTerm(env, argv[1], int).get(1)
  return toTerm(env, a1 * a2)

# Optional `{.raises: [].}` pragma is part of nim's effect system. This verifies
# that the NIF does not raise an exception
func div(env, argc, argv): ErlNifTerm {.nif, arity: 2, raises: [].} =
  let a1 = fromTerm(env, argv[0], int).get(0)
  let a2 = fromTerm(env, argv[1], int)

  if a2 == some(0.int):
    return enif_make_badarg(env)

  return toTerm(env, a1 div a2.unsafeGet())
  
exportNifs("Elixir.NifMath", [
  add,
  subnums,
  mul,
  div
])

Installation

Supported platforms

Continuous integration is run on x86-64 and arm64 linux. Other platforms are not currently tested, although MacOS likely works. Windows likely does not work.

Prerequisites

Nim v1.2.0+. See: installation guide
Erlang/OTP
- Ubuntu: apt-get install erlang-dev
- MacOS: brew install erlang

Installing nimler

Nimler can be installed via nim package manager nimble:

$ nimble install nimler

Using nimler_mix

Github: nimler_mix

Nim and nimler must first be installed. See installation.

Add nimler to mix.exs and run mix deps.get to install nimler from hex.pm

  def deps() do
    [{:nimler, "~> 0.1.1"}]
  end

mix nimler.new to generate scaffold NIF project
mix compile.nimler to compile NIF with nimler

mix nimler.new

Generate basic nimler NIF

Defaults

lib/native is default NIF root

lib/native/nif.nim is default NIF file

lib/native/nim.cfg is default NIF nim configuration. This will be used during compilation. See priv/templates/nim.cfg for current nim.cfg template

mix compile.nimler

Compile NIFs in lib/native

Nimler generates lib/native/nif_wrapper.ex by default

Configuration sample

  def project do
    [
      app: :myproject,
      version: "0.1.0",
      elixir: "~> 1.10",
      start_permanent: Mix.env() == :prod,
      deps: deps(),

      # add the nimler compiler
      compilers: Mix.compilers ++ [:nimler],

      # add optional nimler_config
      nimler_config: nimler_config()
    ]
  end

  def nimler_config() do
    [
      # compile_mode can be one of :debug, :release, :danger
      compile_mode: :debug,

      # compile_flags are passed directly to nim compiler
      # see [priv/templates/nim.cfg](priv/templates/nim.cfg) for default nim cfg
      compile_flags: [
        "--verbosity:2"
      ]
    ]
  end

  def deps() do
    [{:nimler, "~> 0.1.1"}]
  end

Testing

Nimler tests are Elixir NIFs. Elixir is required to run tests.

$ git clone git@github.com:wltsmrz/nimler.git
$ cd nimler
$ nimble build_all # build all test NIFs
$ nimble test_all # run tests

Build status

Automated tests are run via github actions on Ubuntu x64 and arm64.

Target	Status
x86_64-linux
arm64-linux

Nim compile flags

As NIFs are shared libraries, the minimum required configuration is --app:lib. Automated tests are run with the following nim.cfg:

define:nimlerGenWrapperForce
define:nimlerWrapperFilename="NimlerWrapper.ex"
define:release
verbosity:0
app:lib
gc:arc
define:forceBuild
define:noMain
define:noSignalHandler 
opt:speed
stackTrace:off
lineTrace:off
panics:on
warning[GcUnsafe]:off
hint[User]:off
hint[Exec]:off
hint[Link]:off
hint[SuccessX]:off
hint[Conf]:off
hint[Processing]:off
hint[GCStats]:off
hint[GlobalVar]:off

Examples

Basic

Basic example to create add(a, b) NIF. We will use nimler to automatically generate Elixir wrapper module, then test via elixir CLI.

import nimler
import nimler/codec

func add(env: ptr ErlNifEnv, a1: int, a2: int): int {.xnif.} =
  return a1 + a2

exportNifs "Elixir.NumberAdder", [ add ]

Compile nim c --app:lib -d:nimlerGenWrapper nif.nim. Produces libnif.so and NumberAdder.ex
Run elixir -r NumberAdder.ex -e "IO.inspect %{add: NumberAdder.add(1, 2)}"

Note: exportNifs() is a compile-time template that exports an ErlNifEntry to be loaded by Erlang or Elixir

Nimler tests

The tests for nimler itself are Elixir NIFs.

tests/integration erl_nif.h binding coverage
tests/codec conversion between Erlang terms and nim types
tests/positional positional API tests
tests/message message passing to Elixir process
tests/timeslice NIF yielding with enif_schedule_nif

Developer guide

Erlang term conversion

Erlang/Elixir terms are represented in nimler with the opaque type ErlNifTerm. Nimler exposes functions for converting between nim types and Erlang terms.

fromTerm()

fromTerm() produces nim type from ErlNifTerm. fromTerm() returns an Option.

let i_option = env.fromTerm(term, int32)

if i_option.isNone():
  # The term was not successfully read into an int32
else:
  let i = i_option.get()

# Default value of 0 if term is not read successfully
let ii = env.fromTerm(term, int32).get(0)

toTerm()

toTerm() produces ErlNifTerm from nim type.

let term = env.toTerm(10)
let other_term = env.toTerm(10'i32)

Supported Types

Erlang	Elixir	Nimler
Integer	Integer	int, int32, int64, uint, uint32, uint64
Float	Float	float
Atom	Atom	distinct string
Atom	Atom	bool
String	Charlist	seq[char]
Bitstring	String	string
Binary	Binary	seq[byte]
List	List	seq[T]
Tuple	Tuple	tuple
Map	Map	Table[A, B]
PID	PID	ErlPid
Term	Term	ErlTerm

Atoms

let term = env.toTerm(ErlAtom("test"))
# :test

let atom = env.fromTerm(term, ErlAtom).get()
# ErlAtom("test")

Note: The following atom constants are exported from nimler. Note that these are not yet converted to ErlNifTerm:

AtomOk = ErlAtom("ok")

AtomError = ErlAtom("error")

AtomTrue = ErlAtom("true")

AtomFalse = ErlAtom("false")

Booleans

let term = env.toTerm(true)
# :true

let atom = env.fromTerm(term, bool).get()
# true

Charlists

let term = env.toTerm(@"test")
# 'test'

let lst = env.fromTerm(term, seq[char]).get()
# @['t', 'e', 's', 't']

Strings

Strings follow the Elixir pattern of using Erlang bitstring rather than charlists

let term = env.toTerm("test")
# "test"

let str = env.fromTerm(term, string).get()
# "test"

Binaries

let term = env.toTerm(toOpenArrayByte("test", 0, 3))
# <<116, 101, 115, 116>>

let bin = env.fromTerm(term, seq[byte]).get()
# @[116, 101, 115, 116]

Lists

Elements of seq must be of the same type.

let term = env.toTerm(@[1,2,3])
# [1,2,3]

let lst = env.fromTerm(term, seq[int]).get()
# @[1,2,3]

Tuples

Tuples in nim may contain mixed types.

let term = env.toTerm(("test",1,3.14))
# {"test",1,3.14}

let (a,b,c) = env.fromTerm(term, (string, int, float)).get()
# a="test"
# b=1
# c=3.14

Keyword lists

Keyword lists (lists of {Atom, }) are represented with the type ErlKeywords. Passing generic object types to fromTerm() or toTerm() also implies keyword list.

type MyObj = object
  a: int
  b: float
  c: string

let term = toTerm(env, MyObj(a: 1, b: 1.1, c: "test"))
# keyword list: [a: 1, b: 1.1, c: <<"test">>]

let o = fromTerm(env, term, MyObj).get()
# MyObj{a,b,c}

Maps

Maps are represented in nimler with the Table[K, V] type.

import tables

var t = initTable[string, int](4)
t["a"] = 1
t["b"] = 2

let term = env.toTerm(t)
# %{"a" => 1, "b" => 2}

let tt = env.fromTerm(term, Table[string, int]).get()
# {"a": 1, "b": 2}

Opaque terms

Sometimes specifying opaque ErlTerm type is convenient. Example with the positional nimler API:

func addThing(env: ptr ErlNifEnv, myArg: ErlTerm): ErlTerm {.xnif.} =
  # fromTerm(env, myArg, int).get()
  # fromTerm(env, myArg, string).get()
  # fromTerm(env, myArg, seq[byte]).get()

PIDs

Process IDs are represented in nimler with the ErlPid type.

let pid = fromTerm(env, term, ErlPid).get()

Results

Result tuples have a first element either :ok or :error. Nimler functions env.ok() and env.error() accept varargs.

let ok_term = env.ok(env.toTerm(1), env.toTerm(2))
# {:ok, 1, 2}

let err_term = env.error(env.toTerm("Bad thing"))
# {:error, "Bad thing"}

Generating wrapper module

Nimler can generate Erlang or Elixir wrapper at compile time.

Relevant nim compile flags:

-d:nimlerGenWrapper - generate elixir wrapper

-d:nimlerGenWrapperForce - potentially overwrite existing wrapper of the same

-d:nimlerWrapperRoot=/myroot - absolute root directory of the generated wrapper

-d:nimlerWrapperFilename=Wrapper.ex - file name of the generated wrapper

-d:nimlerWrapperLoadInfo={a: 123} - module load info

-d:nimlerWrapperType=elixir - possible values erlang, elixir. Note: to generate erlang wrappers, also remove any Elixir.xxxx prefixes in the call to exportNifs() template

The name of the target Elixir module, and its filename, is based on the module name as exported from nimler and not the name of the .nim source file. export_nifs("Elixir.NumberAdder", ... will produce module named NumberAdder.

The target directory is the same as the .nim source. i.e., -o nim compile flag affects the destination of generated Elixir module. Override with -d:nimlerWrapperRoot.

Erlang/OTP versioning

Erlang/Elixir NIFs are shared libraries that depend on erl_nif.h. Nimler automatically detects installed Erlang/OTP, and tries to produce bindings that are compatible with the Erlang NIF API version detected at compile time.

Using an unsupported function will err during compilation:

Error: enif_term_type() not supported in target NIF version: 2.10.
Requires at least version 2.15.

Target specific erl_nif version

To target a specific version of erl_nif API, compile with: --define:nif_version="x.y".