net.clojars.savya/tokenizers-clj

Idiomatic Clojure wrapper over DJL's HuggingFace tokenizers (native Rust tokenizers via JNI): encode, decode, and count tokens.

Leiningen/Boot

[net.clojars.savya/tokenizers-clj "0.1.0"]

net.clojars.savya/tokenizers-clj {:mvn/version "0.1.0"}

implementation("net.clojars.savya:tokenizers-clj:0.1.0")

<dependency>
  <groupId>net.clojars.savya</groupId>
  <artifactId>tokenizers-clj</artifactId>
  <version>0.1.0</version>
</dependency>

Pushed by
savya on Jun 23, 2026 with this git tree
Recent Versions
- 0.1.0
Dependencies
- ai.djl.huggingface/tokenizers 0.33.0
Homepage
https://github.com/jsavyasachi/tokenizers-clj
License
- Eclipse Public License 1.0
Version Badge
Want to display the latest version of your project on GitHub? Use the markdown code below!
[![Clojars Project](https://img.shields.io/clojars/v/net.clojars.savya/tokenizers-clj.svg)](https://clojars.org/net.clojars.savya/tokenizers-clj)
If you want to include pre-releases and snapshots, use the following markdown code:
[![Clojars Project](https://img.shields.io/clojars/v/net.clojars.savya/tokenizers-clj.svg?include_prereleases)](https://clojars.org/net.clojars.savya/tokenizers-clj)