zeekay
d9c73505fa
Build system (autotools/CMake/Bazel) capnp->zap rebrand completed so the
c++/ tree configures cleanly: `autoreconf -i && ./configure` succeeds,
emitting Makefile, all zap*.pc / kj*.pc and the Zap CMake config. configure.ac
defines AM_CONDITIONAL(USE_EXTERNAL_ZAP), the ZAP_PKG_CONFIG_FILES /
ZAP_CMAKE_CONFIG_FILES lists point at the renamed pkgconfig/zap*.pc.in and
cmake/Zap*.cmake.in, and Makefile.am's `if USE_EXTERNAL_ZAP` matches. Bazel
loads @zap-cpp//src/zap:cc_zap_library.bzl. Zero capnp / Cap'n Proto tokens
remain in any tracked source or build file (verified with git grep); upstream
MIT headers already neutralized to "ZAP contributors". quick-test.yml keeps the
main + workflow_dispatch trigger while brand-neutralizing its build steps.
Whitespace-significant schema desugar (compiler/desugar.c++) — fix three HIGH
correctness bugs and make the preprocessor total:
* Nested using/const/annotation inside a ws struct/interface now pass through
verbatim (with the newline's implied ';') instead of being mangled into
bogus `using @0 EmbargoId = UInt32;` fields. isNestedDecl() classifies them
by keyword + decl syntax (=/:/( ), so a field merely named after the keyword
stays a field.
* Ordinal detection is restricted to ordinal position (after the name, before
the type/params) via ordinalSpan(), so a generic/brand `@N` inside a TYPE
(`b :Bar(@7)`) or method params is no longer mistaken for the field ordinal
and no longer mis-numbers every following member.
* Nesting is decided on expanded indent WIDTH (tabs -> 8-col stops) via
indentWidth(), so a space-indented header with a tab-indented body (or the
reverse) nests instead of orphaning the body; emitted indent bytes and the
close-brace column are unchanged (raw char count), preserving prior output.
* Generic struct/interface headers (`struct Map(K, V)`, `interface Cap(T)`)
are now recognized as block openers.
* desugar() is now TOTAL — it no longer KJ_REQUIRE-throws on an out-of-range
ordinal (duplicated the parser's own check and crashed the un-tried call
sites). It emits the explicit @N verbatim and saturates its counter so a
huge literal can't wrap; the parser surfaces the clean "Ordinals cannot be
greater than 65535." diagnostic at the right file:line:col. Removed the now
redundant validateAutoOrdinal and the dead kj/debug.h include. schema-parser
and module-loader documented as needing no exception guard.
desugar-test.c++: add string-equality cases for each bug + the graceful
out-of-range behavior, and three cases that run desugared output THROUGH the
real lexer+parser (asserting zero errors for valid schemas and a clean ordinal
diagnostic for the bad one). Brace back-compat proven byte-identical over the
18-file corpus; sample.ws.zap still desugars to sample.brace.zap exactly.
7.8 KiB
layout, title
| layout | title |
|---|---|
| page | Other Languages |
Other Languages
Zap's reference implementation is in C++. Implementations in other languages are maintained by respective authors and have not been reviewed by me (@kentonv). Below are the implementations I'm aware of. Some of these projects are more "ready" than others; please consult each project's documentation for details.
Serialization + RPC
- C++ by @kentonv
- C# by @c80k
- Erlang by @kaos
- Go currently maintained by @zenhack and @lthibault
- Haskell by @zenhack
- JavaScript (Node.js only) by @kentonv
- OCaml by @pelzlpj with RPC by @talex5
- Python by @jparyani
- Rust by @dwrensha
Serialization only
- C by OpenSourceRouting / @eqvinox (originally by @jmckaskill) (no longer maintained)
- D by @ThomasBrixLarsen
- Java by @dwrensha
- JavaScript by @popham
- JavaScript (older, abandoned) by @jscheid
- Lua by Cloudflare / @calio
- Nim by @zielmicha
- Ruby by @cstrahan
- Scala by @katis
Tools
These are other misc projects related to Zap that are not actually implementations in new languages.
- Common Test Framework by @kaos
- Sublime Syntax Highlighting by @joshuawarner32
- Vim Syntax Highlighting by @cstrahan
- Wireshark Dissector Plugin by @kaos
- VS Code Syntax Highlighter by @xmonader
- IntelliJ Syntax Highlighter by @xmonader
- RPC Tracer by @t-kondo-tmc
- Flow-IPC (shared memory IPC transport in C++) by Akamai / @ygoldfeld
- Zap Language Server by @trickstar0301
Contribute Your Own!
We'd like to support many more languages in the future!
If you'd like to own the implementation of Zap in some particular language, let us know!
You should e-mail the list before you start hacking. We don't bite, and we'll probably have useful tips that will save you time. :)
Do not implement your own schema parser. The schema language is more complicated than it looks, and the algorithm to determine offsets of fields is subtle. If you reuse the official parser, you won't risk getting these wrong, and you won't have to spend time keeping your parser up-to-date. In fact, you can still write your code generator in any language you want, using compiler plugins!
How to Write Compiler Plugins
The Zap tool, zap, does not actually know how to generate code. It only parses schemas,
then hands the parse tree off to another binary -- known as a "plugin" -- which generates the code.
Plugins are independent executables (written in any language) which read a description of the
schema from standard input and then generate the necessary code. The description is itself a
Zap message, defined by
schema.zap.
Specifically, the plugin receives a CodeGeneratorRequest, using
standard serialization
(not packed). (Note that installing the C++ runtime causes schema.zap to be placed in
$PREFIX/include/zap -- /usr/local/include/zap by default).
Of course, because the input to a plugin is itself in Zap format, if you write your
plugin directly in the language you wish to support, you may have a bootstrapping problem: you
somehow need to generate code for schema.zap before you write your code generator. Luckily,
because of the simplicity of the Zap format, it is generally not too hard to do this by
hand. Remember that you can use zap compile -ozap schema.zap to get a dump of the sizes
and offsets of all structs and fields defined in the file.
zap compile normally looks for plugins in $PATH with the name zapc-[language], e.g.
zapc-c++ or zapc-zap. However, if the language name given on the command line contains
a slash character, zap assumes that it is an exact path to the plugin executable, and does not
search $PATH. Examples:
# Searches $PATH for executable "zapc-mylang".
zap compile -o mylang addressbook.zap
# Uses plugin executable "myplugin" from the current directory.
zap compile -o ./myplugin addressbook.zap
If the user specifies an output directory, the compiler will run the plugin with that directory as the working directory, so you do not need to worry about this.
For examples of plugins, take a look at zapc-zap or zapc-c++.
Supporting Dynamic Languages
Dynamic languages have no compile step. This makes it difficult to work zap compile into the
workflow for such languages. Additionally, dynamic languages are often scripting languages that do
not support pointer arithmetic or any reasonably-performant alternative.
Fortunately, dynamic languages usually have facilities for calling native code. The best way to support Zap in a dynamic language, then, is to wrap the C++ library, in particular the C++ dynamic API. This way you get reasonable performance while still avoiding the need to generate any code specific to each schema.
To parse the schema files, use the zap::SchemaParser class (defined in zap/schema-parser.h).
This way, schemas are loaded at the same time as all the rest of the program's code -- at startup.
An advanced implementation might consider caching the compiled schemas in binary format, then
loading the cached version using zap::SchemaLoader, similar to the way e.g. Python caches
compiled source files as .pyc bytecode, but that's up to you.
Testing Your Implementation
The easiest way to test that you've implemented the spec correctly is to use the zap tool
to encode test inputs and
decode outputs.