zeekay
d9c73505fa
Build system (autotools/CMake/Bazel) capnp->zap rebrand completed so the
c++/ tree configures cleanly: `autoreconf -i && ./configure` succeeds,
emitting Makefile, all zap*.pc / kj*.pc and the Zap CMake config. configure.ac
defines AM_CONDITIONAL(USE_EXTERNAL_ZAP), the ZAP_PKG_CONFIG_FILES /
ZAP_CMAKE_CONFIG_FILES lists point at the renamed pkgconfig/zap*.pc.in and
cmake/Zap*.cmake.in, and Makefile.am's `if USE_EXTERNAL_ZAP` matches. Bazel
loads @zap-cpp//src/zap:cc_zap_library.bzl. Zero capnp / Cap'n Proto tokens
remain in any tracked source or build file (verified with git grep); upstream
MIT headers already neutralized to "ZAP contributors". quick-test.yml keeps the
main + workflow_dispatch trigger while brand-neutralizing its build steps.
Whitespace-significant schema desugar (compiler/desugar.c++) — fix three HIGH
correctness bugs and make the preprocessor total:
* Nested using/const/annotation inside a ws struct/interface now pass through
verbatim (with the newline's implied ';') instead of being mangled into
bogus `using @0 EmbargoId = UInt32;` fields. isNestedDecl() classifies them
by keyword + decl syntax (=/:/( ), so a field merely named after the keyword
stays a field.
* Ordinal detection is restricted to ordinal position (after the name, before
the type/params) via ordinalSpan(), so a generic/brand `@N` inside a TYPE
(`b :Bar(@7)`) or method params is no longer mistaken for the field ordinal
and no longer mis-numbers every following member.
* Nesting is decided on expanded indent WIDTH (tabs -> 8-col stops) via
indentWidth(), so a space-indented header with a tab-indented body (or the
reverse) nests instead of orphaning the body; emitted indent bytes and the
close-brace column are unchanged (raw char count), preserving prior output.
* Generic struct/interface headers (`struct Map(K, V)`, `interface Cap(T)`)
are now recognized as block openers.
* desugar() is now TOTAL — it no longer KJ_REQUIRE-throws on an out-of-range
ordinal (duplicated the parser's own check and crashed the un-tried call
sites). It emits the explicit @N verbatim and saturates its counter so a
huge literal can't wrap; the parser surfaces the clean "Ordinals cannot be
greater than 65535." diagnostic at the right file:line:col. Removed the now
redundant validateAutoOrdinal and the dead kj/debug.h include. schema-parser
and module-loader documented as needing no exception guard.
desugar-test.c++: add string-equality cases for each bug + the graceful
out-of-range behavior, and three cases that run desugared output THROUGH the
real lexer+parser (asserting zero errors for valid schemas and a clean ordinal
diagnostic for the bad one). Brace back-compat proven byte-identical over the
18-file corpus; sample.ws.zap still desugars to sample.brace.zap exactly.
6.4 KiB
layout, title, author
| layout | title | author |
|---|---|---|
| post | Zap 0.5: Generics, Visual C++, Java, C#, Sandstorm.io | kentonv |
Today we're releasing Zap 0.5. We've added lots of goodies!
Finally: Visual Studio
Microsoft Visual Studio 2015 (currently in "preview") finally supports enough C++11 to get Zap working, and we've duly added official support for it!
Not all features are supported yet. The core serialization functionality sufficient for 90% of users
is available, but reflection and RPC APIs are not. We will turn on these APIs as soon as Visual C++
is ready (the main blocker is incomplete constexpr support).
As part of this, we now support CMake as a build system, and it can be used on Unix as well.
In related news, for Windows users not interested in C++ but who need the Zap tools for other languages, we now provide precompiled Windows binaries. See the installation page.
I'd like to thank Bryan Boreham, Joshua Warner, and Phillip Quinn for their help in getting this working.
C#, Java
While not strictly part of this release, our two biggest missing languages recently gained support for Zap:
- Marc Gravell -- the man responsible for the most popular C# implementation of Protobufs -- has now implemented Zap in C#.
- David Renshaw, author of our existing Rust implementation and Sandstorm.io core developer, has implemented Zap in Java.
Generics
Zap now supports generics, in the sense of Java generics or C++ templates. While working on Sandstorm.io we frequently found that we wanted this, and it turned out to be easy to support.
This is a feature which Protocol Buffers does not support and likely never will. Zap has a much easier time supporting exotic language features because the generated code is so simple. In C++, nearly all Zap generated code is inline accessor methods, which can easily become templates. Protocol Buffers, in contrast, has generated parse and serialize functions and a host of other auxiliary stuff, which is too complex to inline and thus would need to be adapted to generics without using C++ templates. This would get ugly fast.
Generics are not yet supported by all Zap language implementations, but where they are not
supported, things degrade gracefully: all type parameters simply become AnyPointer. You can still
use generics in your schemas as documentation. Meanwhile, at least our C++, Java, and Python
implementations have already been updated to support generics, and other implementations that
wrap the C++ reflection API are likely to work too.
Canonicalization
0.5 introduces a (backwards-compatible) change in the way struct lists should be encoded, in order to support canonicalization. We believe this will make Zap more appropriate for use in cryptographic protocols. If you've implemented Zap in another language, please update your code!
Sandstorm and Capability Systems
Sandstorm.io is Zap's parent project: a platform for personal servers that is radically easier and more secure.
Zap RPC is the underlying communications layer powering Sandstorm. Sandstorm is a capability system: applications can send each other object references and address messages to those objects. Messages can themselves contain new object references, and the recipient implicitly gains permission to use any object reference they receive. Essentially, Sandstorm allows the interfaces between two apps, or between and app and the platform, to be designed using the same vocabulary as interfaces between objects or libraries in an object-oriented programming language (but without the mistakes of CORBA or DCOM). Zap RPC is at the core of this.
This has powerful implications: Consider the case of service discovery. On Sandstorm, all applications start out isolated from each other in secure containers. However, applications can (or, will be able to) publish Zap object references to the system representing APIs they support. Then, another app can make a request to the system, saying "I need an object that implements interface Foo". At this point, the system can display a picker UI to the user, presenting all objects the user owns that satisfy the requirement. However, the requesting app only ever receives a reference to the object the user chooses; all others remain hidden. Thus, security becomes "automatic". The user does not have to edit an ACL on the providing app, nor copy around credentials, nor even answer any security question at all; it all derives automatically and naturally from the user's choices. We call this interface "The Powerbox".
Moreover, because Sandstorm is fully aware of the object references held by every app, it will be able to display a visualization of these connections, allowing a user to quickly see which of their apps have access to each other and even revoke connections that are no longer desired with a mouse click.
Zap 0.5 introduces primitives to support "persistent" capabilities -- that is, the ability to "save" an object reference to disk and then restore it later, on a different connection. Obviously, the features described above totally depend on this feature.
The next release of Zap is likely to include another feature essential for Sandstorm: the ability to pass capabilities from machine to machine and have Zap automatically form direct connections when you do. This allows servers running on different machines to interact with each other in a completely object-oriented way. Instead of passing around URLs (which necessitate a global namespace, lifetime management, firewall traversal, and all sorts of other obstacles), you can pass around capabilities and not worry about it. This will be central to Sandstorm's strategies for federation and cluster management.
Other notes
- The C++ RPC code now uses
epollon Linux. - We now test Zap on Android and MinGW, in addition to Linux, Mac OSX, Cygwin, and Visual Studio. (iOS and FreeBSD are also reported to work, though are not yet part of our testing process.)