Emerging intelligent bump-in-the-wire network adapters, iNICs, are
increasingly deployed for accelerating custom network functions in
large-scale data center and cloud environments such as Microsoft Azure.
However, their potential to speed up network-intensive server applications
remains largely unexplored due to the lack of appropriate programming
models and OS abstractions.

GAON is a framework that facilitates accelerating network applications
using iNICs. GAON introduces an *ikernel *abstraction that encapsulates the
code and state of an iNIC-resident application task. The application
dynamically invokes the ikernel by attaching it to a network socket,
thereby rerouting the socket traffic to the ikernel. GAON can securely
execute multiple ikernels of different applications, isolating ikernel
state and network traffic, arbitrating access to network from ikernels and
CPU applications, and providing protected access to the ikernel state from
the owning CPU application.

We prototype GAON on Mellanox Innova 40 Gbps iNICs and develop several
accelerated versions of realistic server workloads. For example, the
transparent key-value store cache ikernel serves 31Mtps (70% of the line rate) at
5 μs 99th percentile latency for 10 byte key/value pairs, seamlessly improving the
host performance by up to 55× and 2.3× for 98% and 60% cache hit rate respectively,
with almost no modification to the original memcached server running on the same
machine.