Sentient Enclaves Framework for Trusted Confidential AI & Crypto Apps

Supported platforms, system requirements:

Sentient Enclaves Framework built for AWS Nitro Enclaves.
Thus supported on EC2 instances with enabled `Nitro Enclaves` option.
Memory requirements should be equal to your application's Docker container image size multiplied by the grade
of two (due to the initrd initramfs ramdisk with rootfs exported from app's Docker container + RAM for enclave).

Sentient Enclaves Framework built with portability of components in mind.

More platforms and cloud based confidential VMs support (including confidential VMs with GPU TEE support)
are coming very soon. (WIP)
                

Overview:

The Sentient Enclaves Framework contains the following components:

Pipeline SLC VSock Communication Protocol

Pipeline VSock secure local channel (SLC) communication protocol, implemented as a client-server application,
that provides remote control of enclave via running shell commands inside the enclave
and provides bidirectional files transmission into/from enclave's file system.

It implements binary protocol over VSock for local communication,
which gives speed, user and developer experience with enclaves on par as Docker experience with containers.
                

Encryption (WIP)

Sentient Enclaves Framework and specifically Pipeline SLC implementation supports strong P2P encryption
and PRE multi-hop re-encryption, between host and enclave, enclave and S3 storage.

Encryption protocol implemented in Pipeline tool on a VSock buffer level and for content protection itself
(data that transferred into and from enclave).

All the data outside the enclave are secured by strong encryption.

Pipeline supports SLC buffer level encryption for secure data transferring without leaks between enclave and host,
and content encryption (encryption/decryption protocol and test tools) between host and outer storage (S3 at the moment),
and multi-hop PRE re-encryption protocol and test tools for data protection between enclave and third-party systems
during data transferring though many points in the cloud or for content protection between many participants/users
in a network and on Sentient platform.

Keys are stored securely in a KMS service and locally can be stored in TPM devices (on EC2 instance). (WIP)
                

Transparent VSock Proxies

Transparent VSock Proxies supports port forwarding and NAT for inbound and outbound TCP streams, and include original
destination address retrieving from TCP stream, with sending/receiving requests/responses to/from original destination
address from request.

It gives full support of networking stack (securely, on demand, with kill-switch (WIP) for sensitive computations
during run-time) for enclaves and for higher level networking protocols over VSock (VirtIO VM sockets) for data exchange
in forward and reverse proxy mode, with ports aggregation up to full-cone scheme support, via NetFilter rules for Linux
kernel networking subsystem.

It is heavily relying on Linux networking stack and custom reproducible Linux kernel builds with networking stack support
for enclaves.

The reverse and forward proxying schemes are supported for p2p proxying, transparent port to VSock forwarding
(many-to-many listeners scheme, with requiring according number of port forwaring proxies to listeners), and fully
transparent proxying (full-cone NAT and port porwarding, with many-to-many listeners scheme, and requiring only one
proxy instance for each side, host and enclave), to provide networking stack access for enclave apps
(in case of forward proxy) and provide confidential services hosted in enclave (in case of reverse proxy).
                

Web server for remote attestation inside the enclave (WIP)

Web server for remote attestation inside the enclave supports proofs generation and proofs verification, for content
(external additional data, like model heavyweight tensor layers and data sets) placed in enclave run-time (in ramdisk
or CoW disk image), signing and verifying of attestation documents (using KMS service from inside the enclave).

The attestation process guarantees that computation process on data, data producing or just data placing, was performed
exactly inside the secure enclave's isolated environment, which prevent leakage of sensitive data and makes computational
processes private and confident.
                

Enclave's file system monitor (WIP)

Enclave's file system monitor (for ramdisk or CoW image) supports unconditional attestation for all outer external
additional data that will be appeared in enclave file system and run-time.

It guarantees that all data, data producers and data handlers in enclave will be attested unconditionally in unattended mode.

This improves guarantees for computational processes and eliminate chances for data leaks and backdoors installation
inside the enclave.

It integrated into web server and FS monitor maintains attestation documents database.
                

Build system for enclave's reproducible builds

Build system for enclaves supports reproducibility of enclave's environment for application running in it, and guarantees
reproducibility of building processes and final isolated enclave environment for apps, during enclave and application
static build phase.

It can be embedded with integration machine (GitHub Actions, any other self-hosted or cloud CI) to perform continuous
integration (CI) process for building and shipping apps and enclaves for them in DevOps pipelining enterprise software
application environment and deployments.

Build system operates with custom but standard Unix/BSD and Linux tools, not relying on any third party or non-standard software.

Build system rebuild reproducibly custom Linux kernel for networking stack support, and rebuild reproducibly of custom
Init system for running apps processes inside enclave and for KVM hypervisor interaction during enclave's boot process,
during enclave's run-time and termination at the enclave's EOL.
                

Usage:

rbuilds.sh script itself should be run from project root directory, to form FS directories properly.

rbuilds.sh script options can be used like this:

./rbuilds.sh --tty --debug --cmd "make_nitro" 2>&1 3>&1

./rbuilds.sh --tty --debug --cmd "make_clear" 2>&1 3>&1
                

Note: 3>&1 needed for --tty flag, as it enables terminal tty device with file descriptor 3 for bash and docker output, 'cause it is executed not in local shell.

For building all stages, run the command:

mkdir -vp ./eif/; \
/usr/bin/time -v -o ./eif/make_build.log \
./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_all" \
2>&1 3>&1 | tee ./eif/make_build.output
                

For different build stages the template command will be the same (as most of this commands use internally same set of variables, set/override by CLI options):

./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_kernel" 2>&1 3>&1

./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_apps" 2>&1 3>&1

./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_init" 2>&1 3>&1

./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_eif" 2>&1 3>&1

./rbuilds.sh --tty --debug --dockerfile ./pipeline-slc-network-al2023.dockerfile --network --init-c --cmd "make_enclave" 2>&1 3>&1
                

For managing enclave's state, run (one of these) commands:

./rbuilds.sh --tty --debug --network --init-c --cmd "run_eif_image_debugmode_cli" 2>&1 3>&1

./rbuilds.sh --tty --debug --network --init-c --cmd "run_eif_image" 2>&1 3>&1

./rbuilds.sh --tty --debug --network --init-c --cmd "attach_console_to_enclave" 2>&1 3>&1

./rbuilds.sh --tty --debug --network --init-c --cmd "list_enclaves" 2>&1 3>&1

./rbuilds.sh --tty --debug --network --init-c --cmd "drop_enclave" 2>&1 3>&1

./rbuilds.sh --tty --debug --network --init-c --cmd "drop_enclaves_all" 2>&1 3>&1
                

For more information about CLI options, interactive shell mode and automation command shell interface, run these usage help commands:

./rbuilds.sh --man

./rbuilds.sh --info
                

Advanced usage:

Also rbuild.sh supporting an interactive shell mode, for interactive builds and commands shell interface mode for interactive automation of builds (and build stages):

Interactive mode shell with timings and console dump - you'll see the shell prompt:

mkdir -vp ./eif/; \
/usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output
                

Automation command shell interface:

{ echo "attach_console_to_enclave"; } | ./rbuild.sh 2>&1

{ echo "list_enclaves"; } | ./rbuild.sh 2>&1

{ echo "drop_enclave"; } | ./rbuild.sh 2>&1

{ echo "drop_enclaves_all"; } | ./rbuild.sh 2>&1
                

Automation command shell interface with timings and console dump:

{ echo "attach_console_to_enclave"; } | /usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output

{ echo "list_enclaves"; } | /usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output

{ echo "drop_enclave"; } | /usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output

{ echo "drop_enclaves_all"; } | /usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output
                

Automation command shell interface for building stages make commands:

{  echo "make eif";
  echo "y";
  echo "y";
  echo "y";
  echo "y";
  echo "y";
  echo "y";
  echo "y"; } | /usr/bin/time -v -o ./make_build.log ./rbuild.sh 2>&1 | tee ./make_build.output

{ echo "make all"; } | /usr/bin/time -v -o ./make_build.log ./rbuild.sh 2>&1 | tee ./make_build.output

mkdir -vp ./eif/; \
{ echo "make all"; } | /usr/bin/time -v -o ./eif/make_build.log ./rbuild.sh 2>&1 | tee ./eif/make_build.output
                

Automation command shell interface with access to local commands execution (use responsibly):

{ echo "lsh"; echo "ls -lah"; } | /usr/bin/time -v -o ./make_build.log ./rbuild.sh 2>&1 | tee ./make_build.output
                

Demonstrations:

Reproducible builds demo:

Reproducible builds demo with full IO:

Enclave's networking demo (WIP):

Visualization of Sentient Enclaves Framework infrastructure, components and its interaction, with diagrams:

Whole Infrastructure bird-eye view:

TEE Infrastructure:

Changelog: what's already done and implemented, release notes

We've got an internal semantic versioning and numerology for releases, according to roadmap release schedule:

Done:

0.1.0 - Pipeline secure local channel protocol, for host to enclave (and vise versa, enclave to host) data transferring,
        and for remote commands execution (with execution results output to logs),
        for managing enclave's environment, providing container-like experience.

0.2.0 - Reproducible builds framework for building customized enclave images (EIF),
        and provide experience for "just use your app dockerfile and build enclave".

0.3.0 - Forward proxies, transparent forward proxies, port forwarding proxies, for TCP (HTTP/HTTPs, etc.) and UDP (DNS, etc.) protocols.
        Providing networking stack for applications in enclave, to forward traffic from apps, from inside the enclave,
        use high-level networking protocols (network/cloud data storage, web, etc.), supported over VSock.

0.4.0 - New updated reproducible builds system for Sentient Enclaves Framework for building customized enclave images (EIF).

0.5.0 - Reverse proxies, transparent reverse proxies (including transparent port forwarding to vsock),
        to support request forwarding into enclave's apps, for providing services (mostly web and other network protocols),
        hosted inside enclave (in isolated memory region and isolated environment from host system).

0.6.0 - Set of reference applications, built with framework - inference server (will include Dobby model),
        fine-tuning server (includes fine-tuning OML library), X agent (chat bot app).
                

Roadmap: what's in progress, in active development and upcoming features, release schedule

WIP (already in development):

0.7.0 - web protocol for RA, with VRF proofs for attestation docs, mt-runtime, mass-production of attestation docs,
        hot cache and cold DB (Sled) integration for storing attestation docs.

0.8.0 - file system monitor, for automagic mass-production of attestation docs, with mt-runtime integration as well.

0.9.0 - cryptography stack, for buffer level SLC and content encryption.

0.10.0 - integration with KMS for key storage, probably TPM module usage for local key storage.

0.11.0 - Advanced logging and log tracing for Pipeline and proxies. Web-RA already included structural logging.
         This will improve logs aggregation, especially in enclave's production mode (without debug console),
         for remote debugging of enclaves, for use in monitoring and log aggregation systems,
         to understand exact places where issues/bugs appeared, to reveal and fix them fast.

The following is a subject to change, i.e. the order of releases and version numbers.

In design stage:

0.12.0 - enclave's init system update, rewritten in Rust.

0.13.0 - Enclaves engine service with web API (Docker API like),
         for enclaves provisioning (EC2 or other cloud instances configuration, including NUMA, Huge Memory Pages allocation, etc.),
         enclaves building (integration with current reproducible builds system, rbuilds build system), enclaves deployment, monitoring.
         Integration with AWS SQS/MQ for deployment tasks tracking and backend systems. Integration with CI (GitHub Actions mostly).

0.14.0 - port of framework to QEMU VM (SEV-SNP, TDX, and GPU TEE support) with EIF images support for running,
         along with qCoW images, block devices and GPU device attestation (NVTrust), FS monitoring and attestation for qCoW images.

0.15.0 - H100, H200 support via QEMU and NVTrust attestation.