The Teaclave's build system utilizes CMake to coordinate compilation, linking,signing, etc. for various components written in different languages (e.g., Rust, C,Python) for different targets (e.g., Linux and SGX). In this document, we willintroduce our build system in details.
Download Windows. SDK Download Linux. SDK Use the SDK to create software solutions that leverage additional protection from platforms with Intel® Software Guard Extensions (Intel® SGX). The kit includes the following resources to get you started. Hi all,Here is the link that you need for this tutorial: Intel SGX SDK download: the version.
# Quick Start
- Download and install CMake, the minimumrequired version is 3.10.
- Open a shell and create a build directory.
- Run the following command to configure with
TEST_MODE
on.
- Build the whole platform.
When making changes, run:
make format
: Format current source code.make run-tests
: Make sure all tests are passed.
You can find more detailed configurations and targets in the following sections.
# Variables and Options
There are a lot of variables and options you can configure to control the buildsystem.
To set a variable or option, you can pass -DXXX=
to cmake
. For example,cmake -DTEST_MODE=ON ..
to enable the TEST_MODE
option.
# Variables
SGX_SDK
: Set (or get from env vars) the path of Intel SGX SDK. Defaults to/opt/sgxsdk
.RUSTFLAGS
: Set (or get from env vars) flags passed to rustc.MESAPY_VERSION
: Set the commit hash to the upstream MesaPy version.RUSTUP_TOOLCHAIN
: Set the Rust toolchain version.CMAKE_BUILD_TYPE
: Set the build type. Defaults to debug.
# Options
COV
: Build with coverage information. Defaults to OFF.OFFLINE
: Compile Rust code withcargo --offline
. Defaults to ON.TEST_MODE
: Build with mock data and disabling some functions for testing.Defaults to OFF.SGX_SIM_MODE
: Build in SGX simulation mode. Defaults to OFF.DCAP
: Use DCAP instead of IAS as the attestation service. Defaults to OFF.GIT_SUBMODULE
: Sync submodules with the upstream repositories. Defaults toON.CLP
: Enablecargo clippy
to lint Rust code during the compilation.Defaults to OFF.DOC
: Generate document withcargo doc
during the compilation. Defaults to OFF.USE_PREBUILT_MESAPY
: Whether to use the prebuilt MesaPy for SGX library. Ifset to OFF, will build the library from the source code. Defaults to ON.
# Targets
The followings are supported targets you can call with make
. For example, to build a specificservice like execution service, you can just run make teaclave_execution_service
.
# App/Enclave
An SGX application has two parts: the app part and the enclave part. You cancompile them separately or together using with these targets:
sgxapp-teaclave_{service_name}
: Build the app part of a service.sgxlib-teaclave_{service_name}
: Build the enclave part of a service.teaclave_{service_name}
: Build (compile, link and sign, etc.) the app andenclave of a service.sgxapp-teaclave_{test_name}
: Build the app part of a test driver.sgxlib-teaclave_{test_name}
: Build the enclave part of a test driver.teaclave_{test_name}
: Build (compile, link, and sign, etc.) the app andenclave of a test driver.
These targets are automatically generated from thecmake/tomls/Cargo.sgx_{}.toml
files. Basically, they are:
test_name
can be:function_tests
,unit_tests
,integration_tests
, etc.service_name
can be:access_control_service
,authentication_service
,storage_service
,execution_service
,frontend_service
,management_service
,scheduler_service
, etc.
# Client SDK
Is There An Sgx Sdk File
Build Teaclave client SDK and install the compiled dynamic/static libraries tothe release folder.
teaclave_client_sdk
: Build the client SDK to both dynamic and staticlibraries.
# Bin
teaclave_cli
: Build the Teclave command line tool.teaclave_dcap_ref_as
: Build the reference implementation of DCAP'sattestation service.teaclave_sgx_tool
: Build the SGX tool.
Above targets are automatically generated from thecmake/tomls/Cargo.unix_app.toml
files.
# Linting
format
: Format all code.clippy
: Runcargo clippy
for linting. Same withmake CLP=1
.
# Doc
doc
: Runcargo doc
to generate documents. Same withmake DOC=1
.
# Tests
run-tests
: Run all test cases.run-integration-tests
: Run integration tests only.run-funtional-tests
: Run functional tests only.run-sdk-tests
: Run tests of client SDK only.run-examples
: Run all examples.cov
: Aggregate coverage results and generate report, needs to config cmakewith-DCOV=ON
.
# Misc
clean
: Cleanup all building intermediates.
# Codebase
You can find source code to learn more about our build system in theCMakeLists.txt
file and the cmake
directories.
1. ECREATE
- [Intel SGX Explained p63] Section 5.3.1. Creation
- [Programming References p21] Section 5.3. ECREATE
An enclave is born when the system software issues the ECREATE
instruction, which turns a free EPC page into the SECS for the new enclave.
ECREATE
copies an SECS structure outside the EPC into an SECS page inside the EPC. The internal structure of SECS is not accessible to software.
Software sets the following fields in the source structure: SECS:BASEADDR
, SECS:SIZE
, and ATTRIBUTES
.
ECREATE
validates the information used to initialize the SECS, and results in a page fault (#PF) or general protection fault (#PF) if the information is not valid.ECREATE
will also fault if SECS target page is in use; already valid; outside the EPC; adresses are not aligned; unused PAGEINFO fields are not zero.
2. EADD
- [Intel SGX Explained p64] Section 5.3.2. Loading
- [Programming References p11] Section 5.3. EADD
The system software can use EADD
instructions to load the initial code and data into the enclave. EADD
is used to create both TCS pages and regular pages.
This function copies a source page from non-enclave memory into the EPC, associates the EPC page with an SECS page residing in the EPC, and stores the linear address and security attributes in EPCM.
EADD
reads its input data from a Page Information (PAGEINFO) structure.
The PAGEINFO structure contains
- The virtual address of the EPC page (
LINADDR
) - The virtual address of the non-EPC page whose contents will be copied into the newly allocated EPC page (
SRCPGE
) - A virtual address that resolves to the SECS of the enclave that will own the page (
SECS
). - The virtual address, pointing to a Security Information (SECINFO) structure, which contains the newly allocated EPC page’s access permissions (R, W, X) and its EPCM page type (
RT_REG
orPT_TCS
).
{: .center-image}
EADD
validates its inputs, and modifies the newly allocated EPC page and its EPCM entry.
EADD
ensures
- The EPC page is not allocated to another enclave.
- The page’s virtual address falls within the enclave’s ELRANGE.
- All the reserved fields in SECINFO are set to zero.
2-1. How a free EPC page is selected in SGX simulation mode?
Simulation implementation might be different from hardware implementation. In simulation mode, ECREATE
allocates all EPC pages via mmap()
.
As SGX simulation simulates SGX behavior by software, it copies EPC page data with virtual address. Hence _EADD()
does not have detailed information about picking a physical EPC page.
2.2. How system software selects a EPC page in SGX hardware mode?
SGX simulation code can’t tell EPC page allocation in detail, as it is also a software, so it cannot use the physical address.
To understand actual implementation, I first tried to understand Intel SGX Programming Reference deeply. There is a table explaining inputs for the instruction.
Table. Instruction Operand Encoding{: .center}
Op/En | EAX | RBX | RCX |
---|---|---|---|
IR | EADD (in) | Address of PAGEINFO (in) | Address of the destination EPC page (in) |
As you see the above table, addresses of PAGEINFO and target EPC page should be saved in the register RBX
and RCX
, respectively. The target EPC page is already determined, which means system software is responsible for selecting one. This is also explained in ISCA ‘15 tutorial slide. [link]
{: .center-image width=“600px”}
Then this means I need to search a code that calls EADD
instruction.
Simulation function which calls EADD
instruction is add_enclave_page()
in sdk/simulation/driver_api/driver_api.cpp
. This is a simulation code because it is in simulation
directory.
Then there should be a function with the same name for hardware?
And yes. There is.
Note that PSW (Platform SoftWare) is for actual hardware. It calls ioctl()
, which calls sgx_ioctl_enclave_add_page()
in linux-sgx-driver. linux-sgx-driver is separately provided in [here].
In linux-sgx-driver/isgx_ioctl.c
,
ioctl()
from enclave_creator_hw.cpp
- calls
isgx_ioctl_enclave_add_page()
atlinux-sgx-driver/isgx_ioctl.c:548
- calls
__enclave_add_page()
atlinux-sgx-driver/isgx_ioctl.c:440
- calls
construct_enclave_page
atlinux-sgx-driver/isgx_ioctl.c:122
- calls
isgx_alloc_epc_page()
atlinux-sgx-driver/isgx_page_cache.c:429
- calls
isgx_alloc_epc_page_fast()
atlinux-sgx-driver/isgx_page_cache.c:411
which picks the first entry from driver’s EPC page list.
isgx Linux SGX driver manages EPC page instances by using the linked list(static LIST_HEAD(isgx_free_list)
at isgx_page_cache.c:25
) and the number of free EPC pages(unsigned int isgx_nr_free_epc_pages
at isgx_page_cache.c:31
).
Is There An Sgx Sdk Free
The type of EPC pages is struct isgx_epc_page
, defined as follows.
Here, pa
is the physical address for the EPC page. How pa
is determined?
Each EPC page that is put into free list is allocated in the function isgx_page_cache_init()
at linux-sgx-driver/isgx_page_cache.c:360
.
Each EPC page has the physical address as start + i
. start
is the first parameter of the function.
The function isgx_page_cache_init()
is called in isgx_init()
at linux-sgx-driver/isgx_main.c:190
.
As shown above, start
is the value of isgx_epc_base
variable.isgx_epc_base
is initialized by the function isgx_init_platform()
at linux-sgx-driver/isgx_main.c:133
.
You can see what isgx_epc_base
value is in your machine, as SGX driver prints it in kernel message buffer by default.
{: .center-image}
My machine tells that 32MiB of EPC is allocated with the physical base memory address 0x80000000
.
All EPC page instance struct isgx_epc_page
has a pa
variable, which contains the physical address of the EPC page, and is initialized as EPC base address + offset.
And, managing page mapping table is also a responsibility of system software. Linux SGX driver insert PTE via calling vm_insert_pfn()
at linux-sgx-driver/isgx_util.c:67
by using epc_page->pa
and expected virtual address enclave_page->addr
as follows.
Is There An Sgx Sdk
To be concluded, when a SGX platform is initialized, several EPC page instances (struct isgx_epc_page
) are allocated to represent all EPC pages. System software manages them as a linked list, called isgx_free_list
. When EADD
is called, system software picks a free EPC page instance from the list, and create a page table entry, pointing the physical address that is saved in epc_page->pa
, with the expected virtual address enclave_page->addr
, a part of user enclave’s ELRANGE.
3. EEXTEND
- [Intel SGX Explained p64] Section 5.3.2. Loading
- [Programming References p31] Section 5.3. EEXTEND
While loading an enclave, the system software will also use the EEXTEND
instruction, which updates the enclave’s measurement used in the software attestation process.
It updates the MRENCLAVE measurement register of an SECS with the measurement of an EXTEND string compromising of “EEXTEND” || ENCLAVEOFFSET || PADDING || 256 bytes of the enclave page.
RCX register contains the effective address of the 256 byte region of an EPC page to be measured.
From Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3D, Part 4:Section 39.1.2. EADD and EEXTEND Interaction
Software can measure a 256 byte region as determined by the by the developer by invoking EEXTEND. Thus to measure an entire 4KB page, system software must execute EEXTEND 16 times. Each invocation of EEXTEND adds to the cryptographic log information about which region is being measured and the measurement of the section.
4. EINIT
- [Intel SGX Explained p64] Section 5.3.3. Initialization
- [Programming References p34] Section 5.3. EINIT
This function is the final instruction executed in the enclave build process. After EINIT, the MRENCLAVE measurement is cimplete, and the enclave is ready to start user code execution using EENTER instruction.
When EINIT
completes successfully, it sets the enclave’s INIT attribute to true. This opens the way for ring 3 application software to execute the enclave’s code, using the SGX instructions.
On the other hand, once INIT is set to true, EADD
cannot be invoked on that enclave anymore, so the system software must load all the pages that make up the enclave’s initial state before executing the EINIT
instruction.
References
- Intel Software Guard Extensions Programming Reference. [link]
- Intel SGX Explained. [link]
- Intel SGX Tutorial Slide presented in ISCA 2015.[link]
- Intel SGX SDK Github Repository. [link]
- Intel SGX Linux Driver Github Repository.[link]
License
All source codes are from Intel SGX SDK Github repository and Intel SGX Linux driver Github repository, released under BSD License 2.0 and GNU General Public License 2.0, respectively.
Intel SGX SDK
Copyright (C) 2011-2017 Intel Corporation. All rights reserved.
Redistribution and use in source and binary forms, with or withoutmodification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution. - Neither the name of the
nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” ANDANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL