The Linera Developer Manual

Welcome to the developer manual of Linera, a decentralized protocol designed for highly scalable Web3 applications.

Linera is currently under active development. This documentation is intended for developers who wish to learn more about Linera and its programming model by prototyping applications on top of the current Software Development Kit (SDK).

We assume that you have a basic understanding of blockchain technology, decentralized applications, and smart contract development. We will provide detailed guides and examples to help you build, test, and deploy your demo applications on your machine in a local instance of the Linera protocol.

We encourage you to join our community and get involved in the development of the Linera ecosystem. You can find more information on our website and on social media channels, including youtube, twitter, telegram, and discord.

Let's get started!

Overview

Linera is a decentralized infrastructure optimized for Web3 applications that require guaranteed performance for an unlimited number of active users.

The core idea of the Linera protocol is to run many lightweight blockchains, called microchains, in parallel in a single set of validators.

How does it work?

In Linera, end users (or rather their wallets) are expected to operate their own microchains, called user chains. The owner of a chain chooses when to add new blocks to the chain and what goes inside the blocks.

Users may add new blocks to their chains in order to process incoming messages from other chains or to execute secure operations on their accounts, for instance to transfer assets to another user.

Importantly, validators ensure that all new blocks are valid. For instance, transfer operations must originate from accounts with sufficient funds; and incoming messages must have been actually sent from another chain. Blocks are verified by validators in the same way for every chain.

A Linera application is a Wasm program that defines its own state and operations. Users can publish bytecode and initialize an application on one chain, and it will be automatically deployed to all chains where it is needed, with a separate state on each chain.

To ensure coordination across chains, an application may rely on asynchronous cross-chain messages. Message payloads are application-specific and opaque to the rest of the system.

The number of applications present on a single chain is not limited. On the same chain, applications are composed as usual using synchronous calls.

The current Linera SDK uses Rust as a source language to create Wasm applications. It relies on the normal Rust toolchains so that Rust programmers can work in their preferred environments.

Linera applications are structured using the familiar notion of Rust crate: the external interfaces of an application (including initialization parameters, operations, messages, and cross-application calls) generally go into the library part of its crate, while the core of each application is compiled into binary files for the Wasm architecture.

How does Linera compare to existing multi-chain infrastructure?

Linera is the first infrastructure designed to support many chains in parallel, and notably an arbitrary number of user chains meant to be operated by user wallets.

In traditional multi-chain infrastructures, each chain usually runs a full blockchain protocol in a separate set of validators. Creating a new chain or exchanging messages between chains is expensive. As a result, the total number of chains is generally limited. Some chains may be specialized to a given use case: these are called "app chains".

In contrast, Linera is optimized for a large number of user chains:

• Users only create blocks in their chain when needed;

• Creating a microchain does not require onboarding validators;

• All chains have the same level of security;

• Microchains communicate efficiently using the internal networks of validators;

• Validators are internally sharded (like a regular web service) and may adjust their capacity elastically by adding or removing internal workers.

The specifications of the Linera protocol (see the whitepaper) also include other types of microchains, called "permissioned" and "public". Public chains are operated by validators and similar to classical blockchains in this regard. Permissioned chains are meant to be used for temporary interactions between users, such as atomic swaps.

Why build on top of Linera?

We believe that many high-value use cases are currently out of reach of existing Web3 infrastructures because of the challenges of serving many active users simultaneously without degrading user experience (unpredictable fees, latency, etc).

Examples of applications that require processing time-sensitive transactions created by many simultaneous users include:

• real-time micro-payments and micro-rewards,

• social data feeds,

• real-time auction systems,

• turn-based games,

• version control systems for software, data pipelines, or AI training pipelines.

Lightweight user chains are instrumental in providing elastic scalability but they have other benefits as well. Because user chains have fewer blocks than traditional blockchains, in Linera, the full-nodes of user chains will be embedded into the users' wallets, typically deployed as a browser extension.

This means that Web UIs connected to a wallet will be able to query the state of the user chain directly (no API provider, no light client) using familiar frameworks (React/GraphQL). Furthermore, wallets will be able to leverage the full node as well for security purposes, including to display meaningful confirmation messages to users.

What is the current state of the development of Linera?

The reference open-source implementation of Linera is under active development. It already includes a Web3 SDK with the necessary features to prototype simple Web3 applications and test them locally on the same machine. Notably, Web UIs (possibly reactive) can already be built on top of Wasm-embedded GraphQL services, and tested locally in the browser.

The main limitations of our current Web3 SDK include:

• Web UIs need to query a local HTTP service acting as a wallet. This setup is meant to be temporary and for testing only: in the future, web UIs will securely connect to a Wallet installed as a browser extension, as usual.

• Gas metering is activated in the Wasm VM but the prices are set to zero by default. Other aspects of the systems that incur costs to validators (e.g. uploading large bytecode) are also not yet protected by fees and/or hard limits.

• Only user chains are currently available for testing and documented in this manual. Support for other types of chain (called "public" and "permissioned") will be added later.

The main development workstreams of Linera, beyond its SDK, can be broken down as follows.

Core Protocol

• User chains
• Permissioned chain (core protocol only)
• Cross-chain messages
• Cross-chain pub/sub channels (initial version)
• Bytecode publishing
• Application creation
• Reconfigurations of validators
• Initial support for gas fees
• Initial support for storage fees and storage limits
• External services to help users create their first chain
• Permissioned chains (adding operation access control, demo of atomic swaps, etc)
• Public chains (adding leader election, inbox constraints, etc)
• Support for easy onboarding of user chains into a new application (removing the need to accept requests)
• Improved pub/sub channels (removing the need to accept subscriptions)
• Blob storage for applications (generalizing bytecode storage)
• Support for archiving chains
• Wallet-friendly chain clients (compile to Wasm/JS, do not maintain execution states for other chains)
• General tokenomics and incentives for all stakeholders
• Governance on the admin chain (e.g. DPoS, onboarding of validators)
• Auditing procedures

Wasm VM integration

• Support for the Wasmer VM
• Support for the Wasmtime VM (experimental)
• Test gas metering and deterministic execution across VMs
• Composing Wasm applications on the same chain (initial version)
• Enhanced composability with "sessions"
• Support for non-blocking (yet deterministic) calls to storage
• Support for read-only GraphQL services in Wasm
• Support for mocked system APIs (initial version)
• More efficient cross-application calls and better reentrancy
• Improve host/guest stub generation to make mocks easier (currently wit-bindgen)
• Compile user full node to Wasm/JS

Storage

• Object management library ("linera-views") on top of Key-Value store abstraction
• Support for Rocksdb
• Experimental support for DynamoDb
• Initial derive macros for GraphQL
• Initial support for ScyllaDb
• Make library fully extensible by users (requires better GraphQL macros)
• Performance benchmarks and improvements (including faster state hashing)
• Production-grade support for the chosen main database
• Support global object locks (needed for dynamic sharding)
• Tooling for debugging
• Make the storage library easy to use outside of Linera

Validator Infrastructure

• Simple TCP/UDP networking (used for benchmarks only)
• GRPC networking
• Basic frontend (aka proxy) supporting fixed internal shards
• Observability
• Initial kubernetes support in CI
• Initial nightly deployment using a cloud provider
• New frontend to support dynamic shard assignment
• Cloud integration to demonstrate elastic scaling

Web3 SDK

• Initial traits for contract and service interfaces
• Support for unit testing
• Support for integration testing
• Local GraphQL service to query and browse system state
• Local GraphQL service to query and browse application states
• Use GraphQL mutations to execute operations and create blocks
• Initial support for unit tests
• Support for integration tests
• Initial ABIs for contract and service interfaces
• Allowing message sender to pay for message execution fees
• Safety programming guidelines (including reentrancy)
• Bindings to use native cryptographic primitives from Wasm
• Allowing applications to pay for user fees
• Allowing applications to use permissioned chains and public chains
• Wallet as a browser extension (no VM)
• Wallet as a browser extension (with Wasm VM)

Getting started

In this section, we will cover the necessary installation steps and a short example to get started with the Linera SDK.

Installation

Let's start with the installation of the Linera development tools.

Overview

The Linera toolchain consist of two crates:

• linera-sdk is the main library to program Linera applications in Rust. It also includes the Wasm test runner binary linera-wasm-test-runner.

• linera-service defines four binaries:

  • linera -- the main client tool, to operate user wallets,
  • linera-proxy -- the Linera proxy, which acts as an ingress for validators,
  • linera-server -- the Linera workers behind the proxy,
  • linera-db -- a command line tool to manage persistent storage.

OS Support

The Linera client and validators run as a set of native binaries. Below is a matrix of supported operating systems.

Prerequisites

The required software may installed as follows on Linux:

• Rust and Wasm

  • curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  • rustup target add wasm32-unknown-unknown

• Protoc

  • curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v21.11/protoc-21.11-linux-x86_64.zip
  • unzip protoc-21.11-linux-x86_64.zip -d $HOME/.local
  • If ~/.local is not in your path, add it: export PATH="$PATH:$HOME/.local/bin"

• On certain Linux distributions, you may have to install development packages such as g++, libclang-dev and libssl-dev.

For MacOS, see the installation section on GitHub.

Installing from crates.io

You may install binaries with

cargo install linera-sdk
cargo install linera-service

and use linera-sdk as a library for Linera Wasm applications:

cargo add linera-sdk

Installing from GitHub

Download the source from GitHub:

git clone https://github.com/linera-io/linera-protocol.git

To install the Linera toolchain locally from source, you may run:

cargo install --path linera-sdk
cargo install --path linera-service

Alternatively, for developing and debugging, you may instead use the binaries compiled in debug mode, e.g. using export PATH="$PWD/target/debug:$PATH".

This manual has been tested against the following commit of the repository:

f2310b4ab7fccc11108971b53368be09996e24ab

Bash helper (optional)

Consider adding the output of linera net helper to your ~/.bash_profile to help with automation.

Getting help

If installation fails, reach out to the team (e.g. on Discord) to help troubleshoot your issue or create an issue on GitHub.

Hello, Linera

This section is about running a local development network, then compiling and deploying your first application from scratch.

By the end of this section, you'll have a microchain running locally and a working application that can be queried using GraphQL.

Starting a Local Test Network

The first step is to start your local development network. A development network consists of a numbers of validators, each of which consist of an ingress proxy (aka a "load balancer") and a number of workers (aka "physical shards").

To start a local network, run the following command:

linera net up

This will start a validator with the default number of shards and create a temporary directory storing the entire network state.

This will set up a number of initial chains and create an initial wallet to operate them.

Using the Initial Test Wallet

linera net up prints Bash statements on its standard output to help you configure your terminal to use the initial wallet of the new test network, for instance:

export LINERA_WALLET="/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/wallet.json"
export LINERA_STORAGE="rocksdb:/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/linera.db"

This wallet is only valid for the lifetime of a single network. Every time a local network is restarted, the wallet needs to be reconfigured.

In the following examples, we assume that the variables LINERA_WALLET and LINERA_STORAGE are both set and point to the initial wallet of the running local network.

Interacting with the Network

The main way of interacting with the network and deploying applications is using the linera client.

To check that the network is working, you can synchronize the balance for your default chain with the rest of the network.

linera sync-balance

You should see an output of 10.

Building an Example Application

Applications running on Linera are Wasm bytecode. Each validator and client has a built-in Wasm virtual machine (VM) which can execute bytecode.

Let's build the counter application from the examples/ subdirectory:

cd examples/counter && cargo build --release

Note: This will automatically build Wasm, not native code, thanks to the configuration file examples/.cargo/config.toml.

Publishing your Application

You can publish the bytecode and create an application using it on your local network using the linera client's publish-and-create command and provide:

1. The location of the contract bytecode
2. The location of the service bytecode
3. The JSON encoded initialization arguments

linera publish-and-create \
  ../target/wasm32-unknown-unknown/release/counter_{contract,service}.wasm \
  --json-argument "42"

Congratulations! You've published your first application on Linera!

Querying your Application

Now let's query your application to get the current counter value. To do that, we need to use the client running in service mode. This will expose a bunch of APIs locally which we can use to interact with applications on the network.

linera service

Navigate to http://localhost:8080 in your browser to access the GraphiQL, the GraphQL IDE. We'll look at this in more detail in a later section; for now, list the applications deployed on your default chain e476… by running:

query {
  applications(
    chainId: "e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65"
  ) {
    id
    description
    link
  }
}

Since we've only deployed one application, the results returned have a single entry.

At the bottom of the returned JSON there is a field link. To interact with your application copy and paste the link into a new browser tab.

Finally, to query the counter value, run:

query {
  value
}

This will return a value of 42, which is the initialization argument we specified when deploying our application.

Core Concepts

We now describe the main concepts of the Linera protocol when it comes to programming Web3 applications.

Microchains

This section provides an introduction to microchains, the main building block of the Linera Protocol. For a more formal treatment refer to the whitepaper.

Background

A microchain is a chain of blocks describing successive changes to a shared state. We will use the terms chain and microchain interchangeably. Linera microchains are similar to the familiar notion of blockchain, with the following important specificities:

• An arbitrary number of microchains can coexist in a Linera network, all sharing the same set of validators and the same level of security. Creating a new microchain only takes one transaction on an existing chain.

• The task of proposing new blocks in a microchain can be assumed either by validators or by end users (or rather their wallets) depending on the configuration of a chain. Specifically, microchains can be single-owner, permissioned, or public, depending on who is authorized to propose blocks.

Cross-Chain Messaging

In traditional networks with a single blockchain, every transaction can access the entire execution state. This is not the case in Linera where the state of a microchain is only affected by its own blocks.

Cross-chain messaging is a way for different microchains to communicate with each other asynchronously. This method allows applications and data to be distributed across multiple chains for better scalability. When an application on one chain sends a message to another chain, a cross-chain request is created. These requests are implemented using remote procedure calls (RPCs) within the validators' internal network, ensuring that each request is executed only once.

Instead of immediately modifying the target chain, messages are placed first in the target chain's inbox. When an owner of the target chain creates its next block in the future, they may reference a selection of messages taken from the current inbox in the new block. This executes the selected messages and applies their messages to the chain state.

Below is an example set of chains sending asynchronous messages to each other over consecutive blocks.

                               ┌───┐     ┌───┐     ┌───┐
                       Chain A │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘
                                                     ▲
                                           ┌─────────┘
                                           │
                               ┌───┐     ┌─┴─┐     ┌───┐
                       Chain B │   ├────►│   ├────►│   │
                               └───┘     └─┬─┘     └───┘
                                           │         ▲
                                           │         │
                                           ▼         │
                               ┌───┐     ┌───┐     ┌─┴─┐
                       Chain C │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘

The Linera protocol allows receivers to discard messages but not to change the ordering of selected messages inside the communication queue between two chains. If a selected message fails to execute, it is skipped during the execution of the receiver's block. The current implementation of the Linera client always selects as many messages as possible from inboxes, and never discards messages.

Chain Ownership Semantics

Only single-owner chains are currently supported in the Linera SDK. However, microchains can create new microchains for other users, and control of a chain can be transferred to another user by changing the owner id. A chain is permanently deactivated when its owner id is set to None.

For more detail and examples on how to open and close chains, see the wallet section on chain management.

Wallets

The command-line tool linera is the main way to interact with a Linera network and manage the user wallets present locally on the system.

As in traditional blockchains, Linera wallets are in charge of holding user private keys. However, instead of signing transactions, Linera wallets are meant to sign blocks and propose them to extend the chains owned by their users.

In practice, wallets include a node which tracks a subset of Linera chains. We will see in the next section how a Linera wallet can run a GraphQL service to expose the state of its chains to web frontends.

Selecting a Wallet

The private state of a wallet is conventionally stored in a file wallet.json, while the state of its the corresponding node are stored in a file linera.db.

To switch between wallets, you may use the --wallet and --storage options of the linera tool, e.g. as in linera --wallet wallet2.json --storage rocksdb:linera2.db.

You may also define the environment variables LINERA_STORAGE and LINERA_WALLET to the same effect. E.g. LINERA_STORAGE=$PWD/wallet2.json and LINERA_WALLET=$PWD/wallet2.json.

Finally, if LINERA_STORAGE_$I and LINERA_WALLET_$I are defined for some number I, you may call linera --with-wallet $I (or linera -w $I for short).

Chain Management

Listing Chains

To list the chains present in your wallet, you may use the command show:

linera wallet show
╭──────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────╮
│ Chain Id                                                         ┆ Latest Block                                                                         │
╞══════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════╡
│ 668774d6f49d0426f610ad0bfa22d2a06f5f5b7b5c045b84a26286ba6bce93b4 ┆ Public Key:         3812c2bf764e905a3b130a754e7709fe2fc725c0ee346cb15d6d261e4f30b8f1 │
│                                                                  ┆ Owner:              c9a538585667076981abfe99902bac9f4be93714854281b652d07bb6d444cb76 │
│                                                                  ┆ Block Hash:         -                                                                │
│                                                                  ┆ Timestamp:          2023-04-10 13:52:20.820840                                       │
│                                                                  ┆ Next Block Height:  0                                                                │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 91c7b394ef500cd000e365807b770d5b76a6e8c9c2f2af8e58c205e521b5f646 ┆ Public Key:         29c19718a26cb0d5c1d28102a2836442f53e3184f33b619ff653447280ccba1a │
│                                                                  ┆ Owner:              efe0f66451f2f15c33a409dfecdf76941cf1e215c5482d632c84a2573a1474e8 │
│                                                                  ┆ Block Hash:         51605cad3f6a210183ac99f7f6ef507d0870d0c3a3858058034cfc0e3e541c13 │
│                                                                  ┆ Timestamp:          2023-04-10 13:52:21.885221                                       │
│                                                                  ┆ Next Block Height:  1                                                                │
╰──────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────╯

Each row represents a chain present in the wallet. On the left is the unique identifier on the chain, and on the right is metadata for that chain associated with the latest block.

Default Chain

Each wallet has a default chain that all commands apply to unless you specify another --chain on the command line.

The default chain is set initially, when the first chain is added to the wallet. You can check the default chain for your wallet by running:

linera wallet show

The Chain Id which is in green text instead of white text is your default chain.

To change the default chain for your wallet, user the set-default command:

linera wallet set-default <chain-id>

Opening a Chain

The Linera protocol defines semantics for how new chains are created, we call this "opening a chain". A chain cannot be opened in a vacuum, it needs to be created by an existing chain on the network.

Open a Chain for Your Own Wallet

To open a chain for your own wallet, you can use the open-chain command:

linera open-chain

This will create a new chain (using the wallet's default chain) and add it to the wallet. Use the wallet show command to see your existing chains.

Open a Chain for Another Wallet

Opening a chain for another wallet requires an extra two steps. Let's initialize a second wallet:

linera --wallet wallet2.json --storage rocksdb:linera2.db wallet init --genesis target/debug/genesis.json

First wallet2 must create an unassigned keypair. The public part of that keypair is then sent to the wallet who is the chain creator.

linera --wallet wallet2.json keygen
6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 # this is the public key for the unassigned keypair

Next, using the public key, wallet can open a chain for wallet2.

linera open-chain --to-public-key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888
e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000
fc9384defb0bcd8f6e206ffda32599e24ba715f45ec88d4ac81ec47eb84fa111

The first line is the message ID specifying the cross-chain message that creates the new chain. of the newly created chain. The second line is the new chain's ID.

Finally, to add the chain to wallet2 for the given unassigned key we use the assign command:

 linera --wallet wallet2.json assign --key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 --message-id e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000

Setting up Extra Wallets Automatically with linera net up

For testing, rather than using linera open-chain and linera assign as above, it is often more convenient to pass the option --extra-wallets N to linera net up.

This option will create create N additional user wallets and output Bash commands to define the environment variables LINERA_{WALLET,STORAGE}_$I where I ranges over 0..=N (I=0 being the wallet for the initial chains).

Once all the environment variables are defined, you may switch between wallets using linera --with-wallet I or linera -w I for short.

Automation in Bash

To automate the process of setting the variables LINERA_WALLET* and LINERA_STORAGE* after creating a local test network in a shell, we provide a Bash helper function linera_spawn_and_read_wallet_variables.

To define the function linera_spawn_and_read_wallet_variables in your shell, run source /dev/stdin <<<"$(linera net helper 2>/dev/null)". You may also add the output of linera net helper to your ~/.bash_profile for future sessions.

Once the function is defined, call linera_spawn_and_read_wallet_variables linera net up instead of linera net up.

Node Service

So far we've seen how to use the Linera client treating it as a binary in your terminal. However, the client also acts as a node which:

1. Executes blocks
2. Exposes an GraphQL API and IDE for dynamically interacting with applications and the system
3. Listens for notifications from validators and automatically updates local chains.

To interact with the node service, run linera in service mode:

linera service

This will run the node service on port 8080 by default (this can be overridden using the --port flag).

A Note on GraphQL

Linera uses GraphQL as the query language for interfacing with different parts of the system. GraphQL enables clients to craft queries such that they receive exactly what they want and nothing more.

GraphQL is used extensively during application development, especially to query the state of an application from a front-end for example.

To learn more about GraphQL check out the official docs.

GraphiQL IDE

Conveniently, the node service exposes a GraphQL IDE called GraphiQL. To use GraphiQL start the node service and navigate to localhost:8080/.

Using the schema explorer on the left of the GraphiQL IDE you can dynamically explore the state of the system and your applications.

GraphQL System API

The node service also exposes a GraphQL API which corresponds to the set of system operations. You can explore the full set of operations by clicking on MutationRoot.

GraphQL Application API

To interact with an application, we run the Linera client in service mode. It exposes a GraphQL API for every application running on any owned chain at localhost:8080/chains/<chain-id>/applications/<application-id>.

Navigating there with your browser will open a GraphiQL interface which enables you to graphically explore the state of your application.

Applications

The programming model of Linera is designed so that developers can take advantage of microchains to scale their applications.

Linera uses the WebAssembly Virtual Machine (Wasm) to execute user applications. Currently, the Linera SDK is focused on the Rust programming language.

The Application Deployment Lifecycle

Linera Applications are designed to be powerful yet re-usable. For this reason there is a distinction between the bytecode and an application instance on the network.

Applications undergo a lifecycle transition aimed at making development easy and flexible:

1. The bytecode is built from a Rust project with the linera-sdk dependency.
2. The bytecode is published to the network on a microchain, and assigned an identifier.
3. A user can create a new application instance, by providing the bytecode identifier and initialization arguments. This process returns an application identifier which can be used to reference and interact with the application.
4. The same bytecode identifier can be used as many times is needed by as many users are needed to create distinct applications.

Importantly, the application deployment lifecycle is abstracted from the user, and an application can be published with a single command:

linera publish-and-create <contract-path> <service-path> <init-args>

This will publish the bytecode as well as initialize the application for you.

Anatomy of an Application

An application is broken into two major components, the contract and the service.

The contract is gas-metered, and is the part of the application which executes operations and messages, make cross-application calls and modifies the application's state. The details are covered in more depth in the SDK docs.

The service is non-metered and read-only. It is used primarily to query the state of an application and populate the presentation layer (think front-end) with the data required for a user interface.

Finally, the application's state is shared by the contract and service in the form of a View, but more on that later.

Operations and Messages

For this section we'll be using a simplified version of the example application called "fungible" where users can send tokens to each other.

At the system-level, interacting with an application can be done via operations and messages.

Operations are defined by an application developer and each application can have a completely different set of operations. Chain owners then actively create operations and put them in their block proposals to interact with an application.

Taking the "fungible token" application as an example, an operation for a user to transfer funds to another user would look like this:

extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Operation {
    /// A transfer from a (locally owned) account to a (possibly remote) account.
    Transfer {
        owner: AccountOwner,
        amount: Amount,
        target_account: Account,
    },
    // Meant to be extended here
}

Messages result from the execution of operations or other messages. Messages can be sent from one chain to another. Block proposers also actively include messages in their block proposal, but unlike with operations, they are only allowed to include them in the right order (possibly skipping some), and only if they were actually created by another chain (or the same chain, earlier).

In our "fungible token" application, a message to credit an account would look like this:

extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Message {
    Credit { owner: AccountOwner, amount: Amount },
    // Meant to be extended here
}

Authentication

Operations are always authenticated and messages may be authenticated. The signer of a block becomes the authenticator of all the operations in that block. As operations are executed by applications, messages can be created to be sent to other chains. When they are created, they can be configured to be authenticated. In that case, the message receives the same authentication as the operation that created it. If handling an incoming message creates new messages, those may also be configured to have the same authentication as the received message.

In other words, the block signer can have its authority propagated across chains through series of messages. This allows applications to safely store user state in chains that the user may not have the authority to produce blocks. The application may also allow only the authorized user to change that state, and not even the chain owner is able to override that.

The figure below shows four chains (A, B, C, D) and some blocks produced in them. Some blocks show the operations and incoming messages they accept, where the authentication is shown inside parenthesis. All operations produced are authenticated by the block proposer, and if these are all single user chains, the proposer is always the chain owner. Messages that have authentication use the one from the operation or message that created it.

One example in the figure is that chain A produced a block with Operation 1, which is authenticated by the owner of chain A (written (A)). That operations sent a message to chain B, and assuming the message was sent with the authentication forwarding enabled, it is received and executed in chain B with the authentication of A. Another example is that chain D produced a block with Operation 2, which is authenticated by the owner of chain D (written (D)). That operation sent a message to chain C, which is executed with authentication of D like the example before. Handling that message in chain C produced a new message, which was sent to chain B. That message, when received by chain B is executed with the authentication of D.

                            ┌───┐     ┌─────────────────┐     ┌───┐
                    Chain A │   ├────►│ Operation 1 (A) ├────►│   │
                            └───┘     └────────┬────────┘     └───┘
                                               │
                                               └────────────┐
                                                            ▼
                                                ┌──────────────────────────┐
                            ┌───┐     ┌───┐     │ Operation 3 (B)          │
                    Chain B │   ├────►│   ├────►│ Message from chain A (A) │
                            └───┘     └───┘     │ Message from chain C (D) │
                                                └──────────────────────────┘
                                                            ▲
                                                   ┌────────┘
                                                   │
                            ┌───┐     ┌──────────────────────────┐     ┌───┐
                    Chain C │   ├────►│ Message from chain D (D) ├────►│   │
                            └───┘     └──────────────────────────┘     └───┘
                                                 ▲
                                     ┌───────────┘
                                     │
                            ┌─────────────────┐     ┌───┐     ┌───┐
                    Chain D │ Operation 2 (A) ├────►│   ├────►│   │
                            └─────────────────┘     └───┘     └───┘

An example where this is used is in the Fungible application, where a Claim operation allows retrieving money from a chain the user does not control (but the user still trusts will produce a block receiving their message). Without the Claim operation, users would only be able to store their tokens on their own chains, and multi-owner and public chains would have their tokens shared between anyone able to produce a block.

With the Claim operation, users can store their tokens on another chain where they're able to produce blocks or where they trust the owner will produce blocks receiving their messages. Only they are able to move their tokens, even on chains where ownership is shared or where they are not able to produce blocks.

Registering an Application across Chains

If Alice is using an application on her chain and starts interacting with Bob via the application, e.g. sends him some tokens using the fungible example, the application automatically gets registered on Bob's chain, too, as soon as he handles the incoming cross-chain messages. After that, he can execute the application's operations on his chain, too, and e.g. send tokens to someone.

But there are also cases where Bob may want to start using an application he doesn't have yet. E.g. maybe Alice regularly makes posts using the social example, and Bob wants to subscribe to her.

In that case, trying to execute an application-specific operation would fail, because the application is not registered on his chain. He needs to request it from Alice first:

linera request-application <application-id> --target-chain-id <alices-chain-id>

Once Alice processes his message (which happens automatically if she is running the client in service mode), he can start using the application.

Writing Linera Applications

In this section, we'll be exploring how to create Web3 applications using the Linera SDK.

We'll use a simple "counter" application as a running example.

We'll focus on the back end of the application, which consists of two main parts: a smart contract and its GraphQL service.

Both the contract and the service of an application are written in Rust using the crate linera-sdk, and compiled to Wasm bytecode.

This section should be seen as a guide versus a reference manual for the SDK. For the reference manual, refer to the documentation of the crate.

Creating a Linera Project

To create your Linera project, use the linera project new command. It sets up the scaffolding and requisite files:

linera project new my-counter

linera project new bootstraps your project by creating the following key files:

• Cargo.toml: your project's manifest filled with the necessary dependencies to create an app;
• src/lib.rs: the application's ABI definition;
• src/state.rs: the application's state;
• src/contract.rs: the application's contract, and the binary target for the contract bytecode;
• src/service.rs: the application's service, and the binary target for the service bytecode.

Creating the Application State

The struct which defines your application's state can be found in src/state.rs.

To represent our counter, we're going to need a single u64. To persist the counter we'll be using Linera's view paradigm.

Views are a little like an ORM, however instead of mapping data structures to a relational database like Postgres, they are instead mapped onto key-value stores like RocksDB.

In vanilla Rust, we might represent our Counter as so:

// do not use this
struct Counter {
  value: u64
}

However, to persist your data, you'll need to replace the existing Application state struct in src/state.rs with the following view:

/// The application state.
#[derive(RootView, GraphQLView)]
#[view(context = "ViewStorageContext")]
pub struct Counter {
    pub value: RegisterView<u64>,
}

and all other occurences of Application in your app.

The RegisterView<T> supports modifying a single value of type T. There are different types of views for different use-cases, but the majority of common data structures have already been implemented:

• A Vec or VecDeque corresponds to a LogView
• A BTreeMap corresponds to a MapView if its values are primitive, or to CollectionView if its values are other vuews;
• A Queue corresponds to a QueueView

For an exhaustive list refer to the Views documentation.

Finally, run cargo check to ensure that your changes compile.

Defining the ABI

The Application Binary Interface (ABI) of a Linera application defines how to interact with this application from other parts of the system. It includes the data structures, data types, and functions exposed by on-chain contracts and services.

ABIs are usually defined in src/lib.rs and compiled across all architectures (Wasm and native).

For a reference guide, check out the documentation of the crate.

Defining a marker struct

The library part of your application (generally in src/lib.rs) must define a public empty struct that implements the Abi trait.

struct CounterAbi;

The Abi trait combines the ContractAbi and ServiceAbi traits to include the types that your application exports.

/// A trait that includes all the types exported by a Linera application (both contract
/// and service).
pub trait Abi: ContractAbi + ServiceAbi {}

Next, we're going to implement each of the two traits.

Contract ABI

The ContractAbi trait defines the data types that your application uses in a contract. Each type represents a specific part of the contract's behavior:

/// A trait that includes all the types exported by a Linera application contract.
pub trait ContractAbi {
    /// Immutable parameters specific to this application (e.g. the name of a token).
    type Parameters: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// Initialization argument passed to a new application on the chain that created it
    /// (e.g. an initial amount of tokens minted).
    ///
    /// To share configuration data on every chain, use [`ContractAbi::Parameters`]
    /// instead.
    type InitializationArgument: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The type of operation executed by the application.
    ///
    /// Operations are transactions directly added to a block by the creator (and signer)
    /// of the block. Users typically use operations to start interacting with an
    /// application on their own chain.
    type Operation: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The type of message executed by the application.
    ///
    /// Messages are executed when a message created by the same application is received
    /// from another chain and accepted in a block.
    type Message: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The argument type when this application is called from another application on the same chain.
    type ApplicationCall: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The argument type when a session of this application is called from another
    /// application on the same chain.
    ///
    /// Sessions are temporary objects that may be spawned by an application call. Once
    /// created, they must be consumed before the current transaction ends.
    type SessionCall: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The type for the state of a session.
    type SessionState: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The response type of an application call.
    type Response: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}

All these types must implement the Serialize, DeserializeOwned, Send, Sync, Debug traits, and have a 'static lifetime.

In our example, we would like to change our InitializationArgument, Operation to u64, like so:

extern crate linera_base;
use linera_base::abi::ContractAbi;
struct CounterAbi;
impl ContractAbi for CounterAbi {
    type InitializationArgument = u64;
    type Parameters = ();
    type Operation = u64;
    type ApplicationCall = ();
    type Message = ();
    type SessionCall = ();
    type Response = ();
    type SessionState = ();
}

Service ABI

The ServiceAbi is in principle very similar to the ContractAbi, just for the service component of your application.

The ServiceAbi trait defines the types used by the service part of your application:

/// A trait that includes all the types exported by a Linera application service.
pub trait ServiceAbi {
    /// Immutable parameters specific to this application (e.g. the name of a token).
    type Parameters: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The type of a query receivable by the application's service.
    type Query: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The response type of the application's service.
    type QueryResponse: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}

For our Counter example, we'll be using GraphQL to query our application so our ServiceAbi should reflect that:

extern crate linera_base;
extern crate async_graphql;
use linera_base::abi::ServiceAbi;
struct CounterAbi;
impl ServiceAbi for CounterAbi {
    type Query = async_graphql::Request;
    type QueryResponse = async_graphql::Response;
    type Parameters = ();
}

Writing the Contract Binary

The contract binary is the first component of a Linera application. It can actually change the state of the application.

To create a contract, we need to implement the Contract trait, which is as follows:

#[async_trait]
pub trait Contract: WithContractAbi + ContractAbi + Send + Sized {
    /// The type used to report errors to the execution environment.
    type Error: Error + From<serde_json::Error> + From<bcs::Error> + 'static;

    /// The desired storage backend used to store the application's state.
    type Storage: ContractStateStorage<Self> + Send + 'static;

    /// Initializes the application on the chain that created it.
    async fn initialize(
        &mut self,
        context: &OperationContext,
        argument: Self::InitializationArgument,
    ) -> Result<ExecutionResult<Self::Message>, Self::Error>;

    /// Applies an operation from the current block.
    async fn execute_operation(
        &mut self,
        context: &OperationContext,
        operation: Self::Operation,
    ) -> Result<ExecutionResult<Self::Message>, Self::Error>;

    /// Applies a message originating from a cross-chain message.
    async fn execute_message(
        &mut self,
        context: &MessageContext,
        message: Self::Message,
    ) -> Result<ExecutionResult<Self::Message>, Self::Error>;

    /// Handles a call from another application.
    async fn handle_application_call(
        &mut self,
        context: &CalleeContext,
        argument: Self::ApplicationCall,
        forwarded_sessions: Vec<SessionId>,
    ) -> Result<ApplicationCallResult<Self::Message, Self::Response, Self::SessionState>, Self::Error>;

    /// Handles a call into a session created by this application.
    async fn handle_session_call(
        &mut self,
        context: &CalleeContext,
        session: Self::SessionState,
        argument: Self::SessionCall,
        forwarded_sessions: Vec<SessionId>,
    ) -> Result<SessionCallResult<Self::Message, Self::Response, Self::SessionState>, Self::Error>;

}

The full trait definition can be found here.

There's quite a bit going on here, so let's break it down and take one method at a time.

For this application, we'll be using the initialize and execute_operation methods.

Initializing our Application

The first thing we need to do is initialize our application by using Contract::initialize.

Contract::initialize is only called once when the application is created and only on the microchain that created the application.

Deployment on other microchains will use the Default implementation of the application state if SimpleStateStorage is used, or the Default value of all sub-views in the state if the ViewStateStorage is used.

For our example application, we'll want to initialize the state of the application to an arbitrary number that can be specified on application creation using its initialization parameters:

    async fn initialize(
        &mut self,
        _context: &OperationContext,
        value: u64,
    ) -> Result<ExecutionResult<Self::Message>, Self::Error> {
        self.value.set(value);
        Ok(ExecutionResult::default())
    }

Implementing the Increment Operation

Now that we have our counter's state and a way to initialize it to any value we would like, a way to increment our counter's value. Changes made by block proposers to application states are broadly called 'operations'.

To create a new operation, we need to use the method Contract::execute_operation. In the counter's case, it will be receiving a u64 which is used to increment the counter:

    async fn execute_operation(
        &mut self,
        _context: &OperationContext,
        operation: u64,
    ) -> Result<ExecutionResult<Self::Message>, Self::Error> {
        let current = self.value.get();
        self.value.set(current + operation);
        Ok(ExecutionResult::default())
    }

Declaring the ABI

Finally, to link our Contract trait implementation with the ABI of the application, the following code is added:

impl WithContractAbi for Counter {
    type Abi = counter::CounterAbi;
}

Writing the Service Binary

The service binary is the second component of a Linera application. It is compiled into a separate Bytecode from the contract and is run independently. It is not metered (meaning that querying an application's service does not consume gas), and can be thought of as a read-only view into your application.

Application states can be arbitrarily complex, and most of the time you don't want to expose this state in its entirety to those who would like to interact with your app. Instead, you might prefer to define a distinct set of queries that can be made against your application.

The Service trait is how you define the interface into your application. The Service trait is defined as follows:

/// The service interface of a Linera application.
#[async_trait]
pub trait Service: WithServiceAbi + ServiceAbi {
    /// Type used to report errors to the execution environment.
    type Error: Error + From<serde_json::Error>;

    /// The desired storage backend used to store the application's state.
    type Storage: ServiceStateStorage;

    /// Executes a read-only query on the state of this application.
    async fn handle_query(
        self: Arc<Self>,
        context: &QueryContext,
        argument: Self::Query,
    ) -> Result<Self::QueryResponse, Self::Error>;
}

The full service trait definition can be found here.

Let's implement Service for our counter application.

First, we want to generate the necessary boilerplate for implementing the service WIT interface, export the necessary resource types and functions so that the host (the process running the bytecode) can call the service. Happily, there is a macro to perform this code generation, so just add the following to service.rs:

linera_sdk::service!(Counter);

Next, we need to implement the Service for Counter. To do this we need to define Service's associated types and implement handle_query, as well as define the Error type:

#[async_trait]
impl Service for Counter {
    type Error = Error;
    type Storage = ViewStateStorage<Self>;

    async fn handle_query(
        self: Arc<Self>,
        _context: &QueryContext,
        request: Request,
    ) -> Result<Response, Self::Error> {
        let schema = Schema::build(
            // implemented in the next section
            QueryRoot { value: *self.value.get() },
            // implemented in the next section
            MutationRoot {},
            EmptySubscription,
        )
        .finish();
        Ok(schema.execute(request).await)
    }
}

/// An error that can occur during the contract execution.
#[derive(Debug, Error)]
pub enum Error {
    /// Invalid query argument; could not deserialize GraphQL request.
    #[error("Invalid query argument; could not deserialize GraphQL request")]
    InvalidQuery(#[from] serde_json::Error),
}

Finally, as before, the following code is needed to incorporate the ABI definitions into your Service implementation:

impl WithServiceAbi for Counter {
    type Abi = counter::CounterAbi;
}

Adding GraphQL compatibility

Finally, we want our application to have GraphQL compatibility. To achieve this we need a QueryRoot for intercepting queries and a MutationRoot for introspection queries for mutations.

struct MutationRoot;

#[Object]
impl MutationRoot {
    async fn increment(&self, value: u64) -> Vec<u8> {
        bcs::to_bytes(&value).unwrap()
    }
}

struct QueryRoot {
    value: u64,
}

#[Object]
impl QueryRoot {
    async fn value(&self) -> &u64 {
        &self.value
    }
}

We haven't included the imports in the above code; they are left as an exercise to the reader (but remember to import async_graphql::Object). If you want the full source code and associated tests check out the examples section on GitHub.

Deploying the Application

To deploy your application, build your contract in release mode with cargo build --release and then use the publish-and-create command while also specifying:

1. The location of the contract bytecode
2. The location of the service bytecode
3. The hex encoded initialization arguments

linera publish-and-create \
  target/wasm32-unknown-unknown/release/my-counter_{contract,service}.wasm \
  --json-argument "42"

Cross-Chain Messages

On Linera, applications are meant to be multi-chain: They are instantiated on every chain where they are used. An application has the same application ID and bytecode everywhere, but a separate state on every chain. To coordinate, the instances can send cross-chain messages to each other. A message sent by an application is always handled by the same application on the target chain: The handling code is guaranteed to be the same as the sending code, but the state may be different.

For your application, you can specify any serializable type as the Message type in your ContractAbi implementation. To send a message, return it among the ExecutionResult's messages:

    pub messages: Vec<OutgoingMessage<Message>>,

The first field specifies either a single destination chain, or a channel, so that it gets sent to all subscribers.

If the second field is true, the callee is allowed to perform actions that require authentication on behalf of the signer of the original block that caused this call.

The third field is the message itself, of the type you specified in the ContractAbi.

You can also use ExecutionResult::with_message and with_authenticated_message for convenience.

During block execution in the sending chain, messages are returned via ExecutionResults. The returned message is then placed in the target chain inbox for processing. There is no guarantee that it will be handled: For this to happen, an owner of the target chain needs to include it in the incoming_messages in one of their blocks. When that happens, the contract's execute_message method gets called on their chain.

Example: Fungible Token

In the fungible example application, such a message can be the transfer of tokens from one chain to another. If the sender includes a Transfer operation on their chain, it decreases their account balance and sends a Credit message to the recipient's chain:

async fn execute_operation(
    &mut self,
    context: &OperationContext,
    operation: Self::Operation,
) -> Result<ExecutionResult<Self::Message>, Self::Error> {
    match operation {
        Operation::Transfer {
            owner,
            amount,
            target_account,
        } => {
            // ...
            self.debit(owner, amount).await?;
            let message = Message::Credit {
                owner: target_account.owner,
                amount,
            };
            Ok(ExecutionResult::default().with_message(target_account.chain_id, message))
        }
        // ...
    }
}

On the recipient's chain, execute_message is called, which increases their account balance.

async fn execute_message(
    &mut self,
    context: &MessageContext,
    message: Message,
) -> Result<ExecutionResult<Self::Message>, Self::Error> {
    match message {
        Message::Credit { owner, amount } => {
            self.credit(owner, amount).await;
            Ok(ExecutionResult::default())
        }
        // ...
    }
}

Calling other Applications

We have seen that cross-chain messages sent by an application on one chain are always handled by the same application on the target chain.

This section is about calling other applications using cross-application calls.

Such calls happen on the same chain and typically use the call_application method implemented by default in the trait Contract:

async fn call_application<A: ContractAbi + Send>(
    &mut self,
    authenticated: bool,
    application: ApplicationId<A>,
    call: &A::ApplicationCall,
    forwarded_sessions: Vec<SessionId>,
) -> Result<(A::Response, Vec<SessionId>), Self::Error> { .. }

The authenticated argument specifies whether the callee is allowed to perform actions that require authentication on behalf of the signer of the original block that caused this call.

The application argument is the callee's application ID, and A is the callee's ABI.

call are the arguments of the application call, in a type defined by the callee.

forwarded_sessions are session data that need to be consumed within this transaction. Sessions will be explained in a separate section.

Example: Crowd-Funding

The crowd-funding example application allows the application creator to launch a campaign with a funding target. That target can be an amount specified in any type of token based on the fungible application. Others can then pledge tokens of that type to the campaign, and if the target is not reached by the deadline, they are refunded.

If Alice used the fungible example to create a Pugecoin application (with an impressionable pug as its mascot), then Bob can create a crowd-funding application, use Pugecoin's application ID as CrowdFundingAbi::Parameters, and specify in CrowdFundingAbi::InitializationArgument that his campaign will run for one week and has a target of 1000 Pugecoins.

Now let's say Carol wants to pledge 10 Pugecoin tokens to Bob's campaign.

First she needs to make sure she has his crowd-funding application on her chain, e.g. using the linera request-application command. This will automatically also register Alice's application on her chain, because it is a dependency of Bob's.

Now she can make her pledge by running the linera service and making a query to Bob's application:

mutation { pledgeWithTransfer(owner: "User:841…6c0", amount: "10") }

This will add a block to Carol's chain containing the pledge operation that gets handled by CrowdFunding::execute_operation, resulting in one cross-application call and two cross-chain messages:

First CrowdFunding::execute_operation calls the fungible application on Carol's chain to transfer 10 tokens to Carol's account on Bob's chain:

self.call_application(
    true,                 // The call is authenticated by Carol, who signed this block.
    Self::fungible_id()?, // The Pugecoin application ID.
    &fungible::ApplicationCall::Transfer {
        owner,            // Carol
        amount,           // 10 tokens
        destination,      // Bob's chain.
    },
    vec![],
).await?;

This causes Fungible::handle_application_call to be run, which will create a cross-chain message sending the amount 10 to the Pugecoin application instance on Bob's chain.

After the cross-application call returns, CrowdFunding::execute_operation continues to create another cross-chain message crowd_funding::Message::PledgeWithAccount, which informs the crowd-funding application on Bob's chain that the 10 tokens are meant for the campaign.

When Bob now adds a block to his chain that handles the two incoming messages, first Fungible::execute_message gets executed, and then CrowdFunding::execute_message. The latter makes another cross-application call to transfer the 10 tokens from Carol's account to the crowd-funding application's account (both on Bob's chain). That is successful because Carol does now have 10 tokens on this chain and she authenticated the transfer indirectly by signing her block. The crowd-funding application now makes a note in its application state on Bob's chain that Carol has pledged 10 Pugecoin tokens.

For the complete code please take a look at the crowd-funding and fungible applications in the examples folder in linera-protocol.

Printing Logs from an Application

Applications can use the log crate to print log messages with different levels of importance. Log messages are useful during development, but they may also be useful for end users. By default the linera service command will log the messages from an application if they are of the "info" importance level or higher (briefly, log::info!, log::warn! and log::error!).

During development it is often useful to log messages of lower importance (such as log::debug! and log::trace!). To enable them, the RUST_LOG environment variable must be set before running linera service. The example below enables trace level messages from applications and enables warning level messages from other parts of the linera binary:

export RUST_LOG="warn,linera_execution::wasm=trace"

Writing Tests

Linera applications can be tested using unit tests or integration tests. Both are a bit different than usual Rust tests. Unit tests are executed inside a WebAssembly virtual machine in an environment that simulates a single microchain and a single application. System APIs are only available if they are mocked using helper functions from linera_sdk::test.

Integration tests run outside a WebAssembly virtual machine, and use a simulated validator for testing. This allows testing interactions between multiple microchains and multiple applications. However, it is also more low-level because testing the application requires interacting with microchains, so execution only happens when blocks are published.

Applications should consider having both types of tests. Unit tests should be used to focus on the application's internals and core functionality. Integration tests should be used to test how the application behaves on a more complex environment that's closer to the real network.

Unit tests

Unit tests can be written beside the application's source code (i.e., inside the src/ directory of the project). There are two major differences to normal Rust unit tests:

• linera project test is used instead of cargo test (or the environment must be configured so that Cargo uses a custom test runner, as described below);
• the #[webassembly_test] attribute is used instead of the usual #[test] attribute.

Manually Configuring the Environment

Running linera project test is easier, but if there's a need to run cargo test explicitly to run the unit tests, Cargo must be configured to use the custom test-runner. The test-runner can be built from the repository.

cd linera-protocol
cargo build -p linera-sdk --bin test-runner --release

The steps above build the test-runner and places the resulting binary at linera-protocol/target/release/test-runner.

With the binary available, the last step is to configure Cargo. There are a few ways to do this. A quick way is to set the CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER environment variable to the path of the binary. A more persistent way is to change one of Cargo's configuration files. As an example, the following file can be placed inside the project's directory at PROJECT_DIR/.cargo/config.toml:

[target.wasm32-unknown-unknown]
runner = "PATH_TO/test-runner"

After configuring the test runner, unit tests can be executed with

cargo test --target wasm32-unknown-unknown

Example

A simple unit test is shown below, which tests if the application's do_something method changes the application state.

#[cfg(test)]
mod tests {
    use crate::state::ApplicationState;
    use webassembly_test::webassembly_test;

    #[webassembly_test]
    fn test_do_something() {
        let mut application = ApplicationState {
            // Configure the application's initial state
            ..ApplicationState::default()
        };

        let result = application.do_something();

        assert!(result.is_ok());
        assert_eq!(application, ApplicationState {
            // Define the application's expected final state
            ..ApplicationState::default()
        });
    }
}

Mocking System APIs

Unit tests run in a constrained environment, so things like access to the key-value store, cross-chain messages and cross-application calls can't be executed. However, they can be simulated using mock APIs. The linera-sdk::test module provides some helper functions to mock the system APIs.

Here's an example mocking the key-value store.

#[cfg(test)]
mod tests {
    use crate::state::ApplicationState;
    use linera_sdk::test::mock_key_value_store;
    use webassembly_test::webassembly_test;

    #[webassembly_test]
    fn test_state_is_not_persisted() {
        let mut storage = mock_key_value_store();

        // Assuming the application uses views
        let mut application = ApplicationState::load(storage.clone())
            .now_or_never()
            .expect("Mock key-value store returns immediately")
            .expect("Failed to load view from mock key-value store");

        // Assuming `do_something` changes the view, but does not persist it
        let result = application.do_something();

        assert!(result.is_ok());

        // Check that the state in memory is different from the state in storage
        assert_ne!(application, ApplicatonState::load(storage));
    }
}

Integration Tests

Integration tests are usually written separately from the application's source code (i.e., inside a tests directory that's beside the src directory). Integration tests are normal Rust integration tests, and they are compiled to the host target instead of the wasm32-unknown-unknown target used for unit tests. This is because unit tests run inside a WebAssembly virtual machine and integration tests run outside a virtual machine, starting isolated virtual machines to run each operation of each block added to each chain.

Integration tests use the helper types from linera_sdk::test to set up a simulated Linera network, and publish blocks to microchains in order to execute the application.

Example

A simple test that sends a message between application instances on different chains is shown below.

#[tokio::test]
async fn test_cross_chain_message() {
    let (validator, application_id) = TestValidator::with_current_application(vec![], vec![]).await;

    let mut sender_chain = validator.get_chain(application_id.creation.chain_id).await;
    let mut receiver_chain = validator.new_chain().await;

    sender_chain
        .add_block(|block| {
            block.with_operation(
                application_id,
                Operation::SendMessageTo(receiver_chain.id()),
            )
        })
        .await;

    receiver_chain.handle_received_messages().await;

    assert_eq!(
        receiver_chain
            .query::<ChainId>(application_id, Query::LastSender)
            .await,
        sender_chain.id(),
    );
}

Advanced Topics

In this section, we present additional topics related to the Linera protocol.

Views

Views are a specific functionality of the Linera system that allow to have data in memory and then seamlessly flush it to an underlying persistent datastore.

The full documentation is available on the crate documentation with all functions having examples.

Concretely, what is provided is the following:

• A trait View that provides load, rollback, clear, flush, delete. The idea is that we can do operation on the data and then flush it to the database storing them.

• Several other traits HashableView, RootView, CryptoHashView, CryptoHashRootView that are important for computing hash.

• A number of standard containers: MapView, SetView, LogView, QueueView, RegisterView that implement the View and HashableView traits.

• Two containers CollectionView and ReentrantCollectionView that are similar to MapView but whose values are views themselves.

• Derive macros that allow to implement the above mentioned traits on struct data types whose entries are views.

Persistent storage

Validators run the servers and the data is stored in persistent storage. As a consequence we need a tool for working with persistent storage and so we have added linera-db for that purpose.

Available persistent storage

The persistent storage that are available right now are RocksDB, DynamoDB and ScyllaDB. Each has its own strengths and weaknesses.

• RocksDB: Data is stored on disk and cannot be shared between shards but is very fast.

• DynamoDB: Data is stored on a remote storage, that has to be on AWS. Data can be shared between shards.

• ScyllaDB: Data is stored on a remote storage. Data can be shared between shards.

There is no fundamental obstacle to the addition of other persistent storage solutions.

In addition, the DynamoDB and ScyllaDB have the notion of a table which means that a given remote location can be used for several completely independent purposes.

The linera-db tool

When operating on a persistent storage some global operations can be required. The command line tool linera-db helps in making them work.

The functionalities are the following:

• list_tables(DynamoDB and ScyllaDB): It lists all the tables that have been created on the persistent storage

• initialize(RocksDB, DynamoDB and ScyllaDB): It initializes a persistent storage.

• check_existence(RocksDB, DynamoDB and ScyllaDB): It tests the existence of a persistent storage. If the error code is 0 then the table exists, if the error code is 1 then the table is absent.

• check_absence(RocksDB, DynamoDB and ScyllaDB): It tests the absence of a persistent storage. If the error code is 0 then the table is absent, if the error code is 1 then the table does not exist.

• delete_all(RocksDB, DynamoDB and ScyllaDB): It deletes all the table of a persistent storage.

• delete_single(DynamoDB and ScyllaDB): It deletes a single table of a persistent storage.

If some error occurs during the operation, then the error code 2 is returned and 0 if everything went fine with the exception of check_existence and check_absence for which the value 1 can occur if the connection with the database was established correctly but the result is not what we expected.

Validators

Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.

In Linera, every chain is backed by the same set of validators and has the same level of security.

The main function of validators is to guarantee the integrity of the infrastructure in the sense that:

• Each block is valid, i.e. it has the correct format, its operations are allowed, the received messages are in the correct order, and e.g. the balance was correctly computed.

• Every message received by one chain was actually sent by another chain.

• If one block on a particular height is certified, no other block on the same height is.

These properties are guaranteed to hold as long as two third of the validators (weighted by their stake) follow the protocol. In the future, deviating from the protocol may cause a validator to be considered malicious and to lose their stake.

Validators also play a role in the liveness of the system by making sure that the history of the chains stays available. However, since validators do not propose blocks on most chains (see next section), they do not guarantee that any particular operation or message will eventually be executed on a chain. Instead, chain owners decide whether and when to propose new blocks, and which operations and messages to include. The current implementation of the Linera client automatically includes all incoming messages in new blocks. The operations are the actions the chain owner explicitly adds, e.g. transfer.

Architecture of a validator

Since every chain uses the same validators, adding more chains does not require adding validators. Instead, it requires each individual validator to scale out by adding more computation units, also known as "workers" or "physical shards".

In the end, a Linera validator resembles a Web2 service made of

• a load balancer (aka ingress/egress), currently implemented by the binary linera-proxy,

• a number of workers, currently implemented by the binary linera-server,

• a shared database, currently implemented by the abstract interface linera-storage.

Example of Linera network

                    │                                             │
                    │                                             │
┌───────────────────┼───────────────────┐     ┌───────────────────┼───────────────────┐
│ validator 1       │                   │     │ validator N       │                   │
│             ┌─────┴─────┐             │     │             ┌─────┴─────┐             │
│             │   load    │             │     │             │   load    │             │
│       ┌─────┤  balancer ├────┐        │     │       ┌─────┤  balancer ├──────┐      │
│       │     └───────────┘    │        │     │       │     └─────┬─────┘      │      │
│       │                      │        │     │       │           │            │      │
│       │                      │        │     │       │           │            │      │
│  ┌────┴─────┐           ┌────┴─────┐  │     │  ┌────┴───┐  ┌────┴────┐  ┌────┴───┐  │
│  │  worker  ├───────────┤  worker  │  │ ... │  │ worker ├──┤  worker ├──┤ worker │  │
│  │    1     │           │    2     │  │     │  │    1   │  │    2    │  │    3   │  │
│  └────┬─────┘           └────┬─────┘  │     │  └────┬───┘  └────┬────┘  └────┬───┘  │
│       │                      │        │     │       │           │            │      │
│       │                      │        │     │       │           │            │      │
│       │     ┌───────────┐    │        │     │       │     ┌─────┴─────┐      │      │
│       └─────┤  shared   ├────┘        │     │       └─────┤  shared   ├──────┘      │
│             │ database  │             │     │             │ database  │             │
│             └───────────┘             │     │             └───────────┘             │
└───────────────────────────────────────┘     └───────────────────────────────────────┘

Inside a validator, components communicate using the internal network of the validator. Notably, workers use direct Remote Procedure Calls (RPCs) with each other to deliver cross-chain messages.

Note that the number of workers may vary for each validator. Both the load balancer and the shared database are represented as a single entity but are meant to scale out in production.

For local testing during development, we currently use a single worker and RocksDB as a database.

Creating New Blocks

In Linera, the responsibility of proposing blocks is separate from the task of validating blocks.

While all chains are validated in the same way, the Linera protocol defines several types of chains, depending on how new blocks are produced.

• The simplest and lowest-latency type of chain is called single-owner chain.

• Other types of Linera chains not currently supported in the SDK include permissioned chains and public chains (see the whitepaper for more context).

For most types of chains (all but public chains), Linera validators do not need to exchange messages with each other.

Instead, the wallets (aka linera clients) of chain owners make the system progress by proposing blocks and actively providing any additional required data to the validators. For instance, client commands such as transfer, publish-bytecode, or open-chain perform multiple steps to append a block containing the token transfer, application publishing, or chain creation operation:

• The Linera client creates a new block containing the desired operation and new incoming messages, if there are any. It also contains the most recent block's hash to designate its parent. The client sends the new block to all validators.

• The validators validate the block, i.e. check that the block satisfies the conditions listed above, and send a cryptographic signature to the client, indicating that they vote to append the new block. But only if they have not voted for a different block on the same height earlier!

• The client ideally receives a vote from every validator, but only a quorum of votes (say, two thirds) are required: These constitute a "certificate", proving that the block was confirmed. The client sends the certificate to every validator.

• The validators "execute" the block: They update their own view of the most recent state of the chain by applying all messages and operations, and if it generated any cross-chain messages, they send these to the appropriate workers.

To guarantee that each incoming message in a block was actually sent by another chain, a validator will, in the second step, only vote for a block if it has already executed the block that sent it. However, when receiving a valid certificate for a block that receives a message it has not seen yet, it will accept and execute the block anyway. The certificate is proof that most other validators have seen the message, so it must be correct.

In the case of single-owner chains, clients must be carefully implemented so that they never propose multiple blocks at the same height. Otherwise, the chain may be stuck: once each of the two conflicting blocks has been signed by enough validators, it becomes impossible to collect a quorum of votes for either block.

In the future, we anticipate that most users will use permissioned chains even if they are the only owners of their chains. Permissioned chains have two confirmation steps instead of one, but it is not possible to accidentally make a chain unextendable. They also allow users to delegate certain admistrative tasks to third-parties, notably to help with epoch changes (i.e. when the validators change if reconfigured).

Glossary

• Address: A unique public alphanumeric identifier used to designate the identity of an entity on the Linera network.

• Admin Chain: The Linera Network has one designated admin chain where validators can join or leave and where new epochs are defined.

• Application: Similar to a smart-contract on Ethereum, an application is code deployed on the Linera network which is executed by all validators. An application has a metered contract which executes 'business logic' and modifies state and an unmetered 'service' which is a read-only view into an application's state.

• Byzantine Fault-Tolerant (BFT): A system which can operate correctly and achieve consensus even if components of the system fail or act maliciously.

• Block Height: The number of blocks preceding a given block on a specific microchain.

• Block Proposal: A candidate block proposed by a chain owner which may be selected at the next block height.

• Bytecode: A collection of bytes corresponding to a program that can be run by the Wasm virtual machine.

• Client: A 'partial node' operated by users which interfaces with the network. In Linera, clients drive the network by proposing new blocks and validators are mostly reactive.

• Certificate: A value with signatures from a quorum of validators. Values can be confirmed blocks, meaning that the block has been added to the chain and is final. There are other values that are used for reaching consensus, before certifying a confirmed block.

• Committee: The set of all validators for a particular epoch, together with their voting weights.

• Chain Owner: The owner of a user chain or permissioned chain. This is represented as the alphanumeric identifier derived from the hash of the owner's public key.

• Channel: A broadcast mechanism enabling publish-subscribe behavior across chains.

• Contract: The metered part of an application which executes business logic and can modify the application's state.

• Cross-Application Call: A call from one application to another on the same chain.

• Cross-Chain Message: A message containing a data payload which is sent from one chain to another. Cross-Chain messages are the asynchronous communication primitive which enable communication on the same application running on different chains.

• Epoch: A period of time when a particular set of validators with particular voting weights can certify new blocks. Since each chain has to transition explicitly from one epoch to the next, epochs can overlap.

• Genesis configuration: The configuration determining the state of a newly created network; the voting weights of the initial set of validators, the initial fee structure, and initial chains that the network starts with.

• Inbox: A commutative data structure storing incoming messages for a given chain.

• Message: See 'Cross-Chain Message'.

• Microchain: A lightweight chain of blocks holding a subset of the network's state running on every validator. This is used interchangeably with 'chain'. All Linera chains are microchains.

• Network: The totality of all protocol participants. A network is the combination of committee, clients and auditors.

• Operation: Operations are transactions directly added to a block by the creator (and signer) of the block. Users typically use operations to start interacting with an application on their own chain.

• Permissioned Chain: A microchain which is owned by more than one user. Users take turns proposing blocks and the likelihood of selection is proportional to their weight.

• Project: The collection of files and dependencies which are built into the bytecode which is instantiated as an application on the Linera Network.

• Public Chain: A microchain with full BFT consensus with a strict set of permissions relied on for the operation of the network,

• Quorum: A set of validators representing > ⅔ of the total stake. A quorum is required to create a certificate.

• Single-Owner Chain: See 'User Chain'.

• Service: An unmetered read-only view into an application's state.

• Shard: A logical subset of all microchains on a given validator. This corresponds directly to a physical worker.

• Stake: An amount of tokens pledged by a validator or auditor, as a collateral to guarantee their honest and correct participation in the network.

• User Chain: Used interchangeably with Single-Owner Chain. User chains are chains which are owned by a single user on the network. Only the chain owner can propose blocks, and therefore only the chain owner can forcibly advance the state of a user chain.

• Validator: Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.

• View: Views are like an Object-Relational Mapping (ORM) for mapping complex types onto key-value stores. Views group complex state changes into a set of elementary operations and commit them atomically. The are full or partial in-memory representations of complex types saved on disk in a key-value store

• Wallet: A file containing a user's public and private keys along with configuration and information regarding the chains they own.

• Web Assembly (Wasm): A binary compilation target and instruction format that runs on a stack based VM. Linera applications are compiled to Wasm and run on Wasm VMs inside validators and clients.

• Web3: A natural evolution of the internet focusing on decentralization by leveraging blockchains and smart contracts.

• Worker: A process which runs a subset of all microchains on a given validator. This corresponds directly to a logical shard.

Glossary

• Address: A unique public alphanumeric identifier used to designate the identity of an entity on the Linera network.

• Admin Chain: The Linera Network has one designated admin chain where validators can join or leave and where new epochs are defined.

• Application: Similar to a smart-contract on Ethereum, an application is code deployed on the Linera network which is executed by all validators. An application has a metered contract which executes 'business logic' and modifies state and an unmetered 'service' which is a read-only view into an application's state.

• Byzantine Fault-Tolerant (BFT): A system which can operate correctly and achieve consensus even if components of the system fail or act maliciously.

• Block Height: The number of blocks preceding a given block on a specific microchain.

• Block Proposal: A candidate block proposed by a chain owner which may be selected at the next block height.

• Bytecode: A collection of bytes corresponding to a program that can be run by the Wasm virtual machine.

• Client: A 'partial node' operated by users which interfaces with the network. In Linera, clients drive the network by proposing new blocks and validators are mostly reactive.

• Certificate: A value with signatures from a quorum of validators. Values can be confirmed blocks, meaning that the block has been added to the chain and is final. There are other values that are used for reaching consensus, before certifying a confirmed block.

• Committee: The set of all validators for a particular epoch, together with their voting weights.

• Chain Owner: The owner of a user chain or permissioned chain. This is represented as the alphanumeric identifier derived from the hash of the owner's public key.

• Channel: A broadcast mechanism enabling publish-subscribe behavior across chains.

• Contract: The metered part of an application which executes business logic and can modify the application's state.

• Cross-Application Call: A call from one application to another on the same chain.

• Cross-Chain Message: A message containing a data payload which is sent from one chain to another. Cross-Chain messages are the asynchronous communication primitive which enable communication on the same application running on different chains.

• Epoch: A period of time when a particular set of validators with particular voting weights can certify new blocks. Since each chain has to transition explicitly from one epoch to the next, epochs can overlap.

• Genesis configuration: The configuration determining the state of a newly created network; the voting weights of the initial set of validators, the initial fee structure, and initial chains that the network starts with.

• Inbox: A commutative data structure storing incoming messages for a given chain.

• Message: See 'Cross-Chain Message'.

• Microchain: A lightweight chain of blocks holding a subset of the network's state running on every validator. This is used interchangeably with 'chain'. All Linera chains are microchains.

• Network: The totality of all protocol participants. A network is the combination of committee, clients and auditors.

• Operation: Operations are transactions directly added to a block by the creator (and signer) of the block. Users typically use operations to start interacting with an application on their own chain.

• Permissioned Chain: A microchain which is owned by more than one user. Users take turns proposing blocks and the likelihood of selection is proportional to their weight.

• Project: The collection of files and dependencies which are built into the bytecode which is instantiated as an application on the Linera Network.

• Public Chain: A microchain with full BFT consensus with a strict set of permissions relied on for the operation of the network,

• Quorum: A set of validators representing > ⅔ of the total stake. A quorum is required to create a certificate.

• Single-Owner Chain: See 'User Chain'.

• Service: An unmetered read-only view into an application's state.

• Shard: A logical subset of all microchains on a given validator. This corresponds directly to a physical worker.

• Stake: An amount of tokens pledged by a validator or auditor, as a collateral to guarantee their honest and correct participation in the network.

• User Chain: Used interchangeably with Single-Owner Chain. User chains are chains which are owned by a single user on the network. Only the chain owner can propose blocks, and therefore only the chain owner can forcibly advance the state of a user chain.

• Validator: Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.

• View: Views are like an Object-Relational Mapping (ORM) for mapping complex types onto key-value stores. Views group complex state changes into a set of elementary operations and commit them atomically. The are full or partial in-memory representations of complex types saved on disk in a key-value store

• Wallet: A file containing a user's public and private keys along with configuration and information regarding the chains they own.

• Web Assembly (Wasm): A binary compilation target and instruction format that runs on a stack based VM. Linera applications are compiled to Wasm and run on Wasm VMs inside validators and clients.

• Web3: A natural evolution of the internet focusing on decentralization by leveraging blockchains and smart contracts.

• Worker: A process which runs a subset of all microchains on a given validator. This corresponds directly to a logical shard.

Videos

We're consistently improving the SDK and creating new material. We recorded the following tutorial videos for the Linera Developer Summer School 2023:

• Build and deploy your first applications

• Add testing, logging, and more to your app

• Use cross-application calls to call other apps

Happy viewing!