The Linera Manual

Welcome to the Linera manual, a decentralized protocol designed for highly scalable, low-latency Web3 applications.

The documentation is split in two parts:

1. The Developers section is intended for developers who wish to learn more about Linera and its programming model by prototyping applications on top of the Linera Rust SDK.
2. The Operators section is intended for operators who wish to run Linera validators.

NEW: Publish and test your Web3 application on the Linera Devnet!

Install the Linera CLI tool then follow the instructions on this page to claim a microchain and publish your first application on the current Devnet.

To join our community and get involved in the development of the Linera ecosystem, check out our GitHub repository, our website, and find us on social media channels such as youtube, twitter, telegram, and discord.

Let's get started!

Developer Manual

This section of the Linera Manual is for developers building applications on Linera.

Getting started

In this section, we will cover the necessary steps to install the Linera toolchain and give a short example to get started with the Linera SDK.

Installation

Let's start with the installation of the Linera development tools.

Overview

The Linera toolchain consist of two crates:

• linera-sdk is the main library to program Linera applications in Rust.

• linera-service defines a number of binaries, including:

  • linera -- the main client tool, used to operate development wallets,
  • linera-proxy -- the proxy service, acting as a public entrypoint for each validator,
  • linera-server -- the service run by each worker of a validator, hidden behind the proxy.

Requirements

The operating systems currently supported by the Linera toolchain can be summarized as follows:

The main prerequisites to install the Linera toolchain are Rust, Wasm, and Protoc. They can be installed as follows on Linux:

• Rust and Wasm

  • curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  • rustup target add wasm32-unknown-unknown

• Protoc

  • curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v21.11/protoc-21.11-linux-x86_64.zip
  • unzip protoc-21.11-linux-x86_64.zip -d $HOME/.local
  • If ~/.local is not in your path, add it: export PATH="$PATH:$HOME/.local/bin"

• On certain Linux distributions, you may have to install development packages such as g++, libclang-dev and libssl-dev.

For MacOS support and for additional requirements needed to test the Linera protocol itself, see the installation section on GitHub.

This manual was tested with the following Rust toolchain:

[toolchain]
channel = "1.77.2"
components = [ "clippy", "rustfmt", "rust-src" ]
targets = [ "wasm32-unknown-unknown" ]
profile = "minimal"

Installing from crates.io

You may install the Linera binaries with

cargo install --locked linera-service@0.11.3

and use linera-sdk as a library for Linera Wasm applications:

cargo add linera-sdk@0.11.3

The version number 0.11.3 corresponds to the current Devnet of Linera and may change frequently.

Installing from GitHub

Download the source from GitHub:

git clone https://github.com/linera-io/linera-protocol.git
cd linera-protocol
git checkout -t origin/devnet_2024_05_07  # Current release branch

To install the Linera toolchain locally from source, you may run:

cargo install --locked --path linera-service

Alternatively, for developing and debugging, you may instead use the binaries compiled in debug mode, e.g. using export PATH="$PWD/target/debug:$PATH".

This manual was tested against the following commit of the repository:

2ada2e77e6a2f3dfa3bd32f4dc609bdadd0fbf3a

Bash helper (optional)

Consider adding the output of linera net helper to your ~/.bash_profile to help with automation.

Getting help

If installation fails, reach out to the team (e.g. on Discord) to help troubleshoot your issue or create an issue on GitHub.

Hello, Linera

This section is about interacting with the Devnet, running a local development network, then compiling and deploying your first application from scratch.

By the end of this section, you'll have a microchain on the Devnet and/or on your local network, and a working application that can be queried using GraphQL.

Using the Devnet

The Linera Devnet is a deployment of the Linera protocol that's useful for developers. It should not be considered stable, and can be restarted from a clean slate and new genesis at any time.

To interact with the Devnet, some tokens are needed. A Faucet service is available to create new microchains and obtain some test tokens. To do so, this must be configured when initializing the wallet:

linera wallet init --with-new-chain --faucet https://faucet.devnet-2024-05-07.linera.net

This creates a new microchain on Devnet with some initial test tokens, and the chain is automatically added to the newly instantiated wallet.

Make sure to use a Linera toolchain compatible with the current Devnet.

Starting a Local Test Network

Another option is to start your own local development network. A development network consists of a number of validators, each of which consist of an ingress proxy (aka. a "load balancer") and a number of workers (aka. "physical shards").

To start a local network, run the following command:

linera net up

This will start a validator with the default number of shards and create a temporary directory storing the entire network state.

This will set up a number of initial chains and create an initial wallet to operate them.

Using the Initial Test Wallet

linera net up prints Bash statements on its standard output to help you configure your terminal to use the initial wallet of the new test network, for instance:

export LINERA_WALLET="/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/wallet.json"
export LINERA_STORAGE="rocksdb:/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/linera.db"

This wallet is only valid for the lifetime of a single network. Every time a local network is restarted, the wallet needs to be reconfigured.

Interacting with the Network

In the following examples, we assume that either the wallet was initialized to interact with the Devnet or the variables LINERA_WALLET and LINERA_STORAGE are both set and point to the initial wallet of the running local network.

The main way of interacting with the network and deploying applications is using the linera client.

To check that the network is working, you can synchronize your default chain with the rest of the network and display the chain balance as follows:

linera sync
linera query-balance

You should see an output number, e.g. 10.

Building an Example Application

Applications running on Linera are Wasm bytecode. Each validator and client has a built-in Wasm virtual machine (VM) which can execute bytecode.

Let's build the counter application from the examples/ subdirectory:

cd examples/counter && cargo build --release --target wasm32-unknown-unknown

Publishing your Application

You can publish the bytecode and create an application using it on your local network using the linera client's publish-and-create command and provide:

1. The location of the contract bytecode
2. The location of the service bytecode
3. The JSON encoded initialization arguments

linera publish-and-create \
  ../target/wasm32-unknown-unknown/release/counter_{contract,service}.wasm \
  --json-argument "42"

Congratulations! You've published your first application on Linera!

Querying your Application

Now let's query your application to get the current counter value. To do that, we need to use the client running in service mode. This will expose a bunch of APIs locally which we can use to interact with applications on the network.

linera service

Navigate to http://localhost:8080 in your browser to access the GraphiQL, the GraphQL IDE. We'll look at this in more detail in a later section; for now, list the applications deployed on your default chain e476… by running:

query {
  applications(
    chainId: "e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65"
  ) {
    id
    description
    link
  }
}

Since we've only deployed one application, the results returned have a single entry.

At the bottom of the returned JSON there is a field link. To interact with your application copy and paste the link into a new browser tab.

Finally, to query the counter value, run:

query {
  value
}

This will return a value of 42, which is the initialization argument we specified when deploying our application.

The Linera Protocol

We now describe the main concepts of the Linera protocol in more details.

Overview

Linera is a decentralized infrastructure optimized for Web3 applications that require guaranteed performance for an unlimited number of active users.

The core idea of the Linera protocol is to run many lightweight blockchains, called microchains, in parallel in a single set of validators.

How does it work?

In Linera, user wallets operate their own microchains. The owner of a chain chooses when to add new blocks to the chain and what goes inside the blocks. Such chains with a single user are called user chains.

Users may add new blocks to their chains in order to process incoming messages from other chains or to execute secure operations on their accounts, for instance to transfer assets to another user.

Importantly, validators ensure that all new blocks are valid. For instance, transfer operations must originate from accounts with sufficient funds; and incoming messages must have been actually sent from another chain. Blocks are verified by validators in the same way for every chain.

A Linera application is a Wasm program that defines its own state and operations. Users can publish bytecode and initialize an application on one chain, and it will be automatically deployed to all chains where it is needed, with a separate state on each chain.

To ensure coordination across chains, an application may rely on asynchronous cross-chain messages. Message payloads are application-specific and opaque to the rest of the system.

                               ┌───┐     ┌───┐     ┌───┐
                       Chain A │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘
                                                     ▲
                                           ┌─────────┘
                                           │
                               ┌───┐     ┌─┴─┐     ┌───┐
                       Chain B │   ├────►│   ├────►│   │
                               └───┘     └─┬─┘     └───┘
                                           │         ▲
                                           │         │
                                           ▼         │
                               ┌───┐     ┌───┐     ┌─┴─┐
                       Chain C │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘

The number of applications present on a single chain is not limited. On the same chain, applications are composed as usual using synchronous calls.

The current Linera SDK uses Rust as a source language to create Wasm applications. It relies on the normal Rust toolchains so that Rust programmers can work in their preferred environments.

How does Linera compare to existing multi-chain infrastructure?

Linera is the first infrastructure designed to support many chains in parallel, and notably an arbitrary number of user chains meant to be operated by user wallets.

In traditional multi-chain infrastructures, each chain usually runs a full blockchain protocol in a separate set of validators. Creating a new chain or exchanging messages between chains is expensive. As a result, the total number of chains is generally limited. Some chains may be specialized to a given use case: these are called "app chains".

In contrast, Linera is optimized for a large number of user chains:

• Users only create blocks in their chain when needed;

• Creating a microchain does not require onboarding validators;

• All chains have the same level of security;

• Microchains communicate efficiently using the internal networks of validators;

• Validators are internally sharded (like a regular web service) and may adjust their capacity elastically by adding or removing internal workers.

Besides user chains, the Linera protocol is designed to support other types of microchains, called "permissioned" and "public" chains. Public chains are operated by validators. In this regard, they are similar to classical blockchains. Permissioned chains are meant to be used for temporary interactions between users, such as atomic swaps.

Why build on top of Linera?

We believe that many high-value use cases are currently out of reach of existing Web3 infrastructures because of the challenges of serving many active users simultaneously without degrading user experience (unpredictable fees, latency, etc).

Examples of applications that require processing time-sensitive transactions created by many simultaneous users include:

• real-time micro-payments and micro-rewards,

• social data feeds,

• real-time auction systems,

• turn-based games,

• version control systems for software, data pipelines, or AI training pipelines.

Lightweight user chains are instrumental in providing elastic scalability but they have other benefits as well. Because user chains have fewer blocks than traditional blockchains, in Linera, the full-nodes of user chains will be embedded into the users' wallets, typically deployed as a browser extension.

This means that Web UIs connected to a wallet will be able to query the state of the user chain directly (no API provider, no light client) using familiar frameworks (React/GraphQL). Furthermore, wallets will be able to leverage the full node as well for security purposes, including to display meaningful confirmation messages to users.

What is the current state of the development of Linera?

The reference open-source implementation of Linera is under active development. It already includes a Web3 SDK with the necessary features to prototype simple Web3 applications and test them locally on the same machine and deploying them to the Devnet.

Web UIs (possibly reactive) can already be built on top of Wasm-embedded GraphQL services, and tested locally in the browser.

The main limitations of our current Web3 SDK include:

• Web UIs need to query a local HTTP service acting as a wallet. This setup is meant to be temporary and for testing only: in the future, web UIs will securely connect to a Wallet installed as a browser extension, as usual.

• Only user chains are currently documented in this manual. Permissioned chains (aka "temporary" chains) were recently added. Support for public chains is in progress.

The main development workstreams of Linera, beyond its SDK, can be broken down as follows.

Core Protocol

• User chains
• Permissioned chains (core protocol only)
• Cross-chain messages
• Cross-chain pub/sub channels (initial version)
• Bytecode publishing
• Application creation
• Reconfigurations of validators
• Support for gas fees
• Support for storage fees and storage limits
• External service (aka. "Faucet") to help users create their first chain
• Permissioned chains (adding operation access control, demo of atomic swaps, etc)
• Avoid repeatedly loading chain states from storage
• Blob storage usable by system and user applications (generalizing/replacing bytecode storage)
• Support for easy onboarding of user chains into a new application (removing the need to accept requests)
• Replace pub/sub channels by data streams (removing the need to accept subscriptions)
• Allow chain clients to control which chains they track (lazily/actively) and execute (do not execute all tracked chains)
• Multi-signed events to facilitate future bridges to external chains
• Public chains (adding leader election, inbox constraints, etc)
• Transaction scripts
• Support for dynamic shard assignment
• Support for archiving chains
• Tokenomics and incentives for all stakeholders
• Governance on the admin chain (e.g. DPoS, onboarding of validators)
• Permissionless auditing protocol

Wasm VM integration

• Support for the Wasmer VM
• Support for the Wasmtime VM (experimental)
• Test gas metering and deterministic execution across VMs
• Composing Wasm applications on the same chain
• Support for non-blocking (yet deterministic) calls to storage
• Support for read-only GraphQL services in Wasm
• Support for mocked system APIs
• Improve host/guest stub generation to make mocks easier
• Support for running Wasm applications in the browser

Storage

• Object management library ("linera-views") on top of Key-Value store abstraction
• Support for Rocksdb
• Experimental support for DynamoDb
• Derive macros for GraphQL
• Support for ScyllaDb
• Make library fully extensible by users (requires better GraphQL macros)
• In-memory storage service for testing purposes
• Support for Web storage (IndexedDB)
• Performance benchmarks and improvements (including faster state hashing)
• Better configuration management
• Local Write-Ahead Log
• Production-grade support for the chosen main database
• Tooling for debugging
• Make the storage library easy to use outside of Linera

Validator Infrastructure

• Simple TCP/UDP networking (used for benchmarks only)
• GRPC networking
• Basic frontend (aka. proxy) supporting fixed internal shards
• Observability
• Kubernetes support in CI
• Deployment using a cloud provider
• Horizontally scalable frontend (aka. proxy)
• Dynamic shard assignment
• Cloud integration to demonstrate elastic scaling

Web3 SDK

• Traits for contract and service interfaces
• Support for unit testing
• Support for integration testing
• Local GraphQL service to query and browse system state
• Local GraphQL service to query and browse application states
• Use GraphQL mutations to execute operations and create blocks
• ABIs for contract and service interfaces
• Allowing message sender to pay for message execution fees
• Wallet as a browser extension (no VM)
• Wallet as a browser extension (with Wasm VM)
• Easier communication with EVM chains
• Bindings to use native cryptographic primitives from Wasm
• Allowing applications to pay for user fees
• Allowing applications to use permissioned chains and public chains

Microchains

This section provides an introduction to microchains, the main building block of the Linera Protocol. For a more formal treatment refer to the whitepaper.

Background

A microchain is a chain of blocks describing successive changes to a shared state. We will use the terms chain and microchain interchangeably. Linera microchains are similar to the familiar notion of blockchain, with the following important specificities:

• An arbitrary number of microchains can coexist in a Linera network, all sharing the same set of validators and the same level of security. Creating a new microchain only takes one transaction on an existing chain.

• The task of proposing new blocks in a microchain can be assumed either by validators or by end users (or rather their wallets) depending on the configuration of a chain. Specifically, microchains can be single-owner, permissioned, or public, depending on who is authorized to propose blocks.

Cross-Chain Messaging

In traditional networks with a single blockchain, every transaction can access the application's entire execution state. This is not the case in Linera where the state of an application is spread across multiple microchains, and the state on any individual microchain is only affected by the blocks of that microchain.

Cross-chain messaging is a way for different microchains to communicate with each other asynchronously. This method allows applications and data to be distributed across multiple chains for better scalability. When an application on one chain sends a message to itself on another chain, a cross-chain request is created. These requests are implemented using remote procedure calls (RPCs) within the validators' internal network, ensuring that each request is executed only once.

Instead of immediately modifying the target chain, messages are placed first in the target chain's inbox. When an owner of the target chain creates its next block in the future, they may reference a selection of messages taken from the current inbox in the new block. This executes the selected messages and applies their messages to the chain state.

Below is an example set of chains sending asynchronous messages to each other over consecutive blocks.

                               ┌───┐     ┌───┐     ┌───┐
                       Chain A │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘
                                                     ▲
                                           ┌─────────┘
                                           │
                               ┌───┐     ┌─┴─┐     ┌───┐
                       Chain B │   ├────►│   ├────►│   │
                               └───┘     └─┬─┘     └───┘
                                           │         ▲
                                           │         │
                                           ▼         │
                               ┌───┐     ┌───┐     ┌─┴─┐
                       Chain C │   ├────►│   ├────►│   │
                               └───┘     └───┘     └───┘

The Linera protocol allows receivers to discard messages but not to change the ordering of selected messages inside the communication queue between two chains. If a selected message fails to execute, the wallet will automatically reject it when proposing the receiver's block. The current implementation of the Linera client always selects as many messages as possible from inboxes, and never discards messages unless they fail to execute.

Chain Ownership Semantics

Active chains can have one or multiple owners. Chains with zero owners are permanently deactivated.

In Linera, the validators guarantee safety: On each chain, at each height, there is at most one unique block.

But liveness—actually adding blocks to a chain at all—relies on the owners. There are different types of rounds and owners, optimized for different use cases:

• First an optional fast round, where a super owner can propose blocks that get confirmed with very particularly low latency, optimal for single-owner chains with no contention.
• Then a number of multi-leader rounds, where all regular owners can propose blocks. This works well even if there is occasional, temporary contention: an owner using multiple devices, or multiple people using the same chain infrequently.
• And finally single-leader rounds: These give each regular chain owner a time slot in which only they can propose a new block, without being hindered by any other owners' proposals. This is ideal for chains with many users that are trying to commit blocks at the same time.

The number of multi-leader rounds is configurable: On chains with fluctuating levels of activity, this allows the system to dynamically switch to single-leader mode whenever all multi-leader rounds fail during periods of high contention. Chains that very often have high activity from multiple owners can set the number of multi-leader rounds to 0.

For more detail and examples on how to open and close chains, see the wallet section on chain management.

Wallets

As in traditional blockchains, Linera wallets are in charge of holding user private keys. However, instead of signing transactions, Linera wallets are meant to sign blocks and propose them to extend the chains owned by their users.

In practice, wallets include a node which tracks a subset of Linera chains. We will see in the next section how a Linera wallet can run a GraphQL service to expose the state of its chains to web frontends.

The command-line tool linera is the main way for developers to interact with a Linera network and manage the user wallets present locally on the system.

Note that this command-line tool is intended mainly for development purposes. Our goal is that end users eventually manage their wallets in a browser extension.

Selecting a Wallet

The private state of a wallet is conventionally stored in a file wallet.json, while the state of its node is stored in a file linera.db.

To switch between wallets, you may use the --wallet and --storage options of the linera tool, e.g. as in linera --wallet wallet2.json --storage rocksdb:linera2.db.

You may also define the environment variables LINERA_STORAGE and LINERA_WALLET to the same effect. E.g. LINERA_STORAGE=$PWD/wallet2.json and LINERA_WALLET=$PWD/wallet2.json.

Finally, if LINERA_STORAGE_$I and LINERA_WALLET_$I are defined for some number I, you may call linera --with-wallet $I (or linera -w $I for short).

Chain Management

Listing Chains

To list the chains present in your wallet, you may use the command show:

linera wallet show
╭──────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────╮
│ Chain ID                                                         ┆ Latest Block                                                                         │
╞══════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════╡
│ 668774d6f49d0426f610ad0bfa22d2a06f5f5b7b5c045b84a26286ba6bce93b4 ┆ Public Key:         3812c2bf764e905a3b130a754e7709fe2fc725c0ee346cb15d6d261e4f30b8f1 │
│                                                                  ┆ Owner:              c9a538585667076981abfe99902bac9f4be93714854281b652d07bb6d444cb76 │
│                                                                  ┆ Block Hash:         -                                                                │
│                                                                  ┆ Timestamp:          2023-04-10 13:52:20.820840                                       │
│                                                                  ┆ Next Block Height:  0                                                                │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 91c7b394ef500cd000e365807b770d5b76a6e8c9c2f2af8e58c205e521b5f646 ┆ Public Key:         29c19718a26cb0d5c1d28102a2836442f53e3184f33b619ff653447280ccba1a │
│                                                                  ┆ Owner:              efe0f66451f2f15c33a409dfecdf76941cf1e215c5482d632c84a2573a1474e8 │
│                                                                  ┆ Block Hash:         51605cad3f6a210183ac99f7f6ef507d0870d0c3a3858058034cfc0e3e541c13 │
│                                                                  ┆ Timestamp:          2023-04-10 13:52:21.885221                                       │
│                                                                  ┆ Next Block Height:  1                                                                │
╰──────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────╯

Each row represents a chain present in the wallet. On the left is the unique identifier on the chain, and on the right is metadata for that chain associated with the latest block.

Default Chain

Each wallet has a default chain that all commands apply to unless you specify another --chain on the command line.

The default chain is set initially, when the first chain is added to the wallet. You can check the default chain for your wallet by running:

linera wallet show

The Chain ID which is in green text instead of white text is your default chain.

To change the default chain for your wallet, user the set-default command:

linera wallet set-default <chain-id>

Opening a Chain

The Linera protocol defines semantics for how new chains are created, we call this "opening a chain". A chain cannot be opened in a vacuum, it needs to be created by an existing chain on the network.

Open a Chain for Your Own Wallet

To open a chain for your own wallet, you can use the open-chain command:

linera open-chain

This will create a new chain (using the wallet's default chain) and add it to the wallet. Use the wallet show command to see your existing chains.

Open a Chain for Another Wallet

Opening a chain for another wallet requires an extra two steps. Let's initialize a second wallet:

linera --wallet wallet2.json --storage rocksdb:linera2.db wallet init --genesis target/debug/genesis.json

First wallet2 must create an unassigned keypair. The public part of that keypair is then sent to the wallet who is the chain creator.

linera --wallet wallet2.json keygen
6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 # this is the public key for the unassigned keypair

Next, using the public key, wallet can open a chain for wallet2.

linera open-chain --to-public-key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888
e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000
fc9384defb0bcd8f6e206ffda32599e24ba715f45ec88d4ac81ec47eb84fa111

The first line is the message ID specifying the cross-chain message that creates the new chain. The second line is the new chain's ID.

Finally, to add the chain to wallet2 for the given unassigned key we use the assign command:

 linera --wallet wallet2.json assign --key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 --message-id e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000

Opening a Chain with Multiple Users

The open-chain commands is a simplified version of open-multi-owner-chain, which gives you fine-grained control over the set and kinds of owners and rounds for the new chain, and the timeout settings for the rounds. E.g. this creates a chain with two owners and two multi-leader rounds.

linera open-multi-owner-chain \
    --chain-id e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000 \
    --owner-public-keys 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 \
                        ca909dcf60df014c166be17eb4a9f6e2f9383314a57510206a54cd841ade455e \
    --multi-leader-rounds 2

The change-ownership command offers the same options to add or remove owners and change round settings for an existing chain.

Setting up Extra Wallets Automatically with linera net up

For testing, rather than using linera open-chain and linera assign as above, it is often more convenient to pass the option --extra-wallets N to linera net up.

This option will create create N additional user wallets and output Bash commands to define the environment variables LINERA_{WALLET,STORAGE}_$I where I ranges over 0..=N (I=0 being the wallet for the initial chains).

Once all the environment variables are defined, you may switch between wallets using linera --with-wallet I or linera -w I for short.

Automation in Bash

To automate the process of setting the variables LINERA_WALLET* and LINERA_STORAGE* after creating a local test network in a shell, we provide a Bash helper function linera_spawn_and_read_wallet_variables.

To define the function linera_spawn_and_read_wallet_variables in your shell, run source /dev/stdin <<<"$(linera net helper 2>/dev/null)". You may also add the output of linera net helper to your ~/.bash_profile for future sessions.

Once the function is defined, call linera_spawn_and_read_wallet_variables linera net up instead of linera net up.

Node Service

So far we've seen how to use the Linera client treating it as a binary in your terminal. However, the client also acts as a node which:

1. Executes blocks
2. Exposes an GraphQL API and IDE for dynamically interacting with applications and the system
3. Listens for notifications from validators and automatically updates local chains.

To interact with the node service, run linera in service mode:

linera service

This will run the node service on port 8080 by default (this can be overridden using the --port flag).

A Note on GraphQL

Linera uses GraphQL as the query language for interfacing with different parts of the system. GraphQL enables clients to craft queries such that they receive exactly what they want and nothing more.

GraphQL is used extensively during application development, especially to query the state of an application from a front-end for example.

To learn more about GraphQL check out the official docs.

GraphiQL IDE

Conveniently, the node service exposes a GraphQL IDE called GraphiQL. To use GraphiQL start the node service and navigate to localhost:8080/.

Using the schema explorer on the left of the GraphiQL IDE you can dynamically explore the state of the system and your applications.

GraphQL System API

The node service also exposes a GraphQL API which corresponds to the set of system operations. You can explore the full set of operations by clicking on MutationRoot.

GraphQL Application API

To interact with an application, we run the Linera client in service mode. It exposes a GraphQL API for every application running on any owned chain at localhost:8080/chains/<chain-id>/applications/<application-id>.

Navigating there with your browser will open a GraphiQL interface which enables you to graphically explore the state of your application.

Applications

The programming model of Linera is designed so that developers can take advantage of microchains to scale their applications.

Linera uses the WebAssembly (Wasm) Virtual Machine to execute user applications. Currently, the Linera SDK is focused on the Rust programming language.

Linera applications are structured using the familiar notion of Rust crate: the external interfaces of an application (including instantiation parameters, operations and messages) generally go into the library part of its crate, while the core of each application is compiled into binary files for the Wasm architecture.

The Application Deployment Lifecycle

Linera Applications are designed to be powerful yet re-usable. For this reason there is a distinction between the bytecode and an application instance on the network.

Applications undergo a lifecycle transition aimed at making development easy and flexible:

1. The bytecode is built from a Rust project with the linera-sdk dependency.
2. The bytecode is published to the network on a microchain, and assigned an identifier.
3. A user can create a new application instance, by providing the bytecode identifier and instantiation arguments. This process returns an application identifier which can be used to reference and interact with the application.
4. The same bytecode identifier can be used as many times needed by as many users needed to create distinct applications.

Importantly, the application deployment lifecycle is abstracted from the user, and an application can be published with a single command:

linera publish-and-create <contract-path> <service-path> <init-args>

This will publish the bytecode as well as instantiate the application for you.

Anatomy of an Application

An application is broken into two major components, the contract and the service.

The contract is gas-metered, and is the part of the application which executes operations and messages, make cross-application calls and modifies the application's state. The details are covered in more depth in the SDK docs.

The service is non-metered and read-only. It is used primarily to query the state of an application and populate the presentation layer (think front-end) with the data required for a user interface.

Operations and Messages

For this section we'll be using a simplified version of the example application called "fungible" where users can send tokens to each other.

At the system-level, interacting with an application can be done via operations and messages.

Operations are defined by an application developer and each application can have a completely different set of operations. Chain owners then actively create operations and put them in their block proposals to interact with an application. Other applications may also call the application by providing an operation for it to execute, this is called a cross-application call and always happens within the same chain. Operations for cross-application calls may return a response value back to the caller.

Taking the "fungible token" application as an example, an operation for a user to transfer funds to another user would look like this:

extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Operation {
    /// A transfer from a (locally owned) account to a (possibly remote) account.
    Transfer {
        owner: AccountOwner,
        amount: Amount,
        target_account: Account,
    },
    // Meant to be extended here
}

Messages result from the execution of operations or other messages. Messages can be sent from one chain to another, always within the same application. Block proposers also actively include messages in their block proposal, but unlike with operations, they are only allowed to include them in the right order (possibly skipping some), and only if they were actually created by another chain (or by a previous block of the same chain).

In our "fungible token" application, a message to credit an account would look like this:

extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Message {
    Credit { owner: AccountOwner, amount: Amount },
    // Meant to be extended here
}

Authentication

Operations in a block are always authenticated and messages may be authenticated. The signer of a block becomes the authenticator of all the operations in that block. As operations are being executed by applications, messages can be created to be sent to other chains. When they are created, they can be configured to be authenticated. In that case, the message receives the same authentication as the operation that created it. If handling an incoming message creates new messages, those may also be configured to have the same authentication as the received message.

In other words, the block signer can have its authority propagated across chains through series of messages. This allows applications to safely store user state on chains that the user may not have the authority to produce blocks. The application may also allow only the authorized user to change that state, and not even the chain owner is able to override that.

The figure below shows four chains (A, B, C, D) and some blocks produced in them. In this example, each chain is owned by a single owner (aka. address). Owners are in charge of producing blocks and sign new blocks using their signing keys. Some blocks show the operations and incoming messages they accept, where the authentication is shown inside parenthesis. All operations produced are authenticated by the block proposer, and if these are all single user chains, the proposer is always the chain owner. Messages that have authentication use the one from the operation or message that created it.

One example in the figure is that chain A produced a block with Operation 1, which is authenticated by the owner of chain A (written (a)). That operation sent a message to chain B, and assuming the message was sent with the authentication forwarding enabled, it is received and executed in chain B with the authentication of (a). Another example is that chain D produced a block with Operation 2, which is authenticated by the owner of chain D (written (d)). That operation sent a message to chain C, which is executed with authentication of (d) like the example before. Handling that message in chain C produced a new message, which was sent to chain B. That message, when received by chain B is executed with the authentication of (d).

                            ┌───┐     ┌─────────────────┐     ┌───┐
       Chain A owned by (a) │   ├────►│ Operation 1 (a) ├────►│   │
                            └───┘     └────────┬────────┘     └───┘
                                               │
                                               └────────────┐
                                                            ▼
                                                ┌──────────────────────────┐
                            ┌───┐     ┌───┐     │ Message from chain A (a) │
       Chain B owned by (b) │   ├────►│   ├────►│ Message from chain C (d) |
                            └───┘     └───┘     │ Operation 3 (b)          │
                                                └──────────────────────────┘
                                                            ▲
                                                   ┌────────┘
                                                   │
                            ┌───┐     ┌──────────────────────────┐     ┌───┐
       Chain C owned by (c) │   ├────►│ Message from chain D (d) ├────►│   │
                            └───┘     └──────────────────────────┘     └───┘
                                                 ▲
                                     ┌───────────┘
                                     │
                            ┌─────────────────┐     ┌───┐     ┌───┐
       Chain D owned by (d) │ Operation 2 (d) ├────►│   ├────►│   │
                            └─────────────────┘     └───┘     └───┘

An example where this is used is in the Fungible application, where a Claim operation allows retrieving money from a chain the user does not control (but the user still trusts will produce a block receiving their message). Without the Claim operation, users would only be able to store their tokens on their own chains, and multi-owner and public chains would have their tokens shared between anyone able to produce a block.

With the Claim operation, users can store their tokens on another chain where they're able to produce blocks or where they trust the owner will produce blocks receiving their messages. Only they are able to move their tokens, even on chains where ownership is shared or where they are not able to produce blocks.

Registering an Application across Chains

If Alice is using an application on her chain and starts interacting with Bob via the application, e.g. sends him some tokens using the fungible example, the application automatically gets registered on Bob's chain, too, as soon as he handles the incoming cross-chain messages. After that, he can execute the application's operations on his chain, too, and e.g. send tokens to someone.

But there are also cases where Bob may want to start using an application he doesn't have yet. E.g. maybe Alice regularly makes posts using the social example, and Bob wants to subscribe to her.

In that case, trying to execute an application-specific operation would fail, because the application is not registered on his chain. He needs to request it from Alice first:

linera request-application <application-id> --target-chain-id <alices-chain-id>

Once Alice processes his message (which happens automatically if she is running the client in service mode), he can start using the application.

Writing Linera Applications

In this section, we'll be exploring how to create Web3 applications using the Linera SDK.

We'll use a simple "counter" application as a running example.

We'll focus on the back end of the application, which consists of two main parts: a smart contract and its GraphQL service.

Both the contract and the service of an application are written in Rust using the crate linera-sdk, and compiled to Wasm bytecode.

This section should be seen as a guide versus a reference manual for the SDK. For the reference manual, refer to the documentation of the crate.

Creating a Linera Project

To create your Linera project, use the linera project new command. The command should be executed outside the linera-protocol folder. It sets up the scaffolding and requisite files:

linera project new my-counter

linera project new bootstraps your project by creating the following key files:

• Cargo.toml: your project's manifest filled with the necessary dependencies to create an app;
• src/lib.rs: the application's ABI definition;
• src/state.rs: the application's state;
• src/contract.rs: the application's contract, and the binary target for the contract bytecode;
• src/service.rs: the application's service, and the binary target for the service bytecode.

Creating the Application State

The struct which defines your application's state can be found in src/state.rs. An application state is the data that is persisted on storage between transactions.

To represent our counter, we're going to need a single u64. To persist the counter we'll be using Linera's view paradigm.

Views are a little like an ORM, however instead of mapping data structures to a relational database like Postgres, they are instead mapped onto key-value stores like RocksDB.

In vanilla Rust, we might represent our Counter as so:

// do not use this
struct Counter {
  value: u64
}

However, to persist your data, you'll need to replace the existing Application state struct in src/state.rs with the following view:

/// The application state.
#[derive(RootView, async_graphql::SimpleObject)]
#[view(context = "ViewStorageContext")]
pub struct Counter {
    pub value: RegisterView<u64>,
}

and all other occurrences of Application in your app.

The RegisterView<T> supports modifying a single value of type T. There are different types of views for different use-cases, but the majority of common data structures have already been implemented:

• A Vec or VecDeque corresponds to a LogView
• A BTreeMap corresponds to a MapView if its values are primitive, or to CollectionView if its values are other views;
• A Queue corresponds to a QueueView

For an exhaustive list refer to the Views documentation.

Finally, run cargo check to ensure that your changes compile.

Defining the ABI

The Application Binary Interface (ABI) of a Linera application defines how to interact with this application from other parts of the system. It includes the data structures, data types, and functions exposed by on-chain contracts and services.

ABIs are usually defined in src/lib.rs and compiled across all architectures (Wasm and native).

For a reference guide, check out the documentation of the crate.

Defining a marker struct

The library part of your application (generally in src/lib.rs) must define a public empty struct that implements the Abi trait.

struct CounterAbi;

The Abi trait combines the ContractAbi and ServiceAbi traits to include the types that your application exports.

/// A trait that includes all the types exported by a Linera application (both contract
/// and service).
pub trait Abi: ContractAbi + ServiceAbi {}

Next, we're going to implement each of the two traits.

Contract ABI

The ContractAbi trait defines the data types that your application uses in a contract. Each type represents a specific part of the contract's behavior:

/// A trait that includes all the types exported by a Linera application contract.
pub trait ContractAbi {
    /// The type of operation executed by the application.
    ///
    /// Operations are transactions directly added to a block by the creator (and signer)
    /// of the block. Users typically use operations to start interacting with an
    /// application on their own chain.
    type Operation: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The response type of an application call.
    type Response: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}

All these types must implement the Serialize, DeserializeOwned, Send, Sync, Debug traits, and have a 'static lifetime.

In our example, we would like to change our Operation to u64, like so:

extern crate linera_base;
use linera_base::abi::ContractAbi;
struct CounterAbi;
impl ContractAbi for CounterAbi {
    type Operation = u64;
    type Response = ();
}

Service ABI

The ServiceAbi is in principle very similar to the ContractAbi, just for the service component of your application.

The ServiceAbi trait defines the types used by the service part of your application:

/// A trait that includes all the types exported by a Linera application service.
pub trait ServiceAbi {
    /// The type of a query receivable by the application's service.
    type Query: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;

    /// The response type of the application's service.
    type QueryResponse: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}

For our Counter example, we'll be using GraphQL to query our application so our ServiceAbi should reflect that:

extern crate linera_base;
extern crate async_graphql;
use linera_base::abi::ServiceAbi;
struct CounterAbi;
impl ServiceAbi for CounterAbi {
    type Query = async_graphql::Request;
    type QueryResponse = async_graphql::Response;
}

Writing the Contract Binary

The contract binary is the first component of a Linera application. It can actually change the state of the application.

To create a contract, we need to create a new type and implement the Contract trait for it, which is as follows:

pub trait Contract: WithContractAbi + ContractAbi + Sized {
    /// The type of message executed by the application.
    type Message: Serialize + DeserializeOwned + Debug;

    /// Immutable parameters specific to this application (e.g. the name of a token).
    type Parameters: Serialize + DeserializeOwned + Clone + Debug;

    /// Instantiation argument passed to a new application on the chain that created it
    /// (e.g. an initial amount of tokens minted).
    type InstantiationArgument: Serialize + DeserializeOwned + Debug;

    /// Creates a in-memory instance of the contract handler.
    async fn load(runtime: ContractRuntime<Self>) -> Self;

    /// Instantiates the application on the chain that created it.
    async fn instantiate(&mut self, argument: Self::InstantiationArgument);

    /// Applies an operation from the current block.
    async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response;

    /// Applies a message originating from a cross-chain message.
    async fn execute_message(&mut self, message: Self::Message);

    /// Finishes the execution of the current transaction.
    async fn store(self);
}

The full trait definition can be found here.

There's quite a bit going on here, so let's break it down and take one method at a time.

For this application, we'll be using the load, execute_operation and store methods.

The Contract Lifecycle

To implement the application contract, we first create a type for the contract:

pub struct CounterContract {
    state: Counter,
    runtime: ContractRuntime<Self>,
}

This type usually contains at least two fields: the persistent state defined earlier and a handle to the runtime. The runtime provides access to information about the current execution and also allows sending messages, among other things. Other fields can be added, and they can be used to store volatile data that only exists while the current transaction is being executed, and discarded afterwards.

When a transaction is executed, the contract type is created through a call to Contract::load method. This method receives a handle to the runtime that the contract can use, and should use it to load the application state. For our implementation, we will load the state and create the CounterContract instance:

    async fn load(runtime: ContractRuntime<Self>) -> Self {
        let state = Counter::load(ViewStorageContext::from(runtime.key_value_store()))
            .await
            .expect("Failed to load state");
        CounterContract { state, runtime }
    }

When the transaction finishes executing successfully, there's a final step where all loaded application contracts are called in order to do any final checks and persist its state to storage. That final step is a call to the Contract::store method, which can be thought of as similar to executing a destructor. In our implementation we will persist the state back to storage:

    async fn store(mut self) {
        self.state.save().await.expect("Failed to save state");
    }

It's possible to do more than just saving the state, and the Contract finalization section provides more details on that.

Instantiating our Application

The first thing that happens when an application is created from a bytecode is that it is instantiated. This is done by calling the contract's Contract::instantiate method.

Contract::instantiate is only called once when the application is created and only on the microchain that created the application.

Deployment on other microchains will use the Default value of all sub-views in the state if the state uses the view paradigm.

For our example application, we'll want to initialize the state of the application to an arbitrary number that can be specified on application creation using its instatiation parameters:

    async fn instantiate(&mut self, value: u64) {
        self.state.value.set(value);
    }

Implementing the Increment Operation

Now that we have our counter's state and a way to initialize it to any value we would like, we need a way to increment our counter's value. Execution requests from block proposers or other applications are broadly called 'operations'.

To handle an operation, we need to implement the Contract::execute_operation method. In the counter's case, the operation it will be receiving is a u64 which is used to increment the counter by that value:

    async fn execute_operation(&mut self, operation: u64) {
        let current = self.value.get();
        self.value.set(current + operation);
    }

Declaring the ABI

Finally, to link our Contract trait implementation with the ABI of the application, the following code is added:

impl WithContractAbi for CounterContract {
    type Abi = counter::CounterAbi;
}

Writing the Service Binary

The service binary is the second component of a Linera application. It is compiled into a separate Bytecode from the contract and is run independently. It is not metered (meaning that querying an application's service does not consume gas), and can be thought of as a read-only view into your application.

Application states can be arbitrarily complex, and most of the time you don't want to expose this state in its entirety to those who would like to interact with your app. Instead, you might prefer to define a distinct set of queries that can be made against your application.

The Service trait is how you define the interface into your application. The Service trait is defined as follows:

pub trait Service: WithServiceAbi + ServiceAbi + Sized {
    /// Immutable parameters specific to this application.
    type Parameters: Serialize + DeserializeOwned + Send + Sync + Clone + Debug + 'static;

    /// Creates a in-memory instance of the service handler.
    async fn new(runtime: ServiceRuntime<Self>) -> Self;

    /// Executes a read-only query on the state of this application.
    async fn handle_query(&self, query: Self::Query) -> Self::QueryResponse;
}

The full service trait definition can be found here.

Let's implement Service for our counter application.

First, we create a new type for the service, similarly to the contract:

pub struct CounterService {
    state: Counter,
}

Just like with the CounterContract type, this type usually has two types: the application state and the runtime. We can omit the fields if we don't use them, so in this example we're omitting the runtime field, since its only used when constructing the CounterService type.

We need to generate the necessary boilerplate for implementing the service WIT interface, export the necessary resource types and functions so that the service can be executed. Fortunately, there is a macro to perform this code generation, so just add the following to service.rs:

linera_sdk::service!(CounterService);

Next, we need to implement the Service trait for CounterService type. The first step is to define the Service's associated type, which is the global parameters specified when the application is instantiated. In our case, the global paramters aren't used, so we can just specify the unit type:

#[async_trait]
impl Service for CounterService {
    type Parameters = ();
}

Also like in contracts, we must implement a load constructor when implementing the Service trait. The constructor receives the runtime handle and should use it to load the application state:

    async fn load(runtime: ServiceRuntime<Self>) -> Self {
        let state = Counter::load(ViewStorageContext::from(runtime.key_value_store()))
            .await
            .expect("Failed to load state");
        Ok(CounterService { state })
    }

Services don't have a store method because they are read-only and can't persist any changes back to the storage.

The actual functionality of the service starts in the handle_query method. We will accept GraphQL queries and handle them using the async-graphql crate. To forward the queries to custom GraphQL handlers we will implement in the next section, we use the following code:

    async fn handle_query(&mut self, request: Request) -> Response {
        let schema = Schema::build(
            // implemented in the next section
            QueryRoot { value: *self.value.get() },
            // implemented in the next section
            MutationRoot {},
            EmptySubscription,
        )
        .finish();
        schema.execute(request).await
    }
}

Finally, as before, the following code is needed to incorporate the ABI definitions into your Service implementation:

impl WithServiceAbi for Counter {
    type Abi = counter::CounterAbi;
}

Adding GraphQL compatibility

Finally, we want our application to have GraphQL compatibility. To achieve this we need a QueryRoot to respond to queries and a MutationRoot for creating serialized Operation values that can be placed in blocks.

In the QueryRoot, we only create a single value query that returns the counter's value:

struct QueryRoot {
    value: u64,
}

#[Object]
impl QueryRoot {
    async fn value(&self) -> &u64 {
        &self.value
    }
}

In the MutationRoot, we only create one increment method that returns a serialized operation to increment the counter by the provided value:

struct MutationRoot;

#[Object]
impl MutationRoot {
    async fn increment(&self, value: u64) -> Vec<u8> {
        bcs::to_bytes(&value).unwrap()
    }
}

We haven't included the imports in the above code; they are left as an exercise to the reader (but remember to import async_graphql::Object). If you want the full source code and associated tests check out the examples section on GitHub.

Deploying the Application

The first step to deploy your application is to configure a wallet. This will determine where the application will be deployed: either to a local net or to the devnet.

Local Net

To configure the local network, follow the steps in the Getting Started section.

Afterwards, the LINERA_WALLET and the LINERA_STORAGE environment variables should be set and can be used in the publish-and-create command to deploy the application while also specifying:

1. The location of the contract bytecode
2. The location of the service bytecode
3. The JSON encoded initialization arguments

linera publish-and-create \
  target/wasm32-unknown-unknown/release/my-counter_{contract,service}.wasm \
  --json-argument "42"

Devnet

To configure the wallet for the devnet while creating a new microchain, the following command can be used:

linera wallet init --with-new-chain --faucet https://faucet.devnet-2024-05-07.linera.net

The Faucet will provide the new chain with some tokens, which can then be used to deploy the application with the publish-and-create command. It requires specifying:

1. The location of the contract bytecode
2. The location of the service bytecode
3. The JSON encoded initialization arguments

linera publish-and-create \
  target/wasm32-unknown-unknown/release/my-counter_{contract,service}.wasm \
  --json-argument "42"

Interacting with the Application

To interact with the deployed application, a node service must be used.

Cross-Chain Messages

On Linera, applications are meant to be multi-chain: They are instantiated on every chain where they are used. An application has the same application ID and bytecode everywhere, but a separate state on every chain. To coordinate, the instances can send cross-chain messages to each other. A message sent by an application is always handled by the same application on the target chain: The handling code is guaranteed to be the same as the sending code, but the state may be different.

For your application, you can specify any serializable type as the Message type in your Contract implementation. To send a message, use the ContractRuntime made available as an argument to the contract's [Contract::load] constructor. The runtime is usually stored inside the contract object, as we did when writing the contract binary. We can then call ContractRuntime::prepare_message to start preparing a message, and then send_to to send it to a destination chain.

    self.runtime
        .prepare_message(message_contents)
        .send_to(destination_chain_id);

It is also possible to send a message to a subscription channel, so that the message is forwarded to the subscribers of that channel. All that has to be done is specify a ChannelName as the destination parameter to send_to.

After block execution in the sending chain, sent messages are placed in the target chains' inboxes for processing. There is no guarantee that it will be handled: For this to happen, an owner of the target chain needs to include it in the incoming_messages in one of their blocks. When that happens, the contract's execute_message method gets called on their chain.

While preparing the message to be sent, it is possible to enable authentication forwarding and/or tracking. Authentication forwarding means that the message is executed by the receiver with the same authenticated signer as the sender of the message, while tracking means that the message is sent back to the sender if the receiver rejects it. The example below enables both flags:

    self.runtime
        .prepare_message(message_contents)
        .with_tracking()
        .with_authentication()
        .send_to(destination_chain_id);

Example: Fungible Token

In the fungible example application, such a message can be the transfer of tokens from one chain to another. If the sender includes a Transfer operation on their chain, it decreases their account balance and sends a Credit message to the recipient's chain:

async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response {
    match operation {
        // ...
        Operation::Transfer {
            owner,
            amount,
            target_account,
        } => {
            self.check_account_authentication(owner)?;
            self.state.debit(owner, amount).await?;
            self.finish_transfer_to_account(amount, target_account, owner)
                .await;
            FungibleResponse::Ok
        }
        // ...
    }
}

async fn finish_transfer_to_account(
    &mut self,
    amount: Amount,
    target_account: Account,
    source: AccountOwner,
) {
    if target_account.chain_id == self.runtime.chain_id() {
        self.state.credit(target_account.owner, amount).await;
    } else {
        let message = Message::Credit {
            target: target_account.owner,
            amount,
            source,
        };
        self.runtime
            .prepare_message(message)
            .with_authentication()
            .with_tracking()
            .send_to(target_account.chain_id);
    }
}

On the recipient's chain, execute_message is called, which increases their account balance.

async fn execute_message(&mut self, message: Message) {
    match message {
        Message::Credit {
            amount,
            target,
            source,
        } => {
            // ...
            self.state.credit(receiver, amount).await;
        }
        // ...
    }
}

Calling other Applications

We have seen that cross-chain messages sent by an application on one chain are always handled by the same application on the target chain.

This section is about calling other applications using cross-application calls.

Such calls happen on the same chain and are made with the ContractRuntime::call_application method:

pub fn call_application<A: ContractAbi + Send>(
    &mut self,
    authenticated: bool,
    application: ApplicationId<A>,
    call: &A::Operation,
) -> A::Response {

The authenticated argument specifies whether the callee is allowed to perform actions that require authentication on behalf of the signer of the original block that caused this call.

The application argument is the callee's application ID, and A is the callee's ABI.

The call argument is the operation requested by the application call.

Example: Crowd-Funding

The crowd-funding example application allows the application creator to launch a campaign with a funding target. That target can be an amount specified in any type of token based on the fungible application. Others can then pledge tokens of that type to the campaign, and if the target is not reached by the deadline, they are refunded.

If Alice used the fungible example to create a Pugecoin application (with an impressionable pug as its mascot), then Bob can create a crowd-funding application, use Pugecoin's application ID as CrowdFundingAbi::Parameters, and specify in CrowdFundingAbi::InstantiationArgument that his campaign will run for one week and has a target of 1000 Pugecoins.

Now let's say Carol wants to pledge 10 Pugecoin tokens to Bob's campaign.

First she needs to make sure she has his crowd-funding application on her chain, e.g. using the linera request-application command. This will automatically also register Alice's Pugecoin application on her chain, because it is a dependency of Bob's.

Now she can make her pledge by running the linera service and making a query to Bob's application:

mutation { pledge(owner: "User:841…6c0", amount: "10") }

This will add a block to Carol's chain containing the pledge operation that gets handled by CrowdFunding::execute_operation, resulting in one cross-application call and two cross-chain messages:

First CrowdFunding::execute_operation calls the fungible application on Carol's chain to transfer 10 tokens to Carol's account on Bob's chain:

// ...
let call = fungible::Operation::Transfer {
    owner,
    amount,
    target_account,
};
// ...
self.runtime
    .call_application(/* authenticated by owner */ true, fungible_id, &call);

This causes Fungible::execute_operation to be run, which will create a cross-chain message sending the amount 10 to the Pugecoin application instance on Bob's chain.

After the cross-application call returns, CrowdFunding::execute_operation continues to create another cross-chain message crowd_funding::Message::PledgeWithAccount, which informs the crowd-funding application on Bob's chain that the 10 tokens are meant for the campaign.

When Bob now adds a block to his chain that handles the two incoming messages, first Fungible::execute_message gets executed, and then CrowdFunding::execute_message. The latter makes another cross-application call to transfer the 10 tokens from Carol's account to the crowd-funding application's account (both on Bob's chain). That is successful because Carol does now have 10 tokens on this chain and she authenticated the transfer indirectly by signing her block. The crowd-funding application now makes a note in its application state on Bob's chain that Carol has pledged 10 Pugecoin tokens.

For the complete code please take a look at the crowd-funding and the fungible application contracts in the examples folder in linera-protocol.

Printing Logs from an Application

Applications can use the log crate to print log messages with different levels of importance. Log messages are useful during development, but they may also be useful for end users. By default the linera service command will log the messages from an application if they are of the "info" importance level or higher (briefly, log::info!, log::warn! and log::error!).

During development it is often useful to log messages of lower importance (such as log::debug! and log::trace!). To enable them, the RUST_LOG environment variable must be set before running linera service. The example below enables trace level messages from applications and enables warning level messages from other parts of the linera binary:

export RUST_LOG="warn,linera_execution::wasm=trace"

Writing Tests

Linera applications can be tested using normal Rust unit tests or integration tests. Unit tests use a mock runtime for execution, so it's useful for testing the application as if it were running by itself on a single chain. Integration tests use a simulated validator for testing. This allows creating chains and adding blocks to them in order to test interactions between multiple microchains and multiple applications.

Applications should consider having both types of tests. Unit tests should be used to focus on the application's internals and core functionality. Integration tests should be used to test how the application behaves on a more complex environment that's closer to the real network.

For Rust tests, the cargo test command can be used to run both the unit and integration tests.

Unit tests

Unit tests are written beside the application's source code (i.e., inside the src directory of the project). The main purpose of a unit test is to test parts of the application in an isolated environment. Anything that's external is usually mocked. When the linera-sdk is compiled with the test feature enabled, the ContractRuntime and SystemRuntime types are actually mock runtimes, and can be configured to return specific values for different tests.

Example

A simple unit test is shown below, which tests if the application contract's do_something method changes the application state.

#[cfg(test)]
mod tests {
    use crate::{ApplicationContract, ApplicationState};
    use linera_sdk::{util::BlockingWait, ContractRuntime};

    #[test]
    fn test_do_something() {
        let runtime = ContractRuntime::new();
        let mut contract = ApplicationContract::load(runtime).blocking_wait();

        let result = contract.do_something();

        // Check that `do_something` succeeded
        assert!(result.is_ok());
        // Check that the state in memory was updated
        assert_eq!(contract.state, ApplicationState {
            // Define the application's expected final state
            ..ApplicationState::default()
        });
        // Check that the state in memory is different from the state in storage
        assert_ne!(
            contract.state,
            ApplicatonState::load(ViewStorageContext::from(runtime.key_value_store()))
        );
    }
}

Integration Tests

Integration tests are usually written separately from the application's source code (i.e., inside a tests directory that's beside the src directory).

Integration tests use the helper types from linera_sdk::test to set up a simulated Linera network, and publish blocks to microchains in order to execute the application.

Example

A simple test that sends a message between application instances on different chains is shown below.

#[tokio::test]
async fn test_cross_chain_message() {
    let parameters = vec![];
    let instantiation_argument = vec![];

    let (validator, application_id) =
        TestValidator::with_current_application(parameters, instantiation_argument).await;

    let mut sender_chain = validator.get_chain(application_id.creation.chain_id).await;
    let mut receiver_chain = validator.new_chain().await;

    sender_chain
        .add_block(|block| {
            block.with_operation(
                application_id,
                Operation::SendMessageTo(receiver_chain.id()),
            )
        })
        .await;

    receiver_chain.handle_received_messages().await;

    assert_eq!(
        receiver_chain
            .query::<ChainId>(application_id, Query::LastSender)
            .await,
        sender_chain.id(),
    );
}

Advanced Topics

In this section, we present additional topics related to the Linera protocol.

Views

Views are a specific functionality of the Linera system that allow to have data in memory and then seamlessly flush it to an underlying persistent datastore.

The full documentation is available on the crate documentation with all functions having examples.

Concretely, what is provided is the following:

• A trait View that provides load, rollback, clear, flush, delete. The idea is that we can do operation on the data and then flush it to the database storing them.

• Several other traits HashableView, RootView, CryptoHashView, CryptoHashRootView that are important for computing hash.

• A number of standard containers: MapView, SetView, LogView, QueueView, RegisterView that implement the View and HashableView traits.

• Two containers CollectionView and ReentrantCollectionView that are similar to MapView but whose values are views themselves.

• Derive macros that allow to implement the above mentioned traits on struct data types whose entries are views.

Persistent storage

Validators run the servers and the data is stored in persistent storage. As a consequence we need a tool for working with persistent storage and so we have added linera-db for that purpose.

Available persistent storage

The persistent storage that are available right now are RocksDB, DynamoDB and ScyllaDB. Each has its own strengths and weaknesses.

• RocksDB: Data is stored on disk and cannot be shared between shards but is very fast.

• DynamoDB: Data is stored on a remote storage, that has to be on AWS. Data can be shared between shards.

• ScyllaDB: Data is stored on a remote storage. Data can be shared between shards.

There is no fundamental obstacle to the addition of other persistent storage solutions.

In addition, the DynamoDB and ScyllaDB have the notion of a table which means that a given remote location can be used for several completely independent purposes.

The linera-db tool

When operating on a persistent storage some global operations can be required. The command line tool linera-db helps in making them work.

The functionalities are the following:

• list_tables(DynamoDB and ScyllaDB): It lists all the tables that have been created on the persistent storage

• initialize(RocksDB, DynamoDB and ScyllaDB): It initializes a persistent storage.

• check_existence(RocksDB, DynamoDB and ScyllaDB): It tests the existence of a persistent storage. If the error code is 0 then the table exists, if the error code is 1 then the table is absent.

• check_absence(RocksDB, DynamoDB and ScyllaDB): It tests the absence of a persistent storage. If the error code is 0 then the table is absent, if the error code is 1 then the table does not exist.

• delete_all(RocksDB, DynamoDB and ScyllaDB): It deletes all the table of a persistent storage.

• delete_single(DynamoDB and ScyllaDB): It deletes a single table of a persistent storage.

If some error occurs during the operation, then the error code 2 is returned and 0 if everything went fine with the exception of check_existence and check_absence for which the value 1 can occur if the connection with the database was established correctly but the result is not what we expected.

Contract Finalization

When a transaction finishes executing successfully, there's a final step where all loaded application contracts have their Contract::store implementation called. This can be seen to be similar to executing a destructor. In that sense, applications may want to perform some final operations after execution finished. While finalizing, contracts may send messages, read and write to the state, but are not allowed to call other applications, because they are all also in the process of finalizing.

While finalizing, contracts can force the transaction to fail by panicking. The block is then rejected, even if the entire transaction's operation had succeeded before the application's Contract::store was called. This allows a contract to reject transactions if other applications don't follow any required constraints it establishes after it responds to a cross-application call.

As an example, a contract that executes a cross-application call with Operation::StartSession may require the same caller to perform another cross-application call with Operation::EndSession before the transaction ends.

pub struct MyContract {
    state: MyState;
    runtime: ContractRuntime<Self>;
    active_sessions: HashSet<ApplicationId>;
}

impl Contract for MyContract {
    type Message = ();
    type InstantiationArgument = ();
    type Parameters = ();

    async fn load(runtime: ContractRuntime<Self>) -> Self {
        let state = MyState::load(ViewStorageContext::from(runtime.key_value_store()))
            .await
            .expect("Failed to load state");

        MyContract {
            state,
            runtime,
            active_sessions: HashSet::new(),
        }
    }

    async fn instantiate(&mut self, (): Self::InstantiationArgument) {}

    async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response {
        let caller = self.runtime
            .authenticated_caller_id()
            .expect("Missing caller ID");

        match operation {
            Operation::StartSession => {
                assert!(
                    self.active_sessions.insert(caller_id),
                    "Can't start more than one session for the same caller"
                );
            }
            Operation::EndSession => {
                assert!(
                    self.active_sessions.remove(&caller_id),
                    "Session was not started"
                );
            }
        }
    }

    async fn execute_message(&mut self, message: Self::Message) -> Result<(), Self::Error> {
        unreachable!("This example doesn't support messages");
    }

    async fn store(&mut self) {
        assert!(
            self.active_sessions.is_empty(),
            "Some sessions have not ended"
        );

        self.state.save().await.expect("Failed to save state");
    }
}

Validators

Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.

In Linera, every chain is backed by the same set of validators and has the same level of security.

The main function of validators is to guarantee the integrity of the infrastructure in the sense that:

• Each block is valid, i.e. it has the correct format, its operations are allowed, the received messages are in the correct order, and e.g. the balance was correctly computed.

• Every message received by one chain was actually sent by another chain.

• If one block on a particular height is certified, no other block on the same height is.

These properties are guaranteed to hold as long as two third of the validators (weighted by their stake) follow the protocol. In the future, deviating from the protocol may cause a validator to be considered malicious and to lose their stake.

Validators also play a role in the liveness of the system by making sure that the history of the chains stays available. However, since validators do not propose blocks on most chains (see next section), they do not guarantee that any particular operation or message will eventually be executed on a chain. Instead, chain owners decide whether and when to propose new blocks, and which operations and messages to include. The current implementation of the Linera client automatically includes all incoming messages in new blocks. The operations are the actions the chain owner explicitly adds, e.g. transfer.

Architecture of a validator

Since every chain uses the same validators, adding more chains does not require adding validators. Instead, it requires each individual validator to scale out by adding more computation units, also known as "workers" or "physical shards".

In the end, a Linera validator resembles a Web2 service made of

• a load balancer (aka. ingress/egress), currently implemented by the binary linera-proxy,

• a number of workers, currently implemented by the binary linera-server,

• a shared database, currently implemented by the abstract interface linera-storage.

Example of Linera network

                    │                                             │
                    │                                             │
┌───────────────────┼───────────────────┐     ┌───────────────────┼───────────────────┐
│ validator 1       │                   │     │ validator N       │                   │
│             ┌─────┴─────┐             │     │             ┌─────┴─────┐             │
│             │   load    │             │     │             │   load    │             │
│       ┌─────┤  balancer ├────┐        │     │       ┌─────┤  balancer ├──────┐      │
│       │     └───────────┘    │        │     │       │     └─────┬─────┘      │      │
│       │                      │        │     │       │           │            │      │
│       │                      │        │     │       │           │            │      │
│  ┌────┴─────┐           ┌────┴─────┐  │     │  ┌────┴───┐  ┌────┴────┐  ┌────┴───┐  │
│  │  worker  ├───────────┤  worker  │  │ ... │  │ worker ├──┤  worker ├──┤ worker │  │
│  │    1     │           │    2     │  │     │  │    1   │  │    2    │  │    3   │  │
│  └────┬─────┘           └────┬─────┘  │     │  └────┬───┘  └────┬────┘  └────┬───┘  │
│       │                      │        │     │       │           │            │      │
│       │                      │        │     │       │           │            │      │
│       │     ┌───────────┐    │        │     │       │     ┌─────┴─────┐      │      │
│       └─────┤  shared   ├────┘        │     │       └─────┤  shared   ├──────┘      │
│             │ database  │             │     │             │ database  │             │
│             └───────────┘             │     │             └───────────┘             │
└───────────────────────────────────────┘     └───────────────────────────────────────┘

Inside a validator, components communicate using the internal network of the validator. Notably, workers use direct Remote Procedure Calls (RPCs) with each other to deliver cross-chain messages.

Note that the number of workers may vary for each validator. Both the load balancer and the shared database are represented as a single entity but are meant to scale out in production.

For local testing during development, we currently use a single worker and RocksDB as a database.

Creating New Blocks

In Linera, the responsibility of proposing blocks is separate from the task of validating blocks.

While all chains are validated in the same way, the Linera protocol defines several types of chains, depending on how new blocks are produced.

• The simplest and lowest-latency type of chain is called single-owner chain.

• Other types of Linera chains not currently supported in the SDK include permissioned chains and public chains (see the whitepaper for more context).

For most types of chains (all but public chains), Linera validators do not need to exchange messages with each other.

Instead, the wallets (aka. linera clients) of chain owners make the system progress by proposing blocks and actively providing any additional required data to the validators. For instance, client commands such as transfer, publish-bytecode, or open-chain perform multiple steps to append a block containing the token transfer, application publishing, or chain creation operation:

• The Linera client creates a new block containing the desired operation and new incoming messages, if there are any. It also contains the most recent block's hash to designate its parent. The client sends the new block to all validators.

• The validators validate the block, i.e. check that the block satisfies the conditions listed above, and send a cryptographic signature to the client, indicating that they vote to append the new block. But only if they have not voted for a different block on the same height earlier!

• The client ideally receives a vote from every validator, but only a quorum of votes (say, two thirds) are required: These constitute a "certificate", proving that the block was confirmed. The client sends the certificate to every validator.

• The validators "execute" the block: They update their own view of the most recent state of the chain by applying all messages and operations, and if it generated any cross-chain messages, they send these to the appropriate workers.

To guarantee that each incoming message in a block was actually sent by another chain, a validator will, in the second step, only vote for a block if it has already executed the block that sent it. However, when receiving a valid certificate for a block that receives a message it has not seen yet, it will accept and execute the block anyway. The certificate is proof that most other validators have seen the message, so it must be correct.

In the case of single-owner chains, clients must be carefully implemented so that they never propose multiple blocks at the same height. Otherwise, the chain may be stuck: once each of the two conflicting blocks has been signed by enough validators, it becomes impossible to collect a quorum of votes for either block.

In the future, we anticipate that most users will use permissioned chains even if they are the only owners of their chains. Permissioned chains have two confirmation steps instead of one, but it is not possible to accidentally make a chain unextendable. They also allow users to delegate certain admistrative tasks to third-parties, notably to help with epoch changes (i.e. when the validators change if reconfigured).

Applications that Handle Assets

In general, if you send tokens to a chain owned by someone else, you rely on them for asset availability: if they don't handle your messages, you don't have access to your tokens.

Fortunately, Linera provides a solution based on temporary chains: if the number of parties who want to participate is limited and known in advance, we can:

• make them all chain owners using the linera change-ownership command,
• allow only one application's operations on the chain,
• and allow only that operation to close the chain, using linera change-application-permissions.

Such an application should have a designated operation or message that causes it to close the chain: when that operation is executed, it should send back all remaining assets, and call the runtime's close_chain method.

Once the chain is closed, owners can still create blocks rejecting messages. That way, even assets that are in flight can be returned.

The matching-engine example application does this:

    async fn execute_operation(&mut self, operation: Operation) -> Self::Response {
        match operation {
            // ...
            Operation::CloseChain => {
                for order_id in self.state.orders.indices().await? {
                    match self.modify_order(order_id, ModifyAmount::All).await {
                        Ok(transfer) => self.send_to(transfer),
                        // Orders with amount zero may have been cleared in an earlier iteration.
                        Err(MatchingEngineError::OrderNotPresent) => continue,
                        Err(error) => return Err(error),
                    }
                }
                self.runtime
                    .close_chain()
                    .expect("The application does not have permissions to close the chain.");
            }
        }
    }

This enables doing atomic swaps using the Matching Engine: if you make a bid, you are guaranteed that at any point in time you can get back either the tokens you are offering or the tokens you bought.

Experimental Topics

In this section, we present experimental topics related to the Linera protocol.

These are still in the works and subject to frequent breaking changes.

Machine Learning on Linera

The Linera application contract / service split allows for securely and efficiently running machine learning models on the edge.

The application's contract retrieves the correct model with all the correctness guarantees enforced by the consensus algorithm, while the client performs inference off-chain, in the un-metered service. Since the service is running on the user's own hardware, it can be implicitly trusted.

Guidelines

The existing examples use the candle framework by Hugging Face as the underlying ML framework.

candle is a minimalist ML framework for Rust with a focus on performance and usability. It also compiles to Wasm and has great support for Wasm both in and outside the browser. Check candle's examples for inspiration on the types of models which are supported.

Getting Started

To add ML capabilities to your existing Linera project, you'll need to add the candle-core, getrandom, rand and tokenizers dependencies to your Linera project:

candle-core = "0.4.1"
getrandom = { version = "0.2.12", default-features = false, features = ["custom"] }
rand = "0.8.5"

Optionally, to run Large Language Models, you'll also need the candle-transformers and transformers crate:

candle-transformers = "0.4.1"
tokenizers = { git = "https://github.com/christos-h/tokenizers", default-features = false, features = ["unstable_wasm"] }

Providing Randomness

ML frameworks use random numbers to perform inference. Linera services run in a Wasm VM which do not have access to the OS Rng. For this reason, we need to manually seed RNG used by candle. We do this by writing a custom getrandom.

Create a file under src/random.rs and add the following:

use std::sync::{Mutex, OnceLock};

use rand::{rngs::StdRng, Rng, SeedableRng};

static RNG: OnceLock<Mutex<StdRng>> = OnceLock::new();

fn custom_getrandom(buf: &mut [u8]) -> Result<(), getrandom::Error> {
    let seed = [0u8; 32];
    RNG.get_or_init(|| Mutex::new(StdRng::from_seed(seed)))
        .lock()
        .expect("failed to get RNG lock")
        .fill(buf);
    Ok(())
}

getrandom::register_custom_getrandom!(custom_getrandom);

This will enable candle and any other crates which rely on getrandom access to a deterministic RNG. If deterministic behaviour is not desired, the System API can be used to seed the RNG from a timestamp.

Loading the model into the Service

Models cannot currently be saved on-chain; for more information see the Limitations below.

To perform model inference, the model must be loaded into the service. To do this we'll use the fetch_url API when a query is made against the service:

impl Service for MyService {
    async fn handle_query(&self, request: Request) -> Response {
        // do some stuff here
        let raw_weights = self.runtime.fetch_url("https://my-model-provider.com/model.bin");
        // do more stuff here
    }
}

This can be served from a local webserver or pulled directly from a model provider such as Hugging Face.

At this point we have the raw bytes which correspond to the models and tokenizer. candle supports multiple formats for storing model weights, both quantized and not (gguf, ggml, safetensors, etc.).

Depending on the model format that you're using, candle exposes convenience functions to convert the bytes into a typed struct which can then be used to perform inference. Below is an example for a non-quantized Llama 2 model:

    fn load_llama_model(cursor: &mut Cursor<Vec<u8>>) -> Result<(Llama, Cache), candle_core::Error> {
        let config = llama2_c::Config::from_reader(cursor)?;
        let weights =
            llama2_c_weights::TransformerWeights::from_reader(cursor, &config, &Device::Cpu)?;
        let vb = weights.var_builder(&config, &Device::Cpu)?;
        let cache = llama2_c::Cache::new(true, &config, vb.pp("rot"))?;
        let llama = Llama::load(vb, config.clone())?;
        Ok((llama, cache))
    }

Inference

Performing inference using candle is not a 'one-size-fits-all' process. Different models require different logic to perform inference so the specifics of how to perform inference are beyond the scope of this document.

Luckily, there are multiple examples which can be used as guidelines on how to perform inference in Wasm:

• Llm Stories
• Generative NFTs
• Candle Wasm Examples

Limitations

Hardware Acceleration

Although SIMD instructions are supported by the service runtime, general purpose GPU hardware acceleration is currently not supported. Therefore, performance in local model inference degraded for larger models.

On-Chain Models

Due to block-size constraints, models need to be stored off-chain until the introduction of the Blob API. The Blob API will enable large binary blobs to be stored on-chain, the correctness and availability of which is guaranteed by the validators.

Maximum Model Size

The maximum size of a model which can be loaded into an application's service is currently constrained by:

1. The addressable memory of the service's Wasm runtime being 4 GiB.
2. Not being able to load models directly to the GPU.

It is recommended that smaller models (50 Mb - 100 Mb) are used at current state of development.

Node Operator Manual

This section of the Linera Manual is meant for operators running Linera validator nodes.

Devnets

This section discusses how to deploy developer networks, aka "Devnets", for testing and development purposes.

Devnets always start from a genesis configuration and an empty state. Validator nodes are usually operated in a single infrastructure for simplicity. Devnets do not handle real assets.

Running devnets with Docker Compose

In this section, we use Docker Compose to run a simple devnet with a single validator.

Docker Compose is a tool for defining and managing multi-container Docker applications. It allows you to describe the services, networks, and volumes of your application in a single YAML file (docker-compose.yml). With Docker Compose, you can easily start, stop, and manage all the containers in your application as a single unit using simple commands like docker-compose up and docker-compose down.

For a more complete setup, consider using Kind as described in the next section.

Installation

This section covers everything you need to install to run a Linera network with Docker Compose.

Note: This section was only tested under Linux.

Docker Compose Requirements

To install Docker Compose see the installing Docker Compose section in the Docker docs.

Installing the Linera Toolchain

To install the Linera Toolchain refer to the installation section.

You want to install the toolchain from GitHub, as you'll be using the repository to run the Docker Compose validator service.

Running with Docker Compose

To run a local devnet with Docker Compose, navigate to the root of the linera-protocol repository and run:

cd docker && ./compose.sh

This will take some time as Docker images are built from the Linera source. When the service is ready, a temporary wallet and database is available under the docker subdirectory.

Referencing these variables with the linera binary will enable you to interact with the devnet:

$ linera --wallet wallet.json --storage rocksdb:linera.db sync
2024-06-07T14:19:32.751359Z  INFO linera: Synchronizing chain information
2024-06-07T14:19:32.771842Z  INFO linera::client_context: Saved user chain states
2024-06-07T14:19:32.771850Z  INFO linera: Synchronized chain information in 20 ms
$ linera --wallet wallet.json --storage rocksdb:linera.db query-balance
2024-06-07T14:19:36.958149Z  INFO linera: Evaluating the local balance of e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65 by staging execution of known incoming messages
2024-06-07T14:19:36.959481Z  INFO linera: Balance obtained after 1 ms
10.

The network is transient, so killing the script will perform a cleanup operation destroying wallets, storage and volumes associated with the network.

Running devnets with kind

In this section, we use kind to run a full devnet (network of validators) locally.

Kind (Kubernetes in Docker) is a tool for running local Kubernetes clusters using Docker container nodes. Kind uses Docker to create a cluster of containers that simulate the Kubernetes control plane and worker nodes, allowing developers to easily create, manage, and test multi-node clusters on their local machine.

Installation

This section covers everything you need to install to run a Linera network with kind.

Linera Toolchain Requirements

The operating systems currently supported by the Linera toolchain can be summarized as follows:

The main prerequisites to install the Linera toolchain are Rust, Wasm, and Protoc. They can be installed as follows on Linux:

• Rust and Wasm

  • curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  • rustup target add wasm32-unknown-unknown

• Protoc

  • curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v21.11/protoc-21.11-linux-x86_64.zip
  • unzip protoc-21.11-linux-x86_64.zip -d $HOME/.local
  • If ~/.local is not in your path, add it: export PATH="$PATH:$HOME/.local/bin"

• On certain Linux distributions, you may have to install development packages such as g++, libclang-dev and libssl-dev.

For MacOS support see the installation section on GitHub.

This manual was tested with the following Rust toolchain:

[toolchain]
channel = "1.77.2"
components = [ "clippy", "rustfmt", "rust-src" ]
targets = [ "wasm32-unknown-unknown" ]
profile = "minimal"

Local Kubernetes Requirements

To run kind locally, you also need the following dependencies:

1. kind
2. kubectl
3. docker
4. helm
5. helm-diff
6. helmfile

Installing the Linera Toolchain

To install the Linera toolchain, download the Linera source from GitHub):

git clone https://github.com/linera-io/linera-protocol.git
cd linera-protocol
git checkout -t origin/devnet_2024_05_07  # Current release branch

and to install the Linera toolchain:

cargo install --locked --path linera-service --features kubernetes

Running with kind

To run a local devnet with kind, navigate to the root of the linera-protocol repository and run:

linera net up --kubernetes

This will take some time as Docker images are built from the Linera source. When the cluster is ready, some text is written to the process output containing the exports required to configure your wallet for the devnet - something like:

export LINERA_WALLET="/tmp/.tmpIOelqk/wallet_0.json"
export LINERA_STORAGE="rocksdb:/tmp/.tmpIOelqk/client_0.db"

Exporting these variables in a new terminal will enable you to interact with the devnet:

$ linera sync-balance
2024-05-21T22:30:12.061199Z  INFO linera: Synchronizing chain information and querying the local balance
2024-05-21T22:30:12.061218Z  WARN linera: This command is deprecated. Use `linera sync && linera query-balance` instead.
2024-05-21T22:30:12.065787Z  INFO linera::client_context: Saved user chain states
2024-05-21T22:30:12.065792Z  INFO linera: Operation confirmed after 4 ms
1000000.

Testnets

This section discusses how to deploy a validator node and join an existing Testnet.

In a Testnet, the validator nodes are run by different operators. Testnets will gain in stability and decentralization over time in preparation of the mainnet launch. Testnets do not handle real assets.

In the initial Testnets of Linera, the set of validator nodes will be managed by the Linera Core team.

Joining an Existing Testnet

In this section, we use Docker Compose to run a validator and join an existing Testnet.

Installation

This section covers everything you need to install to run a Linera validator node with Docker Compose.

Note: This section was only tested under Linux.

Docker Compose Requirements

To install Docker Compose see the installing Docker Compose section in the Docker docs.

Installing the Linera Toolchain

To install the Linera Toolchain refer to the installation section.

You want to install the toolchain from GitHub, as you'll be using the repository to run the Docker Compose validator service.

Setting up a Linera Validator

For the next section, we'll be working out of the docker subdirectory in the linera-protocol repository.

Infrastructure Configuration

Validators run via Docker Compose do not come with a pre-packaged load balancer to perform TLS termination (unlike validators running on Kubernetes).

Thus, it is required of validator operators to provide TLS termination and support long-lived HTTP/2 connections required for the functioning of the Linera notification system.

Finally, the load balancer that performs TLS termination must redirect traffic from 443 to 19100 (the port exposed by the proxy).

Creating your Validator Configuration

Validators are configured using a TOML file. You can use the following template to set up you own validator configuration:

server_config_path = "server.json"
host = "<your-host>" # e.g. my-subdomain.my-domain.net
port = 19100
metrics_host = "proxy"
metrics_port = 21100
internal_host = "proxy"
internal_port = 20100
[external_protocol]
Grpc = "ClearText" # Depending on your load balancer you may need "Tls" here.
[internal_protocol]
Grpc = "ClearText"

[[shards]]
host = "shard"
port = 19100
metrics_host = "shard"
metrics_port = 21100

Genesis Configuration

The genesis configuration describes the committee of validators and chains at the point of network creation. It is required for validators to function.

Initially, the genesis configuration for each Testnet will be found in a public bucket managed by the Linera Protocol core team.

An example can be found here:

wget "https://storage.googleapis.com/linera-io-dev-public/devnet-2024-05-07/genesis.json"

Creating Your Keys

Now that the validator configuration has been created and the genesis configuration is available, the validator private keys can be generated.

To generate the private keys, the linera-server binary is used:

linera-server generate --validators /path/to/validator/configuration.toml

This will generate a file called server.json with the information required for a validator to operate, including a cryptographic keypair.

The public key will be printed after the command has finished executing, for example:

$ linera-server generate --validators /path/to/validator/configuration.toml
2024-07-01T16:51:32.881255Z  INFO linera_version::version_info: Linera protocol: v0.12.0
2024-07-01T16:51:32.881273Z  INFO linera_version::version_info: RPC API hash: p//G+L8e12ZRwUdWoGHWYvWA/03kO0n6gtgKS4D4Q0o
2024-07-01T16:51:32.881274Z  INFO linera_version::version_info: GraphQL API hash: KcS5z1lEg+L9QjcP99l5vNSc7LfCwnwEsfDvMZGJ/PM
2024-07-01T16:51:32.881277Z  INFO linera_version::version_info: WIT API hash: p//G+L8e12ZRwUdWoGHWYvWA/03kO0n6gtgKS4D4Q0o
2024-07-01T16:51:32.881279Z  INFO linera_version::version_info: Source code: https://github.com/linera-io/linera-protocol/tree/44b3e1ab15 (dirty)
2024-07-01T16:51:32.881519Z  INFO linera_server: Wrote server config server.json
92f934525762a9ed99fcc3e3d3e35a825235dae133f2682b78fe22a742bac196 # <- Public Key

The public key, in this case beginning with 92f, must be communicated to the Linera Protocol core team along with the chosen host name for onboarding in the next epoch.

Note: Before being included in the next epoch, validator nodes will receive no traffic from existing users.

Building the Linera Docker image

To build the Linera Docker image, run the following command from the root of the linera-protocol repository:

docker build -f docker/Dockerfile . -t linera

This can take several minutes.

Running a Validator Node

Now that the genesis configuration is available at docker/genesis.json and the server configuration is available at docker/server.json, the validator can be started by running from inside the docker directory:

cd docker && docker compose up -d

This will run the Docker Compose deployment in a detached mode. It can take a few minutes for the ScyllaDB image to be downloaded and started.

Appendix

Congrats on reading this manual to the end! In this section, we will be sharing additional material.

Glossary

• Address: A unique public alphanumeric identifier used to designate the identity of an entity on the Linera network.

• Admin Chain: The Linera Network has one designated admin chain where validators can join or leave and where new epochs are defined.

• Application: Similar to a smart-contract on Ethereum, an application is code deployed on the Linera network which is executed by all validators. An application has a metered contract which executes 'business logic' and modifies state and an unmetered 'service' which is a read-only view into an application's state.

• Byzantine Fault-Tolerant (BFT): A system which can operate correctly and achieve consensus even if components of the system fail or act maliciously.

• Block Height: The number of blocks preceding a given block on a specific microchain.

• Block Proposal: A candidate block proposed by a chain owner which may be selected at the next block height.

• Bytecode: A collection of bytes corresponding to a program that can be run by the Wasm virtual machine.

• Client: The linera program, which is a local node and wallet operated by users to make requests to the network. In Linera, clients drive the network by proposing new blocks and validators are mostly reactive.

• Certificate: A value with signatures from a quorum of validators. Values can be confirmed blocks, meaning that the block has been added to the chain and is final. There are other values that are used for reaching consensus, before certifying a confirmed block.

• Committee: The set of all validators for a particular epoch, together with their voting weights.

• Chain Owner: The owner of a user chain or permissioned chain. This is represented as the alphanumeric identifier derived from the hash of the owner's public key.

• Channel: A broadcast mechanism enabling publish-subscribe behavior across chains.

• Contract: The metered part of an application which executes business logic and can modify the application's state.

• Cross-Application Call: A call from one application to another on the same chain.

• Cross-Chain Message: A message containing a data payload which is sent from one chain to another. Cross-Chain messages are the asynchronous communication primitive which enable communication on the same application running on different chains.

• Devnet: An experimental deployment of the Linera protocol meant for testing and development. In a Devnet, the validator nodes are often run by the same operator for simplicity. Devnets may be shut down and restarted from a genesis configuration any time. Devnets do not handle real assets.

• Epoch: A period of time when a particular set of validators with particular voting weights can certify new blocks. Since each chain has to transition explicitly from one epoch to the next, epochs can overlap.

• Genesis Configuration: The configuration determining the state of a newly created network; the voting weights of the initial set of validators, the initial fee structure, and initial chains that the network starts with.

• Inbox: A commutative data structure storing incoming messages for a given chain.

• Mainnet: A deployment meant to be used in production, with real assets.

• Message: See 'Cross-Chain Message'.

• Microchain: A lightweight chain of blocks holding a subset of the network's state running on every validator. This is used interchangeably with 'chain'. All Linera chains are microchains.

• Network: The totality of all protocol participants. A network is the combination of committee, clients and auditors.

• Operation: Operations are either transactions directly added to a block by the creator (and signer) of the block, or calls to an application from another. Users typically use operations to start interacting with an application on their own chain.

• Permissioned Chain: A microchain which is owned by more than one user. Users take turns proposing blocks and the likelihood of selection is proportional to their weight.

• Project: The collection of files and dependencies which are built into the bytecode which is instantiated as an application on the Linera Network.

• Public Chain: A microchain with full BFT consensus with a strict set of permissions relied on for the operation of the network,

• Quorum: A set of validators representing > ⅔ of the total stake. A quorum is required to create a certificate.

• Single-Owner Chain: See 'User Chain'.

• Service: An unmetered read-only view into an application's state.

• Shard: A logical subset of all microchains on a given validator. This corresponds directly to a physical worker.

• Stake: An amount of tokens pledged by a validator or auditor, as a collateral to guarantee their honest and correct participation in the network.

• Testnet: A deployment of the Linera protocol meant for testing and development. In a Testnet, the validator nodes are operated by multiple operators. Testnets will gain in stability and decentralization over time in preparation of the mainnet launch. Testnets do not handle real assets.

• User Chain: Used interchangeably with Single-Owner Chain. User chains are chains which are owned by a single user on the network. Only the chain owner can propose blocks, and therefore only the chain owner can forcibly advance the state of a user chain.

• Validator: Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.

• View: Views are like an Object-Relational Mapping (ORM) for mapping complex types onto key-value stores. Views group complex state changes into a set of elementary operations and commit them atomically. They are full or partial in-memory representations of complex types saved on disk in a key-value store

• Wallet: A file containing a user's public and private keys along with configuration and information regarding the chains they own.

• WebAssembly (Wasm): A binary compilation target and instruction format that runs on a stack-based VM. Linera applications are compiled to Wasm and run on Wasm VMs inside validators and clients.

• Web3: A natural evolution of the internet focusing on decentralization by leveraging blockchains and smart contracts.

• Worker: A process which runs a subset of all microchains on a given validator. This corresponds directly to a logical shard.

Videos

We're constantly improving the SDK and creating new material. Below are some of our recent tutorial and presentation videos:

Rust on Linera: Spring 2024 Hackathon Kick-Off

Exploring the Linera protocol with our founder, Mathieu Baudet, followed by an overview of the Rust on Linera hackathon specifics and resources.

Rust on Linera: Spring 2024 Hackathon Coding Workshop #1

Building and deploying your first Linera application using the Linera devnet with one of our engineers, Christos Hadjiaslanis.

Rust on Linera: Spring 2024 Hackathon Coding Workshop #2

Making your Linera application robust by adding testing, logging, and more.

Rust on Linera: Spring 2024 Hackathon Coding Workshop #3

Building a meta-fungible token application on Linera.

Secure Reactive Web3 for Everyone

Discussing how user wallets track on-chain data using WebAssembly and microchains with Mathieu Baudet and Andreas Fackler from Eth Prague 2024.

Linera x Movement Labs Integrated Day Party Panel Discussion

Discussing the importance of blockchain and Web3 technologies, challenges and opportunities in the industry, and the significance of security and user empowerment in the new web ecosystem from Consensus 2024.

For the latest videos and highlights, check out our YouTube channel and Twitter Media.

Happy viewing!