The Linera Manual
Welcome to the Linera manual, a decentralized protocol designed for highly scalable, low-latency Web3 applications.
The documentation is split into two parts:
- The Developers section is intended for developers who wish to learn more about Linera and its programming model by prototyping applications using the Linera Rust SDK.
- The Operators section is intended for operators who wish to run Linera validators.
NEW: Publish and test your Web3 application on the Linera Testnet!
Install the Linera CLI tool then follow the instructions on this page to claim a microchain and publish your first application on the current Testnet.
To join our community and get involved in the development of the Linera ecosystem, check out our GitHub repository, our Website, and find us on social media channels such as Youtube, Twitter, Telegram, and Discord.
Let's get started!
Developer Manual
This section of the Linera Manual is for developers building applications on Linera.
Getting started
In this section, we will cover the necessary steps to install the Linera toolchain and give a short example to get started with the Linera SDK.
Installation
Let's start with the installation of the Linera development tools.
Overview
The Linera toolchain consists of two crates:
-
linera-sdk
is the main library to program Linera applications in Rust. -
linera-service
defines a number of binaries, including:linera
-- the main client tool, used to operate development wallets,linera-proxy
-- the proxy service, acting as a public entrypoint for each validator,linera-server
-- the service run by each worker of a validator, hidden behind the proxy.
Requirements
The operating systems currently supported by the Linera toolchain can be summarized as follows:
Linux x86 64-bit | Mac OS (M1 / M2) | Mac OS (x86) | Windows |
---|---|---|---|
✓ Main platform | ✓ Working | ✓ Working | Untested |
The main prerequisites to install the Linera toolchain are Rust, Wasm, and Protoc. They can be installed as follows on Linux:
-
Rust and Wasm
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add wasm32-unknown-unknown
-
Protoc
curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v21.11/protoc-21.11-linux-x86_64.zip
unzip protoc-21.11-linux-x86_64.zip -d $HOME/.local
- If
~/.local
is not in your path, add it:export PATH="$PATH:$HOME/.local/bin"
-
On certain Linux distributions, you may have to install development packages such as
g++
,libclang-dev
andlibssl-dev
.
For MacOS support and for additional requirements needed to test the Linera protocol itself, see the installation section on GitHub.
This manual was tested with the following Rust toolchain:
[toolchain]
channel = "1.81.0"
components = [ "clippy", "rustfmt", "rust-src" ]
targets = [ "wasm32-unknown-unknown" ]
profile = "minimal"
Installing from crates.io
You may install the Linera binaries with
cargo install --locked linera-storage-service@0.13.1
cargo install --locked linera-service@0.13.1 --features storage-service
and use linera-sdk
as a library for Linera Wasm applications:
cargo add linera-sdk@0.13.1
The version number 0.13.1
corresponds to the
current Testnet of Linera. The minor version may change frequently but should
not induce breaking changes.
Installing from GitHub
Download the source from GitHub:
git clone https://github.com/linera-io/linera-protocol.git
cd linera-protocol
git checkout -t origin/testnet_archimedes # Current release branch
To install the Linera toolchain locally from source, you may run:
cargo install --locked --path linera-storage-service
cargo install --locked --path linera-service --features storage-service
Alternatively, for developing and debugging, you may instead use the binaries
compiled in debug mode, e.g. using export PATH="$PWD/target/debug:$PATH"
.
This manual was tested against the following commit of the repository:
f84b66720b92b2eab86622e26b56c3aa2adbc0c0
Bash helper (optional)
Consider adding the output of linera net helper
to your ~/.bash_profile
to
help with automation.
Getting help
If installation fails, reach out to the team (e.g. on Discord) to help troubleshoot your issue or create an issue on GitHub.
Hello, Linera
In this section, you will learn how to interact with the current Testnet, run a local development network, then compile and deploy your first application from scratch.
By the end of this section, you will have a microchain on the Testnet and/or on your local network, and a working application that can be queried using GraphQL.
Using the Testnet
The Linera Testnet is a deployment of the Linera protocol useful for developers and able to host applications.
The current Testnet (codename "Archimedes") is the first deployment of Linera run in partnership with external validators. While it should be considered stable, it will be replaced by a new Testnet when needed. The next Linera Testnet will be restarted from a clean slate and a new genesis block.
To interact with the Testnet, some tokens are needed. A Faucet service is available to create new microchains and obtain some test tokens. To do so, this must be configured when initializing the wallet:
linera wallet init --with-new-chain --faucet https://faucet.testnet-archimedes.linera.net
This creates a new microchain on Testnet with some initial test tokens, and the chain is automatically added to the newly instantiated wallet.
Make sure to use a Linera toolchain compatible with the current Testnet.
Starting a Local Test Network
Another option is to start your own local development network. A development network consists of a number of validators, each of which consists of an ingress proxy (aka. a "load balancer") and a number of workers (aka. "physical shards").
To start a local network, run the following command:
linera net up
This will start a validator with the default number of shards and create a temporary directory storing the entire network state.
This will set up a number of initial chains and create an initial wallet to operate them.
Using the Initial Test Wallet
linera net up
prints Bash statements on its standard output to help you
configure your terminal to use the initial wallet of the new test network, for
instance:
export LINERA_WALLET="/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/wallet.json"
export LINERA_STORAGE="rocksdb:/var/folders/3d/406tbklx3zx2p3_hzzpfqdbc0000gn/T/.tmpvJ6lJI/linera.db"
This wallet is only valid for the lifetime of a single network. Every time a local network is restarted, the wallet needs to be reconfigured.
Interacting with the Network
In the following examples, we assume that either the wallet was initialized to interact with the Devnet or the variables
LINERA_WALLET
andLINERA_STORAGE
are both set and point to the initial wallet of the running local network.
The main way of interacting with the network and deploying applications is using
the linera
client.
To check that the network is working, you can synchronize your default chain with the rest of the network and display the chain balance as follows:
linera sync
linera query-balance
You should see an output number, e.g. 10
.
Building an Example Application
Applications running on Linera are Wasm bytecode. Each validator and client has a built-in Wasm virtual machine (VM) which can execute bytecode.
Let's build the counter
application from the examples/
subdirectory:
cd examples/counter && cargo build --release --target wasm32-unknown-unknown
Publishing your Application
You can publish the bytecode and create an application using it on your local
network using the linera
client's publish-and-create
command and provide:
- The location of the contract bytecode
- The location of the service bytecode
- The JSON encoded initialization arguments
linera publish-and-create \
../target/wasm32-unknown-unknown/release/counter_{contract,service}.wasm \
--json-argument "42"
Congratulations! You've published your first application on Linera!
Querying your Application
Now let's query your application to get the current counter value. To do that, we need to use the client running in service mode. This will expose a bunch of APIs locally which we can use to interact with applications on the network.
linera service
Navigate to http://localhost:8080
in your browser to access the GraphiQL, the
GraphQL IDE. We'll look at this in more detail in a
later section; for now, list
the applications deployed on your default chain e476… by running:
query {
applications(
chainId: "e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65"
) {
id
description
link
}
}
Since we've only deployed one application, the results returned have a single entry.
At the bottom of the returned JSON there is a field link
. To interact with
your application copy and paste the link into a new browser tab.
Finally, to query the counter value, run:
query {
value
}
This will return a value of 42
, which is the initialization argument we
specified when deploying our application.
The Linera Protocol
We now describe the main concepts of the Linera protocol in more details.
Overview
Linera is a decentralized infrastructure optimized for Web3 applications that require guaranteed performance for an unlimited number of active users.
The core idea of the Linera protocol is to run many lightweight blockchains, called microchains, in parallel in a single set of validators.
How does it work?
In Linera, user wallets operate their own microchains. The owner of a chain chooses when to add new blocks to the chain and what goes inside the blocks. Such chains with a single user are called user chains.
Users may add new blocks to their chains in order to process incoming messages from other chains or to execute secure operations on their accounts, for instance to transfer assets to another user.
Importantly, validators ensure that all new blocks are valid. For instance, transfer operations must originate from accounts with sufficient funds; and incoming messages must have been actually sent from another chain. Blocks are verified by validators in the same way for every chain.
A Linera application is a Wasm program that defines its own state and operations. Users can publish bytecode and initialize an application on one chain, and it will be automatically deployed to all chains where it is needed, with a separate state on each chain.
To ensure coordination across chains, an application may rely on asynchronous cross-chain messages. Message payloads are application-specific and opaque to the rest of the system.
┌───┐ ┌───┐ ┌───┐
Chain A │ ├────►│ ├────►│ │
└───┘ └───┘ └───┘
▲
┌─────────┘
│
┌───┐ ┌─┴─┐ ┌───┐
Chain B │ ├────►│ ├────►│ │
└───┘ └─┬─┘ └───┘
│ ▲
│ │
▼ │
┌───┐ ┌───┐ ┌─┴─┐
Chain C │ ├────►│ ├────►│ │
└───┘ └───┘ └───┘
The number of applications present on a single chain is not limited. On the same chain, applications are composed as usual using synchronous calls.
The current Linera SDK uses Rust as a source language to create Wasm applications. It relies on the normal Rust toolchains so that Rust programmers can work in their preferred environments.
How does Linera compare to existing multi-chain infrastructure?
Linera is the first infrastructure designed to support many chains in parallel, and notably an arbitrary number of user chains meant to be operated by user wallets.
In traditional multi-chain infrastructures, each chain usually runs a full blockchain protocol in a separate set of validators. Creating a new chain or exchanging messages between chains is expensive. As a result, the total number of chains is generally limited. Some chains may be specialized to a given use case: these are called "app chains".
In contrast, Linera is optimized for a large number of user chains:
-
Users only create blocks in their chain when needed;
-
Creating a microchain does not require onboarding validators;
-
All chains have the same level of security;
-
Microchains communicate efficiently using the internal networks of validators;
-
Validators are internally sharded (like a regular web service) and may adjust their capacity elastically by adding or removing internal workers.
Besides user chains, the Linera protocol is designed to support other types of microchains, called "permissioned" and "public" chains. Public chains are operated by validators. In this regard, they are similar to classical blockchains. Permissioned chains are meant to be used for temporary interactions between users, such as atomic swaps.
Why build on top of Linera?
We believe that many high-value use cases are currently out of reach of existing Web3 infrastructures because of the challenges of serving many active users simultaneously without degrading user experience (unpredictable fees, latency, etc).
Examples of applications that require processing time-sensitive transactions created by many simultaneous users include:
-
real-time micro-payments and micro-rewards,
-
social data feeds,
-
real-time auction systems,
-
turn-based games,
-
version control systems for software, data pipelines, or AI training pipelines.
Lightweight user chains are instrumental in providing elastic scalability but they have other benefits as well. Because user chains have fewer blocks than traditional blockchains, in Linera, the full-nodes of user chains will be embedded into the users' wallets, typically deployed as a browser extension.
This means that Web UIs connected to a wallet will be able to query the state of the user chain directly (no API provider, no light client) using familiar frameworks (React/GraphQL). Furthermore, wallets will be able to leverage the full node as well for security purposes, including to display meaningful confirmation messages to users.
What is the current state of the development of Linera?
The reference open-source implementation of Linera is under active development. It already includes a Web3 SDK with the necessary features to prototype simple Web3 applications and test them locally on the same machine and deploying them to the Devnet.
Web UIs (possibly reactive) can already be built on top of Wasm-embedded GraphQL services, and tested locally in the browser.
The main limitations of our current Web3 SDK include:
-
Web UIs need to query a local HTTP service acting as a wallet. This setup is meant to be temporary and for testing only: in the future, web UIs will securely connect to a Wallet installed as a browser extension, as usual.
-
Only user chains are currently documented in this manual. Permissioned chains (aka "temporary" chains) were recently added. Support for public chains is in progress.
The main development workstreams of Linera, beyond its SDK, can be broken down as follows.
Core Protocol
- User chains
- Permissioned chains (core protocol only)
- Cross-chain messages
- Cross-chain pub/sub channels (initial version)
- Bytecode publishing
- Application creation
- Reconfigurations of validators
- Support for gas fees
- Support for storage fees and storage limits
- External service (aka. "Faucet") to help users create their first chain
- Permissioned chains (adding operation access control, demo of atomic swaps, etc)
- Avoid repeatedly loading chain states from storage
- Blob storage usable by system and user applications (generalizing/replacing bytecode storage)
- Support for easy onboarding of user chains into a new application (removing the need to accept requests)
- Replace pub/sub channels with data streams (removing the need to accept subscriptions)
- Allow chain clients to control which chains they track (lazily/actively) and execute (do not execute all tracked chains)
- Multi-signed events to facilitate future bridges to external chains
- Public chains (adding leader election, inbox constraints, etc)
- Transaction scripts
- Support for dynamic shard assignment
- Support for archiving chains
- Tokenomics and incentives for all stakeholders
- Governance on the admin chain (e.g. DPoS, onboarding of validators)
- Permissionless auditing protocol
Wasm VM integration
- Support for the Wasmer VM
- Support for the Wasmtime VM (experimental)
- Test gas metering and deterministic execution across VMs
- Composing Wasm applications on the same chain
- Support for non-blocking (yet deterministic) calls to storage
- Support for read-only GraphQL services in Wasm
- Support for mocked system APIs
- Improve host/guest stub generation to make mocks easier
- Support for running Wasm applications in the browser
Storage
- Object management library ("linera-views") on top of Key-Value store abstraction
- Support for Rocksdb
- Experimental support for DynamoDb
- Derive macros for GraphQL
- Support for ScyllaDb
- Make library fully extensible by users (requires better GraphQL macros)
- In-memory storage service for testing purposes
- Support for Web storage (IndexedDB)
- Performance benchmarks and improvements (including faster state hashing)
- Better configuration management
- Local Write-Ahead Log
- Production-grade support for the chosen main database
- Tooling for debugging
- Make the storage library easy to use outside of Linera
Validator Infrastructure
- Simple TCP/UDP networking (used for benchmarks only)
- GRPC networking
- Basic frontend (aka. proxy) supporting fixed internal shards
- Observability
- Kubernetes support in CI
- Deployment using a cloud provider
- Horizontally scalable frontend (aka. proxy)
- Dynamic shard assignment
- Cloud integration to demonstrate elastic scaling
Web3 SDK
- Traits for contract and service interfaces
- Support for unit testing
- Support for integration testing
- Local GraphQL service to query and browse system state
- Local GraphQL service to query and browse application states
- Use GraphQL mutations to execute operations and create blocks
- ABIs for contract and service interfaces
- Allowing message sender to pay for message execution fees
- Wallet as a browser extension (no VM)
- Wallet as a browser extension (with Wasm VM)
- Easier communication with EVM chains
- Bindings to use native cryptographic primitives from Wasm
- Allowing applications to pay for user fees
- Allowing applications to use permissioned chains and public chains
Microchains
This section provides an introduction to microchains, the main building block of the Linera Protocol. For a more formal treatment refer to the whitepaper.
Background
A microchain is a chain of blocks describing successive changes to a shared state. We will use the terms chain and microchain interchangeably. Linera microchains are similar to the familiar notion of blockchain, with the following important specificities:
-
An arbitrary number of microchains can coexist in a Linera network, all sharing the same set of validators and the same level of security. Creating a new microchain only takes one transaction on an existing chain.
-
The task of proposing new blocks in a microchain can be assumed either by validators or by end users (or rather their wallets) depending on the configuration of a chain. Specifically, microchains can be single-owner, permissioned, or public, depending on who is authorized to propose blocks.
Cross-Chain Messaging
In traditional networks with a single blockchain, every transaction can access the application's entire execution state. This is not the case in Linera where the state of an application is spread across multiple microchains, and the state on any individual microchain is only affected by the blocks of that microchain.
Cross-chain messaging is a way for different microchains to communicate with each other asynchronously. This method allows applications and data to be distributed across multiple chains for better scalability. When an application on one chain sends a message to itself on another chain, a cross-chain request is created. These requests are implemented using remote procedure calls (RPCs) within the validators' internal network, ensuring that each request is executed only once.
Instead of immediately modifying the target chain, messages are placed first in the target chain's inbox. When an owner of the target chain creates its next block in the future, they may reference a selection of messages taken from the current inbox in the new block. This executes the selected messages and applies their messages to the chain state.
Below is an example set of chains sending asynchronous messages to each other over consecutive blocks.
┌───┐ ┌───┐ ┌───┐
Chain A │ ├────►│ ├────►│ │
└───┘ └───┘ └───┘
▲
┌─────────┘
│
┌───┐ ┌─┴─┐ ┌───┐
Chain B │ ├────►│ ├────►│ │
└───┘ └─┬─┘ └───┘
│ ▲
│ │
▼ │
┌───┐ ┌───┐ ┌─┴─┐
Chain C │ ├────►│ ├────►│ │
└───┘ └───┘ └───┘
The Linera protocol allows receivers to discard messages but not to change the ordering of selected messages inside the communication queue between two chains. If a selected message fails to execute, the wallet will automatically reject it when proposing the receiver's block. The current implementation of the Linera client always selects as many messages as possible from inboxes, and never discards messages unless they fail to execute.
Chain Ownership Semantics
Active chains can have one or multiple owners. Chains with zero owners are permanently deactivated.
In Linera, the validators guarantee safety: On each chain, at each height, there is at most one unique block.
But liveness—actually adding blocks to a chain at all—relies on the owners. There are different types of rounds and owners, optimized for different use cases:
- First an optional fast round, where a super owner can propose blocks that get confirmed with very particularly low latency, optimal for single-owner chains with no contention.
- Then a number of multi-leader rounds, where all regular owners can propose blocks. This works well even if there is occasional, temporary contention: an owner using multiple devices, or multiple people using the same chain infrequently.
- And finally single-leader rounds: These give each regular chain owner a time slot in which only they can propose a new block, without being hindered by any other owners' proposals. This is ideal for chains with many users that are trying to commit blocks at the same time.
The number of multi-leader rounds is configurable: On chains with fluctuating levels of activity, this allows the system to dynamically switch to single-leader mode whenever all multi-leader rounds fail during periods of high contention. Chains that very often have high activity from multiple owners can set the number of multi-leader rounds to 0.
For more detail and examples on how to open and close chains, see the wallet section on chain management.
Wallets
As in traditional blockchains, Linera wallets are in charge of holding user private keys. However, instead of signing transactions, Linera wallets are meant to sign blocks and propose them to extend the chains owned by their users.
In practice, wallets include a node which tracks a subset of Linera chains. We will see in the next section how a Linera wallet can run a GraphQL service to expose the state of its chains to web frontends.
The command-line tool
linera
is the main way for developers to interact with a Linera network and manage the user wallets present locally on the system.
Note that this command-line tool is intended mainly for development purposes. Our goal is that end users eventually manage their wallets in a browser extension.
Selecting a Wallet
The private state of a wallet is conventionally stored in a file wallet.json
,
while the state of its node is stored in a file linera.db
.
To switch between wallets, you may use the --wallet
and --storage
options of
the linera
tool, e.g. as in
linera --wallet wallet2.json --storage rocksdb:linera2.db
.
You may also define the environment variables LINERA_STORAGE
and
LINERA_WALLET
to the same effect. E.g. LINERA_STORAGE=$PWD/wallet2.json
and
LINERA_WALLET=$PWD/wallet2.json
.
Finally, if LINERA_STORAGE_$I
and LINERA_WALLET_$I
are defined for some
number I
, you may call linera --with-wallet $I
(or linera -w $I
for
short).
Chain Management
Listing Chains
To list the chains present in your wallet, you may use the command show
:
linera wallet show
╭──────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────────────────────╮
│ Chain ID ┆ Latest Block │
╞══════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════╡
│ 668774d6f49d0426f610ad0bfa22d2a06f5f5b7b5c045b84a26286ba6bce93b4 ┆ Public Key: 3812c2bf764e905a3b130a754e7709fe2fc725c0ee346cb15d6d261e4f30b8f1 │
│ ┆ Owner: c9a538585667076981abfe99902bac9f4be93714854281b652d07bb6d444cb76 │
│ ┆ Block Hash: - │
│ ┆ Timestamp: 2023-04-10 13:52:20.820840 │
│ ┆ Next Block Height: 0 │
├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ 91c7b394ef500cd000e365807b770d5b76a6e8c9c2f2af8e58c205e521b5f646 ┆ Public Key: 29c19718a26cb0d5c1d28102a2836442f53e3184f33b619ff653447280ccba1a │
│ ┆ Owner: efe0f66451f2f15c33a409dfecdf76941cf1e215c5482d632c84a2573a1474e8 │
│ ┆ Block Hash: 51605cad3f6a210183ac99f7f6ef507d0870d0c3a3858058034cfc0e3e541c13 │
│ ┆ Timestamp: 2023-04-10 13:52:21.885221 │
│ ┆ Next Block Height: 1 │
╰──────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────────────────────╯
Each row represents a chain present in the wallet. On the left is the unique identifier on the chain, and on the right is metadata for that chain associated with the latest block.
Default Chain
Each wallet has a default chain that all commands apply to unless you specify
another --chain
on the command line.
The default chain is set initially, when the first chain is added to the wallet. You can check the default chain for your wallet by running:
linera wallet show
The Chain ID which is in green text instead of white text is your default chain.
To change the default chain for your wallet, user the set-default
command:
linera wallet set-default <chain-id>
Opening a Chain
The Linera protocol defines semantics for how new chains are created, we call this "opening a chain". A chain cannot be opened in a vacuum, it needs to be created by an existing chain on the network.
Open a Chain for Your Own Wallet
To open a chain for your own wallet, you can use the open-chain
command:
linera open-chain
This will create a new chain (using the wallet's default chain) and add it to
the wallet. Use the wallet show
command to see your existing chains.
Open a Chain for Another Wallet
Opening a chain for another wallet
requires an extra two steps. Let's
initialize a second wallet:
linera --wallet wallet2.json --storage rocksdb:linera2.db wallet init --genesis target/debug/genesis.json
First wallet2
must create an unassigned keypair. The public part of that
keypair is then sent to the wallet
who is the chain creator.
linera --wallet wallet2.json keygen
6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 # this is the public key for the unassigned keypair
Next, using the public key, wallet
can open a chain for wallet2
.
linera open-chain --to-public-key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888
e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000
fc9384defb0bcd8f6e206ffda32599e24ba715f45ec88d4ac81ec47eb84fa111
The first line is the message ID specifying the cross-chain message that creates the new chain. The second line is the new chain's ID.
Finally, to add the chain to wallet2
for the given unassigned key we use the
assign
command:
linera --wallet wallet2.json assign --key 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 --message-id e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000
Opening a Chain with Multiple Users
The open-chain
commands is a simplified version of open-multi-owner-chain
,
which gives you fine-grained control over the set and kinds of owners and rounds
for the new chain, and the timeout settings for the rounds. E.g. this creates a
chain with two owners and two multi-leader rounds.
linera open-multi-owner-chain \
--chain-id e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65010000000000000000000000 \
--owner-public-keys 6443634d872afbbfcc3059ac87992c4029fa88e8feb0fff0723ac6c914088888 \
ca909dcf60df014c166be17eb4a9f6e2f9383314a57510206a54cd841ade455e \
--multi-leader-rounds 2
The change-ownership
command offers the same options to add or remove owners
and change round settings for an existing chain.
Setting up Extra Wallets Automatically with linera net up
For testing, rather than using linera open-chain
and linera assign
as above,
it is often more convenient to pass the option --extra-wallets N
to
linera net up
.
This option will create create N
additional user wallets and output Bash
commands to define the environment variables LINERA_{WALLET,STORAGE}_$I
where
I
ranges over 0..=N
(I=0
being the wallet for the initial chains).
Once all the environment variables are defined, you may switch between wallets
using linera --with-wallet I
or linera -w I
for short.
Automation in Bash
To automate the process of setting the variables LINERA_WALLET*
and
LINERA_STORAGE*
after creating a local test network in a shell, we provide a
Bash helper function linera_spawn_and_read_wallet_variables
.
To define the function linera_spawn_and_read_wallet_variables
in your shell,
run source /dev/stdin <<<"$(linera net helper 2>/dev/null)"
. You may also add
the output of linera net helper
to your ~/.bash_profile
for future sessions.
Once the function is defined, call
linera_spawn_and_read_wallet_variables linera net up
instead of
linera net up
.
Node Service
So far we've seen how to use the Linera client treating it as a binary in your terminal. However, the client also acts as a node which:
- Executes blocks
- Exposes a GraphQL API and IDE for dynamically interacting with applications and the system
- Listens for notifications from validators and automatically updates local chains.
To interact with the node service, run linera
in service
mode:
linera service
This will run the node service on port 8080 by default (this can be overridden
using the --port
flag).
A Note on GraphQL
Linera uses GraphQL as the query language for interfacing with different parts of the system. GraphQL enables clients to craft queries such that they receive exactly what they want and nothing more.
GraphQL is used extensively during application development, especially to query the state of an application from a front-end for example.
To learn more about GraphQL check out the official docs.
GraphiQL IDE
Conveniently, the node service exposes a GraphQL IDE called GraphiQL. To use
GraphiQL start the node service and navigate to localhost:8080/
.
Using the schema explorer on the left of the GraphiQL IDE you can dynamically explore the state of the system and your applications.
GraphQL System API
The node service also exposes a GraphQL API which corresponds to the set of
system operations. You can explore the full set of operations by clicking on
MutationRoot
.
GraphQL Application API
To interact with an application, we run the Linera client in service mode. It
exposes a GraphQL API for every application running on any owned chain at
localhost:8080/chains/<chain-id>/applications/<application-id>
.
Navigating there with your browser will open a GraphiQL interface which enables you to graphically explore the state of your application.
Applications
The programming model of Linera is designed so that developers can take advantage of microchains to scale their applications.
Linera uses the WebAssembly (Wasm) Virtual Machine to execute user applications. Currently, the Linera SDK is focused on the Rust programming language.
Linera applications are structured using the familiar notion of Rust crate: the external interfaces of an application (including instantiation parameters, operations and messages) generally go into the library part of its crate, while the core of each application is compiled into binary files for the Wasm architecture.
The Application Deployment Lifecycle
Linera Applications are designed to be powerful yet re-usable. For this reason there is a distinction between the bytecode and an application instance on the network.
Applications undergo a lifecycle transition aimed at making development easy and flexible:
- The bytecode is built from a Rust project with the
linera-sdk
dependency. - The bytecode is published to the network on a microchain, and assigned an identifier.
- A user can create a new application instance, by providing the bytecode identifier and instantiation arguments. This process returns an application identifier which can be used to reference and interact with the application.
- The same bytecode identifier can be used as many times needed by as many users needed to create distinct applications.
Importantly, the application deployment lifecycle is abstracted from the user, and an application can be published with a single command:
linera publish-and-create <contract-path> <service-path> <init-args>
This will publish the bytecode as well as instantiate the application for you.
Anatomy of an Application
An application is broken into two major components, the contract and the service.
The contract is gas-metered, and is the part of the application which executes operations and messages, make cross-application calls and modifies the application's state. The details are covered in more depth in the SDK docs.
The service is non-metered and read-only. It is used primarily to query the state of an application and populate the presentation layer (think front-end) with the data required for a user interface.
Operations and Messages
For this section we'll be using a simplified version of the example application called "fungible" where users can send tokens to each other.
At the system-level, interacting with an application can be done via operations and messages.
Operations are defined by an application developer and each application can have a completely different set of operations. Chain owners then actively create operations and put them in their block proposals to interact with an application. Other applications may also call the application by providing an operation for it to execute, this is called a cross-application call and always happens within the same chain. Operations for cross-application calls may return a response value back to the caller.
Taking the "fungible token" application as an example, an operation for a user to transfer funds to another user would look like this:
extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Operation {
/// A transfer from a (locally owned) account to a (possibly remote) account.
Transfer {
owner: AccountOwner,
amount: Amount,
target_account: Account,
},
// Meant to be extended here
}
Messages result from the execution of operations or other messages. Messages can be sent from one chain to another, always within the same application. Block proposers also actively include messages in their block proposal, but unlike with operations, they are only allowed to include them in the right order (possibly skipping some), and only if they were actually created by another chain (or by a previous block of the same chain). Messages that originate from the same transaction are included as a single transaction in the receiving block.
In our "fungible token" application, a message to credit an account would look like this:
extern crate serde;
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub enum Message {
Credit { owner: AccountOwner, amount: Amount },
// Meant to be extended here
}
Authentication
Operations in a block are always authenticated and messages may be authenticated. The signer of a block becomes the authenticator of all the operations in that block. As operations are being executed by applications, messages can be created to be sent to other chains. When they are created, they can be configured to be authenticated. In that case, the message receives the same authentication as the operation that created it. If handling an incoming message creates new messages, those may also be configured to have the same authentication as the received message.
In other words, the block signer can have its authority propagated across chains through series of messages. This allows applications to safely store user state on chains that the user may not have the authority to produce blocks. The application may also allow only the authorized user to change that state, and not even the chain owner is able to override that.
The figure below shows four chains (A, B, C, D) and some blocks produced in them. In this example, each chain is owned by a single owner (aka. address). Owners are in charge of producing blocks and sign new blocks using their signing keys. Some blocks show the operations and incoming messages they accept, where the authentication is shown inside parenthesis. All operations produced are authenticated by the block proposer, and if these are all single user chains, the proposer is always the chain owner. Messages that have authentication use the one from the operation or message that created it.
One example in the figure is that chain A produced a block with Operation 1,
which is authenticated by the owner of chain A (written (a)
). That operation
sent a message to chain B, and assuming the message was sent with the
authentication forwarding enabled, it is received and executed in chain B with
the authentication of (a)
. Another example is that chain D produced a block
with Operation 2, which is authenticated by the owner of chain D (written
(d)
). That operation sent a message to chain C, which is executed with
authentication of (d)
like the example before. Handling that message in chain
C produced a new message, which was sent to chain B. That message, when received
by chain B is executed with the authentication of (d)
.
┌───┐ ┌─────────────────┐ ┌───┐
Chain A owned by (a) │ ├────►│ Operation 1 (a) ├────►│ │
└───┘ └────────┬────────┘ └───┘
│
└────────────┐
▼
┌──────────────────────────┐
┌───┐ ┌───┐ │ Message from chain A (a) │
Chain B owned by (b) │ ├────►│ ├────►│ Message from chain C (d) |
└───┘ └───┘ │ Operation 3 (b) │
└──────────────────────────┘
▲
┌────────┘
│
┌───┐ ┌──────────────────────────┐ ┌───┐
Chain C owned by (c) │ ├────►│ Message from chain D (d) ├────►│ │
└───┘ └──────────────────────────┘ └───┘
▲
┌───────────┘
│
┌─────────────────┐ ┌───┐ ┌───┐
Chain D owned by (d) │ Operation 2 (d) ├────►│ ├────►│ │
└─────────────────┘ └───┘ └───┘
An example where this is used is in the Fungible application, where a Claim
operation allows retrieving money from a chain the user does not control (but
the user still trusts will produce a block receiving their message). Without the
Claim
operation, users would only be able to store their tokens on their own
chains, and multi-owner and public chains would have their tokens shared between
anyone able to produce a block.
With the Claim
operation, users can store their tokens on another chain where
they're able to produce blocks or where they trust the owner will produce blocks
receiving their messages. Only they are able to move their tokens, even on
chains where ownership is shared or where they are not able to produce blocks.
Registering an Application across Chains
If Alice is using an application on her chain and starts interacting with Bob
via the application, e.g. sends him some tokens using the fungible
example,
the application automatically gets registered on Bob's chain, too, as soon as he
handles the incoming cross-chain messages. After that, he can execute the
application's operations on his chain, too, and e.g. send tokens to someone.
But there are also cases where Bob may want to start using an application he
doesn't have yet. E.g. maybe Alice regularly makes posts using the social
example, and Bob wants to subscribe to her.
In that case, trying to execute an application-specific operation would fail, because the application is not registered on his chain. He needs to request it from Alice first:
linera request-application <application-id> --target-chain-id <alices-chain-id>
Once Alice processes his message (which happens automatically if she is running the client in service mode), he can start using the application.
Writing Linera Applications
In this section, we'll be exploring how to create Web3 applications using the Linera SDK.
We'll use a simple "counter" application as a running example.
We'll focus on the back end of the application, which consists of two main parts: a smart contract and its GraphQL service.
Both the contract and the service of an application are written in Rust using
the crate linera-sdk
, and compiled to
Wasm bytecode.
This section should be seen as a guide versus a reference manual for the SDK. For the reference manual, refer to the documentation of the crate.
Creating a Linera Project
To create your Linera project, use the linera project new
command. The command
should be executed outside the linera-protocol
folder. It sets up the
scaffolding and requisite files:
linera project new my-counter
linera project new
bootstraps your project by creating the following key
files:
Cargo.toml
: your project's manifest filled with the necessary dependencies to create an app;src/lib.rs
: the application's ABI definition;src/state.rs
: the application's state;src/contract.rs
: the application's contract, and the binary target for the contract bytecode;src/service.rs
: the application's service, and the binary target for the service bytecode.
When writing Linera applications it is a convention to use your app's name as a prefix for names of
trait
,struct
, etc. Hence, in the following manual, we will useCounterContract
,CounterService
, etc.
Creating the Application State
The state of a Linera application consists of onchain data that are persisted between transactions.
The struct
which defines your application's state can be found in
src/state.rs
. To represent our counter, we're going to use a u64
integer.
While we could use a plain data-structure for the entire application state:
struct Counter {
value: u64
}
in general, we prefer to manage persistent data using the concept of "views":
Views allow an application to load persistent data in memory and stage modifications in a flexible way.
Views resemble the persistent objects of an ORM framework, except that they are stored as a set of key-value pairs (instead of a SQL row).
In this case, the placeholder Application
struct in src/state.rs
should be
replaced by
/// The application state.
#[derive(RootView, async_graphql::SimpleObject)]
#[view(context = "ViewStorageContext")]
pub struct Counter {
pub value: RegisterView<u64>,
// Additional fields here will get their own key in storage.
}
and the occurrences of Application
in the rest of the project should be
replaced by Counter
.
The derive macro async_graphql::SimpleObject
is related to GraphQL queries
discussed in the next section.
A RegisterView<T>
supports modifying a single value of type T
. Other data
structures available in the library
linera_views
include:
LogView
for a growing vector of values;QueueView
for queues;MapView
andCollectionView
for associative maps; specifically,MapView
in the case of static values, andCollectionView
when values are other views.
For an exhaustive list of the different constructions, refer to the crate documentation.
Defining the ABI
The Application Binary Interface (ABI) of a Linera application defines how to interact with this application from other parts of the system. It includes the data structures, data types, and functions exposed by on-chain contracts and services.
ABIs are usually defined in src/lib.rs
and compiled across all architectures
(Wasm and native).
For a reference guide, check out the documentation of the crate.
Defining a marker struct
The library part of your application (generally in src/lib.rs
) must define a
public empty struct that implements the Abi
trait.
struct CounterAbi;
The Abi
trait combines the ContractAbi
and ServiceAbi
traits to include
the types that your application exports.
/// A trait that includes all the types exported by a Linera application (both contract
/// and service).
pub trait Abi: ContractAbi + ServiceAbi {}
Next, we're going to implement each of the two traits.
Contract ABI
The ContractAbi
trait defines the data types that your application uses in a
contract. Each type represents a specific part of the contract's behavior:
/// A trait that includes all the types exported by a Linera application contract.
pub trait ContractAbi {
/// The type of operation executed by the application.
///
/// Operations are transactions directly added to a block by the creator (and signer)
/// of the block. Users typically use operations to start interacting with an
/// application on their own chain.
type Operation: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
/// The response type of an application call.
type Response: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}
All these types must implement the Serialize
, DeserializeOwned
, Send
,
Sync
, Debug
traits, and have a 'static
lifetime.
In our example, we would like to change our Operation
to u64
, like so:
extern crate linera_base;
use linera_base::abi::ContractAbi;
struct CounterAbi;
impl ContractAbi for CounterAbi {
type Operation = u64;
type Response = ();
}
Service ABI
The ServiceAbi
is in principle very similar to the ContractAbi
, just for the
service component of your application.
The ServiceAbi
trait defines the types used by the service part of your
application:
/// A trait that includes all the types exported by a Linera application service.
pub trait ServiceAbi {
/// The type of a query receivable by the application's service.
type Query: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
/// The response type of the application's service.
type QueryResponse: Serialize + DeserializeOwned + Send + Sync + Debug + 'static;
}
For our Counter example, we'll be using GraphQL to query our application so our
ServiceAbi
should reflect that:
extern crate linera_base;
extern crate async_graphql;
use linera_base::abi::ServiceAbi;
struct CounterAbi;
impl ServiceAbi for CounterAbi {
type Query = async_graphql::Request;
type QueryResponse = async_graphql::Response;
}
Writing the Contract Binary
The contract binary is the first component of a Linera application. It can actually change the state of the application.
To create a contract, we need to create a new type and implement the Contract
trait for it, which is as follows:
pub trait Contract: WithContractAbi + ContractAbi + Sized {
/// The type of message executed by the application.
type Message: Serialize + DeserializeOwned + Debug;
/// Immutable parameters specific to this application (e.g. the name of a token).
type Parameters: Serialize + DeserializeOwned + Clone + Debug;
/// Instantiation argument passed to a new application on the chain that created it
/// (e.g. an initial amount of tokens minted).
type InstantiationArgument: Serialize + DeserializeOwned + Debug;
/// Creates an in-memory instance of the contract handler.
async fn load(runtime: ContractRuntime<Self>) -> Self;
/// Instantiates the application on the chain that created it.
async fn instantiate(&mut self, argument: Self::InstantiationArgument);
/// Applies an operation from the current block.
async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response;
/// Applies a message originating from a cross-chain message.
async fn execute_message(&mut self, message: Self::Message);
/// Finishes the execution of the current transaction.
async fn store(self);
}
The full trait definition can be found here.
There's quite a bit going on here, so let's break it down and take one method at a time.
For this application, we'll be using the load
, execute_operation
and store
methods.
The Contract Lifecycle
To implement the application contract, we first create a type for the contract:
pub struct CounterContract {
state: Counter,
runtime: ContractRuntime<Self>,
}
This type usually contains at least two fields: the persistent state
defined
earlier and a handle to the runtime. The runtime provides access to information
about the current execution and also allows sending messages, among other
things. Other fields can be added, and they can be used to store volatile data
that only exists while the current transaction is being executed, and discarded
afterwards.
When a transaction is executed, the contract type is created through a call to
Contract::load
method. This method receives a handle to the runtime that the
contract can use, and should use it to load the application state. For our
implementation, we will load the state and create the CounterContract
instance:
async fn load(runtime: ContractRuntime<Self>) -> Self {
let state = Counter::load(runtime.root_view_storage_context())
.await
.expect("Failed to load state");
CounterContract { state, runtime }
}
When the transaction finishes executing successfully, there's a final step where
all loaded application contracts are called in order to do any final checks and
persist its state to storage. That final step is a call to the Contract::store
method, which can be thought of as similar to executing a destructor. In our
implementation we will persist the state back to storage:
async fn store(mut self) {
self.state.save().await.expect("Failed to save state");
}
It's possible to do more than just saving the state, and the Contract finalization section provides more details on that.
Instantiating our Application
The first thing that happens when an application is created from a bytecode is
that it is instantiated. This is done by calling the contract's
Contract::instantiate
method.
Contract::instantiate
is only called once when the application is created and
only on the microchain that created the application.
Deployment on other microchains will use the Default
value of all sub-views in
the state if the state uses the view paradigm.
For our example application, we'll want to initialize the state of the application to an arbitrary number that can be specified on application creation using its instantiation parameters:
async fn instantiate(&mut self, value: u64) {
self.state.value.set(value);
}
Implementing the Increment Operation
Now that we have our counter's state and a way to initialize it to any value we would like, we need a way to increment our counter's value. Execution requests from block proposers or other applications are broadly called 'operations'.
To handle an operation, we need to implement the Contract::execute_operation
method. In the counter's case, the operation it will be receiving is a u64
which is used to increment the counter by that value:
async fn execute_operation(&mut self, operation: u64) {
let current = self.state.value.get();
self.state.value.set(current + operation);
}
Declaring the ABI
Finally, to link our Contract
trait implementation with the ABI of the
application, the following code is added:
impl WithContractAbi for CounterContract {
type Abi = counter::CounterAbi;
}
Writing the Service Binary
The service binary is the second component of a Linera application. It is compiled into a separate Bytecode from the contract and is run independently. It is not metered (meaning that querying an application's service does not consume gas), and can be thought of as a read-only view into your application.
Application states can be arbitrarily complex, and most of the time you don't want to expose this state in its entirety to those who would like to interact with your app. Instead, you might prefer to define a distinct set of queries that can be made against your application.
The Service
trait is how you define the interface into your application. The
Service
trait is defined as follows:
pub trait Service: WithServiceAbi + ServiceAbi + Sized {
/// Immutable parameters specific to this application.
type Parameters: Serialize + DeserializeOwned + Send + Sync + Clone + Debug + 'static;
/// Creates an in-memory instance of the service handler.
async fn new(runtime: ServiceRuntime<Self>) -> Self;
/// Executes a read-only query on the state of this application.
async fn handle_query(&self, query: Self::Query) -> Self::QueryResponse;
}
The full service trait definition can be found here.
Let's implement Service
for our counter application.
First, we create a new type for the service, similarly to the contract:
pub struct CounterService {
state: Counter,
}
Just like with the CounterContract
type, this type usually has two types: the
application state
and the runtime
. We can omit the fields if we don't use
them, so in this example we're omitting the runtime
field, since its only used
when constructing the CounterService
type.
We need to generate the necessary boilerplate for implementing the service
WIT interface,
export the necessary resource types and functions so that the service can be
executed. Fortunately, there is a macro to perform this code generation, so just
add the following to service.rs
:
linera_sdk::service!(CounterService);
Next, we need to implement the Service
trait for CounterService
type. The
first step is to define the Service
's associated type, which is the global
parameters specified when the application is instantiated. In our case, the
global parameters aren't used, so we can just specify the unit type:
#[async_trait]
impl Service for CounterService {
type Parameters = ();
}
Also like in contracts, we must implement a load
constructor when implementing
the Service
trait. The constructor receives the runtime handle and should use
it to load the application state:
async fn load(runtime: ServiceRuntime<Self>) -> Self {
let state = Counter::load(runtime.root_view_storage_context())
.await
.expect("Failed to load state");
Ok(CounterService { state })
}
Services don't have a store
method because they are read-only and can't
persist any changes back to the storage.
The actual functionality of the service starts in the handle_query
method. We
will accept GraphQL queries and handle them using the
async-graphql
crate. To
forward the queries to custom GraphQL handlers we will implement in the next
section, we use the following code:
async fn handle_query(&mut self, request: Request) -> Response {
let schema = Schema::build(
// implemented in the next section
QueryRoot { value: *self.state.value.get() },
// implemented in the next section
MutationRoot {},
EmptySubscription,
)
.finish();
schema.execute(request).await
}
}
Finally, as before, the following code is needed to incorporate the ABI
definitions into your Service
implementation:
impl WithServiceAbi for Counter {
type Abi = counter::CounterAbi;
}
Adding GraphQL compatibility
Finally, we want our application to have GraphQL compatibility. To achieve this
we need a QueryRoot
to respond to queries and a MutationRoot
for creating
serialized Operation
values that can be placed in blocks.
In the QueryRoot
, we only create a single value
query that returns the
counter's value:
struct QueryRoot {
value: u64,
}
#[Object]
impl QueryRoot {
async fn value(&self) -> &u64 {
&self.value
}
}
In the MutationRoot
, we only create one increment
method that returns a
serialized operation to increment the counter by the provided value
:
struct MutationRoot;
#[Object]
impl MutationRoot {
async fn increment(&self, value: u64) -> Vec<u8> {
bcs::to_bytes(&value).unwrap()
}
}
We haven't included the imports in the above code; they are left as an exercise
to the reader (but remember to import async_graphql::Object
). If you want the
full source code and associated tests check out the examples
section
on GitHub.
Deploying the Application
The first step to deploy your application is to configure a wallet. This will determine where the application will be deployed: either to a local net or to the public deployment (i.e. a devnet or a testnet).
Local Net
To configure the local network, follow the steps in the Getting Started section.
Afterwards, the LINERA_WALLET
and the LINERA_STORAGE
environment variables
should be set and can be used in the publish-and-create
command to deploy the
application while also specifying:
- The location of the contract bytecode
- The location of the service bytecode
- The JSON encoded initialization arguments
linera publish-and-create \
target/wasm32-unknown-unknown/release/my-counter_{contract,service}.wasm \
--json-argument "42"
Devnets and Testnets
To configure the wallet for the current testnet while creating a new microchain, the following command can be used:
linera wallet init --with-new-chain --faucet https://faucet.testnet-archimedes.linera.net
The Faucet will provide the new chain with some tokens, which can then be used
to deploy the application with the publish-and-create
command. It requires
specifying:
- The location of the contract bytecode
- The location of the service bytecode
- The JSON encoded initialization arguments
linera publish-and-create \
target/wasm32-unknown-unknown/release/my-counter_{contract,service}.wasm \
--json-argument "42"
Interacting with the Application
To interact with the deployed application, a node service must be used.
Cross-Chain Messages
On Linera, applications are meant to be multi-chain: They are instantiated on every chain where they are used. An application has the same application ID and bytecode everywhere, but a separate state on every chain. To coordinate, the instances can send cross-chain messages to each other. A message sent by an application is always handled by the same application on the target chain: The handling code is guaranteed to be the same as the sending code, but the state may be different.
For your application, you can specify any serializable type as the Message
type in your Contract
implementation. To send a message, use the
ContractRuntime
made available as an argument to the contract's [Contract::load
] constructor.
The runtime is usually stored inside the contract object, as we did when
writing the contract binary. We can then call
ContractRuntime::prepare_message
to start preparing a message, and then
send_to
to send it to a destination chain.
self.runtime
.prepare_message(message_contents)
.send_to(destination_chain_id);
It is also possible to send a message to a subscription channel, so that the
message is forwarded to the subscribers of that channel. All that has to be done
is specify a
ChannelName
as the destination parameter to send_to
.
After block execution in the sending chain, sent messages are placed in the
target chains' inboxes for processing. There is no guarantee that it will be
handled: For this to happen, an owner of the target chain needs to include it in
the incoming_messages
in one of their blocks. When that happens, the
contract's execute_message
method gets called on their chain.
While preparing the message to be sent, it is possible to enable authentication forwarding and/or tracking. Authentication forwarding means that the message is executed by the receiver with the same authenticated signer as the sender of the message, while tracking means that the message is sent back to the sender if the receiver rejects it. The example below enables both flags:
self.runtime
.prepare_message(message_contents)
.with_tracking()
.with_authentication()
.send_to(destination_chain_id);
Example: Fungible Token
In the fungible
example
application, such a message
can be the transfer of tokens from one chain to another. If the sender includes
a Transfer
operation on their chain, it decreases their account balance and
sends a Credit
message to the recipient's chain:
async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response {
match operation {
// ...
Operation::Transfer {
owner,
amount,
target_account,
} => {
self.check_account_authentication(owner)?;
self.state.debit(owner, amount).await?;
self.finish_transfer_to_account(amount, target_account, owner)
.await;
FungibleResponse::Ok
}
// ...
}
}
async fn finish_transfer_to_account(
&mut self,
amount: Amount,
target_account: Account,
source: AccountOwner,
) {
if target_account.chain_id == self.runtime.chain_id() {
self.state.credit(target_account.owner, amount).await;
} else {
let message = Message::Credit {
target: target_account.owner,
amount,
source,
};
self.runtime
.prepare_message(message)
.with_authentication()
.with_tracking()
.send_to(target_account.chain_id);
}
}
On the recipient's chain, execute_message
is called, which increases their
account balance.
async fn execute_message(&mut self, message: Message) {
match message {
Message::Credit {
amount,
target,
source,
} => {
// ...
self.state.credit(receiver, amount).await;
}
// ...
}
}
Calling other Applications
We have seen that cross-chain messages sent by an application on one chain are always handled by the same application on the target chain.
This section is about calling other applications using cross-application calls.
Such calls happen on the same chain and are made with the
ContractRuntime::call_application
method:
pub fn call_application<A: ContractAbi + Send>(
&mut self,
authenticated: bool,
application: ApplicationId<A>,
call: &A::Operation,
) -> A::Response {
The authenticated
argument specifies whether the callee is allowed to perform
actions that require authentication on behalf of the signer of the original
block that caused this call.
The application
argument is the callee's application ID, and A
is the
callee's ABI.
The call
argument is the operation requested by the application call.
Example: Crowd-Funding
The crowd-funding
example application allows the application creator to launch
a campaign with a funding target. That target can be an amount specified in any
type of token based on the fungible
application. Others can then pledge tokens
of that type to the campaign, and if the target is not reached by the deadline,
they are refunded.
If Alice used the fungible
example to create a Pugecoin application (with an
impressionable pug as its mascot), then Bob can create a crowd-funding
application, use Pugecoin's application ID as CrowdFundingAbi::Parameters
, and
specify in CrowdFundingAbi::InstantiationArgument
that his campaign will run
for one week and has a target of 1000 Pugecoins.
Now let's say Carol wants to pledge 10 Pugecoin tokens to Bob's campaign.
First she needs to make sure she has his crowd-funding application on her chain,
e.g. using the linera request-application
command. This will automatically
also register Alice's Pugecoin application on her chain, because it is a
dependency of Bob's.
Now she can make her pledge by running the linera service
and making a query
to Bob's application:
mutation { pledge(owner: "User:841…6c0", amount: "10") }
This will add a block to Carol's chain containing the pledge operation that gets
handled by CrowdFunding::execute_operation
, resulting in one cross-application
call and two cross-chain messages:
First CrowdFunding::execute_operation
calls the fungible
application on
Carol's chain to transfer 10 tokens to Carol's account on Bob's chain:
// ...
let call = fungible::Operation::Transfer {
owner,
amount,
target_account,
};
// ...
self.runtime
.call_application(/* authenticated by owner */ true, fungible_id, &call);
This causes Fungible::execute_operation
to be run, which will create a
cross-chain message sending the amount 10 to the Pugecoin application instance
on Bob's chain.
After the cross-application call returns, CrowdFunding::execute_operation
continues to create another cross-chain message
crowd_funding::Message::PledgeWithAccount
, which informs the crowd-funding
application on Bob's chain that the 10 tokens are meant for the campaign.
When Bob now adds a block to his chain that handles the two incoming messages,
first Fungible::execute_message
gets executed, and then
CrowdFunding::execute_message
. The latter makes another cross-application call
to transfer the 10 tokens from Carol's account to the crowd-funding
application's account (both on Bob's chain). That is successful because Carol
does now have 10 tokens on this chain and she authenticated the transfer
indirectly by signing her block. The crowd-funding application now makes a note
in its application state on Bob's chain that Carol has pledged 10 Pugecoin
tokens.
For the complete code please take a look at the
crowd-funding
and the
fungible
application contracts in the examples
folder in linera-protocol
.
Using Data Blobs
Some applications may want to use static assets, like images or other data: e.g.
the non-fungible
example application implements NFTs, and each NFT has an
associated image.
Data blobs are pieces of binary data that, once published on any chain, can be used on all chains. What format they are in and what they are used for is determined by the application(s) that read(s) them.
You can use the linera publish-data-blob
command to publish the contents of a
file, as an operation in a block on one of your chains. This will print the ID
of the new blob, including its hash. Alternatively, you can run linera service
and use the publishDataBlob
GraphQL mutation.
Applications can now use runtime.read_data_blob(blob_hash)
to read the blob.
This works on any chain, not only the one that published it. The first time your
client executes a block reading a blob, it will download the blob from the
validators if it doesn't already have it locally.
In the case of the NFT app, it is only the service, not the contract, that
actually uses the blob data to display it as an image in the frontend. But we
still want to make sure that the user has the image locally as soon as they
receive an NFT, even if they don't view it yet. This can be achieved by calling
runtime.assert_data_blob_exists(blob_hash)
in the contract: It will make sure
the data is available, without actually loading it.
For the complete code please take a look at the non-fungible
contract
and service.
Printing Logs from an Application
Applications can use the log
crate to print
log messages with different levels of importance. Log messages are useful during
development, but they may also be useful for end users. By default the
linera service
command will log the messages from an application if they are
of the "info" importance level or higher (briefly, log::info!
, log::warn!
and log::error!
).
During development it is often useful to log messages of lower importance (such
as log::debug!
and log::trace!
). To enable them, the RUST_LOG
environment
variable must be set before running linera service
. The example below enables
trace level messages from applications and enables warning level messages from
other parts of the linera
binary:
export RUST_LOG="warn,linera_execution::wasm=trace"
Writing Tests
Linera applications can be tested using normal Rust unit tests or integration tests. Unit tests use a mock runtime for execution, so it's useful for testing the application as if it were running by itself on a single chain. Integration tests use a simulated validator for testing. This allows creating chains and adding blocks to them in order to test interactions between multiple microchains and multiple applications.
Applications should consider having both types of tests. Unit tests should be used to focus on the application's internals and core functionality. Integration tests should be used to test how the application behaves on a more complex environment that's closer to the real network.
For Rust tests, the
cargo test
command can be used to run both the unit and integration tests.
Unit tests
Unit tests are written beside the application's source code (i.e., inside the
src
directory of the project). The main purpose of a unit test is to test
parts of the application in an isolated environment. Anything that's external is
usually mocked. When the linera-sdk
is compiled with the test
feature
enabled, the ContractRuntime
and SystemRuntime
types are actually mock
runtimes, and can be configured to return specific values for different tests.
Example
A simple unit test is shown below, which tests if the application contract's
do_something
method changes the application state.
#[cfg(test)]
mod tests {
use crate::{ApplicationContract, ApplicationState};
use linera_sdk::{util::BlockingWait, ContractRuntime};
#[test]
fn test_do_something() {
let runtime = ContractRuntime::new();
let mut contract = ApplicationContract::load(runtime).blocking_wait();
let result = contract.do_something();
// Check that `do_something` succeeded
assert!(result.is_ok());
// Check that the state in memory was updated
assert_eq!(contract.state, ApplicationState {
// Define the application's expected final state
..ApplicationState::default()
});
// Check that the state in memory is different from the state in storage
assert_ne!(
contract.state,
ApplicationState::load(runtime.root_view_storage_context())
);
}
}
Integration Tests
Integration tests are usually written separately from the application's source
code (i.e., inside a tests
directory that's beside the src
directory).
Integration tests use the helper types from linera_sdk::test
to set up a
simulated Linera network, and publish blocks to microchains in order to execute
the application.
Example
A simple test that sends a message between application instances on different chains is shown below.
#[tokio::test]
async fn test_cross_chain_message() {
let parameters = vec![];
let instantiation_argument = vec![];
let (validator, application_id) =
TestValidator::with_current_application(parameters, instantiation_argument).await;
let mut sender_chain = validator.get_chain(application_id.creation.chain_id).await;
let mut receiver_chain = validator.new_chain().await;
sender_chain
.add_block(|block| {
block.with_operation(
application_id,
Operation::SendMessageTo(receiver_chain.id()),
)
})
.await;
receiver_chain.handle_received_messages().await;
assert_eq!(
receiver_chain
.query::<ChainId>(application_id, Query::LastSender)
.await,
sender_chain.id(),
);
}
Advanced Topics
In this section, we present additional topics related to the Linera protocol.
Contract Finalization
When a transaction finishes executing successfully, there's a final step where
all loaded application contracts have their Contract::store
implementation
called. This can be seen to be similar to executing a destructor. In that sense,
applications may want to perform some final operations after execution finished.
While finalizing, contracts may send messages, read and write to the state, but
are not allowed to call other applications, because they are all also in the
process of finalizing.
While finalizing, contracts can force the transaction to fail by panicking. The
block is then rejected, even if the entire transaction's operation had succeeded
before the application's Contract::store
was called. This allows a contract to
reject transactions if other applications don't follow any required constraints
it establishes after it responds to a cross-application call.
As an example, a contract that executes a cross-application call with
Operation::StartSession
may require the same caller to perform another
cross-application call with Operation::EndSession
before the transaction ends.
pub struct MyContract {
state: MyState;
runtime: ContractRuntime<Self>;
active_sessions: HashSet<ApplicationId>;
}
impl Contract for MyContract {
type Message = ();
type InstantiationArgument = ();
type Parameters = ();
async fn load(runtime: ContractRuntime<Self>) -> Self {
let state = MyState::load(runtime.root_view_storage_context())
.await
.expect("Failed to load state");
MyContract {
state,
runtime,
active_sessions: HashSet::new(),
}
}
async fn instantiate(&mut self, (): Self::InstantiationArgument) {}
async fn execute_operation(&mut self, operation: Self::Operation) -> Self::Response {
let caller = self.runtime
.authenticated_caller_id()
.expect("Missing caller ID");
match operation {
Operation::StartSession => {
assert!(
self.active_sessions.insert(caller_id),
"Can't start more than one session for the same caller"
);
}
Operation::EndSession => {
assert!(
self.active_sessions.remove(&caller_id),
"Session was not started"
);
}
}
}
async fn execute_message(&mut self, message: Self::Message) -> Result<(), Self::Error> {
unreachable!("This example doesn't support messages");
}
async fn store(&mut self) {
assert!(
self.active_sessions.is_empty(),
"Some sessions have not ended"
);
self.state.save().await.expect("Failed to save state");
}
}
Validators
Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.
In Linera, every chain is backed by the same set of validators and has the same level of security.
The main function of validators is to guarantee the integrity of the infrastructure in the sense that:
-
Each block is valid, i.e. it has the correct format, its operations are allowed, the received messages are in the correct order, and e.g. the balance was correctly computed.
-
Every message received by one chain was actually sent by another chain.
-
If one block on a particular height is certified, no other block on the same height is.
These properties are guaranteed to hold as long as two third of the validators (weighted by their stake) follow the protocol. In the future, deviating from the protocol may cause a validator to be considered malicious and to lose their stake.
Validators also play a role in the liveness of the system by making sure that the history of the chains stays available. However, since validators do not propose blocks on most chains (see next section), they do not guarantee that any particular operation or message will eventually be executed on a chain. Instead, chain owners decide whether and when to propose new blocks, and which operations and messages to include. The current implementation of the Linera client automatically includes all incoming messages in new blocks. The operations are the actions the chain owner explicitly adds, e.g. transfer.
Architecture of a validator
Since every chain uses the same validators, adding more chains does not require adding validators. Instead, it requires each individual validator to scale out by adding more computation units, also known as "workers" or "physical shards".
In the end, a Linera validator resembles a Web2 service made of
-
a load balancer (aka. ingress/egress), currently implemented by the binary
linera-proxy
, -
a number of workers, currently implemented by the binary
linera-server
, -
a shared database, currently implemented by the abstract interface
linera-storage
.
Example of Linera network
│ │
│ │
┌───────────────────┼───────────────────┐ ┌───────────────────┼───────────────────┐
│ validator 1 │ │ │ validator N │ │
│ ┌─────┴─────┐ │ │ ┌─────┴─────┐ │
│ │ load │ │ │ │ load │ │
│ ┌─────┤ balancer ├────┐ │ │ ┌─────┤ balancer ├──────┐ │
│ │ └───────────┘ │ │ │ │ └─────┬─────┘ │ │
│ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │
│ ┌────┴─────┐ ┌────┴─────┐ │ │ ┌────┴───┐ ┌────┴────┐ ┌────┴───┐ │
│ │ worker ├───────────┤ worker │ │ ... │ │ worker ├──┤ worker ├──┤ worker │ │
│ │ 1 │ │ 2 │ │ │ │ 1 │ │ 2 │ │ 3 │ │
│ └────┬─────┘ └────┬─────┘ │ │ └────┬───┘ └────┬────┘ └────┬───┘ │
│ │ │ │ │ │ │ │ │
│ │ │ │ │ │ │ │ │
│ │ ┌───────────┐ │ │ │ │ ┌─────┴─────┐ │ │
│ └─────┤ shared ├────┘ │ │ └─────┤ shared ├──────┘ │
│ │ database │ │ │ │ database │ │
│ └───────────┘ │ │ └───────────┘ │
└───────────────────────────────────────┘ └───────────────────────────────────────┘
Inside a validator, components communicate using the internal network of the validator. Notably, workers use direct Remote Procedure Calls (RPCs) with each other to deliver cross-chain messages.
Note that the number of workers may vary for each validator. Both the load balancer and the shared database are represented as a single entity but are meant to scale out in production.
For local testing during development, we currently use a single worker and RocksDB as a database.
Creating New Blocks
In Linera, the responsibility of proposing blocks is separate from the task of validating blocks.
While all chains are validated in the same way, the Linera protocol defines several types of chains, depending on how new blocks are produced.
-
The simplest and lowest-latency type of chain is called single-owner chain.
-
Other types of Linera chains not currently supported in the SDK include permissioned chains and public chains (see the whitepaper for more context).
For most types of chains (all but public chains), Linera validators do not need to exchange messages with each other.
Instead, the wallets (aka. linera
clients) of chain owners make the system
progress by proposing blocks and actively providing any additional required data
to the validators. For instance, client commands such as transfer
,
publish-bytecode
, or open-chain
perform multiple steps to append a block
containing the token transfer, application publishing, or chain creation
operation:
-
The Linera client creates a new block containing the desired operation and new incoming messages, if there are any. It also contains the most recent block's hash to designate its parent. The client sends the new block to all validators.
-
The validators validate the block, i.e. check that the block satisfies the conditions listed above, and send a cryptographic signature to the client, indicating that they vote to append the new block. But only if they have not voted for a different block on the same height earlier!
-
The client ideally receives a vote from every validator, but only a quorum of votes (say, two thirds) are required: These constitute a "certificate", proving that the block was confirmed. The client sends the certificate to every validator.
-
The validators "execute" the block: They update their own view of the most recent state of the chain by applying all messages and operations, and if it generated any cross-chain messages, they send these to the appropriate workers.
To guarantee that each incoming message in a block was actually sent by another chain, a validator will, in the second step, only vote for a block if it has already executed the block that sent it. However, when receiving a valid certificate for a block that receives a message it has not seen yet, it will accept and execute the block anyway. The certificate is proof that most other validators have seen the message, so it must be correct.
In the case of single-owner chains, clients must be carefully implemented so that they never propose multiple blocks at the same height. Otherwise, the chain may be stuck: once each of the two conflicting blocks has been signed by enough validators, it becomes impossible to collect a quorum of votes for either block.
In the future, we anticipate that most users will use permissioned chains even if they are the only owners of their chains. Permissioned chains have two confirmation steps instead of one, but it is not possible to accidentally make a chain unextendable. They also allow users to delegate certain administrative tasks to third-parties, notably to help with epoch changes (i.e. when the validators change if reconfigured).
Applications that Handle Assets
In general, if you send tokens to a chain owned by someone else, you rely on them for asset availability: if they don't handle your messages, you don't have access to your tokens.
Fortunately, Linera provides a solution based on temporary chains: if the number of parties who want to participate is limited and known in advance, we can:
- make them all chain owners using the
linera change-ownership
command, - allow only one application's operations on the chain,
- and allow only that operation to close the chain, using
linera change-application-permissions
.
Such an application should have a designated operation or message that causes it
to close the chain: when that operation is executed, it should send back all
remaining assets, and call the runtime's close_chain
method.
Once the chain is closed, owners can still create blocks rejecting messages. That way, even assets that are in flight can be returned.
The
matching-engine
example application
does this:
async fn execute_operation(&mut self, operation: Operation) -> Self::Response {
match operation {
// ...
Operation::CloseChain => {
for order_id in self.state.orders.indices().await.unwrap() {
match self.modify_order(order_id, ModifyAmount::All).await {
Some(transfer) => self.send_to(transfer),
// Orders with amount zero may have been cleared in an earlier iteration.
None => continue,
}
}
self.runtime
.close_chain()
.expect("The application does not have permissions to close the chain.");
}
}
}
This enables doing atomic swaps using the Matching Engine: if you make a bid, you are guaranteed that at any point in time you can get back either the tokens you are offering or the tokens you bought.
Experimental Topics
In this section, we present experimental topics related to the Linera protocol.
These are still in the works and subject to frequent breaking changes.
Machine Learning on Linera
The Linera application contract / service split allows for securely and efficiently running machine learning models on the edge.
The application's contract retrieves the correct model with all the correctness guarantees enforced by the consensus algorithm, while the client performs inference off-chain, in the un-metered service. Since the service is running on the user's own hardware, it can be implicitly trusted.
Guidelines
The existing examples use the candle
framework by Hugging Face as the underlying ML
framework.
candle
is a minimalist ML framework for Rust with a focus on performance and
usability. It also compiles to Wasm and has great support for Wasm both in and
outside the browser. Check candle's
examples
for inspiration on the types of models which are supported.
Getting Started
To add ML capabilities to your existing Linera project, you'll need to add the
candle-core
, getrandom
, rand
and tokenizers
dependencies to your Linera
project:
candle-core = "0.4.1"
getrandom = { version = "0.2.12", default-features = false, features = ["custom"] }
rand = "0.8.5"
Optionally, to run Large Language Models, you'll also need the
candle-transformers
and transformers
crate:
candle-transformers = "0.4.1"
tokenizers = { git = "https://github.com/christos-h/tokenizers", default-features = false, features = ["unstable_wasm"] }
Providing Randomness
ML frameworks use random numbers to perform inference. Linera services run in a
Wasm VM which do not have access to the OS Rng. For this reason, we need to
manually seed RNG used by candle
. We do this by writing a custom getrandom
.
Create a file under src/random.rs
and add the following:
use std::sync::{Mutex, OnceLock};
use rand::{rngs::StdRng, Rng, SeedableRng};
static RNG: OnceLock<Mutex<StdRng>> = OnceLock::new();
fn custom_getrandom(buf: &mut [u8]) -> Result<(), getrandom::Error> {
let seed = [0u8; 32];
RNG.get_or_init(|| Mutex::new(StdRng::from_seed(seed)))
.lock()
.expect("failed to get RNG lock")
.fill(buf);
Ok(())
}
getrandom::register_custom_getrandom!(custom_getrandom);
This will enable candle
and any other crates which rely on getrandom
access
to a deterministic RNG. If deterministic behaviour is not desired, the System
API can be used to seed the RNG from a timestamp.
Loading the model into the Service
Models cannot currently be saved on-chain; for more information see the
Limitations
below.
To perform model inference, the model must be loaded into the service. To do
this we'll use the fetch_url
API when a query is made against the service:
impl Service for MyService {
async fn handle_query(&self, request: Request) -> Response {
// do some stuff here
let raw_weights = self.runtime.fetch_url("https://my-model-provider.com/model.bin");
// do more stuff here
}
}
This can be served from a local webserver or pulled directly from a model provider such as Hugging Face.
At this point we have the raw bytes which correspond to the models and
tokenizer. candle
supports multiple formats for storing model weights, both
quantized and not (gguf
, ggml
, safetensors
, etc.).
Depending on the model format that you're using, candle
exposes convenience
functions to convert the bytes into a typed struct
which can then be used to
perform inference. Below is an example for a non-quantized Llama 2 model:
fn load_llama_model(cursor: &mut Cursor<Vec<u8>>) -> Result<(Llama, Cache), candle_core::Error> {
let config = llama2_c::Config::from_reader(cursor)?;
let weights =
llama2_c_weights::TransformerWeights::from_reader(cursor, &config, &Device::Cpu)?;
let vb = weights.var_builder(&config, &Device::Cpu)?;
let cache = llama2_c::Cache::new(true, &config, vb.pp("rot"))?;
let llama = Llama::load(vb, config.clone())?;
Ok((llama, cache))
}
Inference
Performing inference using candle
is not a 'one-size-fits-all' process.
Different models require different logic to perform inference so the specifics
of how to perform inference are beyond the scope of this document.
Luckily, there are multiple examples which can be used as guidelines on how to perform inference in Wasm:
Limitations
Hardware Acceleration
Although SIMD instructions are supported by the service runtime, general purpose GPU hardware acceleration is currently not supported. Therefore, performance in local model inference degraded for larger models.
On-Chain Models
Due to block-size constraints, models need to be stored off-chain until the introduction of the Blob API. The Blob API will enable large binary blobs to be stored on-chain, the correctness and availability of which is guaranteed by the validators.
Maximum Model Size
The maximum size of a model which can be loaded into an application's service is currently constrained by:
- The addressable memory of the service's Wasm runtime being 4 GiB.
- Not being able to load models directly to the GPU.
It is recommended that smaller models (50 Mb - 100 Mb) are used at current state of development.
Node Operator Manual
This section of the Linera Manual is meant for operators running Linera validator nodes.
Devnets
This section discusses how to deploy developer networks, aka "Devnets", for testing and development purposes.
Devnets always start from a genesis configuration and an empty state. Validator nodes are usually operated in a single infrastructure for simplicity. Devnets do not handle real assets.
Running devnets with Docker Compose
In this section, we use Docker Compose to run a simple devnet with a single validator.
Docker Compose is a tool for defining and managing multi-container Docker applications. It allows you to describe the services, networks, and volumes of your application in a single YAML file (docker-compose.yml). With Docker Compose, you can easily start, stop, and manage all the containers in your application as a single unit using simple commands like docker-compose up and docker-compose down.
For a more complete setup, consider using Kind as described in the next section.
Installation
This section covers everything you need to install to run a Linera network with Docker Compose.
Note: This section was tested only on Linux.
Docker Compose Requirements
To install Docker Compose see the installing Docker Compose section in the Docker docs.
Installing the Linera Toolchain
To install the Linera Toolchain refer to the installation section.
You want to install the toolchain from GitHub, as you'll be using the repository to run the Docker Compose validator service.
Running with Docker Compose
To run a local devnet with Docker Compose, navigate to the root of the
linera-protocol
repository and run:
cd docker && ./compose.sh
This will take some time as Docker images are built from the Linera source. When
the service is ready, a temporary wallet and database is available under the
docker
subdirectory.
Referencing these variables with the linera
binary will enable you to interact
with the devnet:
$ linera --wallet wallet.json --storage rocksdb:linera.db sync
2024-06-07T14:19:32.751359Z INFO linera: Synchronizing chain information
2024-06-07T14:19:32.771842Z INFO linera::client_context: Saved user chain states
2024-06-07T14:19:32.771850Z INFO linera: Synchronized chain information in 20 ms
$ linera --wallet wallet.json --storage rocksdb:linera.db query-balance
2024-06-07T14:19:36.958149Z INFO linera: Evaluating the local balance of e476187f6ddfeb9d588c7b45d3df334d5501d6499b3f9ad5595cae86cce16a65 by staging execution of known incoming messages
2024-06-07T14:19:36.959481Z INFO linera: Balance obtained after 1 ms
10.
The network is transient, so killing the script will perform a cleanup operation destroying wallets, storage and volumes associated with the network.
Running devnets with kind
In this section, we use kind
to run a full devnet (network of validators)
locally.
Kind (Kubernetes in Docker) is a tool for running local Kubernetes clusters using Docker container nodes. Kind uses Docker to create a cluster of containers that simulate the Kubernetes control plane and worker nodes, allowing developers to easily create, manage, and test multi-node clusters on their local machine.
Installation
This section covers everything you need to install to run a Linera network with
kind.
Linera Toolchain Requirements
The operating systems currently supported by the Linera toolchain can be summarized as follows:
Linux x86 64-bit | Mac OS (M1 / M2) | Mac OS (x86) | Windows |
---|---|---|---|
✓ Main platform | ✓ Working | ✓ Working | Untested |
The main prerequisites to install the Linera toolchain are Rust, Wasm, and Protoc. They can be installed as follows on Linux:
-
Rust and Wasm
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add wasm32-unknown-unknown
-
Protoc
curl -LO https://github.com/protocolbuffers/protobuf/releases/download/v21.11/protoc-21.11-linux-x86_64.zip
unzip protoc-21.11-linux-x86_64.zip -d $HOME/.local
- If
~/.local
is not in your path, add it:export PATH="$PATH:$HOME/.local/bin"
-
On certain Linux distributions, you may have to install development packages such as
g++
,libclang-dev
andlibssl-dev
.
For MacOS support see the installation section on GitHub.
This manual was tested with the following Rust toolchain:
[toolchain]
channel = "1.81.0"
components = [ "clippy", "rustfmt", "rust-src" ]
targets = [ "wasm32-unknown-unknown" ]
profile = "minimal"
Local Kubernetes Requirements
To run kind
locally, you also need the following dependencies:
Installing the Linera Toolchain
To install the Linera
toolchain, download the Linera source from
GitHub:
git clone https://github.com/linera-io/linera-protocol.git
cd linera-protocol
git checkout -t origin/testnet_archimedes # Current release branch
and to install the Linera toolchain:
cargo install --locked --path linera-service --features kubernetes
Running with kind
To run a local devnet with kind
, navigate to the root of the linera-protocol
repository and run:
linera net up --kubernetes
This will take some time as Docker images are built from the Linera source. When the cluster is ready, some text is written to the process output containing the exports required to configure your wallet for the devnet - something like:
export LINERA_WALLET="/tmp/.tmpIOelqk/wallet_0.json"
export LINERA_STORAGE="rocksdb:/tmp/.tmpIOelqk/client_0.db"
Exporting these variables in a new terminal will enable you to interact with the devnet:
$ linera sync-balance
2024-05-21T22:30:12.061199Z INFO linera: Synchronizing chain information and querying the local balance
2024-05-21T22:30:12.061218Z WARN linera: This command is deprecated. Use `linera sync && linera query-balance` instead.
2024-05-21T22:30:12.065787Z INFO linera::client_context: Saved user chain states
2024-05-21T22:30:12.065792Z INFO linera: Operation confirmed after 4 ms
1000000.
Testnets
This section discusses how to deploy a validator node and join an existing Testnet.
In a Testnet, the validator nodes are run by different operators. Testnets will gain in stability and decentralization over time in preparation of the mainnet launch. Testnets do not handle real assets.
In the initial Testnets of Linera, the set of validator nodes will be managed by the Linera Core team.
Joining an Existing Testnet
In this section, we use Docker Compose to run a validator and join an existing Testnet.
Infrastructure Requirements
Validators run via Docker Compose do not come with a pre-packaged load balancer to perform TLS termination (unlike validators running on Kubernetes).
The load balancer configuration must have the following properties:
- Support HTTP/2 connections.
- Support gRPC connections.
- Support long-lived HTTP/2 connections.
- Support a maximum body size of up to 20 MB.
- Provide TLS termination with a certificate signed by a known CA.
Finally, the load balancer that performs TLS termination must redirect traffic
from 443
to 19100
(the port exposed by the proxy).
Using Nginx
Minimum supported version: 1.18.0.
Below is an example Nginx configuration which upholds the infrastructure
requirements found in /etc/nginx/sites-available/default
:
server {
listen 80 http2;
location / {
grpc_pass grpc://127.0.0.1:19100;
}
}
server {
listen 443 ssl http2;
server_name <hostname>; # e.g. my-subdomain.my-domain.net
# SSL certificates
ssl_certificate <ssl-cert-path>; # e.g. /etc/letsencrypt/live/my-subdomain.my-domain.net/fullchain.pem
ssl_certificate_key <ssl-key-path>; # e.g. /etc/letsencrypt/live/my-subdomain.my-domain.net/privkey.pem;
# Proxy traffic to the service running on port 19100.
location / {
grpc_pass grpc://127.0.0.1:19100;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
keepalive_timeout 10m 60s;
grpc_read_timeout 10m;
grpc_send_timeout 10m;
client_header_timeout 10m;
client_body_timeout 10m;
}
Using Caddy
Minimum supported version: v2.4.3
Below is an example Caddy configuration which upholds the infrastructure
requirements found in /etc/caddy/Caddyfile
:
example.com {
reverse_proxy localhost:19100 {
transport http {
versions h2c
read_timeout 10m
write_timeout 10m
}
}
}
ScyllaDB Configuration
ScyllaDB is an open-source distributed NoSQL database built for high-performance and low-latency. Linera validators use ScyllaDB as their persistent storage.
ScyllaDB may require kernel parameters to be modified in order to work. Specifically the number of events allowed in asynchronous I/O contexts.
To set this run:
echo 1048576 > /proc/sys/fs/aio-max-nr
One-Click Deploy
Note: This section was only tested under Linux.
After downloading the linera-protocol
repository, and checking out the testnet
branch testnet_archimedes
you can run
scripts/deploy-validator.sh <hostname>
to deploy a Linera validator.
For example:
$ git fetch origin
$ git checkout -t origin/testnet_archimedes
$ scripts/deploy-validator.sh linera.mydomain.com
The public key will be printed after the command has finished executing, for example:
$ scripts/deploy-validator.sh linera.mydomain.com
...
Public Key: 92f934525762a9ed99fcc3e3d3e35a825235dae133f2682b78fe22a742bac196
The public key, in this case beginning with 92f
, must be communicated to the
Linera Protocol core team along with the chosen host name for onboarding in the
next epoch.
For a more bespoke deployment, refer to the manual installation instructions below.
Note: If you have previously deployed a validator you may need to remove old docker volumes (
docker_linera-scylla-data
anddocker_linera-shared
).
Verifying installation
To verify the installation, you can use the query-validator
command. For
example:
$ linera wallet init --with-new-chain --faucet https://faucet.testnet-archimedes.linera.net
$ linera query-validator grpcs:my-domain.com:443
RPC API hash: kd/Ru73B4ZZjXYkFqqSzoWzqpWi+NX+8IJLXOODjSko
GraphQL API hash: eZqzuBlLT0bcoQUjOCPf2j22NfZUWG95id4pdlUmhgs
WIT API hash: 4/gsw8G+47OUoEWK6hJRGt9R69RanU/OidmX7OKhqfk
Source code: https://github.com/linera-io/linera-protocol/tree/
0cd20d06af5262540535347d4cc6e5952a921d1a6a7f6dd0982159c9311cfb3e
The last line is the hash of the network's genesis configuration.
Manual Installation
This section covers everything you need to install to run a Linera validator node with Docker Compose.
Note: This section was only tested under Linux.
Docker Compose Requirements
To install Docker Compose see the installing Docker Compose section in the Docker docs.
Installing the Linera Toolchain
When installing the Linera Toolchain, you must check out the
testnet_archimedes
branch.
To install the Linera Toolchain refer to the installation section.
You want to install the toolchain from GitHub, as you'll be using the repository to run the Docker Compose validator service.
Setting up a Linera Validator
For the next section, we'll be working out of the docker
subdirectory in the
linera-protocol
repository.
Creating your Validator Configuration
Validators are configured using a TOML file. You can use the following template to set up your own validator configuration:
server_config_path = "server.json"
host = "<your-host>" # e.g. my-subdomain.my-domain.net
port = 19100
metrics_host = "proxy"
metrics_port = 21100
internal_host = "proxy"
internal_port = 20100
[external_protocol]
Grpc = "ClearText" # Depending on your load balancer you may need "Tls" here.
[internal_protocol]
Grpc = "ClearText"
[[shards]]
host = "shard"
port = 19100
metrics_host = "shard"
metrics_port = 21100
Genesis Configuration
The genesis configuration describes the committee of validators and chains at the point of network creation. It is required for validators to function.
Initially, the genesis configuration for each Testnet will be found in a public bucket managed by the Linera Protocol core team.
An example can be found here:
wget "https://storage.googleapis.com/linera-io-dev-public/testnet-archimedes/genesis.json"
Creating Your Keys
Now that the validator configuration has been created and the genesis configuration is available, the validator private keys can be generated.
To generate the private keys, the linera-server
binary is used:
linera-server generate --validators /path/to/validator/configuration.toml
This will generate a file called server.json
with the information required for
a validator to operate, including a cryptographic keypair.
The public key will be printed after the command has finished executing, for example:
$ linera-server generate --validators /path/to/validator/configuration.toml
2024-07-01T16:51:32.881255Z INFO linera_version::version_info: Linera protocol: v0.12.0
2024-07-01T16:51:32.881273Z INFO linera_version::version_info: RPC API hash: p//G+L8e12ZRwUdWoGHWYvWA/03kO0n6gtgKS4D4Q0o
2024-07-01T16:51:32.881274Z INFO linera_version::version_info: GraphQL API hash: KcS5z1lEg+L9QjcP99l5vNSc7LfCwnwEsfDvMZGJ/PM
2024-07-01T16:51:32.881277Z INFO linera_version::version_info: WIT API hash: p//G+L8e12ZRwUdWoGHWYvWA/03kO0n6gtgKS4D4Q0o
2024-07-01T16:51:32.881279Z INFO linera_version::version_info: Source code: https://github.com/linera-io/linera-protocol/tree/44b3e1ab15 (dirty)
2024-07-01T16:51:32.881519Z INFO linera_server: Wrote server config server.json
92f934525762a9ed99fcc3e3d3e35a825235dae133f2682b78fe22a742bac196 # <- Public Key
The public key, in this case beginning with 92f
, must be communicated to the
Linera Protocol core team along with the chosen host name for onboarding in the
next epoch.
Note: Before being included in the next epoch, validator nodes will receive no traffic from existing users.
Building the Linera Docker image
To build the Linera Docker image, run the following command from the root of the
linera-protocol
repository:
docker build --build-arg git_commit="$(git rev-parse --short HEAD)" -f docker/Dockerfile . -t linera
This can take several minutes.
Running a Validator Node
Now that the genesis configuration is available at docker/genesis.json
and the
server configuration is available at docker/server.json
, the validator can be
started by running from inside the docker
directory:
cd docker && docker compose up -d
This will run the Docker Compose deployment in a detached mode. It can take a few minutes for the ScyllaDB image to be downloaded and started.
Appendix
Congrats on reading this manual to the end! In this section, we will be sharing additional material.
Glossary
-
Address: A unique public alphanumeric identifier used to designate the identity of an entity on the Linera network.
-
Admin Chain: The Linera Network has one designated admin chain where validators can join or leave and where new epochs are defined.
-
Application: Similar to a smart-contract on Ethereum, an application is code deployed on the Linera network which is executed by all validators. An application has a metered contract which executes 'business logic' and modifies state and an unmetered 'service' which is a read-only view into an application's state.
-
Byzantine Fault-Tolerant (BFT): A system which can operate correctly and achieve consensus even if components of the system fail or act maliciously.
-
Block Height: The number of blocks preceding a given block on a specific microchain.
-
Block Proposal: A candidate block proposed by a chain owner which may be selected at the next block height.
-
Bytecode: A collection of bytes corresponding to a program that can be run by the Wasm virtual machine.
-
Client: The
linera
program, which is a local node and wallet operated by users to make requests to the network. In Linera, clients drive the network by proposing new blocks and validators are mostly reactive. -
Certificate: A value with signatures from a quorum of validators. Values can be confirmed blocks, meaning that the block has been added to the chain and is final. There are other values that are used for reaching consensus, before certifying a confirmed block.
-
Committee: The set of all validators for a particular epoch, together with their voting weights.
-
Chain Owner: The owner of a user chain or permissioned chain. This is represented as the alphanumeric identifier derived from the hash of the owner's public key.
-
Channel: A broadcast mechanism enabling publish-subscribe behavior across chains.
-
Contract: The metered part of an application which executes business logic and can modify the application's state.
-
Cross-Application Call: A call from one application to another on the same chain.
-
Cross-Chain Message: A message containing a data payload which is sent from one chain to another. Cross-Chain messages are the asynchronous communication primitive which enable communication on the same application running on different chains.
-
Devnet: An experimental deployment of the Linera protocol meant for testing and development. In a Devnet, the validator nodes are often run by the same operator for simplicity. Devnets may be shut down and restarted from a genesis configuration any time. Devnets do not handle real assets.
-
Epoch: A period of time when a particular set of validators with particular voting weights can certify new blocks. Since each chain has to transition explicitly from one epoch to the next, epochs can overlap.
-
Genesis Configuration: The configuration determining the state of a newly created network; the voting weights of the initial set of validators, the initial fee structure, and initial chains that the network starts with.
-
Inbox: A commutative data structure storing incoming messages for a given chain.
-
Mainnet: A deployment meant to be used in production, with real assets.
-
Message: See 'Cross-Chain Message'.
-
Microchain: A lightweight chain of blocks holding a subset of the network's state running on every validator. This is used interchangeably with 'chain'. All Linera chains are microchains.
-
Network: The totality of all protocol participants. A network is the combination of committee, clients and auditors.
-
Operation: Operations are either transactions directly added to a block by the creator (and signer) of the block, or calls to an application from another. Users typically use operations to start interacting with an application on their own chain.
-
Permissioned Chain: A microchain which is owned by more than one user. Users take turns proposing blocks and the likelihood of selection is proportional to their weight.
-
Project: The collection of files and dependencies which are built into the bytecode which is instantiated as an application on the Linera Network.
-
Public Chain: A microchain with full BFT consensus with a strict set of permissions relied on for the operation of the network.
-
Quorum: A set of validators representing > ⅔ of the total stake. A quorum is required to create a certificate.
-
Single-Owner Chain: See 'User Chain'.
-
Service: An unmetered read-only view into an application's state.
-
Shard: A logical subset of all microchains on a given validator. This corresponds directly to a physical worker.
-
Stake: An amount of tokens pledged by a validator or auditor, as a collateral to guarantee their honest and correct participation in the network.
-
Testnet: A deployment of the Linera protocol meant for testing and development. In a Testnet, the validator nodes are operated by multiple operators. Testnets will gain in stability and decentralization over time in preparation of the mainnet launch. Testnets do not handle real assets.
-
User Chain: Used interchangeably with Single-Owner Chain. User chains are chains which are owned by a single user on the network. Only the chain owner can propose blocks, and therefore only the chain owner can forcibly advance the state of a user chain.
-
Validator: Validators run the servers that allow users to download and create blocks. They validate, execute and cryptographically certify the blocks of all the chains.
-
View: Views are like an Object-Relational Mapping (ORM) for mapping complex types onto key-value stores. Views group complex state changes into a set of elementary operations and commit them atomically. They are full or partial in-memory representations of complex types saved on disk in a key-value store
-
Wallet: A file containing a user's public and private keys along with configuration and information regarding the chains they own.
-
WebAssembly (Wasm): A binary compilation target and instruction format that runs on a stack-based VM. Linera applications are compiled to Wasm and run on Wasm VMs inside validators and clients.
-
Web3: A natural evolution of the internet focusing on decentralization by leveraging blockchains and smart contracts.
-
Worker: A process which runs a subset of all microchains on a given validator. This corresponds directly to a logical shard.
Videos
We're constantly improving the SDK and creating new material. Below are some of our recent tutorial and presentation videos:
Rust on Linera: Spring 2024 Hackathon Kick-Off
Exploring the Linera protocol with our founder, Mathieu Baudet, followed by an overview of the Rust on Linera hackathon specifics and resources.
Rust on Linera: Spring 2024 Hackathon Coding Workshop #1
Building and deploying your first Linera application using the Linera devnet with one of our engineers, Christos Hadjiaslanis.
Rust on Linera: Spring 2024 Hackathon Coding Workshop #2
Making your Linera application robust by adding testing, logging, and more.
Rust on Linera: Spring 2024 Hackathon Coding Workshop #3
Building a meta-fungible token application on Linera.
Secure Reactive Web3 for Everyone
Discussing how user wallets track on-chain data using WebAssembly and microchains with Mathieu Baudet and Andreas Fackler from Eth Prague 2024.
Linera x Movement Labs Integrated Day Party Panel Discussion
Discussing the importance of blockchain and Web3 technologies, challenges and opportunities in the industry, and the significance of security and user empowerment in the new web ecosystem from Consensus 2024.
For the latest videos and highlights, check out our YouTube channel and Twitter Media.
Happy viewing!