Skip to main content

​
Introduction

HostOS is the operating system running directly on the host machine hardware. Its primary responsibility is to launch and manage the GuestOS virtual machine using QEMU/libvirt.
HostOS is intentionally limited by design to not perform any trusted Internet Computer Protocol capabilities. All IC protocol operations (replica, orchestrator, etc.) run exclusively in GuestOS.
This minimal design reduces the attack surface and ensures that the critical IC operations are isolated within the GuestOS virtual machine.

​
Core Responsibilities

HostOS has a focused set of responsibilities:

VM Management

Launch and manage the GuestOS virtual machine using QEMU/libvirt

Hardware Access

Provide controlled hardware access to the GuestOS VM

Network Bridge

Bridge networking between physical interfaces and VM

Monitoring

Export host metrics via node_exporter

Remote Access

Provide SSH access for Node Providers/Technicians

Upgrades

Manage HostOS system upgrades

​
Disk Partitioning

HostOS uses a sophisticated partitioning layout with both primary partitions and LVM (Logical Volume Manager):

​
Partition Schema

Primary Partitions          |  LVM (HostOS)                  |  LVM (GuestOS)
----------------------------|--------------------------------|------------------
EFI | Grub                  |  Config | Boot A | Root A | ... |  Config | ...
The system maintains separate A/B partitions for both HostOS and GuestOS, enabling zero-downtime upgrades.

​
Detailed Layout

  • EFI (100 MB): EFI System Partition for bootloader and UEFI data
  • Grub (100 MB): GRUB bootloader configuration
The majority of disk space is allocated to GuestOS, with only about 65 GB reserved for HostOS. This reflects HostOS’s minimal role compared to the GuestOS workload.

​
Partition Configuration

The exact partitioning layout is defined in:
  • ic/ic-os/hostos/partitions.csv
  • ic/ic-os/hostos/volumes.csv

​
Root Partition Characteristics

  • Filesystem: ext4
  • Mount Options: Read-only (ro,errors=remount-ro)
  • fstab Entry: 
    /dev/rootfs     /      ext4    ro,errors=remount-ro   0       1
The read-only root partition prevents tampering and ensures system integrity.

​
Virtual Machine Management

​
QEMU/Libvirt Configuration

HostOS uses libvirt XML format to configure virtual machines. The GuestOS VM configuration is created by code in config::guest_vm_config.
CPU Passthrough: The GuestOS VM is configured with CPU passthrough, allowing direct access to physical CPU features and capabilities. This ensures optimal performance for IC protocol operations.

​
VM Features

  • Virtualization: KVM-accelerated QEMU
  • CPU: Passthrough mode for native performance
  • Memory: Configurable allocation
  • Storage: Virtual disk backed by LVM volumes
  • Networking: Bridged networking with physical interfaces
  • VSOCK: VM socket interface for controlled host communication

​
System Users

HostOS includes several specialized user accounts in addition to standard Ubuntu users:
UsernameHome DirectoryShellDescription
backup/var/lib/backup/bin/bashBackup subnet state
readonly/var/lib/readonly/bin/bashRead-only admin access for Node Providers/Technicians
admin/var/lib/admin/bin/bashFull admin access for Node Providers/Technicians
node_exporter/home/node_exporter/usr/sbin/nologinNode exporter service account
Node Providers and Node Technicians use the readonly and admin accounts to access the node for monitoring and maintenance.

​
Firewall Configuration

HostOS implements a restrictive firewall ruleset using NFTables:
The firewall rules are hard-coded into the disk image. Updating the rules requires proposing and blessing a new disk image.
The ruleset is defined in nftables.conf.

​
Input Rules (Open Ports)

ProtocolPortSourceDescription
TCP22RFC 1918 (private networks)OpenSSH
UDP67RFC 1918 (private networks)DHCP
IPv6 access is restricted to delegated subnets registered in the IC registry, providing additional security.

​
Output Rules (Allowed Destinations)

ProtocolPortDestinationDescription
TCP53AnyDNS
UDP53AnyDNS
UDP123AnyNTP (time synchronization)
TCP80AnyHTTP (download update images)
TCP443AnyHTTPS (download update images)
Outbound connections are limited to essential services: DNS resolution, time synchronization, and downloading system updates.

​
VSOCK Interface

The VSOCK (VM Socket) interface enables controlled communication between GuestOS and HostOS while maintaining isolation.

​
Design Principles

VSOCK provides a controlled environment for GuestOS to communicate with HostOS while maintaining the highest level of isolation between the two operating systems.
All VSOCK commands are initiated from GuestOS. HostOS cannot initiate commands to GuestOS, preventing unauthorized control.
GuestOS is restricted to a limited set of predefined commands, each carefully designed to maintain security.

​
Available Commands

GuestOS can request HostOS to perform specific operations such as:
  • System upgrade coordination
  • Status queries
  • Partition switching for upgrades
  • Controlled hardware operations
For a complete list of VSOCK commands and detailed documentation, see the vsock README.

​
Networking

HostOS manages networking for both itself and the GuestOS VM:

​
Physical Interfaces

  • Direct access to physical network adapters
  • IPv4 configuration via DHCP (RFC 1918 addresses)
  • IPv6 configuration from delegated subnets

​
Virtual Networking

  • Bridge between physical interfaces and GuestOS VM
  • NAT or bridged mode depending on configuration
  • Isolated network namespaces for security

​
DNS and Time

  • DNS resolution for updates and maintenance
  • NTP time synchronization for accurate timestamps
  • Critical for distributed consensus in GuestOS

​
Monitoring and Observability

HostOS exports host metrics through Prometheus node_exporter:
  • Service: node_exporter.service
  • Port: 9100 (IPv6 only, restricted to IC subnets)
  • Metrics: Hardware health, resource utilization, system status
  • TLS: Secured with TLS certificates

​
Available Metrics

  • CPU usage and temperature
  • Memory utilization
  • Disk I/O and space
  • Network throughput and errors
  • System load and uptime

​
Upgrades

HostOS uses the A/B partition scheme for zero-downtime upgrades:

​
Upgrade Process

1

Download

HostOS downloads the new system image via HTTPS
2

Install

New image is written to the inactive partition set (A or B)
3

Verify

Image integrity and signatures are verified
4

Switch

GRUB configuration updated to boot from new partition
5

Reboot

System reboots into the newly upgraded partition
6

Rollback

If upgrade fails, system automatically rolls back to previous partition
HostOS upgrades require a reboot. GuestOS is shut down gracefully before the HostOS reboot.

​
Security Considerations

​
Minimal Attack Surface

HostOS is deliberately minimal:
  • Few installed packages
  • Limited running services
  • No IC protocol operations
  • Restrictive firewall rules

​
Access Control

  • SSH access restricted by source IP
  • IPv4: Only RFC 1918 private networks
  • IPv6: Only delegated IC subnets
  • Separate readonly and admin accounts

​
System Integrity

  • Read-only root filesystem
  • Signed boot components
  • Verified upgrades
  • A/B partitioning for safe rollback

​
Development

The HostOS system configuration is in the ic-os/components/ directory.

​
Adding Files to HostOS

  1. Add files to components/ directory
  2. Enumerate each file in components/hostos.bzl
  3. Add systemd service files if needed

​
Removing Files from HostOS

  1. Remove files from components/ directory
  2. Remove enumeration from components/hostos.bzl
  3. Remove from any other .bzl files that reference them
Some binaries are injected late in the build process after rootfs construction. See ic-os/hostos/defs.bzl for details.

​
Troubleshooting

​
Accessing HostOS

Connect via SSH using the admin or readonly account: 
ssh admin@<node-ipv6-address>

​
Checking VM Status

View GuestOS VM status: 
sudo virsh list --all
sudo virsh dominfo <vm-name>

​
Viewing Logs

HostOS uses systemd for logging: 
# View HostOS logs
journalctl -b

# View specific service logs
journalctl -u <service-name>

# View GuestOS console logs
sudo virsh console <vm-name>

​
Common Issues

Check:
  • Libvirt service status: systemctl status libvirtd
  • VM configuration: virsh dumpxml <vm-name>
  • Available resources: free -h and df -h
Check:
  • Physical interface status: ip link
  • Bridge configuration: ip addr show
  • Firewall rules: nft list ruleset
Verify:
  • Source IP is in allowed range (RFC 1918 or IC subnet)
  • SSH service is running: systemctl status sshd
  • Firewall allows connections

​
Next Steps

Build HostOS

Learn how to build HostOS images

GuestOS

Understand the guest operating system

SetupOS

Learn about initial node setup

IC-OS Overview

Return to IC-OS overview

Build docs developers (and LLMs) love

Get started for freeTalk to us