Skip to main content
Servers Are Up
Virtual Datacenter
PricingFree Tools
Sign inBrowse catalog
Servers Are Up
Virtual Datacenter
PricingFree Tools
Sign inBrowse catalog
Servers Are Up

Your apps, automation and data on hardware we own. Build a private stack with predictable capacity and access through your dashboard, API or SSH.

SLA with service creditsOwned hardwareUS-based

Make room for your next idea

Explore the capacity, apps and access options that fit your workload. Check current availability and your total before you buy.

Sign up freeView Pricing

Products

  • Linux VPS
  • Windows VPS
  • Cloud & SaaS Services
  • Cloud Storage (Azure)
  • Virtual Datacenter
  • AI & Agent Hosting
  • Containers
  • Instant Apps
  • Software Platforms
  • Network builder
  • Game Servers
  • Pricing

Resources

  • Free Network Tools
  • 2026 Pricing Benchmark
  • Live SVG Badges
  • Latency & Speed Test
  • Guides & Tutorials
  • Documentation
  • API Reference
  • MCP Server
  • CLI & SDKs
  • OpenAPI & llms.txt
  • Use Cases
  • Free Migration
  • Uptime & Reliability
  • Datacenter
  • System Status
  • Network
  • Provider Comparisons Hub
  • Compare Providers

Company

  • About Us
  • Enterprise
  • Security Architecture
  • Trust Center
  • Blog
  • FAQ
  • Compliance
  • Startup Program
  • Open-Source Sponsorship
  • Press & Media Kit
  • Referral Program
  • Help Center
  • Contact Us
© Servers Are Up
TermsPrivacySLARefund PolicyDPAAcceptable Use
Service information
Back to Engineering BlogSystems, Linux & KVM
Servers Are Up Engineering·Published 2026-11-11·10 min read·Technical Review Verified

Automated Server Provisioning: Declarative cloud-init, OpenTofu Convergence, and Zero-Downtime Cutovers

automate server provisioning deploy servers with api automate server tasks automated server provisioning
automate server setup
server task automation

Engineering Summary & Direct Answer

Build idempotent server provisioning pipelines with cloud-init user-data, OpenTofu, Ansible, SAU REST (/api/v1) and MCP automation, and blue/green cutovers.

Instant 60s deploy · Browser SSH & WebRDP included
Deploy Linux VPS Compare Full PricingVirtual Datacenter Pools ($12–$380/mo)
On this page (6 sections)
  1. Provisioning as a Rollback-Safe State Machine
  2. Layer 1: Idempotent First-Boot Convergence with cloud-init
  3. Layer 2: Golden Images vs. OpenTofu + Ansible Convergence
  4. Layer 3: API & MCP Provisioning Scripts (/api/v1 + /api/mcp/v1)
  5. Zero-Downtime Blue/Green Cutovers and Fleet Economics
  6. Frequently Asked Questions
  7. Why does SSH connection fail or reject my key right after an instance enters RUNNING state?
  8. How do I test cloud-init changes locally without creating twenty VMs?
  9. How should database schema migrations run during an automated blue/green server cutover?
  10. Where should automated provisioning pipelines store Terraform/OpenTofu state and secrets?

Clicking through web forms and pasting ad-hoc bash snippets into SSH works for your first prototype VM and becomes an operational liability by your fifth. Manual server setup guarantees configuration drift: subtle differences in sysctl parameters, OpenSSH drop-in files, apt package versions, and nftables rules accumulate across staging and production until an routine reboot refuses to bring up networking at 02:00 UTC. Engineering-grade automated server provisioning replaces human folklore with a deterministic, idempotent state machine—covering hypervisor API allocation, first-boot cloud-init convergence, KMS secret hydration, synthetic health gating, and atomic blue/green traffic cutover.

Provisioning as a Rollback-Safe State Machine

Brittle provisioning scripts treat VM creation as a fire-and-forget HTTP POST followed by sleep 45 && ssh root@.... Under real datacenter load, package mirror latency or cloud-init key generation varies across runs; a fixed sleep either races an unconfigured sshd daemon or leaves half-provisioned zombie VMs billing silently when step four fails.

A production provisioning pipeline enforces five explicit state transitions, each guarded by machine-verifiable assertions and an automatic teardown path:

  1. Idempotent Allocation (Requested -> Allocated): Call the compute API with a unique Idempotency-Key header, pinning compute flavor, OS image, VPC network segment, and cloud-init user-data digest so network retries never create duplicate servers.
  2. Early-Boot OS Hardening (Allocated -> CloudInitConverged): Before external users log in, cloud-init creates unprivileged accounts, injects Ed25519 keys, disables password and root SSH authentication, loads default-deny nftables rules, and writes /var/lib/cloud/instance/boot-finished.
  3. Workload & Secret Convergence (CloudInitConverged -> Configured): Ansible playbooks, container pulls, or guest-API file writes place immutable release artifacts and hydrate runtime secrets from KMS into volatile RAM (tmpfs).
  4. Synthetic Readiness Verification (Configured -> Verified): Local and upstream health probes confirm systemd units are active, sockets bind expected loopback/private addresses, and schema migrations match. Any failure before Verified automatically triggers DELETE /api/v1/instances/{id}.
  5. Blue/Green Traffic Shift (Verified -> Active): Reverse proxies, virtual IPs, or load balancers shift live connections to the verified green node, drain the older blue generation, and retire it cleanly.

Layer 1: Idempotent First-Boot Convergence with cloud-init

The golden rule of cloud-init engineering is to push static OS invariants into native declarative modules (users, write_files, apt, packages) rather than stuffing a 300-line imperative shell script into runcmd. Declarative modules run early in the cloud-init boot graph—prior to ssh.service accepting public connections on Ubuntu 24.04 LTS and Debian 12—eliminating the dangerous window where a freshly booted VM exposes default sshd settings to internet scanners.

#cloud-config
# user-data.yaml — Production Ubuntu 24.04 LTS immutable base layer
package_update: true
package_upgrade: true
packages:
  - ca-certificates
  - curl
  - jq
  - nftables
  - fail2ban
  - unattended-upgrades

users:
  - default
  - name: opsadmin
    groups: [sudo, systemd-journal]
    shell: /bin/bash
    sudo: ["ALL=(ALL) NOPASSWD:ALL"]
    lock_passwd: true
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8p2ProdAutomationDeployKey2026 ci@example.com

write_files:
  - path: /etc/ssh/sshd_config.d/90-zero-password.conf
    permissions: "0600"
    owner: root:root
    content: |
      PermitRootLogin no
      PasswordAuthentication no
      KbdInteractiveAuthentication no
      PubkeyAuthentication yes
      AllowAgentForwarding no
      X11Forwarding no
      MaxAuthTries 3
      AllowUsers opsadmin

  - path: /etc/sysctl.d/90-network-hardening.conf
    permissions: "0644"
    owner: root:root
    content: |
      net.ipv4.tcp_syncookies = 1
      net.ipv4.conf.all.rp_filter = 1
      net.ipv4.conf.default.rp_filter = 1
      net.ipv4.conf.all.accept_redirects = 0
      net.ipv6.conf.all.accept_redirects = 0
      net.core.somaxconn = 4096
      vm.swappiness = 10

  - path: /etc/nftables.conf
    permissions: "0644"
    owner: root:root
    content: |
      #!/usr/sbin/nft -f
      flush ruleset
      table inet filter {
        chain input {
          type filter hook input priority 0; policy drop;
          iif "lo" accept
          ct state established,related accept
          ct state invalid drop
          ip protocol icmp accept
          ip6 nexthdr icmpv6 accept
          tcp dport { 22, 80, 443 } accept
        }
        chain forward { type filter hook forward priority 0; policy drop; }
        chain output  { type filter hook output priority 0; policy accept; }
      }

runcmd:
  - [ sysctl, --system ]
  - [ systemctl, enable, --now, nftables ]
  - [ sshd, -t ]
  - [ systemctl, reload, ssh ]
  - [ systemctl, enable, --now, unattended-upgrades ]

Always validate sshd -t before reloading ssh: if a syntax typo sneaks into a template variable, sshd -t aborts the reload cleanly and logs the exact line error to /var/log/cloud-init-output.log—which you can inspect immediately in the Servers Are Up browser console.

Layer 2: Golden Images vs. OpenTofu + Ansible Convergence

As fleets grow beyond a handful of hosts, engineering teams must balance cold-boot speed against image-pipeline maintenance overhead:

Provisioning Architecture Boot-to-Traffic Time CI Maintenance Cost Drift Exposure Ideal Production Use Case
Manual Dashboard + SSH Slow, error-prone Zero CI setup, high ops toil Extreme (snowflake drift) Throwaway debugging sandboxes only
Stock OS + cloud-init + OCI Containers Fast (base OS + docker pull) Low (versioned compose.yaml) Low when host layer stays minimal Stateless web apps, APIs, and AI workers on Linux VPS
OpenTofu + Ansible Convergence Moderate (apt + role tasks) Low (Git PR review + state lock) Very low with scheduled --check audits Databases, reverse proxies, WireGuard hubs, and hybrid fleets
Immutable Golden Images (Blue/Green) Fastest (pre-baked root disk) Higher (image rebuild on every CVE) Near zero (SSH disabled in prod) Strict compliance workloads and rapid horizontal replacement

With OpenTofu, encoding create_before_destroy = true on compute definitions ensures that changing user-data.yaml or upgrading Ubuntu LTS versions provisions and verifies the replacement instance before detaching and destroying the prior node:

# infra/main.tf — Blue/Green immutable replacement lifecycle
terraform {
  required_version = ">= 1.8.0"
}

variable "release_sha" {
  type = string
}

locals {
  cloud_init_raw    = templatefile("${path.module}/user-data.yaml.tftpl", { release_sha = var.release_sha })
  cloud_init_sha256 = sha256(local.cloud_init_raw)
}

resource "terraform_data" "edge_proxy_generation" {
  triggers_replace = [
    local.cloud_init_sha256,
    var.release_sha,
  ]

  input = {
    tier         = "linux-standard" # 4 vCPU, 8 GB RAM, 160 GB persistent storage ($18/mo)
    os_image     = "ubuntu-24.04"
    userdata_sha = local.cloud_init_sha256
  }

  lifecycle {
    create_before_destroy = true
  }
}

When converging stateful configurations on top of provisioned hosts with Ansible, design every task around three rules: never use ansible.builtin.shell without creates: or changed_when:, validate every rendered daemon config via validate: before overwriting /etc, and run nightly ansible-playbook site.yml --check --diff drift audits from CI:

# ansible/roles/caddy_edge/tasks/main.yml
- name: Install Caddy system group and unprivileged user
  ansible.builtin.user:
    name: caddy
    system: true
    shell: /usr/sbin/nologin
    create_home: true
    home: /var/lib/caddy

- name: Render validated Caddyfile atomically
  ansible.builtin.template:
    src: Caddyfile.j2
    dest: /etc/caddy/Caddyfile
    owner: root
    group: caddy
    mode: "0640"
    validate: "/usr/bin/caddy validate --config %s --adapter caddyfile"
  notify: Reload caddy service

- name: Ensure Caddy systemd unit is enabled and active
  ansible.builtin.systemd_service:
    name: caddy
    enabled: true
    state: started

Layer 3: API & MCP Provisioning Scripts (/api/v1 + /api/mcp/v1)

For SaaS platforms provisioning isolated per-customer servers, CI pipelines spinning up branch preview environments, or AI SRE agents executing runbooks, Servers Are Up exposes full programmatic control over REST (/api/v1/instances) and Streamable HTTP MCP (/api/mcp/v1: sau_write_instance_file, sau_exec_instance, sau_ssh_authorize).

Notice a major security advantage of guest-channel automation (PUT /api/v1/instances/{id}/files and POST /api/v1/instances/{id}/exec, mirrored by sau_write_instance_file and sau_exec_instance): your orchestrator can push config files, wait for cloud-init status --wait, and run local HTTP readiness checks on internal private-subnet VMs with zero public IPv4 address ($0 IPv4 cost) and zero inbound SSH ports exposed:

// scripts/blue-green-provision.mjs
import crypto from "node:crypto";

const SAU_API = "https://serversareup.com/api/v1";
const SAU_TOKEN = process.env.SAU_API_TOKEN;
if (!SAU_TOKEN) throw new Error("SAU_API_TOKEN environment variable is required");

async function sauFetch(path, options = {}) {
  const res = await fetch(`${SAU_API}${path}`, {
    ...options,
    headers: {
      Authorization: `Bearer ${SAU_TOKEN}`,
      "Content-Type": "application/json",
      ...(options.headers || {}),
    },
  });
  if (!res.ok) {
    const body = await res.text();
    throw new Error(`${options.method || "GET"} ${path} failed (${res.status}): ${body}`);
  }
  return res.status === 204 ? null : res.json();
}

export async function deployVerifiedGreenServer({ gitSha, productSlug = "linux-basic" }) {
  const idempotencyKey = `bg-${gitSha.slice(0, 12)}-${crypto.randomUUID()}`;
  let candidateId = null;

  try {
    // 1. Allocate green candidate VM deterministically
    const vm = await sauFetch("/instances", {
      method: "POST",
      headers: { "Idempotency-Key": idempotencyKey },
      body: JSON.stringify({
        name: `app-green-${gitSha.slice(0, 8)}`,
        productSlug, // e.g., linux-basic ($10/mo: 2 vCPU, 4 GB RAM, 80 GB persistent storage)
        os: "ubuntu-24.04",
      }),
    });
    candidateId = vm.id;

    // 2. Poll until hypervisor & guest control channel report RUNNING
    let ready = false;
    for (let i = 0; i < 45; i++) {
      const detail = await sauFetch(`/instances/${candidateId}`);
      if (detail.status === "RUNNING") {
        ready = true;
        break;
      }
      if (detail.status === "ERROR") throw new Error("Hypervisor reported ERROR state");
      await new Promise((r) => setTimeout(r, 4000));
    }
    if (!ready) throw new Error("Timed out waiting for RUNNING guest channel");

    // 3. Write release manifest atomically via guest file API (MCP: sau_write_instance_file)
    await sauFetch(`/instances/${candidateId}/files`, {
      method: "PUT",
      body: JSON.stringify({
        path: "/etc/app-release.json",
        mode: "0644",
        content: JSON.stringify({ gitSha, provisionedAt: new Date().toISOString() }, null, 2),
      }),
    });

    // 4. Block on cloud-init completion & local readiness endpoint (MCP: sau_exec_instance)
    const check = await sauFetch(`/instances/${candidateId}/exec`, {
      method: "POST",
      body: JSON.stringify({
        command: "cloud-init status --wait && systemctl is-active nftables && curl -fsS http://127.0.0.1:8080/readyz",
        timeoutSeconds: 180,
      }),
    });
    if (check.exitCode !== 0) {
      throw new Error(`Synthetic verification failed (exit ${check.exitCode}): ${check.stderr}`);
    }

    return { instanceId: candidateId, status: "VERIFIED_GREEN" };
  } catch (err) {
    // 5. Mandatory rollback: delete failed candidate so broken builds never leak spend
    if (candidateId) {
      await sauFetch(`/instances/${candidateId}`, { method: "DELETE" }).catch(() => {});
    }
    throw err;
  }
}

Zero-Downtime Blue/Green Cutovers and Fleet Economics

Once deployVerifiedGreenServer returns VERIFIED_GREEN, cutting traffic over without dropping active HTTP or WebSocket sessions requires updating your upstream reverse proxy (Caddy dynamic upstreams or Nginx reload) to point at the green node's private WireGuard or VPC IP, waiting 30–60 seconds for in-flight requests on the blue node to drain to zero, and only then terminating the blue server.

Choosing the right billing unit on Servers Are Up keeps automated blue/green pipelines cost-efficient:

  • Individual Servers (Linux Cloud VPS): Linux Starter ($6/mo, 2 vCPU / 2 GB RAM / 50 GB persistent storage), Linux Basic ($10/mo, 2 vCPU / 4 GB RAM / 80 GB persistent storage), Linux Standard ($18/mo, 4 vCPU / 8 GB RAM / 160 GB persistent storage), and Linux Pro ($28/mo, 8 vCPU / 16 GB RAM / 320 GB persistent storage).
  • Pooled Capacity for Frequent Blue/Green Cycling (Virtual Datacenter): Reserve a shared capacity block—Hobby Lab ($12/mo, 4 vCPU / 8 GB RAM / 100 GB / 5 instances), Developer Studio ($24/mo, 8 vCPU / 16 GB RAM / 250 GB / 15 instances), or Team Workspace ($95/mo, 24 vCPU / 64 GB RAM / 1 TB / 40 instances)—leaving 25–30% spare pool headroom so green nodes boot alongside blue nodes inside your existing flat monthly pool with zero extra subscription churn.
  • Hybrid Nodes & Secret KMS: Enroll existing bare-metal or multi-cloud hosts into the same API inventory at $3/machine/mo (/dashboard/hybrid) and pull runtime secrets dynamically from SAU Key Vault KMS (/dashboard/key-vault). For autonomous LLM-driven deployment workflows, pair this pipeline with our AI infrastructure management architecture.

For hybrid teams pairing cloud-init automation with on-premises colocation cabinets, software convergence only solves half of the availability equation—standardized physical server rack organization, structured cable labeling, and cold-aisle airflow management prevent thermal throttling and accidental patch-cord disconnects before sau-connector or PXE boot scripts ever execute.

Frequently Asked Questions

Why does SSH connection fail or reject my key right after an instance enters RUNNING state?

Guest OS boot happens in stages: the Linux kernel and network interface come online several seconds before cloud-init finishes writing ~/.ssh/authorized_keys and reloading sshd. Never poll raw TCP port 22 as your sole readiness signal; wait for cloud-init status --wait (or /var/lib/cloud/instance/boot-finished) before running post-boot SSH tasks.

How do I test cloud-init changes locally without creating twenty VMs?

Validate YAML syntax and schema structure first using cloud-init schema --config-file user-data.yaml (included in the cloud-init package on Ubuntu/Debian or inside an Ubuntu container). That catches indentation errors, deprecated module keys, and malformed write_files permissions before making a single hypervisor API call.

How should database schema migrations run during an automated blue/green server cutover?

Apply the expand-and-contract pattern: every database migration deployed before a blue/green cutover must be backward-compatible with the currently running blue code (additive nullable columns, new indexes created CONCURRENTLY, dual-read tolerances). Drop legacy columns only in a subsequent release after the old blue server has been permanently retired.

Where should automated provisioning pipelines store Terraform/OpenTofu state and secrets?

Never commit terraform.tfstate to Git, because state files frequently capture computed attributes and sensitive tokens in cleartext. Store encrypted remote state inside an off-host object container such as Managed Cloud Storage hosted via Microsoft Azure (5 GiB free tier, AES-256-GCM + Azure storage encryption) and inject API credentials at runtime from CI secret stores or SAU Key Vault KMS.

US-Owned Dell PowerEdge R940 KVM Infrastructure

Test & Run Production Workloads on Predictable KVM Compute

Linux VMs from $6/mo · 4 vCPU / 8 GB ECC RAM $18/mo · 8 vCPU / 16 GB RAM $28/mo · Pooled Virtual Datacenters from $12/mo with built-in Browser SSH & WebRDP.

Deploy Linux VPS Compare Full Pricing

Explore With AI & Cite This Page

Open a grounded prompt in your AI assistant or copy a clean Markdown, HTML, or APA reference link.

Ask ChatGPT Ask Perplexity Ask Claude
Share:HN·Reddit·X·LinkedIn
Newer Article

Cloud Cluster Architecture: Raft Quorum Math, etcd Latency, East-West MTU, and K3s Topology

Older Article

Diagnosing Laggy Remote Desktop Sessions: RDP over UDP, H.264 Encoding, MTU Blackholes, and OS Tuning

Related Diagnostic Tools & Engineering Guides

Validate protocols and subnets directly in your browser or continue reading related architecture notes.

Free Browser Tool

SSH Keygen Generator & Cloud-Init YAML Builder

Launch utility
Free Browser Tool

Free WHOIS & RDAP Domain & ASN Lookup Tool

Launch utility
Free Browser Tool

Linux, Windows & Game Server Hardening Checklist

Launch utility

Per-Server vs. Resource-Based Cloud Pricing: Bin-Packing Math and Pooled Virtual Datacenter Economics

Why fixed T-shirt instance billing strands 35-55% of cloud RAM and vCPU, and how pooled Virtual Datacenters turn multi-node infrastructure sizing into an elastic bin-packing problem.

Production Incident Escalation, Self-Service Linux Diagnostics, and the ServersAreUp Support Runbook

A deterministic incident triage runbook for ServersAreUp engineers: isolating SDN vs guest OS faults with ss, journalctl, and mtr, plus escalating via /dashboard/support.

Production Bash Scripting on Linux Servers: Strict Mode, Signal Traps, Atomic Tempfiles, and flock Concurrency

Write defensive, production-grade Bash scripts on Linux servers: set -Eeuo pipefail subtleties, IFS safety, trap cleanup handlers, atomic mktemp writes, flock concurrency locks, and shellcheck.

Secure Remote Access to Your Servers From a Browser

How to get secure remote access to a private Linux or Windows server from a browser: keys, isolation, logging and a setup you can run in one tab.

On this page

  1. Provisioning as a Rollback-Safe State Machine
  2. Layer 1: Idempotent First-Boot Convergence with cloud-init
  3. Layer 2: Golden Images vs. OpenTofu + Ansible Convergence
  4. Layer 3: API & MCP Provisioning Scripts (/api/v1 + /api/mcp/v1)
  5. Zero-Downtime Blue/Green Cutovers and Fleet Economics
  6. Frequently Asked Questions
  7. Why does SSH connection fail or reject my key right after an instance enters RUNNING state?
  8. How do I test cloud-init changes locally without creating twenty VMs?
  9. How should database schema migrations run during an automated blue/green server cutover?
  10. Where should automated provisioning pipelines store Terraform/OpenTofu state and secrets?
Self-Serve Infrastructure

Test & Run Production Workloads on Predictable KVM Compute

Linux VMs from $6/mo · 4 vCPU / 8 GB ECC RAM $18/mo · 8 vCPU / 16 GB RAM $28/mo · Pooled Virtual Datacenters from $12/mo with built-in Browser SSH & WebRDP.

Deploy Linux VPS Compare Full Pricing