Setting up Your Own Linux Server as a Beginner

ByLinna Li

Starter Guide for Setting Up A Linux Server

I’m writing this after setting my first linux server so you’re going to get the leanest setup.

I use the defaults when I can.

If you want to minimize the amount of things you have to learn as you set up your system, my guide intentionally sticks to configurations that work for people coming from MacOS or Windows (because I’m forgetful and have a hard time learning new habits).

I’m also one of those people that naturally have “Marie Kondo mindset”. So I try to remove anything that isn’t completely essential.

In this guide I’m going to give an outline of how went about doing things and call out the configuration decisions I made.

What I’m using my server for

Before we dive in I want to first give a brief overview of what I’m using my linux server for so that you know can decide if this is worth your time.

You gotta set up your server differently depending on the purpose of the server.

The main purpose of my linux server is to run AI Agents in docker containers. No AI will be running on the base operating system.

It will also acting as a Network Attached Storage (NAS) for my media files (photo, video, and audio) so that I have easy access to them to create content with.

I plan to access it through my personal MacAir laptop on my home network and while I’m outside of my house.

The server will be always ON.

Steps to Follow

If your intended purpose is similar to what I just described then let’s dive into it.

Here is the checklist of steps that I followed to get my server up and running.

Create a bootable USB with Linux OS

  • read this installation guide: https://ubuntu.com/server/docs/tutorial/basic-installation/
  • Download the server version of Ubuntu (Intel or AMD 64-bit architecture): https://ubuntu.com/download/server
  • If you’re using Windows to Rufus and download standard for windows x64
  • If you’re using MacOS, use an equivalent alternative to create the bootable USB
  • Got ubuntu server and download intel or arm 64 architecture
  • Device: USB you provided
  • Boot selection: Choose the downladed ubuntu ISO
  • Partition scheme: GPT
  • Target system UEFI (non CDM)

Find a Physical Location for Server

  • Find a place that has an outlet and ethernet cable where your physical server machine will sit (it’s going to be here for a long time)
  • Make sure this is somewhere where NO ONE is going to unplug your server
  • Plug in the bootable USB, connect the computer to a monitor, keyboard, and mouse
  • Turn your computer ON
  • Select and run the bootable drive and go through the OS installation steps
  • Select LVM (Logical Volume Manager) to allow flexible storage management (this is what I choose)
  • You can also unselect it if you want to define a specific partition
  • Choose minimize, entire storage (don’t reset if allocation looks ok)
  • I choose no ubuntu pro
  • I skipped the snaps installs
  • Install OpenSSH (this is required)
  • Choose a name for the computer (this isn’t that important, you can just name it something that describes your physical machine)
  • Choose server name (this is important! name it something that describes the use of computer)
  • Choose a username (this is important! name it something that’s easy to type)
  • Sign in using personal computer with ssh username@ipaddress and provide password
  • You’re computer is set up and ready to play!
  • You can now unplug your bootable USB, monitor, mouse and keyboard because you’re not going interact with it from now on.

SSH connection

  • Copy your ssh key to your server so that you don’t have to provide a password every time you connect by running this code:
ssh-copy-id username@your-server-ip
  • Now you can connect to you linux server by just running the command ssh username@your-server-ip
  • If you want to make it even simpler, add your server details to your ~/.ssh/config
Host server-name
HostName: your-server-ip
User: username
  • Adding the above to your ~/.ssh/config will let you connect to your server by running the command ssh server-name which is quite a bit easier than typing out the entire ip-address
  • You can directly just connect to your server with your computer’s default terminal but I recommend installing tmux and using that alongside your terminal.

If you want a quick guide on how to set up tmux, here is an article.

Remote SSH from outside home network

  • create an account with tailscale and use your Google account or another OAuth to sign in
  • install tailscale on macOS, android, ubuntu server
  • log in with google credentials on all three accounts and access should be enabled immediately
  • It’s that easy!

Install packages

Because I’m using my server for writing and running code, I installed a lot of packages that are needed for software development.

  • sudo apt update && sudo apt upgrade -y
  • sudo apt install git curl wget build-essential -y
  • sudo apt install docker.io docker-compose -y
  • sudo usermod -aG docker $USER
# ─── 1. System packages 
sudo apt-get update && sudo apt-get install -y --no-install-recommends \
    bash \
    ca-certificates \
    coreutils \
    curl \
    findutils \
    gawk \
    git \
    gnupg \
    grep \
    htop \
    iotop \
    jq \
    lsof \
    ncdu \
    net-tools \
    procps \
    ripgrep \
    rsync \
    sed \
    unzip \
    vim \
    zip
  • Install node
# ─── 2. Node.js 22.x 
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key \
    | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_22.x nodistro main" \
    | sudo tee /etc/apt/sources.list.d/nodesource.list
sudo apt-get update && sudo apt-get install -y --no-install-recommends nodejs
  • Configure GIT credentials
git config --global user.email "email"
git config --global user.name "name"
  • Install GitHub CLI
# ─── 4. GitHub CLI 
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
    | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
    | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
sudo apt-get update && sudo apt-get install -y --no-install-recommends gh

Access server directory from your personal laptop

  • SFTP mount with SSHFS (most common)
brew install sshfs
sshfs user@your-server:/ ~/mnt/myserver

Pull and start Docker Containers

  • Create a directory where your docker related files will be stored and folders that your docker container volumes can connect to
  • Now all you have to do is write your dockerfile and docker-compose.yml to start spinning up containers.
  • I say write but if you’re spinning up an open source project or someone else’s app, you’re probably just going to copy and paste from their documentation

I haven’t written an article on docker containers but once I do, you can read the article here.

That’s a Wrap

After setting up my linux server I don’t think I can go back to not having one. It’s one of those life improvements like switching from a bicycle to a car.

But to be honest, it is a technical hurdle that would have taken a lot of research and time to learn if I didn’t have AI.

I basically had to ask Claude questions every step of the way. It would feel extremely tedious if I had to look search through webs forums for all of my thousand questions.

And it would have been extremely bothersome to the person helping me if I had a person to ask questions to, but I didn’t.

My server is still evolving.

Right now I’m running docker containers, and hosting my photo, video and audio files.

I’ve attached a 2TB external storage and will attach another 1TB external storage for extra space. There’s still a lot of files to move over and I’ll be slowly doing that over the next couple of weeks, or month but it’s really not high priority so it’ll take as long as it needs to.

It turns out that I really really enjoy organizing things, both on my computer and in real life. It’s weirdly therapeutic for me.

I know, I’m not normal.

Similar Posts