ABI Knowledge Base

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

infra:overview [2026/04/08 00:23] – created overview avinfra:overview [2026/06/30 15:23] (current) – external edit 127.0.0.1
Line 58: Line 58:
  
 All HPC VMs run on **geonosis** using **bhyve**. The Slurm controller runs in a Devuan VM on **bane**. All HPC VMs run on **geonosis** using **bhyve**. The Slurm controller runs in a Devuan VM on **bane**.
 +
 +All VMs share the same physical CPU: **AMD EPYC 7702 64-Core Processor** (2 sockets, 128 cores / 256 threads on the host).
  
 ==== User-facing VMs (on geonosis) ==== ==== User-facing VMs (on geonosis) ====
  
-^ VM ^ vCPUs ^ RAM ^ OS ^ Slurm Partition ^ Purpose ^ +^ VM ^ vCPUs ^ RAM ^ OS ^ Slurm Partition ^ State ^ Purpose ^ 
-| ssh-01 | 2 | 4G | *TODO* | N/A | Login node (''ssh.abi.am'') | +| ssh-01 | 2 | 4G | *TODO* | N/A | Running | Login node (''ssh.abi.am'') | 
-| ssh-02 | 4 | 8G | *TODO* | N/A | Login node (''ssh.abi.am'') | +| ssh-02 | 4 | 4G | *TODO* | N/A | Running | Login node (''ssh.abi.am'') | 
-| thin-01 | 64 | 384G | *TODO: Devuan/Ubuntu?* | compute (default), thin | General computation | +| thin-01 | 64 | 384G | *TODO* | compute (default), thin | Running | General computation | 
-| thin-02 | 64 | 384G | *TODO* | compute (default), thin | General computation | +| thin-02 | 64 | 384G | *TODO* | compute (default), thin | Running | General computation | 
-| thick-01 | 64 | 768G | *TODO* | compute (default), thick | High-memory computation | +| thick-01 | 64 | 768G | *TODO* | compute (default), thick | Locked | High-memory computation | 
-| dl-01 | 2 | 8G | *TODO* | download | Data downloads only | +| dl-01 | 2 | 8G | *TODO* | download | Locked | Data downloads only | 
-| dl-02 | 2 | 8G | *TODO* | download | Data downloads only | +| dl-02 | 2 | 8G | *TODO* | download | Locked | Data downloads only | 
-| rshiny0 | 4 | 16G | *TODO* | N/A | R Shiny server | +| rshiny0 | 4 | 12G | *TODO* | N/A | Locked | R Shiny server | 
-| rshiny1 | 4 | 16G | *TODO* | N/A | R Shiny server |+| rshiny1 | 4 | 10G | *TODO* | N/A | Locked | R Shiny server | 
 + 
 +**HPC totals:** 210 vCPUs, ~1.5T RAM, 429T storage. 
 + 
 +==== CPU Topology (from ''lscpu'' inside VMs) ==== 
 + 
 +^ Node ^ vCPUs (''nproc'') ^ Sockets ^ Cores/socket ^ Threads/core ^ 
 +| thin-01 | 64 | 2 | 16 | 2 | 
 +| thin-02 | 64 | 2 | 16 | 2 | 
 +| thick-01 | 64 | 2 | 16 | 2 | 
 +| dl-01 | 2 | 2 | 1 | 1 | 
 +| dl-02 | 2 | 2 | 1 | 1 | 
 +| ssh-01 | 2 | 2 | 1 | 1 | 
 +| ssh-02 | 4 | *TODO* | *TODO* | *TODO* | 
 +| rshiny0 | 4 | *TODO* | *TODO* | *TODO* | 
 +| rshiny1 | 4 | *TODO* | *TODO* | *TODO* | 
 + 
 +==== Memory (from ''/proc/meminfo'' inside VMs) ==== 
 + 
 +^ Node ^ MemTotal (kB) ^ ~GiB ^ bhyve allocation ^ 
 +| thin-01 | 396,021,348 | ~377 | 384G | 
 +| thin-02 | 396,021,380 | ~377 | 384G | 
 +| thick-01 | 792,186,300 | ~755 | 768G | 
 +| dl-01 | 8,120,096 | ~7.7 | 8G | 
 +| dl-02 | 8,120,088 | ~7.7 | 8G | 
 +| ssh-01 | 3,997,476 | ~3.8 | 4G | 
 +| ssh-02 | *TODO* | *TODO* | 4G | 
 +| rshiny0 | *TODO* | *TODO* | 12G | 
 +| rshiny1 | *TODO* | *TODO* | 10G | 
 + 
 +==== Storage (ZFS pools on geonosis) ==== 
 + 
 +^ Pool ^ Used ^ Available ^ Refer ^ Mountpoint ^ 
 +| genomic | 135T | 31.4T | 369G | /mnt/genomic | 
 +| zroot | *TODO* | 1.58T | *TODO* | (system + VM images) | 
 + 
 +==== VM Disk Usage (zroot/vm/ datasets) ==== 
 + 
 +^ VM Dataset ^ Used ^ Available ^ Refer ^ 
 +| ssh-01 | 248K | 1.58T | 148K | 
 +| ssh-02 | 268K | 1.58T | 148K | 
 +| thin-01 | 256K | 1.58T | 156K | 
 +| thin-02 | 256K | 1.58T | 156K | 
 +| thick-01 | 252K | 1.58T | 152K | 
 +| dl-01 | 236K | 1.58T | 144K | 
 +| dl-02 | 228K | 1.58T | 144K | 
 +| rshiny0 | 8.77G | 1.58T | 3.88G | 
 +| rshiny1 | 8.85G | 1.58T | 3.91G | 
 + 
 +==== NFS Mounts (visible on compute nodes) ==== 
 + 
 +^ Mount ^ Source ^ Size ^ Used ^ Available ^ Purpose ^ 
 +| /mnt/nas0/proj | genomic (geonosis) | 32T | 256K | 32T | Project storage | 
 +| /mnt/devices/nextseq2000 | genomic (geonosis) | 41T | 8.5T | 32T | Sequencer data | 
 +| /mnt/nas0/user | nas1 (mustafar) | 32T | 256K | 32T | Extended user storage | 
 +| /mnt/home | nas1 (mustafar) | 32T | 384K | 32T | User home directories | 
 +| /mnt/nas1/proj | nas1 (mustafar) | 32T | 128K | 32T | Project data | 
 +| /mnt/nas1/db | nas1 (mustafar) | 33T | 1.8T | 32T | Reference genomes & databases | 
 +| /mnt/bak1/parking | bak1 (hoth) | 109T | 256K | 109T | Backup parking |
  
 ==== Infrastructure VMs ==== ==== Infrastructure VMs ====
  
 ^ VM ^ Host ^ vCPUs ^ RAM ^ OS ^ Purpose ^ ^ VM ^ Host ^ vCPUs ^ RAM ^ OS ^ Purpose ^
-| slurmctld VM | bane | *TODO* | *TODO* | Devuan | Slurm controller daemon (slurmctld) |+| slurmctld | bane | *TODO* | *TODO* | Devuan | Slurm controller daemon (slurmctld) |
  
 ==== Legacy / testing VMs (on geonosis) ==== ==== Legacy / testing VMs (on geonosis) ====
  
-These VMs are for internal IT use. *TODO: Document when needed.*+These VMs are for internal IT use.
  
-^ VM ^ vCPUs ^ RAM ^ Notes ^ +^ VM ^ vCPUs ^ RAM ^ State ^ Notes ^ 
-| comp0 | 32 | 32G | Stopped -- legacy compute node | +| comp0 | 1 | 2G | Stopped | Legacy compute node | 
-| dna0 | 8 | 32G | Stopped -- *TODO* | +| dna0 | 8 | 32G | Stopped | *TODO* | 
-| mitte-dev-01 | 2 | 4G | Stopped -- development/testing |+| mitte-dev-01 | 2 | 4G | Stopped | Development/testing |
  
 ---- ----
Line 203: Line 263:
   * [[infra:monitoring|Monitoring]] -- System health monitoring   * [[infra:monitoring|Monitoring]] -- System health monitoring
   * [[infra:automation|Automation]] -- Configuration management   * [[infra:automation|Automation]] -- Configuration management
-  * [[infra:tips_and_tricks|Admin Tips & Tricks]] -- Quick admin commands+  * [[infra:howto:start|How To's]] -- Guides for common admin and user tasks