Compare commits
12 Commits
main
...
ade8579bae
| Author | SHA1 | Date | |
|---|---|---|---|
| ade8579bae | |||
| 7c63114526 | |||
| 0ad9a305e1 | |||
| 5ade272217 | |||
| 9b4bcbf4e9 | |||
| 98572612c4 | |||
| 16c99e021c | |||
| 099c05107c | |||
| 47582916e5 | |||
| 179287f19d | |||
| f44e7e7f8f | |||
| 49132b8414 |
68
doc/step-001-auto-ssh-add-baremetal-key-after-reboot.txt
Normal file
68
doc/step-001-auto-ssh-add-baremetal-key-after-reboot.txt
Normal file
@@ -0,0 +1,68 @@
|
||||
Step 001: Auto-dodawanie klucza SSH do ssh-agent po restarcie (VPS -> baremetal)
|
||||
|
||||
Cel:
|
||||
- Po restarcie VPS (bez recznego ssh-add) klucz prywatny do baremetala ma byc automatycznie dodany do ssh-agent.
|
||||
- To jest wygodne, gdy po WG chcesz robic deploy/Ansible/ssh bez interakcji.
|
||||
|
||||
Wymagania:
|
||||
- VPS z systemd (Debian/Ubuntu).
|
||||
- Klucz prywatny jest na VPS, np. ~/.ssh/mevnode_baremetal
|
||||
- Klucz NIE powinien miec passphrase (inaczej ssh-add bedzie chcial hasla i usluga oneshot padnie).
|
||||
|
||||
0) Plik klucza (prawa)
|
||||
chmod 600 ~/.ssh/mevnode_baremetal
|
||||
|
||||
1) Wlacz "linger" dla usera (zeby user services startowaly po boot bez logowania)
|
||||
sudo loginctl enable-linger user
|
||||
|
||||
2) Utworz usluge ssh-agent (systemd --user)
|
||||
mkdir -p ~/.config/systemd/user
|
||||
|
||||
cat >~/.config/systemd/user/ssh-agent.service <<'EOF'
|
||||
[Unit]
|
||||
Description=SSH agent
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
|
||||
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
EOF
|
||||
|
||||
3) Utworz usluge ssh-add dla klucza do baremetala (systemd --user)
|
||||
Uwaga: dodajemy ExecStartPre, zeby poczekac az socket agenta bedzie gotowy.
|
||||
|
||||
cat >~/.config/systemd/user/ssh-add-mevnode.service <<'EOF'
|
||||
[Unit]
|
||||
Description=ssh-add mevnode_baremetal
|
||||
After=ssh-agent.service
|
||||
Requires=ssh-agent.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
|
||||
ExecStartPre=/usr/bin/sh -c 'for i in $(seq 1 50); do [ -S "$SSH_AUTH_SOCK" ] && exit 0; sleep 0.1; done; echo "SSH_AUTH_SOCK not ready: $SSH_AUTH_SOCK" >&2; exit 1'
|
||||
ExecStart=/usr/bin/ssh-add %h/.ssh/mevnode_baremetal
|
||||
RemainAfterExit=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
EOF
|
||||
|
||||
4) Reload i wlacz autostart
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now ssh-agent.service ssh-add-mevnode.service
|
||||
|
||||
5) Sprawdz dzialanie
|
||||
systemctl --user status ssh-agent.service ssh-add-mevnode.service --no-pager
|
||||
SSH_AUTH_SOCK="$XDG_RUNTIME_DIR/ssh-agent.socket" ssh-add -l
|
||||
|
||||
6) Test SSH na baremetal (po WG)
|
||||
ssh user@10.66.66.1
|
||||
|
||||
Wskazowka:
|
||||
- Alternatywa bez ssh-agent: wpis w ~/.ssh/config z IdentityFile (nie wymaga ssh-add),
|
||||
ale ten plik opisuje wariant stricte "auto ssh-add po reboot".
|
||||
|
||||
88
doc/step-001-order-wg-then-ssh.txt
Normal file
88
doc/step-001-order-wg-then-ssh.txt
Normal file
@@ -0,0 +1,88 @@
|
||||
Step 001: Kolejnosc wdrozenia (najpierw WireGuard, potem odciecie publicznego SSH)
|
||||
|
||||
Cel: najpierw stawiamy WireGuard i testujemy polaczenie po WG, a dopiero potem ucinamy publiczne SSH.
|
||||
To minimalizuje ryzyko zablokowania sie na baremetalu.
|
||||
|
||||
0) Pre-flight (nie zamykaj obecnej sesji SSH)
|
||||
- Na baremetalu zostaw otwarta aktualna sesje SSH.
|
||||
- Przygotuj drugi terminal do testow.
|
||||
|
||||
Sprawdz publiczny interfejs/IP routing:
|
||||
ip route get 1.1.1.1
|
||||
|
||||
1) WireGuard na baremetalu (nie ruszaj SSH/firewall poza WG portem)
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y wireguard
|
||||
|
||||
Jesli uzywasz skryptu:
|
||||
sudo /usr/local/sbin/wg_serverctl.py gen-server --force
|
||||
sudo /usr/local/sbin/wg_serverctl.py gen-client client1
|
||||
sudo /usr/local/sbin/wg_serverctl.py add client1
|
||||
sudo /usr/local/sbin/wg_serverctl.py export client1
|
||||
|
||||
Uruchom WG na serwerze:
|
||||
sudo systemctl enable --now wg-quick@wg0
|
||||
sudo wg show
|
||||
ip a show wg0
|
||||
|
||||
2) WireGuard na kliencie i test polaczenia
|
||||
Na kliencie:
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y wireguard
|
||||
sudo /usr/local/sbin/wg_client.py install ./client1.conf
|
||||
sudo /usr/local/sbin/wg_client.py up
|
||||
sudo /usr/local/sbin/wg_client.py status
|
||||
|
||||
Testy (z klienta):
|
||||
ping 10.66.66.1
|
||||
ssh user@10.66.66.1
|
||||
|
||||
Dopiero jak to dziala, idziesz dalej.
|
||||
|
||||
3) (Opcjonalnie teraz) UFW: wpusc WG, ale NIE odcinaj jeszcze publicznego SSH
|
||||
Na baremetalu:
|
||||
sudo apt-get install -y ufw
|
||||
sudo ufw allow 51820/udp
|
||||
sudo ufw allow 22/tcp (tymczasowo, zeby nic nie uciac przy wlaczaniu UFW)
|
||||
sudo ufw enable
|
||||
sudo ufw status verbose
|
||||
|
||||
Uwagi:
|
||||
- Jesli `sudo ufw status verbose` pokazuje "Status: inactive", to reguly NIE dzialaja.
|
||||
- W takim wypadku wlacz UFW bezpiecznie (zostawiajac dostep przez WG):
|
||||
|
||||
sudo ufw default deny incoming
|
||||
sudo ufw default allow outgoing
|
||||
sudo ufw allow 51820/udp
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 22 proto tcp
|
||||
sudo ufw enable
|
||||
sudo ufw status verbose
|
||||
|
||||
4) Przestawienie SSH na "tylko WireGuard"
|
||||
1) Najpierw pozwol na SSH po wg0:
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
|
||||
2) Potem ogranicz sshd do WG IP:
|
||||
sudo sshd -t
|
||||
sudoedit /etc/ssh/sshd_config
|
||||
|
||||
Dodaj/ustaw:
|
||||
PasswordAuthentication no
|
||||
PermitRootLogin no
|
||||
ListenAddress 10.66.66.1
|
||||
|
||||
Restart i test:
|
||||
sudo systemctl restart ssh
|
||||
|
||||
Z klienta:
|
||||
ssh user@10.66.66.1
|
||||
|
||||
3) Jak test przeszedl: zablokuj publiczne SSH:
|
||||
sudo ufw deny 22/tcp
|
||||
sudo ufw status verbose
|
||||
|
||||
5) RPC tylko po WG (po instalacji Agave)
|
||||
Docelowo:
|
||||
- bind RPC/WS na 10.66.66.1 (wg0)
|
||||
- UFW: allow 8899/8900 tylko "in on wg0"
|
||||
25
doc/step-001-ssh-copy-key-root-to-user.txt
Normal file
25
doc/step-001-ssh-copy-key-root-to-user.txt
Normal file
@@ -0,0 +1,25 @@
|
||||
Step 001: Jak skopiowac klucze SSH na uzytkownika "user" uruchamiajac komendy jako root
|
||||
|
||||
Cel: skopiowac klucze/konfiguracje SSH z /root/.ssh do /home/user/.ssh albo tylko dodac klucz publiczny do authorized_keys.
|
||||
|
||||
Wariant A: Skopiuj caly katalog /root/.ssh do usera (ustaw prawa)
|
||||
|
||||
sudo install -d -m 700 -o user -g user /home/user/.ssh
|
||||
sudo cp -a /root/.ssh/. /home/user/.ssh/
|
||||
sudo chown -R user:user /home/user/.ssh
|
||||
sudo chmod 700 /home/user/.ssh
|
||||
sudo chmod 600 /home/user/.ssh/authorized_keys 2>/dev/null || true
|
||||
sudo chmod 600 /home/user/.ssh/id_* 2>/dev/null || true
|
||||
sudo chmod 644 /home/user/.ssh/*.pub 2>/dev/null || true
|
||||
sudo chmod 644 /home/user/.ssh/known_hosts 2>/dev/null || true
|
||||
|
||||
Uwaga: kopiowanie prywatnych kluczy (id_ed25519/id_rsa) na innego uzytkownika zwieksza ryzyko. Czasem lepiej uzyc wariantu B.
|
||||
|
||||
Wariant B: Tylko dopisz klucz publiczny roota do authorized_keys usera
|
||||
|
||||
sudo install -d -m 700 -o user -g user /home/user/.ssh
|
||||
sudo install -m 600 -o user -g user /dev/null /home/user/.ssh/authorized_keys
|
||||
sudo cat /root/.ssh/id_ed25519.pub | sudo tee -a /home/user/.ssh/authorized_keys >/dev/null
|
||||
sudo chown user:user /home/user/.ssh/authorized_keys
|
||||
sudo chmod 600 /home/user/.ssh/authorized_keys
|
||||
|
||||
26
doc/step-001-ufw-toggle-ssh-eth0.txt
Normal file
26
doc/step-001-ufw-toggle-ssh-eth0.txt
Normal file
@@ -0,0 +1,26 @@
|
||||
Step 001: Toggle SSH na publicznym interfejsie eth0 (ON/OFF) przez UFW
|
||||
|
||||
Zalozenie:
|
||||
- SSH po WireGuard (wg0) zostaje zawsze wlaczone.
|
||||
- Publiczne SSH po eth0 wlaczasz/wylaczasz tylko regula UFW.
|
||||
- UFW musi byc aktywne (status != inactive).
|
||||
|
||||
1) Zawsze zostaw SSH po wg0:
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
|
||||
2) ON: wlacz SSH po eth0 (public):
|
||||
sudo ufw allow in on eth0 to any port 22 proto tcp
|
||||
sudo ufw status verbose
|
||||
|
||||
3) OFF: wylacz SSH po eth0 (public):
|
||||
sudo ufw deny in on eth0 to any port 22 proto tcp
|
||||
sudo ufw status verbose
|
||||
|
||||
4) (Opcjonalnie) Usuniecie konkretnej reguly:
|
||||
sudo ufw status numbered
|
||||
sudo ufw delete <NUMER_REGULY_DLA_22_NA_eth0>
|
||||
|
||||
Uwaga:
|
||||
- Jesli `sudo ufw status` pokazuje "inactive", to reguly nie dzialaja. Wtedy najpierw:
|
||||
sudo ufw enable
|
||||
|
||||
37
doc/step-001-wg-export-what-next.txt
Normal file
37
doc/step-001-wg-export-what-next.txt
Normal file
@@ -0,0 +1,37 @@
|
||||
Step 001: Co zrobic z plikiem po "wg_serverctl.py export"
|
||||
|
||||
Po stronie serwera, po wykonaniu:
|
||||
sudo /etc/wireguard/wg_serverctl.py export laptop1 --endpoint 149.50.116.219:51820
|
||||
|
||||
Powstaje plik:
|
||||
/etc/wireguard/clients/laptop1.conf
|
||||
|
||||
Co dalej:
|
||||
|
||||
1) Skopiuj plik na klienta (laptop/VPS)
|
||||
Najprosciej przez scp z klienta:
|
||||
scp user@149.50.116.219:/etc/wireguard/clients/laptop1.conf .
|
||||
|
||||
2) Zainstaluj go jako /etc/wireguard/wg0.conf na kliencie
|
||||
sudo install -m 600 laptop1.conf /etc/wireguard/wg0.conf
|
||||
|
||||
3) Podnies WireGuard na kliencie i przetestuj
|
||||
sudo wg-quick up wg0
|
||||
sudo wg show
|
||||
ping 10.66.66.1
|
||||
ssh user@10.66.66.1
|
||||
|
||||
4) Autostart po restarcie (server + client)
|
||||
Domyslnie bywa, ze wg-quick@wg0 jest "disabled" (szczegolnie na kliencie).
|
||||
|
||||
Serwer (Ubuntu/Debian, systemd):
|
||||
sudo systemctl enable --now wg-quick@wg0
|
||||
sudo systemctl is-enabled wg-quick@wg0
|
||||
sudo systemctl status wg-quick@wg0 --no-pager
|
||||
|
||||
Klient (Debian/Ubuntu, systemd):
|
||||
sudo systemctl enable --now wg-quick@wg0
|
||||
sudo systemctl is-enabled wg-quick@wg0
|
||||
|
||||
Wazne:
|
||||
- Dopiero jak ping i ssh po WG dzialaja, przechodzisz do odciecia publicznego SSH na serwerze.
|
||||
116
doc/step-001-wireguard-tools.md
Normal file
116
doc/step-001-wireguard-tools.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Step 001A: WireGuard tools (server/client) + instrukcje reset
|
||||
|
||||
Ten step dodaje dwa skrypty pomocnicze do zarzadzania WireGuard:
|
||||
|
||||
- server: `scripts/wireguard/wg_serverctl.py`
|
||||
- client: `scripts/wireguard/wg_client.py`
|
||||
|
||||
Skrypty trzymaja stan na dysku, zgodnie z konwencja:
|
||||
|
||||
- server keys: `/etc/wireguard/server.key`, `/etc/wireguard/server.pub`
|
||||
- clients: `/etc/wireguard/clients/<name>.key|.pub|.ip|.conf`
|
||||
- generated server config: `/etc/wireguard/wg0.conf`
|
||||
|
||||
Wymagania: `wireguard-tools` (czyli `wg`, `wg-quick`) + `systemd`.
|
||||
|
||||
## Instalacja (server)
|
||||
|
||||
Na bare metalu:
|
||||
|
||||
```bash
|
||||
apt-get update
|
||||
apt-get install -y wireguard
|
||||
```
|
||||
|
||||
Z repo (na hoście) zainstaluj skrypt jako root-only:
|
||||
|
||||
```bash
|
||||
install -m 700 scripts/wireguard/wg_serverctl.py /usr/local/sbin/wg_serverctl.py
|
||||
```
|
||||
|
||||
## Instalacja (client)
|
||||
|
||||
Na kliencie (laptop/VPS):
|
||||
|
||||
```bash
|
||||
apt-get update
|
||||
apt-get install -y wireguard
|
||||
install -m 700 scripts/wireguard/wg_client.py /usr/local/sbin/wg_client.py
|
||||
```
|
||||
|
||||
## Typowy flow (server)
|
||||
|
||||
```bash
|
||||
# 1) wygeneruj klucze serwera
|
||||
wg_serverctl.py gen-server --force
|
||||
|
||||
# 2) wygeneruj klucze klienta (trzymane na serwerze w /etc/wireguard/clients)
|
||||
wg_serverctl.py gen-client client1
|
||||
|
||||
# 3) dodaj klienta do puli, wygeneruj wg0.conf i zrestartuj wg-quick@wg0
|
||||
wg_serverctl.py add client1
|
||||
|
||||
# 4) wyeksportuj client config (do przekazania na klienta)
|
||||
wg_serverctl.py export client1
|
||||
|
||||
# 5) podglad
|
||||
wg_serverctl.py list
|
||||
wg_serverctl.py show client1
|
||||
```
|
||||
|
||||
Plik konfiguracyjny klienta powstaje jako:
|
||||
|
||||
- `/etc/wireguard/clients/client1.conf`
|
||||
|
||||
## Typowy flow (client)
|
||||
|
||||
Skopiuj `client1.conf` na klienta, a potem:
|
||||
|
||||
```bash
|
||||
wg_client.py install ./client1.conf
|
||||
wg_client.py up
|
||||
wg_client.py status
|
||||
```
|
||||
|
||||
## Stop / disable / reset wg0 (Ubuntu 24.04)
|
||||
|
||||
Stop (zostaw config, tylko down):
|
||||
|
||||
```bash
|
||||
systemctl stop wg-quick@wg0
|
||||
```
|
||||
|
||||
Disable autostart:
|
||||
|
||||
```bash
|
||||
systemctl disable wg-quick@wg0
|
||||
```
|
||||
|
||||
Flush (pewny down + usuniecie interfejsu):
|
||||
|
||||
```bash
|
||||
systemctl stop wg-quick@wg0
|
||||
ip link del wg0 2>/dev/null || true
|
||||
ip a show wg0 2>/dev/null || echo "wg0 not present"
|
||||
```
|
||||
|
||||
Hard reset (usuniecie kluczy + konfiguracji):
|
||||
|
||||
```bash
|
||||
systemctl stop wg-quick@wg0
|
||||
systemctl disable wg-quick@wg0
|
||||
ip link del wg0 2>/dev/null || true
|
||||
|
||||
rm -f /etc/wireguard/wg0.conf
|
||||
rm -f /etc/wireguard/server.key /etc/wireguard/server.pub
|
||||
rm -rf /etc/wireguard/clients
|
||||
```
|
||||
|
||||
Reset peerow bez kasowania server key:
|
||||
|
||||
```bash
|
||||
systemctl stop wg-quick@wg0
|
||||
rm -f /etc/wireguard/clients/*.pub /etc/wireguard/clients/*.ip
|
||||
systemctl start wg-quick@wg0
|
||||
```
|
||||
|
||||
198
doc/step-001-wireguard-ufw-ssh-only.md
Normal file
198
doc/step-001-wireguard-ufw-ssh-only.md
Normal file
@@ -0,0 +1,198 @@
|
||||
# Step 001: WireGuard + UFW + SSH tylko po WG (Ubuntu 24.04, 1 NIC)
|
||||
|
||||
Cel: na świeżym Ubuntu 24.04 na bare metalu ustawić:
|
||||
|
||||
- **WireGuard** jako prywatny kanał administracyjny,
|
||||
- **SSH dostępne tylko przez WireGuard**,
|
||||
- przygotować grunt pod **Agave RPC dostępne tylko przez WireGuard** (8899/8900 prywatnie),
|
||||
- zostawić porty klastra Agave na publicznym NIC, żeby node mógł się zsynchronizować.
|
||||
|
||||
Kolejność jest krytyczna: najpierw uruchom WireGuard i sprawdź, że działa, dopiero potem ograniczaj SSH/firewall.
|
||||
|
||||
## Założenia
|
||||
|
||||
- Host ma **1 publiczny interfejs** (np. `enp1s0`).
|
||||
- WireGuard interfejs to `wg0`.
|
||||
- Masz dostęp awaryjny do konsoli (IPMI/KVM) na wypadek pomyłki.
|
||||
|
||||
## Parametry (dostosuj)
|
||||
|
||||
```bash
|
||||
export PUBLIC_IFACE="enp1s0"
|
||||
export WG_IFACE="wg0"
|
||||
|
||||
export WG_SERVER_IP="10.66.66.1/24"
|
||||
export WG_SERVER_IP_ONLY="10.66.66.1"
|
||||
export WG_PORT="51820"
|
||||
|
||||
export WG_CLIENT_IP="10.66.66.2/32"
|
||||
export WG_CLIENT_PUBKEY="<PASTE_CLIENT_PUBLIC_KEY>"
|
||||
|
||||
# Agave/validator ports
|
||||
export RPC_PORT="8899"
|
||||
export WS_PORT="8900"
|
||||
export DYNAMIC_PORT_RANGE="8000:8020"
|
||||
```
|
||||
|
||||
## 1) WireGuard (nie ruszaj jeszcze SSH)
|
||||
|
||||
### Instalacja
|
||||
|
||||
```bash
|
||||
apt-get update
|
||||
apt-get install -y wireguard
|
||||
```
|
||||
|
||||
### Klucze serwera
|
||||
|
||||
```bash
|
||||
umask 077
|
||||
wg genkey | tee /etc/wireguard/server.key >/dev/null
|
||||
cat /etc/wireguard/server.key | wg pubkey | tee /etc/wireguard/server.pub >/dev/null
|
||||
cat /etc/wireguard/server.pub
|
||||
```
|
||||
|
||||
### Konfiguracja `/etc/wireguard/wg0.conf`
|
||||
|
||||
```bash
|
||||
cat >/etc/wireguard/${WG_IFACE}.conf <<EOF
|
||||
[Interface]
|
||||
Address = ${WG_SERVER_IP}
|
||||
ListenPort = ${WG_PORT}
|
||||
PrivateKey = $(cat /etc/wireguard/server.key)
|
||||
|
||||
[Peer]
|
||||
PublicKey = ${WG_CLIENT_PUBKEY}
|
||||
AllowedIPs = ${WG_CLIENT_IP}
|
||||
EOF
|
||||
```
|
||||
|
||||
Start:
|
||||
|
||||
```bash
|
||||
systemctl enable --now wg-quick@${WG_IFACE}
|
||||
wg show
|
||||
ip a show ${WG_IFACE}
|
||||
```
|
||||
|
||||
Test z klienta (po zestawieniu tunelu): `ping 10.66.66.1`.
|
||||
|
||||
## 2) UFW: public tylko WG + porty klastra; prywatnie RPC/SSH po wg0
|
||||
|
||||
Instalacja:
|
||||
|
||||
```bash
|
||||
apt-get install -y ufw
|
||||
```
|
||||
|
||||
Public: pozwól na WireGuard:
|
||||
|
||||
```bash
|
||||
ufw allow ${WG_PORT}/udp
|
||||
```
|
||||
|
||||
Public: pozwól na porty klastra Agave (żeby node mógł syncować).
|
||||
|
||||
Uwaga: Solana/Agave używa głównie **UDP** w tym zakresie; jeśli będziesz miał problemy z reachability, rozważ dopuszczenie też TCP w tym samym zakresie.
|
||||
|
||||
```bash
|
||||
ufw allow in on "${PUBLIC_IFACE}" to any port ${DYNAMIC_PORT_RANGE}/udp
|
||||
```
|
||||
|
||||
Prywatnie (wg0): RPC/WS:
|
||||
|
||||
```bash
|
||||
ufw allow in on "${WG_IFACE}" to any port ${RPC_PORT} proto tcp
|
||||
ufw allow in on "${WG_IFACE}" to any port ${WS_PORT} proto tcp
|
||||
```
|
||||
|
||||
Na tym etapie NIE blokuj jeszcze publicznego SSH (żeby się nie zablokować).
|
||||
|
||||
Włącz UFW:
|
||||
|
||||
```bash
|
||||
ufw enable
|
||||
ufw status verbose
|
||||
```
|
||||
|
||||
## 3) SSH tylko po WireGuard (dopiero gdy WG działa)
|
||||
|
||||
### 3.1) UFW: pozwól na SSH po wg0
|
||||
|
||||
```bash
|
||||
ufw allow in on "${WG_IFACE}" to any port 22 proto tcp
|
||||
```
|
||||
|
||||
### 3.2) sshd: nasłuch tylko na WG IP
|
||||
|
||||
Edytuj `sshd_config`:
|
||||
|
||||
```bash
|
||||
sed -n '1,220p' /etc/ssh/sshd_config
|
||||
```
|
||||
|
||||
Dodaj (lub ustaw) na końcu pliku:
|
||||
|
||||
```conf
|
||||
PasswordAuthentication no
|
||||
PermitRootLogin no
|
||||
ListenAddress 10.66.66.1
|
||||
```
|
||||
|
||||
Zanim zrestartujesz SSH:
|
||||
|
||||
- miej otwartą obecną sesję SSH,
|
||||
- zestaw WG na kliencie,
|
||||
- przygotuj drugi terminal do testu po `10.66.66.1`.
|
||||
|
||||
Restart:
|
||||
|
||||
```bash
|
||||
systemctl restart ssh
|
||||
ss -lntp | grep -E ':22\\b' || true
|
||||
```
|
||||
|
||||
Test z klienta po WG:
|
||||
|
||||
```bash
|
||||
ssh <user>@10.66.66.1
|
||||
```
|
||||
|
||||
Jeśli działa: zablokuj publiczny SSH:
|
||||
|
||||
```bash
|
||||
ufw deny 22/tcp
|
||||
ufw status verbose
|
||||
```
|
||||
|
||||
## 4) Kernel/sysctl baseline pod Agave (na tym etapie bez uruchamiania Agave)
|
||||
|
||||
Minimalne wartości, które już mamy w Ansible (i kilka dodatkowych bezpiecznych):
|
||||
|
||||
```bash
|
||||
cat >/etc/sysctl.d/90-agave.conf <<'EOF'
|
||||
fs.file-max = 2000000
|
||||
vm.max_map_count = 1000000
|
||||
net.core.somaxconn = 65535
|
||||
net.core.netdev_max_backlog = 250000
|
||||
net.core.rmem_max = 134217728
|
||||
net.core.wmem_max = 134217728
|
||||
EOF
|
||||
|
||||
sysctl --system
|
||||
```
|
||||
|
||||
Limity `nofile`:
|
||||
|
||||
```bash
|
||||
cat >/etc/security/limits.d/90-solana.conf <<'EOF'
|
||||
* soft nofile 1000000
|
||||
* hard nofile 1000000
|
||||
EOF
|
||||
```
|
||||
|
||||
## 5) Co dalej
|
||||
|
||||
- Instalacja i uruchomienie Agave: patrz `doc/etap-007-manual-baremetal-rpc-commands.md`.
|
||||
- Narzedzia do generowania konfiguracji WireGuard (server/client) + reset wg0: patrz `doc/step-001-wireguard-tools.md`.
|
||||
- Docelowo: bind RPC/WS do WG IP (zamiast `0.0.0.0`) i spójne, minimalne reguły firewall.
|
||||
261
doc/step-002-plan-solana-rpc-mainnet-private.txt
Normal file
261
doc/step-002-plan-solana-rpc-mainnet-private.txt
Normal file
@@ -0,0 +1,261 @@
|
||||
Step 002: Plan wdrozenia Solana RPC (Agave mainnet, non-voting, private RPC + Geyser->Postgres + monitoring + NOC tmux)
|
||||
|
||||
Zalozenia (default):
|
||||
- WG iface: wg0, WG subnet: 10.66.66.0/24, WG IP baremetala: 10.66.66.1
|
||||
- Public iface: eth0, public IP: 149.50.116.219
|
||||
- RPC: 8899 (HTTP), WS: 8900 (WS = RPC_PORT+1)
|
||||
- Postgres (lokalnie): 5432 (DB: geyser, user: geyser)
|
||||
- (Optional) Yellowstone gRPC: 10000 (tylko po WG)
|
||||
- Prometheus: 9090, Grafana: 3000, node_exporter: 9100 (tylko po WG)
|
||||
- Dynamic port range (cluster): 8000-8020/udp (public eth0)
|
||||
- User: solana
|
||||
- Dirs:
|
||||
- /var/lib/solana (ledger/accounts)
|
||||
- /etc/solana (configs)
|
||||
- /var/log/solana (logs)
|
||||
|
||||
0) Preconditions (check)
|
||||
# WG dziala
|
||||
ip -4 a show wg0
|
||||
wg show
|
||||
|
||||
# SSH po WG dziala (z laptop/VPS):
|
||||
# ssh user@10.66.66.1
|
||||
|
||||
1) Packages
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
tmux btop \
|
||||
sysstat iotop \
|
||||
ethtool nstat \
|
||||
chrony curl jq git ca-certificates \
|
||||
ripgrep \
|
||||
smartmontools nvme-cli \
|
||||
build-essential pkg-config libssl-dev \
|
||||
clang llvm-dev libclang-dev \
|
||||
cmake protobuf-compiler libudev-dev zlib1g-dev
|
||||
|
||||
2) Users/Dirs
|
||||
sudo getent group solana >/dev/null || sudo groupadd --system solana
|
||||
sudo id -u solana >/dev/null 2>&1 || sudo useradd --system --gid solana \
|
||||
--home-dir /var/lib/solana --create-home --shell /bin/bash solana
|
||||
|
||||
sudo install -d -o root -g root -m 0755 /etc/solana /opt/solana/bin
|
||||
sudo install -d -o solana -g solana -m 0750 \
|
||||
/var/lib/solana /var/lib/solana/ledger /var/lib/solana/accounts /var/log/solana
|
||||
|
||||
# identity permissions (po utworzeniu pliku):
|
||||
# sudo chown solana:solana /var/lib/solana/identity.json
|
||||
# sudo chmod 600 /var/lib/solana/identity.json
|
||||
|
||||
2A) Storage (rekomendowane)
|
||||
# Variant A: jesli /dev/nvme0n1 jest wolny i chcesz go podzielic 50/50:
|
||||
# doc/step-002a-storage-nvme0n1-xfs.txt
|
||||
#
|
||||
# Variant B (mpabi): masz juz 2x NVMe z XFS:
|
||||
# - SOLACCT na accounts (nvme0n1p1)
|
||||
# - SOLLEDG na ledger (nvme1n1p1)
|
||||
# doc/step-002a-storage-existing-solacct-solledg-xfs.txt
|
||||
|
||||
3) Firewall/WG binding (private-only)
|
||||
# UFW: deny everything inbound by default
|
||||
sudo ufw default deny incoming
|
||||
sudo ufw default allow outgoing
|
||||
|
||||
# WireGuard (public)
|
||||
sudo ufw allow 51820/udp
|
||||
|
||||
# SSH only via WG
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 22 proto tcp
|
||||
|
||||
# RPC/WS only via WG
|
||||
sudo ufw allow in on wg0 to any port 8899 proto tcp
|
||||
sudo ufw allow in on wg0 to any port 8900 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 8899 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 8900 proto tcp
|
||||
|
||||
# Yellowstone gRPC only via WG
|
||||
sudo ufw allow in on wg0 to any port 10000 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 10000 proto tcp
|
||||
|
||||
# Monitoring only via WG
|
||||
sudo ufw allow in on wg0 to any port 9100 proto tcp
|
||||
sudo ufw allow in on wg0 to any port 9090 proto tcp
|
||||
sudo ufw allow in on wg0 to any port 3000 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 9100 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 9090 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 3000 proto tcp
|
||||
|
||||
# Cluster ports (public, inbound) for sync (UDP)
|
||||
sudo ufw allow in on eth0 to any port 8000:8020 proto udp
|
||||
# (opcjonalnie) jesli bedziesz mial problemy z reachability:
|
||||
# sudo ufw allow in on eth0 to any port 8000:8020 proto tcp
|
||||
|
||||
sudo ufw enable
|
||||
sudo ufw status verbose
|
||||
|
||||
4) Validator (Agave) config + systemd (non-voting RPC-only)
|
||||
# sysctl baseline (Agave startup checks)
|
||||
sudo tee /etc/sysctl.d/90-agave.conf >/dev/null <<'EOF'
|
||||
net.core.rmem_max = 134217728
|
||||
net.core.wmem_max = 134217728
|
||||
EOF
|
||||
sudo sysctl --system
|
||||
|
||||
# ulimit baseline
|
||||
sudo tee /etc/security/limits.d/90-solana.conf >/dev/null <<'EOF'
|
||||
* soft nofile 1000000
|
||||
* hard nofile 1000000
|
||||
EOF
|
||||
|
||||
# Agave tools (solana-keygen etc.) for solana user
|
||||
sudo -u solana -H bash -lc 'sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)"'
|
||||
|
||||
# Copy binaries to /opt/solana/bin (avoid symlink permission gotchas)
|
||||
sudo cp -a /var/lib/solana/.local/share/solana/install/active_release/bin/. /opt/solana/bin/
|
||||
sudo find /opt/solana/bin -maxdepth 1 -type f -exec chmod 0755 {} \;
|
||||
|
||||
/opt/solana/bin/solana --version || true
|
||||
/opt/solana/bin/solana-keygen --version
|
||||
|
||||
# Build agave-validator (pin tag + toolchain)
|
||||
# Uwaga: dla nowszych stable release'ow binarka moze nie byc publikowana,
|
||||
# wiec build ze zrodel jest wymagany.
|
||||
# doc/step-002f-agave-build-from-source-latest.txt
|
||||
|
||||
# Identity (create once)
|
||||
if [ ! -f /var/lib/solana/identity.json ]; then
|
||||
sudo -u solana -H /opt/solana/bin/solana-keygen new --no-passphrase -o /var/lib/solana/identity.json
|
||||
sudo chown solana:solana /var/lib/solana/identity.json
|
||||
sudo chmod 600 /var/lib/solana/identity.json
|
||||
fi
|
||||
|
||||
# Genesis hash (mainnet)
|
||||
GENESIS_HASH="$(curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getGenesisHash"}' \
|
||||
https://api.mainnet-beta.solana.com | jq -r .result)"
|
||||
echo "$GENESIS_HASH"
|
||||
# Oczekiwane (mainnet): 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d
|
||||
|
||||
# systemd unit (bind RPC to WG/localhost only)
|
||||
# UWAGA: w unit file NIE uzywaj `\\` na koncu linii w ExecStart.
|
||||
# Jesli wpiszesz `\\`, agave-validator dostanie literalny argument `\` i padnie z:
|
||||
# "error: The subcommand '\\' wasn't recognized"
|
||||
#
|
||||
# Najprosciej: ExecStart w 1 linii, bez "kontynuacji" backslashami.
|
||||
# Ale: zeby uniknac problemow z wklejaniem/lamaniem linii, ponizej jest bezpieczna wersja wielolinijkowa.
|
||||
sudo tee /etc/systemd/system/agave-validator.service >/dev/null <<'EOF'
|
||||
[Unit]
|
||||
Description=Agave validator (mainnet, RPC-only, non-voting)
|
||||
After=network-online.target wg-quick@wg0.service
|
||||
Wants=network-online.target wg-quick@wg0.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=solana
|
||||
Group=solana
|
||||
WorkingDirectory=/var/lib/solana
|
||||
Environment=RUST_LOG=info
|
||||
LimitNOFILE=1048576
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStopSec=120
|
||||
ExecStart=/opt/solana/bin/agave-validator \
|
||||
--identity /var/lib/solana/identity.json \
|
||||
--no-voting \
|
||||
--private-rpc \
|
||||
--ledger /var/lib/solana/ledger \
|
||||
--accounts /var/lib/solana/accounts \
|
||||
--log /var/log/solana/validator.log \
|
||||
--bind-address 0.0.0.0 \
|
||||
--rpc-bind-address 10.66.66.1 \
|
||||
--rpc-port 8899 \
|
||||
--dynamic-port-range 8000-8020 \
|
||||
--entrypoint entrypoint.mainnet-beta.solana.com:8001 \
|
||||
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d \
|
||||
--full-rpc-api \
|
||||
--limit-ledger-size 50000000
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/var/lib/solana/ledger /var/lib/solana/accounts /var/log/solana /var/lib/solana
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now agave-validator
|
||||
|
||||
systemctl is-active agave-validator
|
||||
journalctl -u agave-validator -n 80 --no-pager
|
||||
|
||||
# Jesli padnie na genesis (brak genesis.tar.bz2):
|
||||
# doc/step-002c-genesis-download-mainnet.txt
|
||||
#
|
||||
# Jesli padnie na snapshot (mainnet wymaga snapshot):
|
||||
# doc/step-002d-snapshot-download-mainnet.txt
|
||||
#
|
||||
# Jesli chcesz najpierw odpalic validator z CLI (test), zanim przepniesz na systemd:
|
||||
# doc/step-002e-agave-validator-run-cli.txt
|
||||
|
||||
5) Geyser plugin -> Postgres
|
||||
# Krok po kroku:
|
||||
# doc/step-003-geyser-plugin-postgres.txt
|
||||
#
|
||||
# W skrocie:
|
||||
# - budujesz plugin .so i wrzucasz do /opt/solana/plugins/
|
||||
# - tworzysz /etc/solana/geyser-postgres.json
|
||||
# - dopisujesz do ExecStart:
|
||||
# --geyser-plugin-config /etc/solana/geyser-postgres.json
|
||||
|
||||
6) Drift DLOB (2 markets) (on VPS, not baremetal) (placeholder)
|
||||
# VPS env:
|
||||
# PERP_MARKETS_TO_LOAD=0,75
|
||||
# USE_GRPC=true
|
||||
# GRPC_ENDPOINT=http://10.66.66.1:10000
|
||||
|
||||
7) Monitoring (bind to WG only) (placeholder)
|
||||
# node_exporter (prefer bind to WG):
|
||||
# sudo apt-get install -y prometheus-node-exporter
|
||||
# sudo systemctl enable --now prometheus-node-exporter
|
||||
#
|
||||
# Prometheus/Grafana (bind to 10.66.66.1):
|
||||
# sudo apt-get install -y prometheus grafana
|
||||
# sudo systemctl enable --now prometheus grafana-server
|
||||
|
||||
8) NOC tmux (btop) (placeholder)
|
||||
# Create tmux session:
|
||||
# btop
|
||||
# iostat -x 1
|
||||
# vmstat 1
|
||||
# nstat -az
|
||||
# ethtool -S eth0 | head -n 40 (wrap in watch)
|
||||
# journalctl -fu agave-validator
|
||||
# while true; do curl getHealth/getSlot; sleep 1; done
|
||||
|
||||
9) Troubleshooting checklist
|
||||
# Services:
|
||||
systemctl status agave-validator --no-pager
|
||||
journalctl -u agave-validator -n 200 --no-pager
|
||||
|
||||
# Sockets and binds:
|
||||
ss -lntp | egrep ':22\\b|:8899\\b|:8900\\b|:10000\\b|:9100\\b|:9090\\b|:3000\\b' || true
|
||||
|
||||
# Firewall:
|
||||
sudo ufw status verbose
|
||||
|
||||
# RPC local:
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getHealth"}' \
|
||||
http://127.0.0.1:8899 | jq .
|
||||
|
||||
5-min quick verification checklist
|
||||
wg show
|
||||
sudo ufw status verbose
|
||||
systemctl is-active agave-validator
|
||||
ss -lntp | egrep ':8899\\b|:8900\\b|:10000\\b|:9090\\b|:3000\\b|:9100\\b' || true
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getSlot"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
214
doc/step-002-solana-rpc-install.txt
Normal file
214
doc/step-002-solana-rpc-install.txt
Normal file
@@ -0,0 +1,214 @@
|
||||
Step 002: Instalacja Solana RPC (Agave) na baremetalu (po WireGuard)
|
||||
|
||||
Stan startowy:
|
||||
- Masz dzialajacy WireGuard: z laptopa/VPS pingujesz 10.66.66.1 i logujesz sie: `ssh user@10.66.66.1`
|
||||
- SSH po publicznym eth0 jest odciete (UFW deny na eth0:22), SSH dziala tylko po wg0.
|
||||
- RPC ma byc prywatny: porty 8899/8900 tylko po wg0, ale node ma miec publiczne porty klastra (UDP dynamic range) zeby syncowal.
|
||||
|
||||
UWAGA o prywatnosci:
|
||||
- RPC/WS bindujemy na wg0 (10.66.66.1), zeby NIE wystawic RPC na publicznym eth0 nawet gdy UFW jest wylaczone.
|
||||
- Dodatkowo i tak robimy UFW "deny" na eth0:8899/8900 (defense-in-depth).
|
||||
|
||||
Parametry (dostosuj, przyklad mainnet):
|
||||
- PUBLIC_IFACE=eth0
|
||||
- WG_IFACE=wg0
|
||||
- WG_IP=10.66.66.1
|
||||
- ENDPOINT_RPC_PORT=8899
|
||||
- ENDPOINT_WS_PORT=8900 (WS = RPC_PORT+1)
|
||||
- DYNAMIC_PORT_RANGE="8000:8020"
|
||||
|
||||
0) Storage (rekomendowane)
|
||||
Jesli masz osobny NVMe na dane (ledger/accounts), zrob najpierw:
|
||||
doc/step-002a-storage-nvme0n1-xfs.txt
|
||||
albo (mpabi, 2x NVMe z juz istniejacym XFS):
|
||||
doc/step-002a-storage-existing-solacct-solledg-xfs.txt
|
||||
|
||||
1) Bazowe pakiety
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
chrony curl jq git \
|
||||
ripgrep \
|
||||
smartmontools nvme-cli \
|
||||
build-essential pkg-config libssl-dev \
|
||||
clang llvm-dev libclang-dev \
|
||||
cmake protobuf-compiler libudev-dev zlib1g-dev
|
||||
|
||||
2) Uzytkownik i katalogi
|
||||
sudo getent group solana >/dev/null || sudo groupadd --system solana
|
||||
sudo id -u solana >/dev/null 2>&1 || sudo useradd --system --gid solana \
|
||||
--home-dir /var/lib/solana --create-home --shell /bin/bash solana
|
||||
|
||||
sudo install -d -o root -g root -m 0755 /etc/solana /opt/solana/bin
|
||||
sudo install -d -o solana -g solana -m 0750 \
|
||||
/var/lib/solana /var/lib/solana/ledger /var/lib/solana/accounts /var/log/solana
|
||||
|
||||
# Jesli ledger/accounts sa mountpointami (Step 002A), upewnij sie, ze root FS jest writable dla usera solana:
|
||||
sudo chown solana:solana /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
sudo chmod 750 /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
|
||||
3) Sysctl + ulimit (baseline pod Agave)
|
||||
sudo tee /etc/sysctl.d/90-agave.conf >/dev/null <<'EOF'
|
||||
fs.file-max = 2000000
|
||||
vm.max_map_count = 1000000
|
||||
net.core.somaxconn = 65535
|
||||
net.core.netdev_max_backlog = 250000
|
||||
net.core.rmem_max = 134217728
|
||||
net.core.wmem_max = 134217728
|
||||
EOF
|
||||
sudo sysctl --system
|
||||
|
||||
sudo tee /etc/security/limits.d/90-solana.conf >/dev/null <<'EOF'
|
||||
* soft nofile 1000000
|
||||
* hard nofile 1000000
|
||||
EOF
|
||||
|
||||
4) Firewall (UFW): RPC prywatnie po wg0, porty klastra publicznie po eth0
|
||||
sudo ufw default deny incoming
|
||||
sudo ufw default allow outgoing
|
||||
|
||||
# WireGuard z internetu
|
||||
sudo ufw allow 51820/udp
|
||||
|
||||
# SSH tylko po wg0 (zakladamy, ze tak chcesz)
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 22 proto tcp
|
||||
|
||||
# RPC/WS tylko po wg0
|
||||
sudo ufw allow in on wg0 to any port 8899 proto tcp
|
||||
sudo ufw allow in on wg0 to any port 8900 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 8899 proto tcp
|
||||
sudo ufw deny in on eth0 to any port 8900 proto tcp
|
||||
|
||||
# Porty klastra (inbound) na eth0 - UDP
|
||||
sudo ufw allow in on eth0 to any port 8000:8020 proto udp
|
||||
|
||||
# (Opcjonalnie) jesli bedziesz mial problemy z reachability:
|
||||
# sudo ufw allow in on eth0 to any port 8000:8020 proto tcp
|
||||
|
||||
sudo ufw enable
|
||||
sudo ufw status verbose
|
||||
|
||||
5) Narzedzia Solana/Anza (dla `solana`): solana-keygen itd.
|
||||
sudo -u solana -H bash -lc 'sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)"'
|
||||
|
||||
# Weryfikacja, ze installer utworzyl active_release/bin (bez exit, zeby nie ubijac pane/tmux):
|
||||
sudo -u solana -H bash -lc 'ls -ld ~/.local/share/solana/install/active_release/bin && ls -1 ~/.local/share/solana/install/active_release/bin | head'
|
||||
|
||||
# Kopiuj binarki do /opt/solana/bin (zamiast symlinkow - mniej problemow z uprawnieniami)
|
||||
sudo cp -a /var/lib/solana/.local/share/solana/install/active_release/bin/. /opt/solana/bin/
|
||||
sudo find /opt/solana/bin -maxdepth 1 -type f -exec chmod 0755 {} \;
|
||||
|
||||
/opt/solana/bin/solana-keygen --version
|
||||
|
||||
6) Build `agave-validator` ze zrodel i instalacja do /opt/solana/bin
|
||||
# Najnowszy stable release dla mainnet wybieraj z GitHub Releases (tag v3.0.x).
|
||||
# Krok po kroku (w tym automatyczny wybor tagu):
|
||||
doc/step-002f-agave-build-from-source-latest.txt
|
||||
|
||||
7) Identity (tylko raz) + backup poza hostem
|
||||
if [ ! -f /var/lib/solana/identity.json ]; then
|
||||
sudo -u solana -H bash -lc \
|
||||
"/opt/solana/bin/solana-keygen new --no-passphrase -o /var/lib/solana/identity.json"
|
||||
sudo chmod 0600 /var/lib/solana/identity.json
|
||||
sudo chown solana:solana /var/lib/solana/identity.json
|
||||
fi
|
||||
|
||||
sudo -u solana -H bash -lc \
|
||||
"/opt/solana/bin/solana-keygen pubkey /var/lib/solana/identity.json"
|
||||
|
||||
8) (Opcjonalnie, rekomendowane) Genesis hash z publicznego RPC
|
||||
Mainnet:
|
||||
GENESIS_HASH="$(curl -sS -H 'content-type: application/json' \
|
||||
--data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"getGenesisHash\"}' \
|
||||
https://api.mainnet-beta.solana.com | jq -r .result)"
|
||||
echo "$GENESIS_HASH"
|
||||
|
||||
Testnet:
|
||||
GENESIS_HASH="$(curl -sS -H 'content-type: application/json' \
|
||||
--data '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"getGenesisHash\"}' \
|
||||
https://api.testnet.solana.com | jq -r .result)"
|
||||
echo "$GENESIS_HASH"
|
||||
|
||||
9) Systemd unit: /etc/systemd/system/agave-validator.service
|
||||
Minimalny unit (RPC node, non-voting). RPC/WS bind na wg0 (10.66.66.1).
|
||||
Jesli widzisz w logach:
|
||||
"error: The subcommand '\\' wasn't recognized"
|
||||
to zrob:
|
||||
doc/step-002b-agave-validator-systemd.txt
|
||||
|
||||
Jesli chcesz najpierw odpalic recznie z CLI (test) przed systemd:
|
||||
doc/step-002e-agave-validator-run-cli.txt
|
||||
|
||||
sudo tee /etc/systemd/system/agave-validator.service >/dev/null <<'EOF'
|
||||
[Unit]
|
||||
Description=Agave validator (mainnet, RPC-only, non-voting)
|
||||
After=network-online.target wg-quick@wg0.service
|
||||
Wants=network-online.target wg-quick@wg0.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=solana
|
||||
Group=solana
|
||||
WorkingDirectory=/var/lib/solana
|
||||
Environment=RUST_LOG=info
|
||||
LimitNOFILE=1048576
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStopSec=120
|
||||
ExecStart=/opt/solana/bin/agave-validator \
|
||||
--identity /var/lib/solana/identity.json \
|
||||
--no-voting \
|
||||
--private-rpc \
|
||||
--ledger /var/lib/solana/ledger \
|
||||
--accounts /var/lib/solana/accounts \
|
||||
--log /var/log/solana/validator.log \
|
||||
--bind-address 0.0.0.0 \
|
||||
--rpc-bind-address 10.66.66.1 \
|
||||
--rpc-port 8899 \
|
||||
--dynamic-port-range 8000-8020 \
|
||||
--entrypoint entrypoint.mainnet-beta.solana.com:8001 \
|
||||
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d \
|
||||
--full-rpc-api \
|
||||
--limit-ledger-size 50000000
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/var/lib/solana/ledger /var/lib/solana/accounts /var/log/solana /var/lib/solana
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now agave-validator
|
||||
|
||||
Jesli chcesz dodac genesis hash:
|
||||
- dopisz w ExecStart linie:
|
||||
--expected-genesis-hash <GENESIS_HASH>
|
||||
|
||||
10) Weryfikacja
|
||||
systemctl is-active agave-validator
|
||||
journalctl -u agave-validator -n 200 --no-pager
|
||||
|
||||
# z samego hosta (getHealth moze zwracac blad dopoki node nie dogoni klastra):
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getVersion"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
|
||||
# z laptopa/VPS po WG:
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getSlot"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
|
||||
Jesli cos nie dziala:
|
||||
- `sudo wg show` (czy WG jest OK)
|
||||
- `sudo ufw status verbose` (czy RPC nie jest blokowane po wg0)
|
||||
- `journalctl -u agave-validator -n 200 --no-pager` (logi Agave)
|
||||
|
||||
Typowe fixy:
|
||||
- "error: The subcommand '\\' wasn't recognized":
|
||||
doc/step-002b-agave-validator-systemd.txt
|
||||
- "failed to open genesis" / brak `genesis.tar.bz2`:
|
||||
doc/step-002c-genesis-download-mainnet.txt
|
||||
- "did you forget to provide a snapshot" (mainnet bez snapshot nie ruszy):
|
||||
doc/step-002d-snapshot-download-mainnet.txt
|
||||
51
doc/step-002a-storage-existing-solacct-solledg-xfs.txt
Normal file
51
doc/step-002a-storage-existing-solacct-solledg-xfs.txt
Normal file
@@ -0,0 +1,51 @@
|
||||
Step 002A (variant): Storage na 2x NVMe (XFS juz istnieje) -> SOLACCT=accounts, SOLLEDG=ledger
|
||||
|
||||
Context (wg hosta mpabi):
|
||||
- /dev/nvme0n1p1: XFS LABEL=SOLACCT (nowy dysk na accounts)
|
||||
- /dev/nvme1n1p1: XFS LABEL=SOLLEDG (stary dysk ~63% used na ledger)
|
||||
|
||||
Cel:
|
||||
- NIE formatowac (bez kasowania danych), tylko zamontowac i spiac w /etc/fstab:
|
||||
- /var/lib/solana/accounts -> SOLACCT
|
||||
- /var/lib/solana/ledger -> SOLLEDG
|
||||
|
||||
0) Check (zanim dotkniesz fstab)
|
||||
lsblk -f
|
||||
sudo blkid /dev/nvme0n1p1 /dev/nvme1n1p1
|
||||
|
||||
# Upewnij sie, ze NIC nie jest zamontowane w /var/lib/solana/*:
|
||||
findmnt /var/lib/solana/accounts || true
|
||||
findmnt /var/lib/solana/ledger || true
|
||||
|
||||
1) Mountpointy
|
||||
sudo getent group solana >/dev/null || sudo groupadd --system solana
|
||||
sudo id -u solana >/dev/null 2>&1 || sudo useradd --system --gid solana \
|
||||
--home-dir /var/lib/solana --create-home --shell /bin/bash solana
|
||||
|
||||
sudo install -d -o solana -g solana -m 0750 /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
2) /etc/fstab (po UUID)
|
||||
UUID_ACCTS="$(sudo blkid -s UUID -o value /dev/nvme0n1p1)"
|
||||
UUID_LEDGER="$(sudo blkid -s UUID -o value /dev/nvme1n1p1)"
|
||||
|
||||
echo "UUID=$UUID_ACCTS /var/lib/solana/accounts xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
echo "UUID=$UUID_LEDGER /var/lib/solana/ledger xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
|
||||
sudo mount -a
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
3) Permissions (po mount) - krytyczne
|
||||
sudo chown solana:solana /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
sudo chmod 750 /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
4) Weryfikacja
|
||||
df -hT /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
findmnt /var/lib/solana/accounts
|
||||
findmnt /var/lib/solana/ledger
|
||||
stat -c '%U:%G %a %n' /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
5) Uwaga (bezpieczenstwo / dane)
|
||||
- Jesli na SOLLEDG masz stare dane ledgera z innej instalacji, to validator moze startowac na "starym stanie".
|
||||
Wtedy najlepiej wyczyscic ledger przed pobraniem snapshotow:
|
||||
doc/step-002d-snapshot-download-mainnet.txt (sekcja czyszczenia)
|
||||
|
||||
64
doc/step-002a-storage-nvme0n1-xfs.txt
Normal file
64
doc/step-002a-storage-nvme0n1-xfs.txt
Normal file
@@ -0,0 +1,64 @@
|
||||
Step 002A: Storage pod Solana data (nvme0n1 -> XFS -> /var/lib/solana/{ledger,accounts})
|
||||
|
||||
Cel:
|
||||
- system zostaje na md1 (/)
|
||||
- dane Solany (ledger/accounts) ida na osobny NVMe: /dev/nvme0n1
|
||||
|
||||
UWAGA: kroki nizej KASUJA dane na /dev/nvme0n1.
|
||||
|
||||
0) Check (czy nvme0n1 jest do wyczyszczenia)
|
||||
lsblk /dev/nvme0n1
|
||||
sudo wipefs -n /dev/nvme0n1
|
||||
sudo lsblk -f /dev/nvme0n1
|
||||
|
||||
1) Partycje (50/50)
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y parted
|
||||
|
||||
sudo parted -s /dev/nvme0n1 mklabel gpt
|
||||
sudo parted -s /dev/nvme0n1 mkpart primary 0% 50%
|
||||
sudo parted -s /dev/nvme0n1 mkpart primary 50% 100%
|
||||
sudo partprobe /dev/nvme0n1
|
||||
sudo udevadm settle
|
||||
|
||||
lsblk /dev/nvme0n1
|
||||
ls -l /dev/nvme0n1p1 /dev/nvme0n1p2
|
||||
|
||||
2) Format XFS
|
||||
Uwaga: mkfs.xfs na Ubuntu ma limit label <= 12 znakow.
|
||||
|
||||
sudo mkfs.xfs -f -L sol_ledger /dev/nvme0n1p1
|
||||
sudo mkfs.xfs -f -L sol_accts /dev/nvme0n1p2
|
||||
sudo blkid /dev/nvme0n1p1 /dev/nvme0n1p2
|
||||
|
||||
3) Mountpointy + fstab (po UUID)
|
||||
sudo install -d -o solana -g solana -m 0750 /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
|
||||
UUID_LEDGER="$(sudo blkid -s UUID -o value /dev/nvme0n1p1)"
|
||||
UUID_ACCTS="$(sudo blkid -s UUID -o value /dev/nvme0n1p2)"
|
||||
|
||||
echo "UUID=$UUID_LEDGER /var/lib/solana/ledger xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
echo "UUID=$UUID_ACCTS /var/lib/solana/accounts xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
|
||||
sudo mount -a
|
||||
|
||||
# systemd pokazuje hint po edycji fstab - to jest OK, ale zrob reload:
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
4) WAŻNE: ustaw ownera na zamontowanych filesystemach
|
||||
Po mount, root filesystemu jest domyslnie root:root 755, wiec validator uruchamiany jako solana nie zapisze ledger/accounts.
|
||||
|
||||
sudo chown solana:solana /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
sudo chmod 750 /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
|
||||
5) Weryfikacja
|
||||
df -hT /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
findmnt /var/lib/solana/ledger
|
||||
findmnt /var/lib/solana/accounts
|
||||
|
||||
stat -c '%U:%G %a %n' /var/lib/solana/ledger /var/lib/solana/accounts
|
||||
|
||||
Dla tego hosta (z wdrozenia):
|
||||
- /dev/nvme0n1p1 (LABEL=sol_ledger) UUID=56a2c342-3162-43f2-9649-e0dd25db870d
|
||||
- /dev/nvme0n1p2 (LABEL=sol_accts) UUID=ae5d6625-0910-489a-bbe3-6fd156439e03
|
||||
|
||||
79
doc/step-002b-agave-validator-systemd.txt
Normal file
79
doc/step-002b-agave-validator-systemd.txt
Normal file
@@ -0,0 +1,79 @@
|
||||
Step 002B: agave-validator pod systemd (RPC-only, non-voting) + fix na problemy z ExecStart
|
||||
|
||||
Ten krok naprawia 2 typowe bledy po wklejeniu unit-a:
|
||||
1) "error: The subcommand '\\' wasn't recognized"
|
||||
2) RPC nie startuje, bo ExecStart zostal "polamany" (brakuje --rpc-port albo --log ma sciezke do katalogu)
|
||||
|
||||
Przyczyna:
|
||||
- W unit file kontynuacja linii w ExecStart to JEDEN backslash na koncu linii: `\`
|
||||
- NIE uzywaj `\\` (dwa backslashe) bo wtedy do programu trafia literalny argument `\` i validator pada.
|
||||
- Dodatkowo: unikaj bardzo dlugich linii, bo clipboard/paste potrafi je rozciac (i systemd wtedy ignoruje reszte).
|
||||
|
||||
0) Stop/wycisz restart loop
|
||||
sudo systemctl disable --now agave-validator
|
||||
sudo systemctl reset-failed agave-validator
|
||||
|
||||
1) Wrzuc poprawny unit (wielolinijkowy ExecStart z poprawna kontynuacja `\`)
|
||||
sudo tee /etc/systemd/system/agave-validator.service >/dev/null <<'EOF'
|
||||
[Unit]
|
||||
Description=Agave validator (mainnet, RPC-only, non-voting)
|
||||
After=network-online.target wg-quick@wg0.service
|
||||
Wants=network-online.target wg-quick@wg0.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=solana
|
||||
Group=solana
|
||||
WorkingDirectory=/var/lib/solana
|
||||
Environment=RUST_LOG=info
|
||||
LimitNOFILE=1048576
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
TimeoutStopSec=120
|
||||
ExecStart=/opt/solana/bin/agave-validator \
|
||||
--identity /var/lib/solana/identity.json \
|
||||
--no-voting \
|
||||
--private-rpc \
|
||||
--ledger /var/lib/solana/ledger \
|
||||
--accounts /var/lib/solana/accounts \
|
||||
--log /var/log/solana/validator.log \
|
||||
--bind-address 0.0.0.0 \
|
||||
--rpc-bind-address 10.66.66.1 \
|
||||
--rpc-port 8899 \
|
||||
--dynamic-port-range 8000-8025 \
|
||||
--entrypoint entrypoint.mainnet-beta.solana.com:8001 \
|
||||
--entrypoint entrypoint2.mainnet-beta.solana.com:8001 \
|
||||
--entrypoint entrypoint3.mainnet-beta.solana.com:8001 \
|
||||
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d \
|
||||
--full-rpc-api \
|
||||
--limit-ledger-size 50000000 \
|
||||
--geyser-plugin-config /etc/solana/yellowstone-geyser.json
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/var/lib/solana/ledger /var/lib/solana/accounts /var/log/solana /var/lib/solana
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
2) Reload + start
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now agave-validator
|
||||
|
||||
3) Weryfikacja (na hoście)
|
||||
# Sprawdz, czy systemd widzi kompletne argv[] (ma byc --rpc-port 8899 i poprawna sciezka --log):
|
||||
systemctl show agave-validator -p ExecStart --no-pager
|
||||
|
||||
sudo systemctl status agave-validator --no-pager -l
|
||||
|
||||
# Czy porty sluchaja (RPC/WS na wg0):
|
||||
sudo ss -lntp | egrep ':8899\\b|:8900\\b' || true
|
||||
|
||||
# getHealth moze zwracac blad dopoki node nie dogoni klastra, wiec sprawdz getVersion:
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getVersion"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
|
||||
# Logi:
|
||||
sudo journalctl -u agave-validator -n 120 --no-pager
|
||||
34
doc/step-002c-genesis-download-mainnet.txt
Normal file
34
doc/step-002c-genesis-download-mainnet.txt
Normal file
@@ -0,0 +1,34 @@
|
||||
Step 002C: Download genesis (mainnet) do /var/lib/solana/ledger (fix na "failed to open genesis")
|
||||
|
||||
Objaw w logach:
|
||||
Failed to load genesis_config ... missing genesis.bin
|
||||
Extracting "/var/lib/solana/ledger/genesis.tar.bz2"...
|
||||
Failed to start validator: failed to open genesis ... No such file or directory
|
||||
|
||||
Fix:
|
||||
- Zaciagnij genesis archive na dysk i zrestartuj serwis.
|
||||
|
||||
1) Zatrzymaj validator (zeby nie robil restart loop)
|
||||
sudo systemctl stop agave-validator
|
||||
|
||||
2) Pobierz genesis.tar.bz2 do ledger (jako user `solana`)
|
||||
sudo -u solana -H bash -lc '
|
||||
cd /var/lib/solana/ledger
|
||||
rm -f genesis.tar.bz2 genesis.bin
|
||||
curl -sSfL https://api.mainnet-beta.solana.com/genesis.tar.bz2 -o genesis.tar.bz2
|
||||
ls -lh genesis.tar.bz2
|
||||
'
|
||||
|
||||
Oczekiwane:
|
||||
- plik genesis.tar.bz2 istnieje (ok. kilkadziesiat KB)
|
||||
|
||||
3) Start + logi
|
||||
sudo systemctl start agave-validator
|
||||
systemctl status agave-validator --no-pager -l
|
||||
sudo journalctl -u agave-validator -n 80 --no-pager
|
||||
|
||||
4) RPC check (local/WG)
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getVersion"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
|
||||
49
doc/step-002d-snapshot-download-mainnet.txt
Normal file
49
doc/step-002d-snapshot-download-mainnet.txt
Normal file
@@ -0,0 +1,49 @@
|
||||
Step 002D: Download snapshot (mainnet) do /var/lib/solana/ledger (wymagane na mainnet)
|
||||
|
||||
Objaw (bez snapshot):
|
||||
Failed to start validator: ... did you forget to provide a snapshot
|
||||
|
||||
UWAGA:
|
||||
- Snapshot full jest DUZY (setki GB). Ten krok moze trwac dlugo.
|
||||
- Pobieramy full + incremental z oficjalnego endpointu (api.mainnet-beta.solana.com).
|
||||
|
||||
0) Stop validator
|
||||
sudo systemctl stop agave-validator
|
||||
|
||||
1) (Opcjonalnie, polecane) wyczysc stare dane ledger (zostaw genesis.*)
|
||||
sudo -u solana -H bash -lc '
|
||||
cd /var/lib/solana/ledger
|
||||
rm -rf rocksdb snapshots accounts_index accounts_hash_cache admin.rpc ledger.lock
|
||||
ls -la | head
|
||||
'
|
||||
|
||||
2) Pobierz FULL + INCREMENTAL snapshot do ledger (jako user `solana`)
|
||||
sudo -u solana -H bash -lc '
|
||||
set -euo pipefail
|
||||
cd /var/lib/solana/ledger
|
||||
|
||||
FULL_REL="$(curl -sI https://api.mainnet-beta.solana.com/snapshot.tar.bz2 | awk "/^location:/{print \\$2}" | tr -d "\\r")"
|
||||
INC_REL="$(curl -sI https://api.mainnet-beta.solana.com/incremental-snapshot.tar.bz2 | awk "/^location:/{print \\$2}" | tr -d "\\r")"
|
||||
|
||||
echo "FULL_REL=$FULL_REL"
|
||||
echo "INC_REL=$INC_REL"
|
||||
|
||||
FULL_URL="https://api.mainnet-beta.solana.com${FULL_REL}"
|
||||
INC_URL="https://api.mainnet-beta.solana.com${INC_REL}"
|
||||
|
||||
FULL_FILE="$(basename "$FULL_REL")"
|
||||
INC_FILE="$(basename "$INC_REL")"
|
||||
|
||||
echo "Downloading FULL: $FULL_FILE"
|
||||
curl -fSL "$FULL_URL" -o "$FULL_FILE"
|
||||
ls -lh "$FULL_FILE"
|
||||
|
||||
echo "Downloading INC: $INC_FILE"
|
||||
curl -fSL "$INC_URL" -o "$INC_FILE"
|
||||
ls -lh "$INC_FILE"
|
||||
'
|
||||
|
||||
3) Start validator + logi
|
||||
sudo systemctl start agave-validator
|
||||
sudo journalctl -u agave-validator -n 80 --no-pager
|
||||
|
||||
71
doc/step-002e-agave-validator-run-cli.txt
Normal file
71
doc/step-002e-agave-validator-run-cli.txt
Normal file
@@ -0,0 +1,71 @@
|
||||
Step 002E: Start agave-validator z linii polecen (test przed systemd)
|
||||
|
||||
Cel:
|
||||
- Odpalic `agave-validator` recznie (w tmux), potwierdzic ze RPC dziala.
|
||||
- Dopiero potem przepiac na systemd (Step 002B).
|
||||
|
||||
Preconditions:
|
||||
- Masz genesis + snapshoty:
|
||||
- doc/step-002c-genesis-download-mainnet.txt
|
||||
- doc/step-002d-snapshot-download-mainnet.txt
|
||||
- WireGuard dziala (RPC ma byc prywatne po wg0).
|
||||
|
||||
0) Upewnij sie, ze systemd nie trzyma procesu na portach
|
||||
```bash
|
||||
sudo systemctl stop agave-validator || true
|
||||
sudo systemctl disable agave-validator || true
|
||||
```
|
||||
|
||||
1) Odpal w tmux (zeby proces nie zginal po rozlaczeniu SSH)
|
||||
```bash
|
||||
tmux new -s agave-rpc
|
||||
```
|
||||
|
||||
2) Start (dzialajaca komenda z wdrozenia)
|
||||
```bash
|
||||
sudo bash -lc '
|
||||
set -euo pipefail
|
||||
ulimit -n 1048576
|
||||
exec sudo -u solana -H /opt/solana/bin/agave-validator \
|
||||
--identity /var/lib/solana/identity.json \
|
||||
--no-voting \
|
||||
--private-rpc \
|
||||
--ledger /var/lib/solana/ledger \
|
||||
--accounts /var/lib/solana/accounts \
|
||||
--log - \
|
||||
--bind-address 0.0.0.0 \
|
||||
--rpc-bind-address 10.66.66.1 \
|
||||
--rpc-port 8899 \
|
||||
--dynamic-port-range 8000-8020 \
|
||||
--entrypoint entrypoint.mainnet-beta.solana.com:8001 \
|
||||
--entrypoint entrypoint2.mainnet-beta.solana.com:8001 \
|
||||
--entrypoint entrypoint3.mainnet-beta.solana.com:8001 \
|
||||
--expected-genesis-hash 5eykt4UsFv8P8NJdTREpY1vzqKqZKvdpKuc147dw2N9d \
|
||||
--full-rpc-api \
|
||||
--limit-ledger-size 50000000
|
||||
'
|
||||
```
|
||||
|
||||
3) Weryfikacja (w drugim oknie / innym terminalu)
|
||||
```bash
|
||||
ss -lnt | egrep ':8899\\b|:8900\\b' || true
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getVersion"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
```
|
||||
|
||||
4) Detach z tmux
|
||||
- `Ctrl+b d`
|
||||
|
||||
5) Stop (gdy chcesz zatrzymac recznie)
|
||||
- `tmux attach -t agave-rpc`
|
||||
- `Ctrl+c` (zatrzyma validator)
|
||||
- `exit`
|
||||
|
||||
Next:
|
||||
- Jak komenda dziala stabilnie, wrzuc unit i odpal przez systemd:
|
||||
doc/step-002b-agave-validator-systemd.txt
|
||||
|
||||
70
doc/step-002f-agave-build-from-source-latest.txt
Normal file
70
doc/step-002f-agave-build-from-source-latest.txt
Normal file
@@ -0,0 +1,70 @@
|
||||
Step 002F: Build Agave (agave-validator) z najnowszych zrodel (pin do releasu) i instalacja do /opt/solana/bin
|
||||
|
||||
Cel:
|
||||
- Zbudowac `agave-validator` ze zrodel z pinu do konkretnego releasu (stable dla mainnet).
|
||||
- Nie robic "floating HEAD" (zeby nie wjechal update bez kontroli).
|
||||
|
||||
Konwencja:
|
||||
- "najnowsza wersja" = najnowszy stabilny release dla mainnet z GitHub Releases repo `anza-xyz/agave`.
|
||||
(Zweryfikuj przed startem: tag typu v3.0.x.)
|
||||
|
||||
0) Packages (build deps)
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
build-essential pkg-config libssl-dev \
|
||||
clang llvm-dev libclang-dev \
|
||||
cmake protobuf-compiler libudev-dev zlib1g-dev \
|
||||
curl jq git ca-certificates
|
||||
|
||||
1) Uzytkownik + katalogi
|
||||
sudo getent group solana >/dev/null || sudo groupadd --system solana
|
||||
sudo id -u solana >/dev/null 2>&1 || sudo useradd --system --gid solana \
|
||||
--home-dir /var/lib/solana --create-home --shell /bin/bash solana
|
||||
|
||||
sudo install -d -o root -g root -m 0755 /opt/solana/bin
|
||||
sudo install -d -o solana -g solana -m 0750 /var/lib/solana/src
|
||||
|
||||
2) Wybierz tag releasu (stable dla mainnet)
|
||||
# Opcja A (manual): sprawdz Releases i wpisz tag, np.:
|
||||
AGAVE_VERSION_TAG="v3.0.13"
|
||||
|
||||
# Opcja B (CLI): pobierz liste tagow i wybierz najwyzszy semver (bez rc/beta)
|
||||
# (Uwaga: to korzysta z network; jesli nie masz neta - uzyj opcji A.)
|
||||
AGAVE_VERSION_TAG="$(
|
||||
git ls-remote --refs --tags https://github.com/anza-xyz/agave.git 'v*' \
|
||||
| awk -F/ '{print $NF}' \
|
||||
| rg -v '(rc|beta|alpha)' \
|
||||
| sort -V \
|
||||
| tail -n1
|
||||
)"
|
||||
echo "AGAVE_VERSION_TAG=$AGAVE_VERSION_TAG"
|
||||
|
||||
3) Rust toolchain
|
||||
# Rekomendacja: uzyj rustup (Agave ma pinned toolchain w repo).
|
||||
sudo -u solana -H bash -lc 'command -v rustup >/dev/null 2>&1 || (curl -sSfL https://sh.rustup.rs | sh -s -- -y)'
|
||||
|
||||
4) Clone + build (pin do tagu)
|
||||
AGAVE_SRC_DIR="/var/lib/solana/src/agave-${AGAVE_VERSION_TAG}"
|
||||
|
||||
sudo -u solana -H bash -lc "
|
||||
set -euo pipefail
|
||||
export PATH=/var/lib/solana/.cargo/bin:\$PATH
|
||||
rm -rf \"${AGAVE_SRC_DIR}\"
|
||||
git clone --depth 1 --branch \"${AGAVE_VERSION_TAG}\" https://github.com/anza-xyz/agave.git \"${AGAVE_SRC_DIR}\"
|
||||
cd \"${AGAVE_SRC_DIR}\"
|
||||
|
||||
# Agave ma wrapper `./cargo`, ktory bierze pinned toolchain z repo.
|
||||
./cargo build --release -p agave-validator
|
||||
"
|
||||
|
||||
5) Install do /opt/solana/bin
|
||||
sudo install -o root -g root -m 0755 \
|
||||
"${AGAVE_SRC_DIR}/target/release/agave-validator" \
|
||||
/opt/solana/bin/agave-validator
|
||||
|
||||
/opt/solana/bin/agave-validator --version
|
||||
|
||||
6) Uwagi (operacyjne)
|
||||
- Jesli budujesz pluginy (Geyser), trzymaj je na tym samym pinned releasie i najlepiej w tej samej "epoce" API.
|
||||
- Po update do nowego releasu: buduj validator + pluginy ponownie i restartuj systemd.
|
||||
|
||||
264
doc/step-002g-mpabi-runbook-agave-rpc-geyser-postgres.txt
Normal file
264
doc/step-002g-mpabi-runbook-agave-rpc-geyser-postgres.txt
Normal file
@@ -0,0 +1,264 @@
|
||||
Runbook (mpabi): Agave RPC (mainnet, private) + Geyser plugin -> Postgres
|
||||
|
||||
Cel:
|
||||
- Na mpabi odpalic prywatny Agave RPC (non-voting) po WireGuard (wg0).
|
||||
- Zbudowac Agave ze zrodel (pinned release).
|
||||
- (Po starcie) dodac Geyser plugin zapisujacy do PostgreSQL.
|
||||
|
||||
Skrot (kolejnosc):
|
||||
1) Preconditions: WG + UFW (SSH tylko po wg0).
|
||||
2) Storage: SOLACCT->/var/lib/solana/accounts, SOLLEDG->/var/lib/solana/ledger (+ decyzja: wipe czy kontynuacja).
|
||||
3) Pakiety + sysctl/limits.
|
||||
4) /opt/solana/bin: Anza tools (solana-keygen itd.).
|
||||
5) Build Agave ze zrodel (pin tag) + install do /opt/solana/bin/agave-validator.
|
||||
6) Identity (0600).
|
||||
7) Genesis + snapshot (mainnet).
|
||||
8) Systemd: agave-validator (User=solana) + RPC smoke test po WG.
|
||||
9) Geyser plugin -> Postgres + dopiecie do systemd + weryfikacja.
|
||||
|
||||
Konwencja:
|
||||
- Komendy uruchamiasz jako user: `user` (z sudo).
|
||||
- Validator dziala jako uzytkownik systemowy: `solana`.
|
||||
- Biny instalujemy do /opt/solana/bin (root-owned, stabilne sciezki).
|
||||
- Dane runtime:
|
||||
- ledger: /var/lib/solana/ledger (SOLLEDG = /dev/nvme1n1p1)
|
||||
- accounts: /var/lib/solana/accounts (SOLACCT = /dev/nvme0n1p1)
|
||||
|
||||
Parametry hosta (mpabi):
|
||||
- Public iface: enp6s0, public IP: 149.50.116.219
|
||||
- WireGuard iface: wg0, WG IP: 10.66.66.1
|
||||
- RPC: 8899, WS: 8900
|
||||
- Cluster inbound UDP: 8000-8020 (na public)
|
||||
|
||||
=====================
|
||||
0) Preconditions
|
||||
=====================
|
||||
1) WireGuard up:
|
||||
ip -4 a show wg0
|
||||
wg show
|
||||
|
||||
2) SSH powinno byc tylko po wg0 (defense-in-depth):
|
||||
sudo ufw status verbose
|
||||
|
||||
3) Upewnij sie, ze masz sudo do operacji operatorskich (systemctl/journalctl/ufw).
|
||||
|
||||
=====================
|
||||
1) Storage: mount SOLACCT/SOLLEDG
|
||||
=====================
|
||||
UWAGA: ten wariant NIE formatuje dyskow (bez kasowania), tylko montuje istniejace XFS.
|
||||
|
||||
UUIDs (wg lsblk -f):
|
||||
- SOLACCT: 055bf4e7-00f0-45a0-a995-2413b14428c4 (/dev/nvme0n1p1)
|
||||
- SOLLEDG: 6ac7a8c6-efc7-4803-a3dc-a66ea2ef0125 (/dev/nvme1n1p1)
|
||||
|
||||
1) (Opcjonalnie) obejrzyj co zajmuje SOLLEDG (zanim podepniesz pod /var/lib/solana/ledger):
|
||||
sudo mkdir -p /mnt/solledg
|
||||
sudo mount /dev/nvme1n1p1 /mnt/solledg
|
||||
sudo du -sh /mnt/solledg/* 2>/dev/null | sort -h | tail
|
||||
sudo umount /mnt/solledg
|
||||
|
||||
2) Utworz usera solana + mountpointy:
|
||||
sudo getent group solana >/dev/null || sudo groupadd --system solana
|
||||
sudo id -u solana >/dev/null 2>&1 || sudo useradd --system --gid solana \
|
||||
--home-dir /var/lib/solana --create-home --shell /bin/bash solana
|
||||
|
||||
sudo install -d -o solana -g solana -m 0750 /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
3) Dodaj wpisy do /etc/fstab (sprawdz czy nie ma juz duplikatow):
|
||||
grep -nE '(/var/lib/solana/(accounts|ledger))' /etc/fstab || true
|
||||
|
||||
UUID_ACCTS="055bf4e7-00f0-45a0-a995-2413b14428c4"
|
||||
UUID_LEDGER="6ac7a8c6-efc7-4803-a3dc-a66ea2ef0125"
|
||||
|
||||
echo "UUID=$UUID_ACCTS /var/lib/solana/accounts xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
echo "UUID=$UUID_LEDGER /var/lib/solana/ledger xfs noatime,nodiratime 0 2" | sudo tee -a /etc/fstab >/dev/null
|
||||
|
||||
sudo mount -a
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
4) Permissions (krytyczne):
|
||||
sudo chown solana:solana /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
sudo chmod 750 /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
5) Verify:
|
||||
df -hT /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
findmnt /var/lib/solana/accounts
|
||||
findmnt /var/lib/solana/ledger
|
||||
stat -c '%U:%G %a %n' /var/lib/solana/accounts /var/lib/solana/ledger
|
||||
|
||||
DECYZJA: co robimy z SOLLEDG (63% used)?
|
||||
- Opcja A: zostawiamy (jesli to stary ledger Solany i chcesz kontynuowac).
|
||||
- Opcja B: czyscimy i robimy sync od nowa (destrukcyjne, rekomendowane dla powtarzalnosci):
|
||||
sudo systemctl stop agave-validator 2>/dev/null || true
|
||||
sudo -u solana -H bash -lc '
|
||||
cd /var/lib/solana/ledger
|
||||
rm -rf rocksdb snapshots accounts_index accounts_hash_cache admin.rpc ledger.lock
|
||||
'
|
||||
|
||||
=====================
|
||||
2) Pakiety + sysctl/limits
|
||||
=====================
|
||||
1) Pakiety:
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
chrony curl jq git ca-certificates ripgrep \
|
||||
smartmontools nvme-cli \
|
||||
build-essential pkg-config libssl-dev \
|
||||
clang llvm-dev libclang-dev \
|
||||
cmake protobuf-compiler libudev-dev zlib1g-dev
|
||||
|
||||
2) Sysctl:
|
||||
sudo tee /etc/sysctl.d/90-agave.conf >/dev/null <<'EOF'
|
||||
fs.file-max = 2000000
|
||||
vm.max_map_count = 1000000
|
||||
net.core.somaxconn = 65535
|
||||
net.core.netdev_max_backlog = 250000
|
||||
net.core.rmem_max = 134217728
|
||||
net.core.wmem_max = 134217728
|
||||
EOF
|
||||
sudo sysctl --system
|
||||
|
||||
3) ulimit:
|
||||
sudo tee /etc/security/limits.d/90-solana.conf >/dev/null <<'EOF'
|
||||
* soft nofile 1000000
|
||||
* hard nofile 1000000
|
||||
EOF
|
||||
|
||||
=====================
|
||||
3) Firewall (UFW) dla prywatnego RPC
|
||||
=====================
|
||||
Docelowo:
|
||||
- SSH 22/tcp: tylko wg0
|
||||
- RPC 8899/tcp, WS 8900/tcp: tylko wg0 (dodatkowo bind na 10.66.66.1)
|
||||
- WireGuard: 51820/udp (public)
|
||||
- Cluster sync: 8000-8020/udp (public)
|
||||
|
||||
Komendy (jesli jeszcze nie ustawione):
|
||||
sudo ufw default deny incoming
|
||||
sudo ufw default allow outgoing
|
||||
sudo ufw allow 51820/udp
|
||||
|
||||
sudo ufw allow in on wg0 to any port 22 proto tcp
|
||||
sudo ufw deny in on enp6s0 to any port 22 proto tcp
|
||||
|
||||
sudo ufw allow in on wg0 to any port 8899 proto tcp
|
||||
sudo ufw allow in on wg0 to any port 8900 proto tcp
|
||||
sudo ufw deny in on enp6s0 to any port 8899 proto tcp
|
||||
sudo ufw deny in on enp6s0 to any port 8900 proto tcp
|
||||
|
||||
sudo ufw allow in on enp6s0 to any port 8000:8020 proto udp
|
||||
|
||||
sudo ufw enable
|
||||
sudo ufw status verbose
|
||||
|
||||
=====================
|
||||
4) Instalacja narzedzi Solana/Anza do /opt/solana/bin
|
||||
=====================
|
||||
1) Installer (jako solana):
|
||||
sudo install -d -o root -g root -m 0755 /opt/solana/bin
|
||||
sudo -u solana -H bash -lc 'sh -c "$(curl -sSfL https://release.anza.xyz/stable/install)"'
|
||||
|
||||
2) Skopiuj biny do /opt/solana/bin:
|
||||
sudo cp -a /var/lib/solana/.local/share/solana/install/active_release/bin/. /opt/solana/bin/
|
||||
sudo find /opt/solana/bin -maxdepth 1 -type f -exec chmod 0755 {} \;
|
||||
|
||||
3) Verify:
|
||||
/opt/solana/bin/solana --version || true
|
||||
/opt/solana/bin/solana-keygen --version
|
||||
|
||||
=====================
|
||||
5) Build + install agave-validator (pinned latest stable)
|
||||
=====================
|
||||
Patrz:
|
||||
- trade-iac/doc/step-002f-agave-build-from-source-latest.txt
|
||||
|
||||
Skrót:
|
||||
1) Wybierz TAG (np. v3.0.x) i zbuduj jako solana do /var/lib/solana/src/
|
||||
2) Zainstaluj binarke do /opt/solana/bin/agave-validator
|
||||
3) Verify:
|
||||
/opt/solana/bin/agave-validator --version
|
||||
|
||||
=====================
|
||||
6) Identity
|
||||
=====================
|
||||
1) Utworz raz:
|
||||
if [ ! -f /var/lib/solana/identity.json ]; then
|
||||
sudo -u solana -H /opt/solana/bin/solana-keygen new --no-passphrase -o /var/lib/solana/identity.json
|
||||
sudo chown solana:solana /var/lib/solana/identity.json
|
||||
sudo chmod 0600 /var/lib/solana/identity.json
|
||||
fi
|
||||
|
||||
2) Zapisz pubkey do notatek:
|
||||
sudo -u solana -H /opt/solana/bin/solana-keygen pubkey /var/lib/solana/identity.json
|
||||
|
||||
=====================
|
||||
7) Genesis + snapshot (mainnet)
|
||||
=====================
|
||||
1) Genesis:
|
||||
patrz: trade-iac/doc/step-002c-genesis-download-mainnet.txt
|
||||
|
||||
2) Snapshot (wymagany na mainnet):
|
||||
patrz: trade-iac/doc/step-002d-snapshot-download-mainnet.txt
|
||||
|
||||
=====================
|
||||
8) Systemd: agave-validator (RPC-only)
|
||||
=====================
|
||||
Unit template:
|
||||
- trade-iac/doc/step-002b-agave-validator-systemd.txt
|
||||
|
||||
Wazne:
|
||||
- `User=solana`
|
||||
- `--rpc-bind-address 10.66.66.1`
|
||||
- `--private-rpc`
|
||||
- nie uzywaj `\\` w ExecStart (tylko pojedynczy `\` na koncu linii)
|
||||
|
||||
Start/verify:
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now agave-validator
|
||||
systemctl is-active agave-validator
|
||||
sudo journalctl -u agave-validator -n 120 --no-pager
|
||||
|
||||
RPC smoke test (po WG):
|
||||
curl -sS -H 'content-type: application/json' \
|
||||
--data '{"jsonrpc":"2.0","id":1,"method":"getVersion"}' \
|
||||
http://10.66.66.1:8899 | jq .
|
||||
|
||||
Manual run (exports + komenda w plikach na mpabi):
|
||||
# Zawiera "solana export" (PATH + SOLANA_RPC_URL) i helper `agave_run()`:
|
||||
# - /etc/solana/agave-env.sh
|
||||
# - /etc/solana/agave-validator.args
|
||||
sudo -u solana -H bash -lc 'source /etc/solana/agave-env.sh; agave_run'
|
||||
|
||||
=====================
|
||||
9) Geyser plugin -> PostgreSQL
|
||||
=====================
|
||||
Krok po kroku:
|
||||
- trade-iac/doc/step-003-geyser-plugin-postgres.txt
|
||||
|
||||
Skrót:
|
||||
1) Postgres:
|
||||
sudo apt-get install -y postgresql
|
||||
sudo systemctl enable --now postgresql
|
||||
|
||||
2) DB/user:
|
||||
sudo -u postgres psql -v ON_ERROR_STOP=1 <<'SQL'
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'geyser') THEN
|
||||
CREATE ROLE geyser LOGIN PASSWORD 'CHANGE_ME';
|
||||
END IF;
|
||||
END $$;
|
||||
CREATE DATABASE geyser OWNER geyser;
|
||||
SQL
|
||||
|
||||
3) Build plugin (.so) i instaluj do /opt/solana/plugins/
|
||||
4) Skopiuj example JSON config do /etc/solana/geyser-postgres.json, ustaw DSN, selektory.
|
||||
5) Dopisz do unit ExecStart:
|
||||
--geyser-plugin-config /etc/solana/geyser-postgres.json
|
||||
i zrestartuj:
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart agave-validator
|
||||
|
||||
Verify:
|
||||
sudo journalctl -u agave-validator -n 200 --no-pager | rg -i 'geyser|plugin|postgres' || true
|
||||
sudo -u postgres psql -d geyser -c '\\dt' | head
|
||||
122
doc/step-003-geyser-plugin-postgres.txt
Normal file
122
doc/step-003-geyser-plugin-postgres.txt
Normal file
@@ -0,0 +1,122 @@
|
||||
Step 003: Geyser plugin -> PostgreSQL (Agave) na prywatnym RPC node
|
||||
|
||||
Cel:
|
||||
- Wlaczyc Geyser plugin zapisujacy eventy do Postgresa.
|
||||
- Na poczatek: dzialajace end-to-end (plugin laduje sie w validatorze + dane wpadaja do DB).
|
||||
|
||||
Wazne ryzyka:
|
||||
- Pisanie wszystkiego do Postgresa na mainnet generuje OGROMNY write load.
|
||||
Jezeli DB bedzie na tym samym hoście co validator, latwo o IO contention i spadek performance sync.
|
||||
- Rekomendacja: startuj od selektorow (filtrow), np. tylko programy ktore bot potrzebuje.
|
||||
|
||||
0) Preconditions
|
||||
- Agave dziala pod systemd:
|
||||
- doc/step-002b-agave-validator-systemd.txt
|
||||
- Masz pinned Agave release (tag) i go nie zmieniasz "w locie":
|
||||
- doc/step-002f-agave-build-from-source-latest.txt
|
||||
|
||||
1) Postgres (lokalnie, minimalny setup)
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y postgresql postgresql-contrib
|
||||
sudo systemctl enable --now postgresql
|
||||
|
||||
# DB + user (dostosuj haslo; nie commituj do repo)
|
||||
sudo -u postgres psql -v ON_ERROR_STOP=1 <<'SQL'
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'geyser') THEN
|
||||
CREATE ROLE geyser LOGIN PASSWORD 'CHANGE_ME';
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
CREATE DATABASE geyser OWNER geyser;
|
||||
SQL
|
||||
|
||||
2) Build plugin ze zrodel (Anza Agave plugin Postgres)
|
||||
# Repo:
|
||||
# https://github.com/anza-xyz/agave-geyser-plugin-postgres
|
||||
#
|
||||
# Uwaga: plugin i validator musza byc kompatybilne wersjami interfejsu.
|
||||
# Najbezpieczniej: build pluginu po update Agave i test "plugin loaded".
|
||||
#
|
||||
# Opcjonalnie: pin plugin do tagu/commit (np. pod ten sam release co Agave).
|
||||
# PLUGIN_REF="main" # albo tag/commit
|
||||
|
||||
sudo install -d -o root -g root -m 0755 /opt/solana/plugins
|
||||
sudo install -d -o solana -g solana -m 0750 /var/lib/solana/src
|
||||
|
||||
PLUGIN_SRC_DIR="/var/lib/solana/src/agave-geyser-plugin-postgres"
|
||||
|
||||
sudo -u solana -H bash -lc "
|
||||
set -euo pipefail
|
||||
export PATH=/var/lib/solana/.cargo/bin:\$PATH
|
||||
rm -rf \"${PLUGIN_SRC_DIR}\"
|
||||
git clone https://github.com/anza-xyz/agave-geyser-plugin-postgres.git \"${PLUGIN_SRC_DIR}\"
|
||||
cd \"${PLUGIN_SRC_DIR}\"
|
||||
if [ \"${PLUGIN_REF:-main}\" != \"main\" ]; then git checkout \"${PLUGIN_REF}\"; fi
|
||||
cargo build --release
|
||||
"
|
||||
|
||||
# Typowa nazwa .so (sprawdz `ls target/release/*.so`):
|
||||
sudo cp -a "${PLUGIN_SRC_DIR}"/target/release/*.so /opt/solana/plugins/
|
||||
sudo chmod 0755 /opt/solana/plugins/*.so
|
||||
|
||||
3) Schema w Postgres
|
||||
# Plugin repo ma SQL do stworzenia schematu. Najpierw znajdz plik:
|
||||
sudo -u solana -H bash -lc '
|
||||
cd /var/lib/solana/src/agave-geyser-plugin-postgres
|
||||
find . -maxdepth 3 -type f -iname \"*schema*.sql\" -o -type f -iname \"create*.sql\" | sort
|
||||
'
|
||||
|
||||
# Potem wykonaj jako postgres na DB geyser (podmien sciezke na znaleziony plik):
|
||||
# sudo -u postgres psql -d geyser -f /var/lib/solana/src/agave-geyser-plugin-postgres/scripts/create_schema.sql
|
||||
|
||||
4) Konfig pluginu (JSON)
|
||||
# Najbezpieczniej: skopiuj example config z repo pluginu i zmien tylko:
|
||||
# - sciezke do .so
|
||||
# - connection string
|
||||
# - selektory (na poczatek mozesz dac \"*\" tylko na smoke test, potem zawęz)
|
||||
#
|
||||
# Znajdz przyklad w repo:
|
||||
sudo -u solana -H bash -lc '
|
||||
cd /var/lib/solana/src/agave-geyser-plugin-postgres
|
||||
find . -maxdepth 4 -type f -iname \"*.json\" | sort
|
||||
rg -n \"connection_str|libpath|accounts_selector|transaction_selector|selector\" -S . || true
|
||||
'
|
||||
|
||||
sudo install -d -o root -g root -m 0755 /etc/solana
|
||||
|
||||
# Start od skopiowania example (podmien sciezke jesli repo ma inna nazwe pliku):
|
||||
# sudo cp -a /var/lib/solana/src/agave-geyser-plugin-postgres/config.json /etc/solana/geyser-postgres.json
|
||||
|
||||
# Uprawnienia: config zawiera haslo, ale validator (User=solana) musi go czytac.
|
||||
if [ -f /etc/solana/geyser-postgres.json ]; then
|
||||
sudo chown root:solana /etc/solana/geyser-postgres.json
|
||||
sudo chmod 0640 /etc/solana/geyser-postgres.json
|
||||
fi
|
||||
|
||||
5) Wlaczenie pluginu w agave-validator (systemd)
|
||||
# Sprawdz, jak na Twojej wersji nazywa sie flaga:
|
||||
/opt/solana/bin/agave-validator --help | rg -n \"geyser|plugin\" || true
|
||||
|
||||
# Docelowo (w unit file) dopisz:
|
||||
# --geyser-plugin-config /etc/solana/geyser-postgres.json
|
||||
#
|
||||
# Nastepnie:
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart agave-validator
|
||||
|
||||
6) Weryfikacja
|
||||
sudo journalctl -u agave-validator -n 200 --no-pager | rg -i \"geyser|plugin|postgres\" || true
|
||||
|
||||
# Czy w DB pojawiaja sie tabele (zalezy od schema.sql):
|
||||
sudo -u postgres psql -d geyser -c \"\\dt\" | head
|
||||
|
||||
# Czy rosną inserty (przyklad - dostosuj nazwy tabel):
|
||||
# sudo -u postgres psql -d geyser -c 'select count(*) from account;'
|
||||
|
||||
7) Performance (minimum sanity)
|
||||
- Jezeli validator zacznie znacząco odstawać, zacznij od:
|
||||
- zawężenia selektorow (unikaj \"*\" po smoke tescie)
|
||||
- zmniejszenia batch/threads
|
||||
- przeniesienia DB na osobny host (lub przynajmniej na osobny NVMe)
|
||||
93
scripts/wireguard/wg_client.py
Executable file
93
scripts/wireguard/wg_client.py
Executable file
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from datetime import datetime
|
||||
|
||||
WG_DIR = Path("/etc/wireguard")
|
||||
WG0 = WG_DIR / "wg0.conf"
|
||||
|
||||
|
||||
def require_root():
|
||||
if os.geteuid() != 0:
|
||||
print("ERROR: run as root (sudo).", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run(cmd, check=True, capture=False):
|
||||
return subprocess.run(
|
||||
cmd,
|
||||
check=check,
|
||||
text=True,
|
||||
stdout=subprocess.PIPE if capture else None,
|
||||
stderr=subprocess.PIPE if capture else None,
|
||||
)
|
||||
|
||||
|
||||
def backup_if_exists(p: Path):
|
||||
if p.exists():
|
||||
ts = datetime.now().strftime("%Y-%m-%d_%H%M%S")
|
||||
b = p.with_name(p.name + f".bak.{ts}")
|
||||
shutil.copy2(p, b)
|
||||
|
||||
|
||||
def cmd_install(src: str):
|
||||
srcp = Path(src)
|
||||
if not srcp.exists():
|
||||
raise RuntimeError(f"Missing source config: {srcp}")
|
||||
WG_DIR.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(WG_DIR, 0o700)
|
||||
backup_if_exists(WG0)
|
||||
shutil.copy2(srcp, WG0)
|
||||
os.chmod(WG0, 0o600)
|
||||
os.chown(WG0, 0, 0)
|
||||
print(f"OK: installed {WG0}")
|
||||
|
||||
|
||||
def cmd_up():
|
||||
run(["wg-quick", "up", "wg0"], check=True)
|
||||
|
||||
|
||||
def cmd_down():
|
||||
run(["wg-quick", "down", "wg0"], check=False)
|
||||
|
||||
|
||||
def cmd_status():
|
||||
run(["wg", "show"], check=False)
|
||||
run(["ip", "a", "show", "wg0"], check=False)
|
||||
|
||||
|
||||
def main():
|
||||
require_root()
|
||||
p = argparse.ArgumentParser(prog="wg_client.py")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
i = sub.add_parser("install")
|
||||
i.add_argument("config", help="Path to clientX.conf from server export")
|
||||
|
||||
sub.add_parser("up")
|
||||
sub.add_parser("down")
|
||||
sub.add_parser("status")
|
||||
|
||||
args = p.parse_args()
|
||||
|
||||
try:
|
||||
if args.cmd == "install":
|
||||
cmd_install(args.config)
|
||||
elif args.cmd == "up":
|
||||
cmd_up()
|
||||
elif args.cmd == "down":
|
||||
cmd_down()
|
||||
elif args.cmd == "status":
|
||||
cmd_status()
|
||||
except Exception as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
419
scripts/wireguard/wg_serverctl.py
Executable file
419
scripts/wireguard/wg_serverctl.py
Executable file
@@ -0,0 +1,419 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional, List
|
||||
|
||||
WG_KEY_RE = re.compile(r"^[A-Za-z0-9+/]{43}=$")
|
||||
|
||||
|
||||
@dataclass
|
||||
class Cfg:
|
||||
wg_dir: Path = Path("/etc/wireguard")
|
||||
clients_dir: Path = Path("/etc/wireguard/clients")
|
||||
server_key: Path = Path("/etc/wireguard/server.key")
|
||||
server_pub: Path = Path("/etc/wireguard/server.pub")
|
||||
wg0_conf: Path = Path("/etc/wireguard/wg0.conf")
|
||||
|
||||
wg_addr: str = "10.66.66.1/24"
|
||||
wg_port: int = 51820
|
||||
keepalive: int = 25
|
||||
|
||||
client_net_prefix: str = "10.66.66."
|
||||
client_ip_start: int = 2
|
||||
client_ip_end: int = 254
|
||||
|
||||
|
||||
def require_root():
|
||||
if os.geteuid() != 0:
|
||||
print("ERROR: run as root (sudo).", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run(
|
||||
cmd: List[str],
|
||||
capture: bool = False,
|
||||
check: bool = True,
|
||||
input_text: Optional[str] = None,
|
||||
) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
cmd,
|
||||
text=True,
|
||||
input=input_text,
|
||||
stdout=subprocess.PIPE if capture else None,
|
||||
stderr=subprocess.PIPE if capture else None,
|
||||
check=check,
|
||||
)
|
||||
|
||||
|
||||
def ensure_dirs(cfg: Cfg):
|
||||
cfg.wg_dir.mkdir(parents=True, exist_ok=True)
|
||||
cfg.clients_dir.mkdir(parents=True, exist_ok=True)
|
||||
os.chmod(cfg.wg_dir, 0o700)
|
||||
os.chmod(cfg.clients_dir, 0o700)
|
||||
|
||||
|
||||
def read_first_nonempty_line(p: Path) -> str:
|
||||
if not p.exists():
|
||||
return ""
|
||||
for line in p.read_text(encoding="utf-8", errors="ignore").splitlines():
|
||||
line = line.strip().replace("\r", "")
|
||||
if line:
|
||||
return line
|
||||
return ""
|
||||
|
||||
|
||||
def validate_key(k: str, what: str):
|
||||
if not WG_KEY_RE.match(k):
|
||||
raise ValueError(f"{what} has invalid WireGuard key format: {k!r}")
|
||||
|
||||
|
||||
def chmod600_root(p: Path):
|
||||
os.chmod(p, 0o600)
|
||||
os.chown(p, 0, 0)
|
||||
|
||||
|
||||
def backup_if_exists(p: Path):
|
||||
if p.exists():
|
||||
ts = datetime.now().strftime("%Y-%m-%d_%H%M%S")
|
||||
b = p.with_name(p.name + f".bak.{ts}")
|
||||
shutil.copy2(p, b)
|
||||
|
||||
|
||||
def gen_priv_key_to(path: Path):
|
||||
priv = run(["wg", "genkey"], capture=True).stdout.strip()
|
||||
validate_key(priv, "generated private key")
|
||||
path.write_text(priv + "\n", encoding="utf-8")
|
||||
chmod600_root(path)
|
||||
|
||||
|
||||
def pubkey_from_priv(priv: str) -> str:
|
||||
validate_key(priv, "private key")
|
||||
pub = run(["wg", "pubkey"], capture=True, input_text=priv + "\n").stdout.strip()
|
||||
validate_key(pub, "generated public key")
|
||||
return pub
|
||||
|
||||
|
||||
def detect_public_ip() -> str:
|
||||
# Works on Ubuntu/Debian in typical setups
|
||||
out = run(
|
||||
[
|
||||
"bash",
|
||||
"-lc",
|
||||
"ip route get 1.1.1.1 | awk '{for (i=1;i<=NF;i++) if ($i==\"src\") {print $(i+1); exit}}'",
|
||||
],
|
||||
capture=True,
|
||||
).stdout.strip()
|
||||
if not out:
|
||||
raise RuntimeError("Could not detect server public IP.")
|
||||
return out
|
||||
|
||||
|
||||
def restart_wg(unit: str = "wg-quick@wg0"):
|
||||
run(["systemctl", "restart", unit], capture=False, check=True)
|
||||
# optional status (non-fatal)
|
||||
try:
|
||||
run(["systemctl", "--no-pager", "-l", "status", unit], capture=False, check=False)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def list_clients(cfg: Cfg) -> List[str]:
|
||||
pubs = sorted(cfg.clients_dir.glob("*.pub"))
|
||||
return [p.stem for p in pubs]
|
||||
|
||||
|
||||
def used_octets(cfg: Cfg) -> set:
|
||||
s = set()
|
||||
for f in cfg.clients_dir.glob("*.ip"):
|
||||
v = read_first_nonempty_line(f)
|
||||
if v.isdigit():
|
||||
s.add(int(v))
|
||||
return s
|
||||
|
||||
|
||||
def alloc_octet(cfg: Cfg) -> int:
|
||||
used = used_octets(cfg)
|
||||
for i in range(cfg.client_ip_start, cfg.client_ip_end + 1):
|
||||
if i not in used:
|
||||
return i
|
||||
raise RuntimeError("No free IPs left in pool.")
|
||||
|
||||
|
||||
def gen_server(cfg: Cfg, force: bool):
|
||||
ensure_dirs(cfg)
|
||||
if not force and (cfg.server_key.exists() or cfg.server_pub.exists()):
|
||||
raise RuntimeError("Server key already exists. Use --force to overwrite.")
|
||||
gen_priv_key_to(cfg.server_key)
|
||||
priv = read_first_nonempty_line(cfg.server_key)
|
||||
pub = pubkey_from_priv(priv)
|
||||
cfg.server_pub.write_text(pub + "\n", encoding="utf-8")
|
||||
chmod600_root(cfg.server_pub)
|
||||
print("OK: generated server keys:")
|
||||
print(f" - {cfg.server_key} (PRIVATE)")
|
||||
print(f" - {cfg.server_pub} (PUBLIC)")
|
||||
print("Server public key:")
|
||||
print(pub)
|
||||
|
||||
|
||||
def gen_client(cfg: Cfg, name: str, force: bool):
|
||||
ensure_dirs(cfg)
|
||||
keyf = cfg.clients_dir / f"{name}.key"
|
||||
pubf = cfg.clients_dir / f"{name}.pub"
|
||||
if not force and (keyf.exists() or pubf.exists()):
|
||||
raise RuntimeError(f"Client {name} exists. Use --force to overwrite.")
|
||||
gen_priv_key_to(keyf)
|
||||
priv = read_first_nonempty_line(keyf)
|
||||
pub = pubkey_from_priv(priv)
|
||||
pubf.write_text(pub + "\n", encoding="utf-8")
|
||||
chmod600_root(pubf)
|
||||
print("OK: generated client keys:")
|
||||
print(f" - {keyf} (PRIVATE)")
|
||||
print(f" - {pubf} (PUBLIC)")
|
||||
print("Client public key:")
|
||||
print(pub)
|
||||
|
||||
|
||||
def generate_wg0_conf(cfg: Cfg):
|
||||
ensure_dirs(cfg)
|
||||
if not cfg.server_key.exists():
|
||||
raise RuntimeError("Missing server.key. Run gen-server first.")
|
||||
server_priv = read_first_nonempty_line(cfg.server_key)
|
||||
validate_key(server_priv, "server private key")
|
||||
|
||||
backup_if_exists(cfg.wg0_conf)
|
||||
|
||||
lines: List[str] = []
|
||||
lines += [
|
||||
"[Interface]",
|
||||
f"Address = {cfg.wg_addr}",
|
||||
f"ListenPort = {cfg.wg_port}",
|
||||
f"PrivateKey = {server_priv}",
|
||||
]
|
||||
|
||||
for name in sorted(list_clients(cfg)):
|
||||
pubf = cfg.clients_dir / f"{name}.pub"
|
||||
ipf = cfg.clients_dir / f"{name}.ip"
|
||||
if not ipf.exists():
|
||||
raise RuntimeError(f"Client {name} has no .ip file (run add {name}).")
|
||||
pub = read_first_nonempty_line(pubf)
|
||||
validate_key(pub, f"client {name} pubkey")
|
||||
octet = read_first_nonempty_line(ipf)
|
||||
if not octet.isdigit():
|
||||
raise RuntimeError(f"Invalid ip octet in {ipf}: {octet!r}")
|
||||
o = int(octet)
|
||||
if o < 2 or o > 254:
|
||||
raise RuntimeError(f"Invalid ip octet in {ipf}: {o}")
|
||||
lines += [
|
||||
"",
|
||||
"[Peer]",
|
||||
f"# {name}",
|
||||
f"PublicKey = {pub}",
|
||||
f"AllowedIPs = {cfg.client_net_prefix}{o}/32",
|
||||
f"PersistentKeepalive = {cfg.keepalive}",
|
||||
]
|
||||
|
||||
cfg.wg0_conf.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
chmod600_root(cfg.wg0_conf)
|
||||
print(f"OK: generated {cfg.wg0_conf}")
|
||||
|
||||
|
||||
def add_client(cfg: Cfg, name: str, pubkey: Optional[str], ip: Optional[str], no_restart: bool):
|
||||
ensure_dirs(cfg)
|
||||
pubf = cfg.clients_dir / f"{name}.pub"
|
||||
ipf = cfg.clients_dir / f"{name}.ip"
|
||||
|
||||
# resolve pubkey
|
||||
if pubkey:
|
||||
pubkey = pubkey.strip()
|
||||
validate_key(pubkey, "provided client public key")
|
||||
pubf.write_text(pubkey + "\n", encoding="utf-8")
|
||||
chmod600_root(pubf)
|
||||
else:
|
||||
if not pubf.exists():
|
||||
raise RuntimeError(f"Missing {pubf}. Run gen-client {name} or provide --pubkey.")
|
||||
pubkey = read_first_nonempty_line(pubf)
|
||||
validate_key(pubkey, f"client {name} pubkey")
|
||||
|
||||
# resolve ip octet
|
||||
if ip is None:
|
||||
octet = alloc_octet(cfg)
|
||||
else:
|
||||
ip = ip.strip()
|
||||
if ip.startswith(cfg.client_net_prefix):
|
||||
ip = ip[len(cfg.client_net_prefix) :]
|
||||
if not ip.isdigit():
|
||||
raise RuntimeError("--ip must be an octet (2..254) or full IP like 10.66.66.X")
|
||||
octet = int(ip)
|
||||
if octet < 2 or octet > 254:
|
||||
raise RuntimeError("--ip must be 2..254")
|
||||
if octet in used_octets(cfg):
|
||||
raise RuntimeError(f"IP octet {octet} already used")
|
||||
|
||||
ipf.write_text(f"{octet}\n", encoding="utf-8")
|
||||
chmod600_root(ipf)
|
||||
|
||||
print(f"OK: added peer {name} -> {cfg.client_net_prefix}{octet}/32")
|
||||
generate_wg0_conf(cfg)
|
||||
if not no_restart:
|
||||
restart_wg()
|
||||
|
||||
|
||||
def del_client(cfg: Cfg, name: str, no_restart: bool):
|
||||
ensure_dirs(cfg)
|
||||
pubf = cfg.clients_dir / f"{name}.pub"
|
||||
ipf = cfg.clients_dir / f"{name}.ip"
|
||||
# keep .key on purpose
|
||||
if pubf.exists():
|
||||
pubf.unlink()
|
||||
if ipf.exists():
|
||||
ipf.unlink()
|
||||
print(f"OK: removed peer {name} (pub/ip). Private key kept if existed.")
|
||||
generate_wg0_conf(cfg)
|
||||
if not no_restart:
|
||||
restart_wg()
|
||||
|
||||
|
||||
def export_client_conf(cfg: Cfg, name: str, endpoint: Optional[str]):
|
||||
ensure_dirs(cfg)
|
||||
keyf = cfg.clients_dir / f"{name}.key"
|
||||
ipf = cfg.clients_dir / f"{name}.ip"
|
||||
|
||||
if not keyf.exists():
|
||||
raise RuntimeError(f"Missing {keyf}. Run gen-client {name} on server or create key on client.")
|
||||
if not ipf.exists():
|
||||
raise RuntimeError(f"Missing {ipf}. Run add {name} first.")
|
||||
if not cfg.server_pub.exists():
|
||||
raise RuntimeError("Missing server.pub. Run gen-server first.")
|
||||
|
||||
client_priv = read_first_nonempty_line(keyf)
|
||||
validate_key(client_priv, f"client {name} private key")
|
||||
server_pub = read_first_nonempty_line(cfg.server_pub)
|
||||
validate_key(server_pub, "server public key")
|
||||
|
||||
octet = int(read_first_nonempty_line(ipf))
|
||||
if endpoint is None:
|
||||
endpoint = f"{detect_public_ip()}:{cfg.wg_port}"
|
||||
|
||||
out = cfg.clients_dir / f"{name}.conf"
|
||||
content = "\n".join(
|
||||
[
|
||||
"[Interface]",
|
||||
f"Address = {cfg.client_net_prefix}{octet}/32",
|
||||
f"PrivateKey = {client_priv}",
|
||||
"",
|
||||
"[Peer]",
|
||||
f"PublicKey = {server_pub}",
|
||||
f"Endpoint = {endpoint}",
|
||||
# Default: only reach the server's WG IP via tunnel.
|
||||
"AllowedIPs = 10.66.66.1/32",
|
||||
f"PersistentKeepalive = {cfg.keepalive}",
|
||||
"",
|
||||
]
|
||||
)
|
||||
out.write_text(content, encoding="utf-8")
|
||||
chmod600_root(out)
|
||||
print(f"OK: wrote {out}")
|
||||
|
||||
|
||||
def cmd_list(cfg: Cfg):
|
||||
ensure_dirs(cfg)
|
||||
names = list_clients(cfg)
|
||||
if not names:
|
||||
print("(no clients)")
|
||||
return
|
||||
for n in names:
|
||||
ipf = cfg.clients_dir / f"{n}.ip"
|
||||
octet = read_first_nonempty_line(ipf) if ipf.exists() else "?"
|
||||
print(f"{n}\t{cfg.client_net_prefix}{octet}/32")
|
||||
|
||||
|
||||
def cmd_show(cfg: Cfg, name: str):
|
||||
ensure_dirs(cfg)
|
||||
pubf = cfg.clients_dir / f"{name}.pub"
|
||||
ipf = cfg.clients_dir / f"{name}.ip"
|
||||
keyf = cfg.clients_dir / f"{name}.key"
|
||||
print(f"Name: {name}")
|
||||
if ipf.exists():
|
||||
print(f"IP: {cfg.client_net_prefix}{read_first_nonempty_line(ipf)}/32")
|
||||
else:
|
||||
print("IP: (not assigned)")
|
||||
if pubf.exists():
|
||||
print(f"PUB: {read_first_nonempty_line(pubf)}")
|
||||
else:
|
||||
print("PUB: (missing)")
|
||||
print(f"KEY: {'exists' if keyf.exists() else 'missing'} ({keyf})")
|
||||
|
||||
|
||||
def main():
|
||||
require_root()
|
||||
cfg = Cfg()
|
||||
p = argparse.ArgumentParser(prog="wg_serverctl.py")
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
s = sub.add_parser("gen-server")
|
||||
s.add_argument("--force", action="store_true")
|
||||
|
||||
g = sub.add_parser("gen-client")
|
||||
g.add_argument("name")
|
||||
g.add_argument("--force", action="store_true")
|
||||
|
||||
a = sub.add_parser("add")
|
||||
a.add_argument("name")
|
||||
a.add_argument("--pubkey", help="Client public key (base64, ends with =)")
|
||||
a.add_argument("--ip", help="IP octet (2..254) or full 10.66.66.X")
|
||||
a.add_argument("--no-restart", action="store_true")
|
||||
|
||||
d = sub.add_parser("del")
|
||||
d.add_argument("name")
|
||||
d.add_argument("--no-restart", action="store_true")
|
||||
|
||||
ap = sub.add_parser("apply")
|
||||
ap.add_argument("--no-restart", action="store_true")
|
||||
|
||||
sub.add_parser("list")
|
||||
|
||||
sh = sub.add_parser("show")
|
||||
sh.add_argument("name")
|
||||
|
||||
ex = sub.add_parser("export")
|
||||
ex.add_argument("name")
|
||||
ex.add_argument("--endpoint", help="Override endpoint ip:port (default: auto-detect public ip)")
|
||||
|
||||
args = p.parse_args()
|
||||
|
||||
try:
|
||||
if args.cmd == "gen-server":
|
||||
gen_server(cfg, args.force)
|
||||
elif args.cmd == "gen-client":
|
||||
gen_client(cfg, args.name, args.force)
|
||||
elif args.cmd == "add":
|
||||
add_client(cfg, args.name, args.pubkey, args.ip, args.no_restart)
|
||||
elif args.cmd == "del":
|
||||
del_client(cfg, args.name, args.no_restart)
|
||||
elif args.cmd == "apply":
|
||||
generate_wg0_conf(cfg)
|
||||
if not args.no_restart:
|
||||
restart_wg()
|
||||
elif args.cmd == "list":
|
||||
cmd_list(cfg)
|
||||
elif args.cmd == "show":
|
||||
cmd_show(cfg, args.name)
|
||||
elif args.cmd == "export":
|
||||
export_client_conf(cfg, args.name, args.endpoint)
|
||||
else:
|
||||
p.print_help()
|
||||
except Exception as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
Reference in New Issue
Block a user