Author SHA1 Message Date
Ruakij bd24f01110 Let a host name its CrowdSec machine at the LAPI with crowdsec.machine_name, the short hostname of doris is dev 2026-10-21 06:10:01 +00:00
Ruakij 1ede3d3329 Read the fail2ban sshd jail from the journal, Debian 12+ has no auth.log and the jail failed to start 2026-10-13 16:36:08 +00:00
Ruakij deb3521bfa Add a proxmox_no_nag role that hides the subscription dialog and reapplies it from an apt hook after every widget-toolkit update 2026-10-13 15:55:56 +00:00
Ruakij c8cac22d12 Run the cilium status check in check mode too and never report it as changed 2026-10-09 14:37:30 +00:00
Ruakij 6fe03c5f70 Skip the cilium CLI download when it is installed already 2026-10-09 14:37:30 +00:00
Ruakij 15b4e54bd5 Read resolv.conf for the k3s nameservers in check mode too 2026-10-09 14:37:30 +00:00
Ruakij c016459922 Write the k3s registries.yaml only on a semantic change 2026-10-09 14:37:29 +00:00
Ruakij 4d1725f17f Deploy a registries.yaml from registries_config on k3s servers and restart k3s on its change 2026-10-09 14:37:29 +00:00
Ruakij 80658c632f Add the cilium network plugin to the kubernetes role, disabling servicelb, and collect the disabled k3s components in one disable list 2026-10-09 14:37:29 +00:00
Ruakij 60b67f748e Install the k3s servers one at a time with retries from the latest channel, enable the service and link the kubeconfig to ~/.kube/config 2026-10-09 14:37:28 +00:00
Ruakij 4a76aa3a2e Set node-name, node-external-ip and labels_extra in the k3s config and config_extra for agents 2026-10-09 14:37:27 +00:00
Ruakij e9f1153482 Add clean_resolv_conf to the kubernetes role to give k3s a resolv.conf with only the nameserver lines 2026-10-09 14:37:27 +00:00
Ruakij ca1fcc16b6 Fall back to the default IPv4 and IPv6 address when the k3s node IP interface has none 2026-10-09 14:37:27 +00:00
Ruakij fb94fd5997 Deploy the calico operator v3.28.0 2026-10-09 14:37:26 +00:00
Ruakij 713d38d77d Shrink the default IPv6 cluster CIDR of k3s to fd42::/112 2026-10-09 14:37:26 +00:00
Ruakij bc71066de1 Run the k3s install of the kubernetes role only on target_host when it is set 2026-10-09 14:37:26 +00:00
Ruakij 5eae9ca348 Drop the docker dependency and the helm install from the kubernetes role 2026-10-09 14:37:25 +00:00
Ruakij 844e017f9e Add a tuned role that installs tuned and sets the throughput-performance profile 2026-10-09 14:37:25 +00:00
Ruakij 381f3ab7c9 Start the wireguard-ipv6-converter again whenever its bound service starts via a drop-in and always restart it on deploy 2026-10-09 14:37:24 +00:00
Ruakij 6390e6288b Disable SSH password auth in the common role only when sshd is active 2026-10-09 14:37:24 +00:00
Ruakij 13d60c4f9b Install nmon and fail2ban and upgrade packages on every distribution in the common role 2026-10-09 14:37:24 +00:00
Ruakij cb09f77b68 Enable TCP BBR with the fq qdisc on kernels from 4.9 in the common role 2026-10-09 14:37:23 +00:00
Ruakij 84fd52b678 Generate a random IPv6 stable secret once per host for the default interface in the common role 2026-10-09 14:37:23 +00:00
Ruakij c7eceae087 Ensure the .bashrc exists, expand aliases after sudo and color grep in the common role aliases 2026-10-09 14:37:23 +00:00
Ruakij 23ed82b113 Own the k3s config.yaml in the kubernetes role template, write it only on a semantic change and run the installer only when the pinned k3s version differs 2026-10-09 14:13:21 +00:00
Ruakij 37e21ead36 Expose the etcd metrics on the node IP of the etcd members of k3s 2026-10-09 12:56:24 +00:00
Ruakij 2798853153 Raise the inotify limits of the k3s nodes to 65536 instances and 4194304 watches, except in LXC 2026-10-09 12:55:46 +00:00
Ruakij e9ba272d70 Manage the kubelet image garbage collection drop-in and its config-dir kubelet-arg in the kubernetes role 2026-10-09 12:54:41 +00:00
Ruakij b62992f21e Cap the systemd journal at 512M on disk, 128M in memory and two weeks in the common role 2026-10-09 12:53:44 +00:00
Ruakij fd73949412 Whitelist the public_ips of every inventory host in crowdsec and keep a LAPI allowlist equal to the whitelist 2026-10-08 09:19:51 +00:00
Ruakij d345ccf585 Add a pre_command to kopia_backup for host dump scripts and install gpg for the apt key 2026-10-08 08:51:23 +00:00
Ruakij b45a07da56 Add forward_guard to host_firewall for a forward chain that drops traffic to guarded guests outside the allowed interfaces, ports and ICMP types 2026-10-08 08:48:40 +00:00
Ruakij 9ded6814c3 Pin the fstrim timer and the monthly ZFS trim and scrub per host in the common role to keep them out of the backup window 2026-10-07 22:42:08 +00:00
Ruakij a9a60e3106 Keep the clock on systemd-timesyncd and the timezone on UTC in the common role 2026-10-07 21:22:01 +00:00
Ruakij 5da89fdf29 Run the kopia_backup timer at 03:30 UTC by default since the hosts run in different timezones 2026-10-07 21:19:09 +00:00
Ruakij 9e2e6cbb99 Add IPv4 port forwards and masquerade to host_firewall in an ip host_nat table 2026-10-07 21:10:35 +00:00
Ruakij c9d2334906 Add a kopia_backup role for the daily Kopia snapshot script and timer against an existing S3 repository 2026-10-07 21:00:25 +00:00
Ruakij c87af201aa Add a crowdsec role for agents and the nftables bouncer against a central LAPI 2026-10-07 11:02:59 +00:00
Ruakij cfe87002c5 Replace the nftables role with host_firewall, one atomic inet host input table with zones, geo sets and lockout rollback 2026-10-07 11:02:59 +00:00
Ruakij 27e9d4a1ab Merge branch 'role_nftables' 2023-06-21 17:35:09 +02:00
Ruakij 755f9b2e1a Initial role 2023-05-21 20:57:43 +02:00
Ruakij 753f456ef3 Merge branch 'role_kubernetes-k3s' 2023-04-14 10:21:06 +02:00
Ruakij 093612f3a7 Change restart-check to started/restarted check 2023-04-14 10:20:16 +02:00
Ruakij a92409c56f Add failed_when to deploy calico operator 2023-04-14 10:01:42 +02:00
Ruakij f50e3ac33c Use first node's IP for joining cluster 2023-04-14 09:50:39 +02:00
Ruakij 668ff23ee6 Fix service task wrong usage 2023-04-14 09:42:46 +02:00
Ruakij c2c6a2872f Fix conditional for changed after install 2023-04-12 23:12:10 +02:00
Ruakij c1c7ec9e56 Remove workaround as k3s is now at 1.26 2023-04-12 22:39:43 +02:00
Ruakij 550f6868ff Fix old usage of network_plugin var 2023-04-12 20:49:38 +02:00
Ruakij 3fe288f6a5 Merge branch 'role_wireguard-ipv6-converter' 2023-04-12 20:40:01 +02:00
Ruakij ab9220d042 Add pause & gather_facts at end when service was started 2023-04-12 20:39:49 +02:00
Ruakij 15ad7920d4 Merge branch 'role_wireguard-ipv6-converter' 2023-04-12 20:24:27 +02:00
Ruakij 7012e1ea2b Separate enable and re-/start service for fine-control 2023-04-12 20:23:08 +02:00
Ruakij 13ebd48c5d Add register for task download 2023-04-12 20:22:34 +02:00
Ruakij 1a76b94a46 Add download-version 2023-04-12 14:58:43 +02:00
Ruakij 1b765689e6 Make sure daemon is reloaded before service-start 2023-04-12 14:50:14 +02:00
Ruakij 5b607df2de Add checks if var is set 2023-04-12 14:47:02 +02:00
Ruakij 521b76453a Add start to naming 2023-04-12 14:45:55 +02:00
Ruakij 7f503d983a Fix download-location 2023-04-12 14:45:43 +02:00
Ruakij b143d9c848 Fix checks 2023-04-12 14:45:31 +02:00
Ruakij f95bcbc38d change defaults 2023-04-12 14:44:10 +02:00
Ruakij 248a3c08b8 Initial role-data 2023-04-12 13:36:14 +02:00
Ruakij f2c86dc22d Merge branch 'role_kubernetes-k3s' 2023-04-05 20:20:03 +02:00
Ruakij a79f2cac8a Merge branch 'role_netbird_client' 2023-04-05 20:19:44 +02:00
Ruakij c8f90f0f8d Update calico 2023-04-05 20:15:31 +02:00
Ruakij 41570ea40d Create new block for network-stuff 2023-04-05 20:14:36 +02:00
Ruakij a3c887748a Move network-helper to own file independend from calico 2023-04-05 20:14:12 +02:00
Ruakij d113625fa8 Fix env-value not being string 2023-04-05 20:08:25 +02:00
Ruakij dadd077723 Fix service and conditional 2023-04-05 14:13:22 +02:00
Ruakij 0d43d07ad4 Add extra-config option 2023-04-05 14:12:56 +02:00
Ruakij d6f8f975bb Reload when config changed, but install already done 2023-04-05 13:32:14 +02:00
Ruakij 7c86a5d77d Add register for config 2023-04-05 13:30:22 +02:00
Ruakij 8c4e3c2401 Update routingtabletowg and use new sync feature 2023-03-30 14:54:49 +02:00
Ruakij b46d35c8a5 Add labels 2023-03-30 14:54:28 +02:00
Ruakij 791ad96849 Add ipv6-check to calico deploy 2023-03-19 15:17:45 +01:00
Ruakij fc3d9845d6 Fix undeterministic node-selection but uses group 2023-03-19 15:17:02 +01:00
Ruakij 590b75ac23 Add quotes to token-usage for special chars 2023-03-19 14:01:19 +01:00
Ruakij 0c82504299 Separate getting name and ips to fix bug easily 2023-03-19 14:00:46 +01:00
Ruakij 2fee9a1747 Only enable ipv6 when available and activated 2023-03-19 14:00:15 +01:00
Ruakij fb44c39969 Add install of often-used packets 2023-03-19 13:58:47 +01:00
Ruakij 5452303992 Remove netmaker from dependency 2023-03-19 13:58:05 +01:00
Ruakij 4321d78cf8 Add comments to variables 2023-03-19 13:57:57 +01:00
Ruakij f9a859e95c Add ingress-option 2023-03-19 13:56:26 +01:00
Ruakij fd302e4ebc Move regather facts to join when changed 2023-03-19 13:47:01 +01:00
Ruakij b5729caa0e Add wait for interface to come up 2023-03-19 13:46:44 +01:00
Ruakij dca40ed835 Remove throttle 2023-03-19 13:46:21 +01:00
Ruakij 95ddd04a86 Fix join command 2023-03-19 10:39:06 +01:00
Ruakij 911bc47acb Initial role stuff 2023-03-17 15:58:49 +01:00
Ruakij e5920b3ddf Add network-plugin option 2023-03-17 15:57:48 +01:00
Ruakij f3e381aca3 Move netmaker to deprecated 2023-01-18 13:23:51 +01:00
Ruakij 233eadaf40 Merge branch 'role_netmaker_server' 2023-01-10 09:47:34 +01:00
Ruakij e5ebc2ad5f Merge branch 'role_common' 2022-11-02 16:41:10 +01:00
Ruakij 98c51c6fc1 Merge branch 'role_kubernetes-k3s' 2022-11-02 16:40:52 +01:00
Ruakij 6b59bf6c75 Merge branch 'role_netmaker' 2022-11-02 16:40:36 +01:00
Ruakij 1b2af7cf6c Merge branch 'role_netmaker_server' 2022-11-02 16:40:25 +01:00
Ruakij d9cf3d2066 Re-gatherfacts at the end for other plays 2022-11-02 16:38:13 +01:00
Ruakij f42bce9b6b Add changed-detection 2022-11-02 16:37:53 +01:00
Ruakij 0fc5dbb791 Initial role-data 2022-11-02 16:29:55 +01:00
Ruakij 975746e7d7 Add IPv6-network to common 2022-11-02 10:27:06 +01:00
Ruakij 247fdec7ae fixme: hotfix for multi-master netmaker-server
netmaker doesnt handle concurrent joins to different server-nodes well and will duplicate addresses
2022-10-27 01:23:45 +02:00
Ruakij 9cb2e88193 Merge branch 'role_netmaker' 2022-10-27 01:16:57 +02:00
Ruakij 25ceb0f456 Merge branch 'role_netmaker_server' 2022-10-27 01:16:55 +02:00
Ruakij fcc4f1ed18 Fix task 2022-10-21 15:40:00 +02:00
Ruakij de0e220004 Add defaults-var-file 2022-10-21 15:32:20 +02:00
Ruakij f9cc97a8f2 Add CA to trust-store 2022-10-21 15:32:08 +02:00
Ruakij 811fc22eef Delete unnecessary task-file 2022-10-21 15:31:37 +02:00
Ruakij 4cb418e2b6 Add role netmaker (netclient) 2022-10-17 14:51:52 +02:00
93 changed files with 2228 additions and 41 deletions
@@ -0,0 +1,4 @@
netclient:
# Token to join default-network
# leave empty to ignore
join_network_token:
+3
View File
@@ -0,0 +1,3 @@
---
dependencies: []
#- role: docker
+4
View File
@@ -0,0 +1,4 @@
- name: Deploy CA Certificate
ansible.builtin.copy:
src: secret_files/netmaker_server/ca/ca.crt
dest: /etc/ssl/certs/netmaker-ca.pem
+25
View File
@@ -0,0 +1,25 @@
- name: Install Packages
# when: docker_file.stat.exists == False
package:
name:
- gpg
- gpg-agent
- name: Add netmaker-key
apt_key:
url: https://apt.netmaker.org/gpg.key
state: present
- name: Add netmaker-repository
apt_repository:
repo: "deb https:apt.netmaker.org stable main"
state: present
filename: netmaker
update_cache: yes
- name: Install wireguard & netclient
package:
name:
- wireguard
- netclient
state: latest
@@ -0,0 +1,7 @@
- name: Join netmaker-network
when: "netclient.join_network_token is defined"
command: "netclient join -t {{ netclient.join_network_token }}"
failed_when: command.rc != 0
changed_when: "'starting wireguard' in command.stdout"
register: command
throttle: 1
+8
View File
@@ -0,0 +1,8 @@
- import_tasks: ./certs.yml
- import_tasks: ./install.yml
- import_tasks: ./join-network.yml
- name: Gather facts to get changes
ansible.builtin.gather_facts:
+14
View File
@@ -0,0 +1,14 @@
ipv6_stable_secret_seed: "@JW7rQ^P3*f^KRkX3744DZQJb3DfMgZJJMtGc!Jmy!ECBqb^QzAMYLHD8dLvf8m%2bRSRFhgY#SXzAgnxWTvHv#ti6b5PngR^k2gHbQW*%dJ9sdeKM$H77Gymh&xamNN"
# Times of the heavy disk jobs, for hosts on a shared HDD pool that keep them out of the backup window.
disk_maintenance:
# systemd OnCalendar for fstrim.timer; unset keeps the distribution schedule.
fstrim_on_calendar:
# Cron "minute hour" of the monthly ZFS trim and scrub from zfsutils-linux; unset keeps 00:24.
zfs_cron_time:
# journald.conf [Journal] settings of a drop-in, in this order; null leaves journald untouched.
journald_size_cap:
SystemMaxUse: 512M
RuntimeMaxUse: 128M
MaxRetentionSec: 2week
+10
View File
@@ -1,2 +1,12 @@
- name: reload_sysctl
command: sysctl --system
- name: restart_journald
systemd:
name: systemd-journald
state: restarted
- name: restart_fail2ban
systemd:
name: fail2ban
state: restarted
+36 -15
View File
@@ -1,20 +1,41 @@
- name: Ensure .bashrc exists
file:
path: "{{ ansible_facts.env.HOME }}/.bashrc"
state: touch
mode: "0644"
- name: General aliases
blockinfile:
path: "{{ ansible_facts.env.HOME }}/.bashrc"
marker: "# {mark} ANSIBLE MANAGED BLOCK | General aliases"
block: |
alias clr="clear"
alias hgrep="history | grep"
alias syslog="tail -f --lines=100 /var/log/syslog"
alias cp="rsync -hlAXEptgoDS --numeric-ids --info=progress2"
path: "{{ ansible_facts.env.HOME }}/.bashrc"
marker: "# {mark} ANSIBLE MANAGED BLOCK | General aliases"
block: |
alias clr="clear"
alias hgrep="history | grep"
alias syslog="tail -f --lines=100 /var/log/syslog"
alias cp="rsync -hlAXEptgoDS --numeric-ids --info=progress2"
# We can tell bash to check the next word after the alias (i.e sudo) by adding a space to the end of the alias value. https://askubuntu.com/questions/22037/aliases-not-available-when-using-sudo
alias sudo='sudo '
- name: ls aliases and colors
blockinfile:
path: "{{ ansible_facts.env.HOME }}/.bashrc"
marker: "# {mark} ANSIBLE MANAGED BLOCK | ls aliases and colors"
block: |
export LS_OPTIONS='--color=auto'
eval "`dircolors`"
alias ls='ls $LS_OPTIONS'
alias ll='ls $LS_OPTIONS -l'
alias l='ls $LS_OPTIONS -la'
path: "{{ ansible_facts.env.HOME }}/.bashrc"
marker: "# {mark} ANSIBLE MANAGED BLOCK | ls aliases and colors"
block: |
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
alias ls='ls --color=auto'
#alias dir='dir --color=auto'
#alias vdir='vdir --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
fi
export LS_OPTIONS='--color=auto'
eval "`dircolors`"
alias ls='ls $LS_OPTIONS'
alias ll='ls $LS_OPTIONS -l'
alias l='ls $LS_OPTIONS -la'
+35
View File
@@ -0,0 +1,35 @@
- name: Pin the fstrim timer
when: disk_maintenance.fstrim_on_calendar | default(none) is not none
block:
- name: Create the fstrim timer drop-in directory
file:
path: /etc/systemd/system/fstrim.timer.d
state: directory
- name: Set the fstrim timer schedule
copy:
dest: /etc/systemd/system/fstrim.timer.d/schedule.conf
content: |
[Timer]
OnCalendar=
OnCalendar={{ disk_maintenance.fstrim_on_calendar }}
RandomizedDelaySec=0
# A catch-up run after a boot or a timezone change would land at any time.
Persistent=false
register: fstrim_timer
- name: Restart the fstrim timer
when: fstrim_timer.changed
systemd:
name: fstrim.timer
daemon_reload: yes
state: restarted
- name: Schedule the monthly ZFS trim and scrub
when: disk_maintenance.zfs_cron_time | default(none) is not none
lineinfile:
path: /etc/cron.d/zfsutils-linux
regexp: '^\S+ \S+ (\S+ \* \* root .*/usr/lib/zfs-linux/{{ item }};.*)$'
line: '{{ disk_maintenance.zfs_cron_time }} \1'
backrefs: yes
loop: [trim, scrub]
+9
View File
@@ -0,0 +1,9 @@
# Debian 12+ ships no syslog daemon, so there is no /var/log/auth.log and the
# file-based sshd jail refuses to start; read sshd from the journal instead.
- name: Read the sshd jail from the journal
copy:
dest: /etc/fail2ban/jail.d/sshd-systemd.conf
content: |
[sshd]
backend = systemd
notify: restart_fail2ban
+17
View File
@@ -0,0 +1,17 @@
- name: Cap the journal size
when: journald_size_cap | default(none) is not none
block:
- name: Create the journald drop-in directory
file:
path: /etc/systemd/journald.conf.d
state: directory
- name: Set the journal size cap
copy:
dest: /etc/systemd/journald.conf.d/10-size-cap.conf
content: |
[Journal]
{% for key, value in journald_size_cap.items() %}
{{ key }}={{ value }}
{% endfor %}
notify: restart_journald
Regular → Executable
+11
View File
@@ -1,7 +1,18 @@
- import_tasks: ./packages.yml
- import_tasks: ./fail2ban.yml
- import_tasks: ./ssh.yml
- import_tasks: ./packages.yml
- import_tasks: ./timesync.yml
- import_tasks: ./aliases.yml
- import_tasks: ./networking.yml
- import_tasks: ./disk_maintenance.yml
- import_tasks: ./journald.yml
tags: journald
+59
View File
@@ -0,0 +1,59 @@
- name: Set sysctl settings for ip-forwarding
copy:
dest: "/etc/sysctl.d/ip-forwarding.conf"
content: |
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
notify: reload_sysctl
- name: Check if the sysctl configuration file exists
stat:
path: "/etc/sysctl.d/ipv6-slaac-address-generation.conf"
register: sysctl_conf
- block:
# - name: Generate IPv6 stable-secret segments
# set_fact:
# stable_secret_segments: "{{ stable_secret_segments | default() + '%0x' | format(range(0, 2**(16*2)) | random(seed=ipv6_stable_secret_seed+inventory_hostname+(stable_secret_segments | default()))) }}"
# loop: "{{ range(4) }}"
# - name: Format IPv6 stable-secret segments
# set_fact:
# ipv6_stable_secret: "{{ stable_secret_segments | regex_replace('(.{4})(?=.)', '\\1:') }}"
- name: Generate random IPv6 stable-secret
set_fact:
ipv6_stable_secret: "{{ 'fd' + (lookup('password', '/dev/null length=30 chars=hex')).split('.')[0][:7] + '::' + (lookup('password', '/dev/null length=4 chars=hex')) }}"
- name: Set sysctl settings for ipv6-address-generation
copy:
dest: "/etc/sysctl.d/ipv6-slaac-address-generation.conf"
content: |
{% if ansible_facts.default_ipv6.interface is defined %}
net.ipv6.conf.{{ ansible_facts.default_ipv6.interface }}.addr_gen_mode = 2
net.ipv6.conf.{{ ansible_facts.default_ipv6.interface }}.stable_secret = {{ ipv6_stable_secret }}
{% else %}
net.ipv6.conf.{{ ansible_facts.default_ipv4.interface }}.addr_gen_mode = 2
net.ipv6.conf.{{ ansible_facts.default_ipv4.interface }}.stable_secret = {{ ipv6_stable_secret }}
{% endif %}
notify: reload_sysctl
when:
- ansible_facts.default_ipv6 is defined
- not sysctl_conf.stat.exists
- name: Set sysctl settings to override ipv6-slaac with enabled forwarding
copy:
dest: "/etc/sysctl.d/ipv6-slaac-override.conf"
content: |
net.ipv6.conf.all.accept_ra = 2
notify: reload_sysctl
- name: Set sysctl settings for TCP BBR
copy:
dest: "/etc/sysctl.d/tcp-bbr.conf"
content: |
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
when:
- ansible_kernel is version('4.9', '>=')
notify: reload_sysctl
+15 -15
View File
@@ -1,20 +1,20 @@
- name: Update Packages
apt:
update_cache: yes
upgrade: yes
when: ansible_facts.distribution == "Debian"
update_cache: yes
upgrade: yes
- name: Install Packages
package:
name:
- gpg
- htop
- iotop
- slurm
- sudo
- screen
- curl
- rsync
- zstd
state: latest
when: ansible_facts.distribution == "Debian"
name:
- gpg
- htop
- iotop
- slurm
- nmon
- sudo
- screen
- curl
- rsync
- zstd
- fail2ban
state: latest
+19 -11
View File
@@ -1,12 +1,20 @@
- name: Disable SSH password auth
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^PasswordAuthentication\s*yes'
line: "PasswordAuthentication no"
register: sshd_config
- name: Check if SSH daemon is active
systemd:
name: sshd
state: started
register: sshd_status
- name: Restart SSH daemon
service:
name: sshd
state: restarted
when: sshd_config.changed
- block:
- name: Disable SSH password auth
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^PasswordAuthentication\s*yes'
line: "PasswordAuthentication no"
register: sshd_config
- name: Restart SSH daemon
service:
name: sshd
state: restarted
when: sshd_config.changed
when: sshd_status.status.ActiveState == 'active'
+19
View File
@@ -0,0 +1,19 @@
# Containers share the host kernel clock and cannot discipline it themselves.
- name: Configure time synchronisation
when: ansible_facts['virtualization_role'] != 'guest' or ansible_facts['virtualization_type'] not in ['lxc', 'docker', 'podman', 'container']
block:
- name: Install systemd-timesyncd
package:
name: systemd-timesyncd
state: present
- name: Enable systemd-timesyncd
systemd:
name: systemd-timesyncd
enabled: yes
state: started
# Logs and local-time timers line up across hosts.
- name: Set the timezone to UTC
community.general.timezone:
name: Etc/UTC
+28
View File
@@ -0,0 +1,28 @@
---
crowdsec:
# Central LAPI; hosts run only the agent and the bouncer.
lapi_url: ""
registration_token: ""
bouncer_api_key: ""
# Machine name at the LAPI; empty is the short hostname.
machine_name: ""
# name -> hub collection, merged across group_vars/host_vars (null drops one).
collections: {}
# name -> hub scenario outside the collections.
scenarios: {}
# path below /etc/crowdsec -> content, for local parsers and scenarios.
local_files: {}
# name -> CIDR (single addresses as /32 or /128) never acted upon. Joined by
# the public_ips (same format) of every inventory host, then whitelisted by
# this agent and, when lapi_allowlist is set, allowlisted on the LAPI.
whitelist: {}
# LAPI allowlist kept equal to the whitelist; it also drops alerts on ranges
# holding a whitelisted address, which a parser whitelist cannot.
lapi_allowlist: ""
# Host whose k3s kubectl reaches the LAPI pod (crowdsec/crowdsec-lapi).
lapi_kubectl_host: ""
# name -> acquisition datasource, written to /etc/crowdsec/acquis.d/<name>.yaml.
acquisition: {}
# packagecloud has no trixie suite yet; the bookworm build is static Go.
apt_suite_map:
trixie: bookworm
+10
View File
@@ -0,0 +1,10 @@
---
- name: Restart crowdsec
ansible.builtin.systemd:
name: crowdsec.service
state: restarted
- name: Restart crowdsec-firewall-bouncer
ansible.builtin.systemd:
name: crowdsec-firewall-bouncer.service
state: restarted
+223
View File
@@ -0,0 +1,223 @@
---
- name: Require LAPI settings
ansible.builtin.assert:
that:
- crowdsec.lapi_url | length > 0
- crowdsec.registration_token | length > 0
- crowdsec.bouncer_api_key | length > 0
fail_msg: crowdsec.lapi_url, crowdsec.registration_token and crowdsec.bouncer_api_key must be set
- name: Create /etc/apt/keyrings
ansible.builtin.file:
path: /etc/apt/keyrings
state: directory
mode: "0755"
- name: Add the CrowdSec apt key
ansible.builtin.get_url:
url: https://packagecloud.io/crowdsec/crowdsec/gpgkey
dest: /etc/apt/keyrings/crowdsec.asc
mode: "0644"
- name: Add the CrowdSec apt repository
ansible.builtin.copy:
dest: /etc/apt/sources.list.d/crowdsec.list
mode: "0644"
content: >
deb [signed-by=/etc/apt/keyrings/crowdsec.asc]
https://packagecloud.io/crowdsec/crowdsec/{{ ansible_distribution | lower }}/
{{ crowdsec.apt_suite_map[ansible_distribution_release] | default(ansible_distribution_release) }} main
register: crowdsec_repo
# The postinst asks these; without them it registers a local machine for a
# local LAPI and enrols in the central CAPI.
- name: Preseed the package for a remote LAPI
ansible.builtin.debconf:
name: crowdsec
question: "{{ item.question }}"
vtype: "{{ item.vtype }}"
value: "{{ item.value }}"
loop:
- { question: crowdsec/lapi, vtype: boolean, value: "false" }
- { question: crowdsec/capi, vtype: boolean, value: "false" }
- { question: crowdsec/lapi_host, vtype: string, value: "{{ crowdsec.lapi_url | urlsplit('netloc') }}" }
- name: Create the CrowdSec config directories
ansible.builtin.file:
path: "{{ item }}"
state: directory
mode: "0755"
loop:
- /etc/crowdsec
- /etc/crowdsec/acquis.d
- /etc/crowdsec/bouncers
- /etc/crowdsec/parsers/s02-enrich
- /etc/systemd/system/crowdsec.service.d
- /etc/systemd/system/crowdsec-firewall-bouncer.service.d
- /etc/systemd/system/nftables.service.d
- name: Disable the local API server
ansible.builtin.copy:
dest: /etc/crowdsec/config.yaml.local
mode: "0644"
content: |
api:
server:
enable: false
notify: Restart crowdsec
# The postinst starts the agent before it is registered; an empty credentials
# file then makes the start a no-op instead of a failed package install.
- name: Start the agent only once registered
ansible.builtin.copy:
dest: /etc/systemd/system/crowdsec.service.d/registered.conf
mode: "0644"
content: |
[Unit]
ConditionFileNotEmpty=/etc/crowdsec/local_api_credentials.yaml
register: crowdsec_dropin
# Start, reload and stop of nftables.service flush the whole ruleset with the
# bouncer tables; a restart recreates them with all decisions. Ordering after
# nftables.service makes the queued restarts wait for its job to finish.
- name: Restart the bouncer after nftables.service flushes the ruleset
ansible.builtin.copy:
dest: "{{ item.dest }}"
mode: "0644"
content: "{{ item.content }}"
loop:
- dest: /etc/systemd/system/nftables.service.d/crowdsec-firewall-bouncer.conf
content: |
[Service]
ExecStartPost=-/bin/systemctl --no-block try-restart crowdsec-firewall-bouncer.service
ExecReload=-/bin/systemctl --no-block try-restart crowdsec-firewall-bouncer.service
ExecStopPost=-/bin/systemctl --no-block try-restart crowdsec-firewall-bouncer.service
- dest: /etc/systemd/system/crowdsec-firewall-bouncer.service.d/nftables.conf
content: |
[Unit]
After=nftables.service
loop_control:
label: "{{ item.dest }}"
register: crowdsec_bouncer_dropins
# Present before the package install, so the postinst finds an API key and
# does not try to create one on a local LAPI.
- name: Configure the firewall bouncer
ansible.builtin.template:
src: crowdsec-firewall-bouncer.yaml.local.j2
dest: /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml.local
mode: "0600"
notify: Restart crowdsec-firewall-bouncer
- name: Install the parser whitelist
ansible.builtin.template:
src: whitelist.yaml.j2
dest: /etc/crowdsec/parsers/s02-enrich/host-whitelist.yaml
mode: "0644"
notify: Restart crowdsec
- name: Install the acquisition files
ansible.builtin.copy:
dest: "/etc/crowdsec/acquis.d/{{ item.key }}.yaml"
mode: "0644"
content: "{{ item.value | to_nice_yaml }}"
loop: "{{ crowdsec.acquisition | dict2items | rejectattr('value', 'none') }}"
notify: Restart crowdsec
- name: Reload systemd
ansible.builtin.systemd:
daemon_reload: true
when: crowdsec_dropin is changed or crowdsec_bouncer_dropins is changed
- name: Install the agent and the nftables bouncer
ansible.builtin.apt:
name: [crowdsec, crowdsec-firewall-bouncer-nftables]
state: present
update_cache: "{{ crowdsec_repo is changed }}"
# The postinst otherwise runs service detection, which needs credentials that only the registration below creates
environment:
CROWDSEC_SETUP_UNATTENDED_DISABLE: "1"
- name: Read the agent credentials
ansible.builtin.command: grep -q '^login:' /etc/crowdsec/local_api_credentials.yaml
register: crowdsec_registered
changed_when: false
failed_when: false
check_mode: false
- name: Register the agent at the LAPI
ansible.builtin.command: >-
cscli lapi register --url {{ crowdsec.lapi_url }}
--token {{ crowdsec.registration_token }} --machine {{ crowdsec.machine_name | default(ansible_hostname, true) }}
when: crowdsec_registered.rc != 0
no_log: true
notify: Restart crowdsec
- name: List the installed hub items
ansible.builtin.command: cscli hub list -o json
register: crowdsec_hub_json
changed_when: false
# cscli is missing in check mode before the first install.
failed_when: crowdsec_hub_json.rc != 0 and not ansible_check_mode
check_mode: false
- name: Install the hub items
ansible.builtin.command: "cscli {{ item }} install {{ crowdsec_missing | join(' ') }}"
vars:
crowdsec_wanted: "{{ crowdsec[item] | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique }}"
crowdsec_present: "{{ (crowdsec_hub_json.stdout | default('{}', true) | from_json)[item] | default([], true) | map(attribute='name') }}"
crowdsec_missing: "{{ crowdsec_wanted | difference(crowdsec_present) }}"
loop: [collections, scenarios]
when: crowdsec_missing | length > 0
notify: Restart crowdsec
- name: Install the local parsers and scenarios
ansible.builtin.copy:
dest: "/etc/crowdsec/{{ item.key }}"
mode: "0644"
content: "{{ item.value }}"
loop: "{{ crowdsec.local_files | dict2items | rejectattr('value', 'none') }}"
loop_control:
label: "{{ item.key }}"
notify: Restart crowdsec
- name: Enable the agent and the bouncer
ansible.builtin.systemd:
name: "{{ item }}"
enabled: true
state: started
loop: [crowdsec.service, crowdsec-firewall-bouncer.service]
- name: Sync the LAPI allowlist
when: crowdsec.lapi_allowlist | length > 0
run_once: true
delegate_to: "{{ crowdsec.lapi_kubectl_host }}"
vars:
crowdsec_lapi_cscli: k3s kubectl -n crowdsec exec deploy/crowdsec-lapi -c crowdsec-lapi -- cscli
block:
- name: Read the LAPI allowlist
ansible.builtin.command: "{{ crowdsec_lapi_cscli }} allowlists inspect {{ crowdsec.lapi_allowlist }} -o json"
register: crowdsec_allowlist_json
changed_when: false
failed_when: false
check_mode: false
- name: Create the LAPI allowlist
ansible.builtin.command: >-
{{ crowdsec_lapi_cscli }} allowlists create {{ crowdsec.lapi_allowlist }}
-d "Internal ranges and the public addresses of all hosts, managed by Ansible"
when: crowdsec_allowlist_json.rc != 0
# cscli stores single addresses without their /32 or /128.
- name: Update the LAPI allowlist
ansible.builtin.command: "{{ crowdsec_lapi_cscli }} allowlists {{ item.action }} {{ crowdsec.lapi_allowlist }} {{ item.cidrs | join(' ') }}"
vars:
crowdsec_allowlist_wanted: "{{ crowdsec_whitelist | map('regex_replace', '/(32|128)$', '') | list }}"
crowdsec_allowlist_present: >-
{{ ((crowdsec_allowlist_json.stdout | default('{}', true) | from_json).get('items') or []) | map(attribute='value') | list }}
loop:
- { action: add, cidrs: "{{ crowdsec_allowlist_wanted | difference(crowdsec_allowlist_present) }}" }
- { action: remove, cidrs: "{{ crowdsec_allowlist_present | difference(crowdsec_allowlist_wanted) }}" }
loop_control:
label: "{{ item.action }} {{ item.cidrs | join(' ') }}"
when: item.cidrs | length > 0
@@ -0,0 +1,8 @@
# {{ ansible_managed }}
api_url: {{ crowdsec.lapi_url }}/
api_key: {{ crowdsec.bouncer_api_key }}
mode: nftables
# forward covers Docker-published ports and Proxmox guests.
nftables_hooks:
- input
- forward
+9
View File
@@ -0,0 +1,9 @@
# {{ ansible_managed }}
name: local/host-whitelist
description: Internal ranges and the public addresses of all hosts
whitelist:
reason: internal or admin source
cidr:
{% for cidr in crowdsec_whitelist %}
- "{{ cidr }}"
{% endfor %}
+4
View File
@@ -0,0 +1,4 @@
---
crowdsec_whitelist: >-
{{ (crowdsec.whitelist | dict2items | rejectattr('value', 'none') | map(attribute='value') | list
+ groups['all'] | map('extract', hostvars, 'public_ips') | select('defined') | flatten) | unique }}
+46
View File
@@ -0,0 +1,46 @@
---
# Port and address maps are name -> value so group_vars and host_vars levels
# merge into one map (hash_behaviour = merge); a null value drops an entry.
# Ports are numbers or "first-last" ranges.
firewall:
public:
tcp: {}
udp: {}
# Accepted only from the geo sets or the admin set.
geo:
tcp: {}
# Accepted on wt0 before the public zone is evaluated.
vpn:
tcp: {}
udp: {}
admin:
v4: {}
v6: {}
# Interface names (nft wildcards allowed) whose traffic is trusted.
internal_ifaces: {}
# Guest bridges (Proxmox vmbr*): DHCP, router solicitations and the public
# zone only, never the geo ports.
guest_ifaces: {}
# Kubernetes ring: pod interfaces, and node NetBird IPs / pod CIDRs on wt0.
cluster:
ifaces: {}
v4: {}
v6: {}
# IPv4 port forwards: name -> {daddr, to, tcp: ports, udp: ports}, ports as in
# the maps above. Forwarded traffic passes the forward hook, not input.
dnat: {}
# IPv4 source NAT: name -> nft match, e.g. 'oifname "vmbr0"'.
masquerade: {}
# Forwarded traffic to guarded guests: name -> {ifaces, v4, v6}, where v4 and
# v6 are {daddr, tcp: ports, icmp: icmp or icmpv6 types}. Replies and traffic
# in on ifaces pass, the listed ports and types pass, the rest to daddr drops;
# traffic to other addresses is not filtered.
forward_guard: {}
ssh_rate_limit: "10/minute burst 5 packets"
# EU-27 plus CH, NO, GB, IS, LI (ipdeny country codes). Empty leaves the geo
# ports to the admin set; a load peaks at about 60 MB of kernel memory, too much for a 1 GB host.
geo_countries: [at, be, bg, hr, cy, cz, dk, ee, fi, fr, de, gr, hu, ie, it, lv, lt, lu, mt, nl, pl, pt, ro, sk, si, es, se, ch, "no", gb, is, li]
rollback_seconds: 120
# name -> shell command that has to exit 0 through the new rule set before the
# rollback is disarmed; retried for a quarter of the rollback window.
confirm: {}
+8
View File
@@ -0,0 +1,8 @@
[Unit]
Description=Refresh the geo sets of the host firewall
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/nft-host-geo
+10
View File
@@ -0,0 +1,10 @@
[Unit]
Description=Daily refresh of the host firewall geo sets
[Timer]
OnCalendar=daily
RandomizedDelaySec=6h
Persistent=true
[Install]
WantedBy=timers.target
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
# Started by a transient timer before a rule set load; Ansible stops the timer
# once the new rule set is confirmed.
cd /etc/nftables.d || exit 1
mv -f host.nft host.nft.rolled-back
if [ -s host.nft.good ]; then
cp host.nft.good host.nft && nft -f host.nft && exit 0
fi
nft delete table inet host
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=Host input firewall and NAT (nftables tables inet host and ip host_nat)
# Same early-boot placement as nftables.service. Its "flush ruleset" on start,
# reload and stop would wipe this table, so load after it and follow all three.
DefaultDependencies=no
Wants=network-pre.target
Before=network-pre.target shutdown.target
After=nftables.service
PartOf=nftables.service
ReloadPropagatedFrom=nftables.service
Conflicts=shutdown.target
ConditionPathExists=/etc/nftables.d/host.nft
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/nft -f /etc/nftables.d/host.nft
ExecReload=/usr/sbin/nft -f /etc/nftables.d/host.nft
ExecStop=-/usr/sbin/nft delete table inet host
[Install]
WantedBy=sysinit.target
+168
View File
@@ -0,0 +1,168 @@
---
- name: Install nftables and curl
ansible.builtin.apt:
name: [nftables, curl]
state: present
- name: Check that this node is part of its own cluster ring
ansible.builtin.assert:
that: ansible_wt0.ipv4.address in (firewall.cluster.v4 | dict2items | map(attribute='value') | list)
fail_msg: "wt0 address {{ ansible_wt0.ipv4.address }} is missing from firewall.cluster.v4"
when: firewall.cluster.v4 | default({}, true) | length > 0 and ansible_wt0.ipv4.address is defined
- name: Remove the include block of the old nftables role
ansible.builtin.blockinfile:
path: /etc/nftables.conf
marker: "# {mark} ANSIBLE MANAGED BLOCK - nftables"
state: absent
- name: Remove the rule files of the old nftables role
ansible.builtin.file:
path: /etc/nftables/ansible-managed
state: absent
- name: Create /etc/nftables.d
ansible.builtin.file:
path: /etc/nftables.d
state: directory
mode: "0755"
- name: Install the systemd units
ansible.builtin.copy:
src: "{{ item }}"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop: [nft-host.service, nft-host-geo.service, nft-host-geo.timer]
register: firewall_units
- name: Install the rollback script
ansible.builtin.copy:
src: nft-host-rollback
dest: /usr/local/sbin/nft-host-rollback
mode: "0755"
- name: Geo sets
when: firewall.geo_countries | length > 0
block:
- name: Install the geo set updater
ansible.builtin.template:
src: nft-host-geo.j2
dest: /usr/local/sbin/nft-host-geo
mode: "0755"
- name: Build the geo sets on first run
ansible.builtin.command: /usr/local/sbin/nft-host-geo
args:
creates: /etc/nftables.d/host-geo.nft
- name: Read the geo sets file
ansible.builtin.stat:
path: /etc/nftables.d/host-geo.nft
register: firewall_geo
# Without geo sets the public SSH port would only answer NetBird and the admin set.
- name: Require non-empty geo sets
ansible.builtin.assert:
that: firewall_geo.stat.exists and firewall_geo.stat.size > 0
fail_msg: /etc/nftables.d/host-geo.nft is missing or empty, refusing to apply the firewall
# Check mode neither installs the updater nor builds the sets.
when: not ansible_check_mode
- name: Enable the geo set timer
ansible.builtin.systemd:
name: nft-host-geo.timer
enabled: true
state: started
daemon_reload: "{{ firewall_units is changed }}"
# Check mode does not install the unit.
ignore_errors: "{{ ansible_check_mode }}"
- name: Stop the geo set refresh
when: firewall.geo_countries | length == 0
block:
- name: Disable the geo set timer
ansible.builtin.systemd:
name: nft-host-geo.timer
enabled: false
state: stopped
daemon_reload: "{{ firewall_units is changed }}"
- name: Remove the geo sets file
ansible.builtin.file:
path: /etc/nftables.d/host-geo.nft
state: absent
- name: Render the rule set
ansible.builtin.template:
src: host.nft.j2
dest: /etc/nftables.d/host.nft
mode: "0644"
validate: nft -c -f %s
register: firewall_rules
- name: Read the firewall unit state
ansible.builtin.command: systemctl is-active nft-host.service
register: firewall_active
changed_when: false
failed_when: false
check_mode: false
- name: Apply the rule set with a rollback timer
when:
- firewall_rules is changed or firewall_units is changed or firewall_active.stdout != 'active'
- not ansible_check_mode
block:
# A failed run of the transient unit keeps its name taken for systemd-run.
- name: Clear a leftover rollback timer
ansible.builtin.shell: >-
systemctl stop nft-host-rollback.timer;
systemctl reset-failed nft-host-rollback.timer nft-host-rollback.service;
true
changed_when: false
# Unless disarmed below, this moves the rejected rule file aside so a reboot
# does not load it again, and reloads the last confirmed one; without one
# it drops the table.
- name: Arm the rollback
ansible.builtin.command: systemd-run --on-active={{ firewall.rollback_seconds }} --unit=nft-host-rollback /usr/local/sbin/nft-host-rollback
changed_when: false
- name: Load the rule set
ansible.builtin.systemd:
name: nft-host.service
enabled: true
state: reloaded
daemon_reload: true
# A reused SSH control socket would hide a lockout, so test a fresh connection.
- name: Drop the SSH connection
ansible.builtin.meta: reset_connection
- name: Reconnect through the new rule set
ansible.builtin.wait_for_connection:
timeout: "{{ (firewall.rollback_seconds / 2) | int }}"
- name: Check the services behind the new rule set
ansible.builtin.shell: "{{ item.value }}"
loop: "{{ firewall.confirm | default({}, true) | dict2items | rejectattr('value', 'none') }}"
loop_control:
label: "{{ item.key }}"
register: firewall_confirm
until: firewall_confirm.rc == 0
retries: "{{ (firewall.rollback_seconds / 20) | int }}"
delay: 5
changed_when: false
# Fails when the timer already fired, so a slow check cannot pass for a rolled-back host.
- name: Disarm the rollback
ansible.builtin.shell: >-
test "$(systemctl show --property=SubState --value nft-host-rollback.timer)" = waiting &&
systemctl stop nft-host-rollback.timer
changed_when: true
- name: Keep the rule set as the last confirmed one
ansible.builtin.copy:
src: /etc/nftables.d/host.nft
dest: /etc/nftables.d/host.nft.good
remote_src: true
mode: "0644"
+169
View File
@@ -0,0 +1,169 @@
#!/usr/sbin/nft -f
# {{ ansible_managed }}
{% macro list(map) -%}
{{ map | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | map('string') | unique | join(', ') }}
{%- endmacro %}
{% macro addr_set(name, type, map) -%}
set {{ name }} {
type {{ type }}; flags interval; auto-merge;
{% if list(map) %}
elements = { {{ list(map) }} }
{% endif %}
}
{%- endmacro %}
{% set cluster = firewall.cluster | default({}, true) %}
{% set cluster_ifaces = cluster.ifaces | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique | list %}
{% set internal_ifaces = firewall.internal_ifaces | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique | list %}
{% set guest_ifaces = firewall.guest_ifaces | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique | list %}
{% set guest_match = 'iifname { "' ~ (guest_ifaces | join('", "')) ~ '" }' %}
{% set dnat = firewall.dnat | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | list %}
{% set masquerade = firewall.masquerade | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique | list %}
{% set forward_guard = firewall.forward_guard | default({}, true) | dict2items | rejectattr('value', 'none') | list %}
# Declare-then-delete makes every load one atomic transaction that replaces
# only these tables; Cilium, Docker, NetBird and CrowdSec tables stay untouched.
table inet host {}
delete table inet host
table ip host_nat {}
delete table ip host_nat
{% if firewall.geo_countries %}
include "/etc/nftables.d/host-geo.nft"
{% endif %}
table inet host {
{{ addr_set('admin_v4', 'ipv4_addr', firewall.admin.v4) }}
{{ addr_set('admin_v6', 'ipv6_addr', firewall.admin.v6) }}
{{ addr_set('cluster_v4', 'ipv4_addr', cluster.v4) }}
{{ addr_set('cluster_v6', 'ipv6_addr', cluster.v6) }}
set ssh_meter_v4 {
type ipv4_addr; flags dynamic; timeout 10m; size 65535;
}
set ssh_meter_v6 {
type ipv6_addr; flags dynamic; timeout 10m; size 65535;
}
chain input {
type filter hook input priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iif lo accept
icmp type { echo-request, destination-unreachable, time-exceeded, parameter-problem } accept
icmpv6 type { echo-request, destination-unreachable, packet-too-big, time-exceeded, parameter-problem, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, mld-listener-query } accept
udp sport 67 udp dport 68 accept
meta nfproto ipv6 udp sport 547 udp dport 546 accept
{% if cluster_ifaces or list(cluster.v4) or list(cluster.v6) %}
# ponytail: the cluster ring trusts every port between nodes and pods;
# per-port lists (6443, 10250, 2379-2380, 4240, 8472, ...) are the upgrade path.
{% for iface in cluster_ifaces %}
iifname "{{ iface }}" accept
{% endfor %}
iifname "wt0" ip saddr @cluster_v4 accept
iifname "wt0" ip6 saddr @cluster_v6 accept
{% endif %}
{% if internal_ifaces %}
# ponytail: internal bridges are trusted on every port; per-port lists are the upgrade path.
{% for iface in internal_ifaces %}
iifname "{{ iface }}" accept
{% endfor %}
{% endif %}
{% if guest_ifaces %}
# Pods on k3s guests egress through the node IP, so guest bridges get
# only DHCP and router solicitations beyond the public zone.
{{ guest_match }} udp dport 67 accept
{{ guest_match }} icmpv6 type nd-router-solicit accept
{% endif %}
# VPN peers get the vpn ports, then everything public.
iifname "wt0" jump vpn
jump public
}
chain vpn {
{% if list(firewall.vpn.tcp) %}
tcp dport { {{ list(firewall.vpn.tcp) }} } accept
{% endif %}
{% if list(firewall.vpn.udp) %}
udp dport { {{ list(firewall.vpn.udp) }} } accept
{% endif %}
}
chain public {
{% if list(firewall.public.tcp) %}
tcp dport { {{ list(firewall.public.tcp) }} } accept
{% endif %}
{% if list(firewall.public.udp) %}
udp dport { {{ list(firewall.public.udp) }} } accept
{% endif %}
{% if list(firewall.geo.tcp) %}
tcp dport { {{ list(firewall.geo.tcp) }} } jump geo
{% endif %}
}
chain geo {
{% if guest_ifaces %}
# Guest IPv6 addresses sit inside the geo sets.
{{ guest_match }} return
{% endif %}
ip saddr @admin_v4 accept
ip6 saddr @admin_v6 accept
tcp dport 22 ct state new update @ssh_meter_v4 { ip saddr limit rate over {{ firewall.ssh_rate_limit }} } drop
# One IPv6 client usually holds a whole /64.
tcp dport 22 ct state new update @ssh_meter_v6 { ip6 saddr & ffff:ffff:ffff:ffff:: limit rate over {{ firewall.ssh_rate_limit }} } drop
{% if firewall.geo_countries %}
ip saddr @geo_v4 accept
ip6 saddr @geo_v6 accept
{% endif %}
}
{% if forward_guard %}
chain forward {
type filter hook forward priority filter; policy accept;
{% for guard in forward_guard %}
# {{ guard.key }}
{% set ifaces = guard.value.ifaces | default({}, true) | dict2items | rejectattr('value', 'none') | map(attribute='value') | unique | list %}
{% for fam, l3, icmp in [('v4', 'ip', 'icmp'), ('v6', 'ip6', 'icmpv6')] if list((guard.value[fam] | default({}, true)).daddr) %}
{% set zone = guard.value[fam] %}
{% set match = l3 ~ ' daddr { ' ~ list(zone.daddr) ~ ' }' %}
{{ match }} ct state established,related accept
{% if ifaces %}
{{ match }} iifname { "{{ ifaces | join('", "') }}" } accept
{% endif %}
{% if list(zone.tcp) %}
{{ match }} tcp dport { {{ list(zone.tcp) }} } accept
{% endif %}
{% if list(zone.icmp) %}
{{ match }} {{ icmp }} type { {{ list(zone.icmp) }} } accept
{% endif %}
{{ match }} drop
{% endfor %}
{% endfor %}
}
{% endif %}
}
{% if dnat or masquerade %}
table ip host_nat {
chain prerouting {
type nat hook prerouting priority dstnat; policy accept;
{% for fwd in dnat %}
{% for proto in ['tcp', 'udp'] if list(fwd[proto]) %}
ip daddr {{ fwd.daddr }} {{ proto }} dport { {{ list(fwd[proto]) }} } dnat to {{ fwd.to }}
{% endfor %}
{% endfor %}
}
chain postrouting {
type nat hook postrouting priority srcnat; policy accept;
{% for match in masquerade %}
{{ match }} masquerade
{% endfor %}
}
}
{% endif %}
+46
View File
@@ -0,0 +1,46 @@
#!/bin/sh
# {{ ansible_managed }}
# Rebuilds the geo sets of table inet host from ipdeny. Any failed download or
# check exits before the swap, so the previous file stays in place.
set -eu
out=/etc/nftables.d/host-geo.nft
countries="{{ firewall.geo_countries | join(' ') }}"
prefix='^[0-9a-f.:]+/[0-9]+$'
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
fetch() {
curl -fsS --retry 3 --max-time 120 -o "$2" "$1"
grep -Eq "$prefix" "$2" || { echo "no prefixes in $1" >&2; exit 1; }
}
for cc in $countries; do
fetch "https://www.ipdeny.com/ipblocks/data/aggregated/$cc-aggregated.zone" "$tmp/$cc.v4"
fetch "https://www.ipdeny.com/ipv6/ipaddresses/aggregated/$cc-aggregated.zone" "$tmp/$cc.v6"
done
elements() {
cat "$@" | grep -E "$prefix" | sed 's/$/,/'
}
{
echo "# Generated by $0, do not edit."
echo 'table inet host {'
echo ' set geo_v4 { type ipv4_addr; flags interval; auto-merge; elements = {'
elements "$tmp"/*.v4
echo ' } }'
echo ' set geo_v6 { type ipv6_addr; flags interval; auto-merge; elements = {'
elements "$tmp"/*.v6
echo ' } }'
echo '}'
} >"$tmp/host-geo.nft"
nft -c -f "$tmp/host-geo.nft"
install -m 0644 "$tmp/host-geo.nft" "$out.new"
mv -f "$out.new" "$out"
if systemctl -q is-active nft-host.service; then
systemctl reload nft-host.service
fi
+31
View File
@@ -0,0 +1,31 @@
---
kopia_backup:
# S3 repository; it is created, and its policies and maintenance are set, outside this role.
endpoint: tier1.backups.ruekov.eu
bucket: ""
access_key: ""
secret_key: ""
password: ""
# Snapshot owner is root@<hostname>; several hosts can share one repository.
hostname: "{{ inventory_hostname_short }}"
# Kopia defaults when unset.
content_cache_size_mb:
metadata_cache_size_mb:
# Paths to snapshot; missing ones are skipped at run time.
sources: []
# source path -> ignore rules, replacing the policy rules on every run (null drops one).
ignore: {}
# Command run first, e.g. a host script writing dumps into a source; a failure fails the run, the snapshot still runs.
pre_command: ""
# Database dumps written to /var/backups/dumps before the snapshot, which then includes that directory.
# Each item: file, container, command (run with sh -c inside the container, output is the dump).
dumps: []
# Snapshot the newest k3s etcd snapshot, staged in /var/backups/etcd.
etcd_snapshot: false
# Explicit UTC: the hosts run in different timezones.
on_calendar: "*-*-* 03:30 UTC"
randomized_delay: ""
# Extra units the service starts after, e.g. docker.service for dumps.
after: []
# Nice=10 and idle IO scheduling.
idle_priority: true
+95
View File
@@ -0,0 +1,95 @@
---
- name: Require the repository settings
ansible.builtin.assert:
that:
- kopia_backup.bucket | length > 0
- kopia_backup.access_key | length > 0
- kopia_backup.secret_key | length > 0
- kopia_backup.password | length > 0
- kopia_backup.sources | length > 0
fail_msg: kopia_backup.bucket, access_key, secret_key, password and sources must be set
- name: Create /etc/apt/keyrings
ansible.builtin.file:
path: /etc/apt/keyrings
state: directory
mode: "0755"
- name: Install gpg for the apt key
ansible.builtin.apt:
name: gpg
state: present
- name: Add the Kopia apt key
ansible.builtin.shell: curl -fsSL https://kopia.io/signing-key | gpg --dearmor -o /etc/apt/keyrings/kopia-keyring.gpg
args:
creates: /etc/apt/keyrings/kopia-keyring.gpg
- name: Add the Kopia apt repository
ansible.builtin.copy:
dest: /etc/apt/sources.list.d/kopia.list
mode: "0644"
content: |
deb [signed-by=/etc/apt/keyrings/kopia-keyring.gpg] http://packages.kopia.io/apt/ stable main
register: kopia_backup_repo
- name: Install Kopia
ansible.builtin.apt:
name: kopia
state: present
update_cache: "{{ kopia_backup_repo is changed }}"
- name: Read the repository connection
ansible.builtin.command: kopia repository status
environment:
KOPIA_CHECK_FOR_UPDATES: "false"
register: kopia_backup_status
changed_when: false
failed_when: false
check_mode: false
- name: Connect the repository
ansible.builtin.command:
argv: "{{ argv + (cache if kopia_backup.content_cache_size_mb else []) }}"
vars:
argv:
- kopia
- repository
- connect
- s3
- --endpoint={{ kopia_backup.endpoint }}
- --bucket={{ kopia_backup.bucket }}
- --override-hostname={{ kopia_backup.hostname }}
- --override-username=root
cache:
- --content-cache-size-mb={{ kopia_backup.content_cache_size_mb }}
- --metadata-cache-size-mb={{ kopia_backup.metadata_cache_size_mb }}
# Secrets through the environment, flags would show them in the process list
environment:
AWS_ACCESS_KEY_ID: "{{ kopia_backup.access_key }}"
AWS_SECRET_ACCESS_KEY: "{{ kopia_backup.secret_key }}"
KOPIA_PASSWORD: "{{ kopia_backup.password }}"
KOPIA_CHECK_FOR_UPDATES: "false"
when: kopia_backup_status.rc != 0
no_log: true
- name: Install the backup script
ansible.builtin.template:
src: kopia-backup.j2
dest: /usr/local/sbin/kopia-backup
mode: "0755"
- name: Install the systemd units
ansible.builtin.template:
src: "{{ item }}.j2"
dest: "/etc/systemd/system/{{ item }}"
mode: "0644"
loop: [kopia-backup.service, kopia-backup.timer]
register: kopia_backup_units
- name: Enable the timer
ansible.builtin.systemd:
name: kopia-backup.timer
enabled: true
state: started
daemon_reload: "{{ kopia_backup_units is changed }}"
+58
View File
@@ -0,0 +1,58 @@
#!/bin/bash
# {{ ansible_managed }}
# Daily Kopia backup of {{ kopia_backup.hostname }} into the bucket {{ kopia_backup.bucket }} on {{ kopia_backup.endpoint }}.
set -uo pipefail
export KOPIA_CHECK_FOR_UPDATES=false
failed=0
sources=()
{% if kopia_backup.pre_command %}
{{ kopia_backup.pre_command }} || { echo "pre_command failed" >&2; failed=1; }
{% endif %}
{% if kopia_backup.etcd_snapshot %}
# Only the latest etcd snapshot; the live etcd dir is not consistent on disk and older snapshots only add churn
etcd=/var/backups/etcd
latest=$(ls -t /var/lib/rancher/k3s/server/db/snapshots/etcd-snapshot-* 2>/dev/null | head -1)
if [ -n "$latest" ]; then
install -d -m700 "$etcd"
rm -f "$etcd/etcd-snapshot"
ln "$latest" "$etcd/etcd-snapshot" 2>/dev/null || cp -p "$latest" "$etcd/etcd-snapshot" || failed=1
fi
{% endif %}
{% if kopia_backup.dumps %}
dumps=/var/backups/dumps
install -d -m700 "$dumps"
dump() { # file, container, command run inside the container
if docker exec "$2" sh -c "$3" > "$dumps/$1.tmp"; then
mv "$dumps/$1.tmp" "$dumps/$1"
else
echo "dump of $1 failed" >&2; rm -f "$dumps/$1.tmp"; failed=1
fi
}
{% for d in kopia_backup.dumps %}
dump {{ d.file | quote }} {{ d.container | quote }} {{ d.command | quote }}
{% endfor %}
{% endif %}
{% set ignores = kopia_backup.ignore | dict2items | rejectattr('value', 'none') | list %}
{% if ignores %}
# Ignore rules of a source, replaced as a whole (--clear-ignore runs after --add-ignore within one call).
# Kopia sorts the rules, so negations (!) cannot be relied on to come after the rule they override.
ignore() {
local path=$1; shift
kopia policy set "$path" --clear-ignore >/dev/null 2>&1 &&
kopia policy set "$path" "${@/#/--add-ignore=}" >/dev/null 2>&1 ||
{ echo "setting the ignore rules of $path failed" >&2; failed=1; }
}
{% for i in ignores %}
ignore {{ i.key | quote }} {{ i.value | map('quote') | join(' ') }}
{% endfor %}
{% endif %}
for p in {{ kopia_backup.sources | map('quote') | join(' ') }}{{ ' "$etcd"' if kopia_backup.etcd_snapshot }}{{ ' "$dumps"' if kopia_backup.dumps }}; do
[ -d "$p" ] && sources+=("$p")
done
kopia snapshot create "${sources[@]}" || failed=1
exit $failed
@@ -0,0 +1,14 @@
# {{ ansible_managed }}
[Unit]
Description=Kopia backup to {{ kopia_backup.endpoint }}
After={{ (['network-online.target'] + kopia_backup.after) | join(' ') }}
Wants=network-online.target
[Service]
Type=oneshot
Environment=HOME=/root
{% if kopia_backup.idle_priority %}
Nice=10
IOSchedulingClass=idle
{% endif %}
ExecStart=/usr/local/sbin/kopia-backup
@@ -0,0 +1,13 @@
# {{ ansible_managed }}
[Unit]
Description=Daily Kopia backup to {{ kopia_backup.endpoint }}
[Timer]
OnCalendar={{ kopia_backup.on_calendar }}
{% if kopia_backup.randomized_delay %}
RandomizedDelaySec={{ kopia_backup.randomized_delay }}
{% endif %}
Persistent=true
[Install]
WantedBy=timers.target
+65
View File
@@ -0,0 +1,65 @@
---
kubernetes:
ipPool:
ipv4:
# Minimum: /24
cluster_cidr: 10.42.0.0/16
service_cidr: 10.43.0.0/16
ipv6:
# Minimum: /120
cluster_cidr: fd42::/112
service_cidr: fd43::/112
# Interface to grab node-IPv4/v6 from
nodeIp_interface: <interface to grab nodeIp from>
control_plane:
dns_name: <control-plane dns-reachable-name>
token: <shared token for nodes to join>
network:
# Copies the current /etc/resolv.conf whilst only keeping "nameserver" lines
# can be used to remove filter
clean_resolv_conf: false
# One of [none, flannel, calico, cilium]
plugin: calico
# Helper for networking
helper:
# https://github.com/Ruakij/RoutingTableToWg
# Translates received-routes from e.g. BGP to wireguard-allowedips
# Helpful, when nodeIp_interface is a wireguard-interface
routingtabletowg: false
# One of [none, traefik-ingress, nginx-ingress]
ingress_controller: traefik-ingress
# k3s version to install, e.g. v1.37.1+k3s1. An installed k3s is replaced only
# when this is set and differs; empty installs the latest when k3s is missing.
version: ""
# Extra config of the nodes running etcd, left out with disable-etcd
etcd_config_extra:
# Without it etcd serves metrics on 127.0.0.1 only, out of reach of vmagent
etcd-expose-metrics: true
config_extra:
# etcd-tuning
# heartbeat: 0.5-1.5x of rtt
# election: 10x- of heartbeat
etcd-arg:
heartbeat-interval: 500
election-timeout: 5000
registries_config:
# Defines the names and endpoints of registries
# When embedded-registry is enabled, every registry needs to be defined here to allow this node to distribute those images.
#mirrors:
# Enable distribution for all registries
#"*": {}
# Defines authentication and extra settings for the registries
#config:
+33
View File
@@ -0,0 +1,33 @@
@startuml
rectangle "Control-Plane" as control_plane {
rectangle "Node" as sn1 {
component "netclient" as sn1_netclient
component etcd as sn1_etcd
component "k3s-server" as sn1_k3s_server
sn1_k3s_server - sn1_etcd
}
rectangle "Node" as sn2 {
component "netclient" as sn2_netclient
component etcd as sn2_etcd
component "k3s-server" as sn2_k3s_server
sn2_k3s_server - sn2_etcd
}
sn1_netclient -- sn2_netclient
sn1_etcd -- sn2_etcd
}
rectangle "Workers" {
rectangle "Node" as an1 {
component "netclient" as an1_netclient
component "k3s-agent" as sn1_k3s_agent
}
}
@enduml
+35
View File
@@ -0,0 +1,35 @@
# Copyright 2018-2022 Docker Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
disabled_plugins = []
#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0
#[grpc]
# address = "/run/containerd/containerd.sock"
# uid = 0
# gid = 0
#[debug]
# address = "/run/containerd/debug.sock"
# uid = 0
# gid = 0
# level = "info"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
+19
View File
@@ -0,0 +1,19 @@
- name: reload_sysctl
command: sysctl --system
- name: restart_containerd
ansible.builtin.service:
name: containerd
state: restarted
- name: reload_networking
service:
name: networking
state: restarted
async: 5
poll: 0
notify: wait_for_connection
- name: wait_for_connection
wait_for_connection:
delay: 5
+3
View File
@@ -0,0 +1,3 @@
---
#dependencies:
# - role: docker
+68
View File
@@ -0,0 +1,68 @@
- name: Create k3s-folder
ansible.builtin.file:
path: /etc/rancher/k3s/
state: directory
mode: "0755"
- name: Read the live k3s config
slurp:
src: /etc/rancher/k3s/config.yaml
register: live_config
failed_when: false
# A rewrite restarts k3s, so the file is written only when k3s would read other
# settings: the live files differ from the rendering in comments, order and
# whitespace.
- name: Deploy k3s config
ansible.builtin.template:
src: k3s/{{ type }}/config.yaml.jinja2
dest: /etc/rancher/k3s/config.yaml
register: config
when: >-
live_config.content is not defined or
(live_config.content | b64decode | from_yaml) != (lookup('template', 'k3s/' ~ type ~ '/config.yaml.jinja2') | from_yaml)
- name: Read the installed k3s version
command: k3s --version
register: k3s_installed
changed_when: false
failed_when: false
check_mode: false
- name: Download install-script
get_url:
url: https://get.k3s.io
dest: /root/k3s_install.sh
mode: "744"
# todo: update when file changed?
- name: Read the live registries.yaml
slurp:
src: /etc/rancher/k3s/registries.yaml
register: live_registries
failed_when: false
when: "type == 'server'"
# Written only on a change k3s would see, like config.yaml
- name: Deploy registries.yaml
ansible.builtin.template:
src: k3s/server/registries.yaml.j2
dest: /etc/rancher/k3s/registries.yaml
register: registries_config
when:
- "type == 'server'"
- >-
live_registries.content is not defined or
(live_registries.content | b64decode | from_yaml) != (lookup('template', 'k3s/server/registries.yaml.j2') | from_yaml)
- import_tasks: ./install/server/setup_network.yml
when: "type == 'server'"
#- import_tasks: ./install/server/install_helm.yml
# when: "type == 'server'"
- import_tasks: ./install/server/install_k3s.yml
when: "type == 'server'"
- import_tasks: ./install/agent/install_k3s.yml
when: "type == 'agent'"
@@ -0,0 +1,15 @@
- name: Install K3s agent
command: /root/k3s_install.sh {{ type }}
environment:
INSTALL_K3S_VERSION: "{{ kubernetes.version }}"
when: k3s_installed.rc != 0 or (kubernetes.version != '' and ('k3s version ' ~ kubernetes.version ~ ' ') not in k3s_installed.stdout)
register: command
changed_when: "'No change detected' not in command.stdout"
until: "command is not failed"
retries: 2
delay: 10
- name: Make sure service is started / restarted on config change
service:
name: k3s-agent
state: "{{ 'restarted' if not command.changed and config.changed else 'started' }}"
@@ -0,0 +1,17 @@
- name: Add Balto key
apt_key:
url: https://baltocdn.com/helm/signing.asc
state: present
- name: Add Balto Repository
apt_repository:
repo: "deb https://baltocdn.com/helm/stable/debian/ all main"
state: present
filename: kubernetes
update_cache: yes
- name: Install helm
package:
name:
- helm
state: latest
@@ -0,0 +1,50 @@
- block:
- name: Install K3s-server
command: /root/k3s_install.sh {{ type }}
environment:
INSTALL_K3S_CHANNEL: latest
INSTALL_K3S_VERSION: "{{ kubernetes.version }}"
when: k3s_installed.rc != 0 or (kubernetes.version != '' and ('k3s version ' ~ kubernetes.version ~ ' ') not in k3s_installed.stdout)
register: command
changed_when: "'No change detected' not in command.stdout"
until: "command is not failed"
retries: 2
delay: 10
- name: Make sure service is started / restarted on config change
service:
name: k3s
enabled: true
state: "{{ 'restarted' if not command.changed and (config.changed or registries_config.changed) else 'started' }}"
- name: Waiting for K3s-server to accept connections
ansible.builtin.wait_for:
host: "{{ inventory_hostname }}"
port: 6443
state: started
throttle: 1
- name: Ensure the .kube directory exists
file:
path: "~/.kube"
state: directory
mode: "0755"
- name: Create symbolic link for k3s.yaml
file:
src: "/etc/rancher/k3s/k3s.yaml"
dest: "~/.kube/config"
state: link
- name: Deploy calico
import_tasks: ./roles/kubernetes/tasks/install/server/network-plugin/deploy_calico.yml
when:
- "kubernetes.network.plugin == 'calico'"
- name: Deploy cilium
import_tasks: ./roles/kubernetes/tasks/install/server/network-plugin/deploy_cilium.yml
when:
- "kubernetes.network.plugin == 'cilium'"
- name: Deploy network-helpers
import_tasks: ./roles/kubernetes/tasks/install/server/network-plugin/deploy_network_helper.yml
@@ -0,0 +1,19 @@
- name: Deploy calico operator
command: kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/tigera-operator.yaml
register: command
changed_when: "'created' in command.stdout"
run_once: true
failed_when:
- "command.rc == 1 and 'AlreadyExists' not in command.stderr"
- name: Deploy calico ressource template
ansible.builtin.template:
src: ./k3s/server/network-plugin/calico/custom-ressource.yml.jinja2
dest: /root/calico-ressource.yml
run_once: true
- name: Deploy calico ressource
command: kubectl apply -f /root/calico-ressource.yml
register: command
changed_when: "'created' in command.stdout"
run_once: true
@@ -0,0 +1,71 @@
- name: Check if already installed in cluster
command: "cilium status"
register: cilium_status
ignore_errors: true
changed_when: false
check_mode: false
run_once: true
- name: Check if the Cilium CLI is installed
stat:
path: /usr/local/bin/cilium
register: cilium_cli
- name: Get Cilium CLI version
ansible.builtin.command:
argv:
- curl
- -s
- https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt
register: cilium_cli_version
check_mode: no
changed_when: false
run_once: true
- name: Determine CLI architecture
set_fact:
cli_arch: "{{ 'arm64' if ansible_architecture == 'aarch64' else 'amd64' }}"
run_once: true
- name: Download and verify Cilium CLI
block:
- name: Download Cilium CLI tar.gz file
get_url:
url: "https://github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_version.stdout }}/cilium-linux-{{ cli_arch }}.tar.gz"
dest: "/tmp/cilium-linux-{{ cli_arch }}.tar.gz"
checksum: "sha256:{{ lookup('url', 'https://github.com/cilium/cilium-cli/releases/download/' + cilium_cli_version.stdout + '/cilium-linux-' + cli_arch + '.tar.gz.sha256sum', wantlist=True)[0].split(' ')[0] }}"
mode: '0755'
- name: Extract Cilium CLI tar.gz to /usr/local/bin
unarchive:
src: "/tmp/cilium-linux-{{ cli_arch }}.tar.gz"
dest: "/usr/local/bin"
remote_src: yes
#extra_opts: ["--strip-components=1"]
mode: 777
# An installed CLI is left alone, not upgraded to the latest release
when: cilium_cli_version.stdout is defined and not cilium_cli.stat.exists
become: yes
- name: Clean up temporary files
file:
path: "/tmp/cilium-linux-{{ cli_arch }}.tar.gz"
state: absent
# Install cilium via cli
- block:
- name: Install cilium
command: >
cilium install
--set
imap.mode=kubernetes,
ipv6.enabled=true,
#ipam.operator.clusterPoolIPv4PodCIDRList={{ kubernetes.ipPool.ipv4.cluster_cidr }},
#ipam.operator.clusterPoolIPv6PodCIDRList={{ kubernetes.ipPool.ipv6.cluster_cidr }}
- name: Wait for cilium
command: cilium status --wait
changed_when: false
run_once: true
when:
- cilium_status.rc != 0
@@ -0,0 +1,7 @@
- name: Deploy service-file for routing-table to wireguard-translation
ansible.builtin.template:
src: ./k3s/server/network-plugin/calico/routingtabletowg.yml.jinja2
dest: /var/lib/rancher/k3s/server/manifests/routingtabletowg.yml
mode: u=rw,g=r,o=r
run_once: true
when: "kubernetes.network.helper.routingtabletowg"
@@ -0,0 +1,6 @@
- name: Set control-plane-dns-endpoint towards local-ip
blockinfile:
path: /etc/hosts
marker: "# {mark} ANSIBLE MANAGED BLOCK | k3s"
block: |
{{ nodeip_ipv4 }} {{ kubernetes.control_plane.dns_name }}
+6
View File
@@ -0,0 +1,6 @@
- import_tasks: ./prerequisites.yml
- import_tasks: ./node.yml
- import_tasks: ./install.yml
when: target_host is not defined or inventory_hostname == target_host
+32
View File
@@ -0,0 +1,32 @@
- name: Kubelet image garbage collection
tags: kubelet_gc
block:
- name: Create the kubelet drop-in directory
file:
path: /etc/rancher/k3s/kubelet.conf.d
state: directory
mode: "0755"
- name: Set the image garbage collection thresholds
copy:
dest: /etc/rancher/k3s/kubelet.conf.d/10-image-gc.conf
mode: "0644"
content: |
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
imageGCHighThresholdPercent: 70
imageGCLowThresholdPercent: 55
imageMaximumGCAge: 168h
# Every pod watching its config files takes inotify instances and watches from
# the per-user pool. An LXC container inherits these from its host. Only this
# file is loaded: sysctl --system would also apply distribution defaults such as
# rp_filter that differ from the live values Cilium relies on.
- name: Raise the inotify limits
tags: inotify
when: ansible_facts['virtualization_type'] != 'lxc'
ansible.posix.sysctl:
name: "{{ item.key }}"
value: "{{ item.value }}"
sysctl_file: /etc/sysctl.d/90-inotify.conf
loop: "{{ {'fs.inotify.max_user_instances': 65536, 'fs.inotify.max_user_watches': 4194304} | dict2items }}"
+61
View File
@@ -0,0 +1,61 @@
#- name: Load br_netfilter kernel-module
# modprobe:
# name: br_netfilter
# state: present
- name: Set sysctl settings for iptables bridged traffic
copy:
dest: "/etc/sysctl.d/kubernetes.conf"
content: |
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.conf.all.forwarding=1
net.ipv6.conf.all.forwarding=1
notify: reload_sysctl
#- name: Disable swap
# command: swapoff -a
- name: Install required packages
package:
name:
#- containerd
#- iptables
# For Longhorn:
- nfs-common
- open-iscsi
state: present
- import_tasks: ./prerequisites/containerd.yml
- name: Gather interface-name
set_fact:
interface: "{{ kubernetes.ipPool.nodeIp_interface | replace('-', '_') }}"
- name: Getting nodeIp-data from interface
set_fact:
nodeip_ipv4: >-
{{ (ansible_facts[interface].ipv4.address)
if (ansible_facts is defined and ansible_facts.get(interface) is defined and ansible_facts[interface].ipv4 is defined)
else (ansible_default_ipv4.address | default('')) }}
nodeip_ipv6: >-
{{ (ansible_facts[interface].ipv6[0].address)
if (ansible_facts is defined and ansible_facts.get(interface) is defined and ansible_facts[interface].ipv6 is defined and (ansible_facts[interface].ipv6 | length) > 0)
else (ansible_default_ipv6.address | default('')) }}
- block:
- name: Read resolv.conf
shell: "grep '^nameserver' /etc/resolv.conf"
register: nameserver_line
changed_when: false
check_mode: false
- name: Write /etc/resolv-k3s.conf with only nameservers
copy:
content: "{{ nameserver_line.stdout }}"
dest: /etc/resolv-k3s.conf
when: kubernetes.network.clean_resolv_conf is true
- name: Run handlers to reload configurations
meta: flush_handlers
@@ -0,0 +1,24 @@
- name: Check if containerd-service exists & is started
service:
name: containerd
state: started
ignore_errors: true
register: containerd_status
- name: Install containerd when not exists
package:
name:
- containerd
when: containerd_status is failed
- name: Create containerd config-folder
file:
path: /etc/containerd
state: directory
- name: Deploy containerd-config
ansible.builtin.copy:
src: containerd_config.toml
dest: /etc/containerd/config.toml
mode: u=rw,g=r,o=r
notify: restart_containerd
@@ -0,0 +1,53 @@
{% set disabled_components = [] %}
## Base ##
node-name: {{ ansible_fqdn }}
# Join cluster
server: https://{{ hostvars[groups['kubernetes'][0]]['nodeip_ipv4'] }}:6443
token: '{{ kubernetes.token }}'
# Networking
{% if ansible_default_ipv6 is defined %}
node-external-ip: {{ ansible_default_ipv4.address }},{{ ansible_default_ipv6.address }}
{% else %}
node-external-ip: {{ ansible_default_ipv4.address }}
{% endif %}
{% if kubernetes.network.clean_resolv_conf %}
resolv-conf: /etc/resolv-k3s.conf
{% endif %}
{% if nodeip_ipv6 != "" and kubernetes.ipPool.ipv6 is defined %}
node-ip: {{ nodeip_ipv4 }},{{ nodeip_ipv6 }}
{% else %}
node-ip: {{ nodeip_ipv4 }}
{% endif %}
# Network-plugin
{% if kubernetes.network.plugin == "flannel" %}
{% else %}
{% if kubernetes.network.plugin == "cilium" %}
# Cilium requires servicelb to be disabled when replacing kube-proxy
{% set _ = disabled_components.append('servicelb') %}
{% endif %}
{% endif %}
## Label
# Region & DC
node-label:
{% if region is defined %}
- topology.kubernetes.io/region={{ region }}
{% endif %}
{% if zone is defined %}
- topology.kubernetes.io/zone={{ zone }}
{% endif %}
{% for item in kubernetes.labels_extra %}
- {{ item }}
{% endfor %}
# Disabled components
{% if disabled_components %}
disable: {{ disabled_components | join(',') }}
{% endif %}
{{ kubernetes.config_extra | to_yaml }}
@@ -0,0 +1,91 @@
{% set disabled_components = [] %}
## Base ##
node-name: {{ ansible_fqdn }}
{% if inventory_hostname == groups['kubernetes'][0] %}
# Initialize with internal etcd
cluster-init: true
{% else %}
# Join cluster
server: https://{{ hostvars[groups['kubernetes'][0]]['nodeip_ipv4'] }}:6443
{% endif %}
token: '{{ kubernetes.token }}'
tls-san:
- {{ kubernetes.control_plane.dns_name }}
# Networking
{% if ansible_default_ipv6 is defined and ansible_default_ipv6.address is defined %}
node-external-ip: {{ ansible_default_ipv4.address }},{{ ansible_default_ipv6.address }}
{% else %}
node-external-ip: {{ ansible_default_ipv4.address }}
{% endif %}
{% if kubernetes.network.clean_resolv_conf %}
resolv-conf: /etc/resolv-k3s.conf
{% endif %}
{% if nodeip_ipv6 != "" and kubernetes.ipPool.ipv6 is defined %}
node-ip: {{ nodeip_ipv4 }},{{ nodeip_ipv6 }}
cluster-cidr: {{ kubernetes.ipPool.ipv4.cluster_cidr }},{{ kubernetes.ipPool.ipv6.cluster_cidr }}
service-cidr: {{ kubernetes.ipPool.ipv4.service_cidr }},{{ kubernetes.ipPool.ipv6.service_cidr }}
{% else %}
node-ip: {{ nodeip_ipv4 }}
cluster-cidr: {{ kubernetes.ipPool.ipv4.cluster_cidr }}
service-cidr: {{ kubernetes.ipPool.ipv4.service_cidr }}
{% endif %}
# Pin the apiserver advertise-address to the mesh IP: node-external-ip above
# otherwise gets picked, leaking a LAN-only address into the default/kubernetes
# Endpoints object that other nodes cannot route to.
advertise-address: {{ nodeip_ipv4 }}
egress-selector-mode: disabled
# Network-plugin
{% if kubernetes.network.plugin == "flannel" %}
flannel-backend: vxlan
{% else %}
disable-network-policy: true
flannel-backend: none
{% if kubernetes.network.plugin == "cilium" %}
# Cilium requires servicelb to be disabled when replacing kube-proxy
{% set _ = disabled_components.append('servicelb') %}
{% endif %}
{% endif %}
# Ingress-plugin
{% if kubernetes.ingress_controller != "traefik-ingress" %}
{% set _ = disabled_components.append('traefik') %}
{% endif %}
## Label
# Region & DC
node-label:
{% if region is defined %}
- topology.kubernetes.io/region={{ region }}
{% endif %}
{% if zone is defined %}
- topology.kubernetes.io/zone={{ zone }}
{% endif %}
{% if kubernetes.labels_extra is defined %}
{% for item in kubernetes.labels_extra %}
- {{ item }}
{% endfor %}
{% endif %}
# Disabled components
{% if disabled_components %}
disable: {{ disabled_components | join(',') }}
{% endif %}
{{ kubernetes.config_extra | to_yaml }}
# Image garbage collection tuning (drop-in KubeletConfiguration)
kubelet-arg:
- config-dir=/etc/rancher/k3s/kubelet.conf.d
{% if not kubernetes.config_extra['disable-etcd'] | default(false) %}
# etcd members only
{{ kubernetes.etcd_config_extra | to_nice_yaml(indent=2) }}
{% endif %}
@@ -0,0 +1,34 @@
# This section includes base Calico installation configuration.
# For more information, see: https://projectcalico.docs.tigera.io/master/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: 26
cidr: {{ kubernetes.ipPool.ipv4.cluster_cidr }}
encapsulation: None
natOutgoing: Enabled
nodeSelector: all()
{% if nodeip_ipv6 != "" and kubernetes.ipPool.ipv6 is defined %}
- blockSize: 122
cidr: {{ kubernetes.ipPool.ipv6.cluster_cidr }}
encapsulation: None
natOutgoing: Enabled
nodeSelector: all()
{% endif %}
---
# This section configures the Calico API server.
# For more information, see: https://projectcalico.docs.tigera.io/master/reference/installation/api#operator.tigera.io/v1.APIServer
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec: {}
@@ -0,0 +1,45 @@
# https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: routingtabletowg
namespace: calico-system
labels:
app: routingtabletowg
spec:
selector:
matchLabels:
app: routingtabletowg
template:
metadata:
labels:
app: routingtabletowg
spec:
tolerations:
# this toleration is to have the daemonset runnable on master nodes
# remove it if your masters can't run pods
- key: node-role.kubernetes.io/master
effect: NoSchedule
hostNetwork: true
containers:
- name: routingtabletowg
image: "ruakij/routingtabletowg:0.2.0"
env:
- name: INTERFACE
value: {{ kubernetes.ipPool.nodeIp_interface }}
- name: FILTER_PROTOCOL
value: bird
- name: PERIODIC_SYNC
value: '300'
securityContext:
capabilities:
add:
- NET_ADMIN
resources:
requests:
cpu: 10m
memory: 10Mi
limits:
cpu: 20m
memory: 20Mi
---
@@ -0,0 +1 @@
{{ kubernetes.registries_config | to_nice_yaml(indent=2) }}
@@ -0,0 +1,6 @@
netbird_client:
# Key and url to join a network
# leave empty to ignore
join_network:
setup_key:
management_url:
+26
View File
@@ -0,0 +1,26 @@
- name: Install Packages
# when: docker_file.stat.exists == False
package:
name:
- ca-certificates
- curl
- gnupg
- name: Add netbird-key
apt_key:
url: https://pkgs.wiretrustee.com/debian/public.key
state: present
- name: Add netbird-repository
apt_repository:
repo: "deb https://pkgs.wiretrustee.com/debian stable main"
state: present
filename: netbird
update_cache: yes
- name: Install wireguard & netbird
package:
name:
- wireguard
- netbird
state: latest
+16
View File
@@ -0,0 +1,16 @@
- name: Join netbird-network
when: "netbird_client.join_network.setup_key is defined"
command: "netbird up --management-url {{ netbird_client.join_network.management_url }} --setup-key {{ netbird_client.join_network.setup_key }}"
failed_when: command.rc != 0
changed_when: "'Connected' in command.stdout"
register: command
- name: Wait for netbird-interface to exist
wait_for:
path: "/sys/class/net/wt0"
state: present
when: command.changed
- name: Gather facts to get changes
ansible.builtin.gather_facts:
when: command.changed
+4
View File
@@ -0,0 +1,4 @@
- import_tasks: ./install.yml
- import_tasks: ./join-network.yml
+1
View File
@@ -0,0 +1 @@
DPkg::Post-Invoke { "/usr/local/sbin/pve-no-nag"; };
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
# Disables the "No valid subscription" dialog of the Proxmox web UI. Called by the apt hook
# 86pve-no-nag, because every proxmox-widget-toolkit update restores the file. Without a match
# (changed upstream code) the dialog returns and nothing breaks.
f=/usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js
[ -f "$f" ] && sed -i 's/checked_command: function (orig_cmd) {$/& orig_cmd(); return;/' "$f"
exit 0
+15
View File
@@ -0,0 +1,15 @@
- name: Install the subscription nag remover
copy:
src: pve-no-nag
dest: /usr/local/sbin/pve-no-nag
mode: "0755"
- name: Run it after every dpkg run
copy:
src: 86pve-no-nag
dest: /etc/apt/apt.conf.d/86pve-no-nag
mode: "0644"
- name: Remove the subscription nag
command: /usr/local/sbin/pve-no-nag
changed_when: false
+2
View File
@@ -0,0 +1,2 @@
tuned:
profile:
+6
View File
@@ -0,0 +1,6 @@
- name: Install Packages
package:
name:
- tuned
- tuned-utils
- tuned-utils-systemtap
+5
View File
@@ -0,0 +1,5 @@
- import_tasks: ./install-packages.yml
- import_tasks: ./setup-service.yml
- import_tasks: ./set-profile.yml
+5
View File
@@ -0,0 +1,5 @@
- name: Set profile
command: tuned-adm profile throughput-performance
register: command
failed_when:
- "command.rc != 0"
+5
View File
@@ -0,0 +1,5 @@
- name: Make sure service is started
service:
name: tuned
enabled: true
state: "started"
@@ -0,0 +1,12 @@
wireguard_ipv6_converter:
version: latest
# see https://github.com/Ruakij/wg-ipv6-converter#31-environment
setup:
interface: wg0
#ipv6_format: fc12::%02x%02x:%02x%02x/%d
#filter_prefix: 100.100
#recheck_interval: 60s
service:
#bindTo: netbird.service
+11
View File
@@ -0,0 +1,11 @@
- name: Get architecture
set_fact:
arch: "{{ 'amd64' if ansible_architecture == 'x86_64' else 'arm64' }}"
versionUri: "{% if wireguard_ipv6_converter.version == 'latest' %}latest/download{% else %}download/{{ wireguard_ipv6_converter.version }}{% endif %}"
- name: Download binary
get_url:
url: https://github.com/Ruakij/wg-ipv6-converter/releases/{{ versionUri }}/wg-ipv6-converter_{{ arch }}
dest: /usr/local/bin/wg-ipv6-converter
mode: "744"
register: deployDownload
+3
View File
@@ -0,0 +1,3 @@
- import_tasks: ./deploy.yml
- import_tasks: ./setup-service.yml
@@ -0,0 +1,47 @@
- name: Deploy service
ansible.builtin.template:
src: wg-ipv6-conv.service.jinja2
dest: /etc/systemd/system/wg-ipv6-converter_{{ wireguard_ipv6_converter.setup.interface }}.service
register: serviceFile
- name: Enable service
ansible.builtin.service:
name: wg-ipv6-converter_{{ wireguard_ipv6_converter.setup.interface }}
daemon-reload: true
enabled: true
- name: Ensure drop-in directory for the bound service exists
ansible.builtin.file:
path: /etc/systemd/system/{{ wireguard_ipv6_converter.service.bindTo }}.d
state: directory
mode: "0755"
when: "wireguard_ipv6_converter.service.bindTo is defined"
- name: Deploy start-propagation drop-in for the bound service
ansible.builtin.template:
src: wg-ipv6-conv-restart.conf.jinja2
dest: /etc/systemd/system/{{ wireguard_ipv6_converter.service.bindTo }}.d/10-wg-ipv6-converter_{{ wireguard_ipv6_converter.setup.interface }}.conf
mode: "0644"
when: "wireguard_ipv6_converter.service.bindTo is defined"
register: upholdsFile
- name: Reload systemd for drop-in
ansible.builtin.systemd:
daemon_reload: true
when: "upholdsFile.changed"
- name: Start service if interface exists already
ansible.builtin.service:
name: wg-ipv6-converter_{{ wireguard_ipv6_converter.setup.interface }}
state: "restarted"
register: service
when: "wireguard_ipv6_converter.setup.interface in ansible_interfaces"
- name: Pause for 5s to wait for program to have run
ansible.builtin.pause:
seconds: 5
when: "service.changed"
- name: Gather facts to get changes
ansible.builtin.gather_facts:
when: "service.changed"
@@ -0,0 +1,7 @@
# BindsTo= in the converter unit only propagates stop, so a restart of
# {{ wireguard_ipv6_converter.service.bindTo }} leaves the converter dead and the re-created
# interface without its IPv6 addresses. Start it again whenever
# {{ wireguard_ipv6_converter.service.bindTo }} starts. --no-block because the converter orders
# itself After={{ wireguard_ipv6_converter.service.bindTo }}.
[Service]
ExecStartPost=/bin/systemctl --no-block start wg-ipv6-converter_{{ wireguard_ipv6_converter.setup.interface }}.service
@@ -0,0 +1,29 @@
[Unit]
Description=WireGuard IPv6 converter for {{ wireguard_ipv6_converter.setup.interface }}
{% if wireguard_ipv6_converter.service.bindTo is defined %}
BindsTo={{ wireguard_ipv6_converter.service.bindTo }}
After={{ wireguard_ipv6_converter.service.bindTo }}
{% endif %}
[Service]
Type=simple
{% if wireguard_ipv6_converter.service.bindTo is defined %}
ExecStartPre=/bin/sleep 10
{% endif %}
ExecStart=/usr/local/bin/wg-ipv6-converter
Restart=always
RestartSec=30
Environment="INTERFACE={{ wireguard_ipv6_converter.setup.interface }}"
{% if wireguard_ipv6_converter.setup.ipv6_format is defined %}
Environment="IPV6_FORMAT={{ wireguard_ipv6_converter.setup.ipv6_format }}"
{% endif %}
{% if wireguard_ipv6_converter.setup.filter_prefix is defined %}
Environment="FILTER_PREFIX={{ wireguard_ipv6_converter.setup.filter_prefix }}"
{% endif %}
{% if wireguard_ipv6_converter.setup.recheck_interval is defined %}
Environment="RECHECK_INTERVAL={{ wireguard_ipv6_converter.setup.recheck_interval }}"
{% endif %}
[Install]
WantedBy=multi-user.target