The base needs infrastructure to be operational.
One key basic needs of every human beeing besides a room, power, water and Mate is of cause Internet
When we had the go to move into the room there was already a Internet connection and we are allowed to use it.
The three parties in the building should be segmented from each other.
The resulting infrastructure description should be generic and the concept usable by other Hackerspaces or even smaller companies.
The WAN port of the Fritzbox is unusable, because it is only usable for DSL. We have FTTB
Every party will get one port on the Fritzbox.
The fritzbox must stay at the base. It will be a shared environment, but it cannot be avoided for now.
The Qnap needs to be moved to the top floor.
New highlevel layout
+-------------+
| Fritzbox |
| 192.168.1.1 |
+-+----+----+-+
| | |
+------------------+ | +-----------------------+
| | |
3rd floor | 2nd floor| 1st+floor|
| | |USB-NIC (vmbr1, external)
| | |
+---------+---------+ +---------+---------+ +--------------+--------------------------------------+
| WLAN Router | | WLAN Router | | |
| 192.168.1.30 | | 192.168.1.20 | | +------------------+------------------+
| NAT | | NAT | | |192.168.1.10 vmbr1|192.168.1.11 vmbr1|
| 192.168.30.0/24 | | 192.168.20.0/24 | | Proxmox | +------+ | +------+ |
+-------------------+ +-------------------+ | 192.168.11.10 | pfSense VM | rproxy01 |
| vmbr0 | +------+ | +------+ |
| |192.168.10.1 vmbr0| |
| |192.168.11.1 vmbr2| |
| +------------------+------------------+
| |
+-----------+---+-------------------------------------+
| |
Virtual Servers (vmbr2, servers)| |Onboard-NIC (vmbr0, Internal)
| |
+----------+ +----------------+
| |
192.168.10.0/24| |192.168.11.0/24
| |
+--------+-------+ +-------+------+
| Virtual Switch | | Switch |
+------+--+------+ +-----+---+----+
| | | |
+----+ +----+ +-----+ +-----+
| | | |
+----------+--+ +-----+-------+ +-----+------+ +-----+------+
| auth VM | | chat VM | | AP | | Freifunk |
|192.168.10.11| |192.168.10.12| |192.168.11.3| | |
+-------------+ +-------------+ +------------+ +------------+
Fritzbox
192.168.1.0/24
This is the shared network and it belongs to the building itself.
The Fritzbox itself is managed by trinitor
Every floor can have 10 static IPs in the shared network
1st floor = 192.168.1.10 - 19
2nd floor = 192.168.1.20 - 29
3rd floor = 192.168.1.30 - 39
There is also a DHCP range, but it is more for emergency use.
Our /dev/base port will be connected to a Proxmox server = our virtual environment.
This port will be dedicated to a VM where we have a firewall running. In this case a pfSense.
The firewall VM will have three virtual NICs.
1. WAN = Fritzbox = Internet
2. virtual switch = Server = vSwitch with no physical NICs attached. All VMs will be connected to the vSwitch. The traffic will stay inside the host.
3. physical switch = the /dev/base internal network
The firewall will protect the /dev/base infrastructure from the Internet and the other parties in the building.
The VMs are in a segmented VLAN to have them also controlled in a better way.
| IP | Hostname | Comment |
|---|---|---|
| External | ||
| 192.168.1.1 | router | fritzbox |
| Firewall | ||
| 192.168.1.10 | fw01 | WAN = fritzbox connection |
| 192.168.10.1 | fw01 | virtual switch |
| 192.168.11.1 | fw01 | physical switch |
| Hardware | ||
| 192.168.11.2 | switch01 | |
| 192.168.11.3 | accesspoint01 | radius AP |
| 192.168.11.4 | accesspoint01 | freifunk AP |
| 192.168.11.10 | virt01 | proxmox server |
| 192.168.11.11 | virt02 | proxmox server |
| 192.168.11.20 | music01 | RuneAudio RPi |
| Virtual Machines | ||
| 192.168.1.11 | rproxy01 | nginx reverse proxy |
| 192.168.10.11 | auth01 | UCS |
| 192.168.10.12 | chat01 | rocket chat |
There are some options (ESXi, XenServer, oVirt, …), but the simplest one seems to be Proxmox.
An old Dell Notebook with an additional USB NIC will be used for now.
Network
| Interface | Comment |
|---|---|
| Network Cards | |
| eno1 | internal NIC - connected to the /dev/base switch |
| enx9410… | wireless (unused) |
| enx9ceb… | USB NIC - connected to the Fritzbox |
| Bridges (virtual switches) | |
| vmbr0 | Internal Bridge, eno1, 192.168.11.10 |
| vmbr1 | External Bridge, enx9ceb, no IP assigned |
| vmbr2 | Server Bridge, no NIC connected, no IP assigned |
Other settings
vi /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet consoleblank=10"
grub-mkconfig -o /boot/grub/grub.cfg
Management:
https://192.168.11.10:8006
There is also a second Proxmox server with only one NIC.
It is running on a MacMini and will only be turned on if needed to safe power.
Proxmox runs on Apple Hardware, but there are some driver issues.
Disable modules:
cat >> /etc/modprobe.d/pve-blacklist.conf <<EOF blacklist pcspkr blacklist b43 EOF
don't enable readondriver (which breaks to console)
vi /etc/defaults/grub
GRUB_CMDLINE_LINUX_DEFAULT="nomodeset debug"
update-grub
Management:
https://192.168.11.11:8006
pfsense installed inside a VM
Default settings
Network Setup
| Proxmox Bridge | Proxmox device name | Internal device name | Interface Name | IP |
|---|---|---|---|---|
| vmbr1 | net0 | em0 | WAN | 192.168.1.10/24 |
| vmbr0 | net1 | em1 | LAN | 192.168.11.1/24 |
| vmbr2 | net2 | em2 | SERVER | 192.168.10.1/24 |
DHCP
enabled for LAN and SERVER
192.168.10.100-199
192.168.11.100-199
Firewall Rules
NAT
disabled on pfsense. The House Network is routed, so we could share resources (printers?)
Static routes are configured on the fritzbox for this reason
Management:
https://192.168.10.1
https://192.168.11.1
Structure:
Groups:
LDAP:
Management:
https://192.168.10.11
The Access Point will broadcast three SSIDs.
| /dev/base PSK | WPA2/AES Personal | fallback |
| /dev/base | WPA2/AES Enterprise | member network |
| /dev/guest | WPA2/AES Personal | guest only |
For not both share the same subnet and are not segmented.
Advantage: might be easier to change the password, implement radius
Longterm there should be a Freifunk for guests
Segmenting the guest network means we have to use VLANs in Proxmox, pfsense, switch, and access point.
Without VLANs we can just take the cable from the AP and plug it into the Fritzbox.
With VLANs we have no easy fallback in case theres an issue with the Proxmox server
| SSID | /dev/base |
| Version | WPA2 |
| Encryption | AES |
| Radius Server IP | 192.168.10.11 |
| Radius Port | 1812 |
| Radius Password | xxx |
| Group Key Update Period | 0 |
vi /etc/freeradius/clients.conf
client 192.168.11.3 {
secret=xxx
shortname=ap01
}
univention-radius-check-access –username=trinitor
LDAP:
{"filter": "(&(objectCategory=person)(objectclass=user)(memberOf:1.2.840.113556.1.4.1941:=CN=devbase_delegate_access_rocketchat,CN=Groups,DC=devbase,DC=org)(sAMAccountName=#{username}))", "scope": "sub", "userDN": "cn=s-rocketchat,cn=users,dc=devbase,dc=org", "password": "xxx"}
Login:
http://192.168.10.12:3000/
There will be multiple services on the internal network with need certificates
Lets encrypt should be used when possible
port 80 will point to one server
Fritzbox configuration:
fw01.devbase.org configuration
Server Configuration:
iface ens18 inet static address 192.168.1.11/24 gateway 192.168.1.1 dns-nameservers 192.168.1.1
server {
listen 80;
server_name chat.devbase.org;
index index.html index.htm;
location / {
alias /var/www/chat_devbase.org/;
}
}
server {
#client_max_body_size 80M;
listen 443 ssl default_server;
server_name chat.devbase.org;
ssl on;
ssl_certificate /etc/letsencrypt/live/chat.devbase.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/chat.devbase.org/privkey.pem;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://192.168.1.10:3000;
proxy_redirect off;
}
}
client 192.168.11.1 {
secret=xxx
shortname=fw01
}
Alias /nextcloud "/var/www/nextcloud/"
<Directory /var/www/nextcloud/>
Options +FollowSymlinks
AllowOverride All
<IfModule mod_dav.c>
Dav off
</IfModule>
SetEnv HOME /var/www/nextcloud
SetEnv HTTP_HOME /var/www/nextcloud
</Directory>
EOF
* ln -s /etc/apache2/sites-available/nextcloud.conf /etc/apache2/sites-enabled/
* mysql_secure_installation
* mysql -u root -p
* CREATE USER 'nextcloud'@'localhost' IDENTIFIED BY 's3cret';
* CREATE DATABASE nextcloud;
* GRANT ALL PRIVILEGES ON nextcloud.* TO 'nextcloud'@'localhost';
* cat >> /etc/php/7.0/apache2/php.ini <<EOF
<code>
opcache.enable=1
opcache.enable_cli=1
opcache.interned_strings_buffer=8
opcache.max_accelerated_files=10000
opcache.memory_consumption=128
opcache.save_comments=1
opcache.revalidate_freq=1
EOF
gitlab_rails['ldap_enabled'] = false
gitlab_rails['ldap_servers'] = YAML.load <<-'EOS'
main: # 'main' is the GitLab 'provider ID' of this LDAP server
label: 'LDAP'
host: 'auth01.devbase.org'
port: 389
uid: 'sAMAccountName'
bind_dn: 'CN=s-gitlab,CN=Users,DC=devbase,DC=org'
password: 'mylittlepassword'
encryption: 'start_tls' # "start_tls" or "simple_tls" or "plain"
verify_certificates: false
active_directory: true
allow_username_or_email_login: false
lowercase_usernames: false
block_auto_created_users: false
base: 'CN=Users,DC=devbase,DC=org'
# user_filter: ''
EOS
Background music should not be interrupted by rebooting clients or phones leaving the building.
A dedicated client should play the music and controllable by all kind of devices.
audio_output {
type "httpd"
name "My HTTP Stream"
encoder "flac" # optional, vorbis or lame
port "8000"
bind_to_address "0.0.0.0" # optional, IPv4 or IPv6
quality "5.0" # do not define if bitrate is defined
# bitrate "128" # do not define if quality is defined
format "44100:16:1"
max_clients "0" # optional 0=no limit
}
EOF
Management:
http://music01.devbase.org
To manage the Unfi WiFi access points a Controller is needed