User Tools

Site Tools


projects:base-infrastructure

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
projects:base-infrastructure [2017-09-12 13:45]
trinitor
projects:base-infrastructure [2018-05-13 08:58] (current)
trinitor [gitlab]
Line 20: Line 20:
 New highlevel layout New highlevel layout
 <code> <code>
-                           +------------+ +                           +-------------+ 
-                            Fritzbox +                             Fritzbox  
-                           +-+----+----+++                           | 192.168.1.1 
 +                           +-+----+----+-+
                                 |    |                                 |    |
-          +------------------+    |    +---------------------+ +          +------------------+    |    +-----------------------+ 
-          |                                                +          |                                                  
-3rd floor |              2nd floor|                1st+floor | +3rd floor |              2nd floor|                   1st+floor| 
-          |                                                +          |                                                  |USB-NIC (vmbr1, external) 
-+---------+---------+   +---------+---------+                | +          |                                                  
-|    WLAN Router    |   |    WLAN Router    |        +-------+--------------------+ ++---------+---------+   +---------+---------+   +--------------+--------------------------------------+ 
-|   192.168.1.30    |     192.168.1.20    |        |                            | +|    WLAN Router    |      WLAN Router    |                                                       | 
-|        NAT        |   |        NAT        |        |               +------------+ +|   192.168.1.30    |     192.168.1.20    |                 +------------------+------------------+ 
-|  192.168.30.0/24  |    192.168.20.0/24        |    Proxmox    |192.168.1.10+|        NAT        |          NAT        |                 |192.168.1.10 vmbr1|192.168.1.11 vmbr1| 
-+-------------------+   +-------------------+        | 192.168.11.10 |  +------+  | +|  192.168.30.0/24  |    192.168.20.0/24   |    Proxmox    |     +------+         +------+     
-                                                                   | pfSense VM | ++-------------------+   +-------------------+   | 192.168.11.10 |    pfSense VM    |     rproxy01     | 
-                                                                   |  +------+  +                                                |     vmbr0         +------+         +------+     
-                                                     |               |192.168.10.1| +                                                |               |192.168.10.1 vmbr0|                  
-                                                     +-------------------+---+----+ +                                                              |192.168.11.1 vmbr2|                  | 
-                                                                         |   | +                                                |               +------------------+------------------+ 
-                                                 +-----------------------+   +---++                                                |                                                     | 
 +                                                +-----------+---+-------------------------------------+ 
 +                                                              | 
 +                            Virtual Servers (vmbr2, servers)|   |Onboard-NIC (vmbr0, Internal) 
 +                                                            |   | 
 +                                                 +----------+   +----------------+
                                                                                |                                                                                |
                                   192.168.10.0/24|                               |192.168.11.0/24                                   192.168.10.0/24|                               |192.168.11.0/24
Line 49: Line 55:
                                           +----+  +----+                 +-----+   +-----+                                           +----+  +----+                 +-----+   +-----+
                                           |            |                               |                                           |            |                               |
-                               +----------+--+   +-----+-------+    +----+-----+   +-----+----+ +                               +----------+--+   +-----+-------+   +-----+------+  +-----+------+ 
-                                 auth VM      chat VM    |       AP      | Freifunk | +                                 auth VM      chat VM    |       AP       Freifunk  
-                               |192.168.10.11|   |192.168.10.12|                        +                               |192.168.10.11|   |192.168.10.12|   |192.168.11.3            
-                               +-------------+   +-------------+    +----------+   +----------++                               +-------------+   +-------------+   +------------+  +------------+
  
 </code> </code>
Line 61: Line 67:
 This is the shared network and it belongs to the building itself. \\ This is the shared network and it belongs to the building itself. \\
 The Fritzbox itself is managed by trinitor \\ The Fritzbox itself is managed by trinitor \\
-Be biggest challenge will the the fight for port forwarding. \\ 
-A shared webserver in the shared network as a reverse proxy could solve the 80/443 fight. \\ 
-Will be done when we have the need. 
  
 Every floor can have 10 static IPs in the shared network \\ Every floor can have 10 static IPs in the shared network \\
Line 95: Line 98:
 |192.168.11.4  |accesspoint01 |freifunk AP | |192.168.11.4  |accesspoint01 |freifunk AP |
 |192.168.11.10 |virt01        |proxmox server | |192.168.11.10 |virt01        |proxmox server |
 +|192.168.11.11 |virt02        |proxmox server |
 +|192.168.11.20 |music01       |RuneAudio RPi |
 ^Virtual Machines ^^^ ^Virtual Machines ^^^
 +|192.168.1.11  |rproxy01      |nginx reverse proxy |
 |192.168.10.11 |auth01        |UCS | |192.168.10.11 |auth01        |UCS |
 |192.168.10.12 |chat01        |rocket chat | |192.168.10.12 |chat01        |rocket chat |
  
-==== Virtualisation ====+==== Virtualisation (main) ====
 There are some options (ESXi, XenServer, oVirt, ...), but the simplest one seems to be Proxmox. \\ There are some options (ESXi, XenServer, oVirt, ...), but the simplest one seems to be Proxmox. \\
 An old Dell Notebook with an additional USB NIC will be used for now. \\ An old Dell Notebook with an additional USB NIC will be used for now. \\
Line 121: Line 127:
 Management: \\ Management: \\
 [[https://192.168.11.10:8006]] [[https://192.168.11.10:8006]]
 +
 +==== Virtualisation (secondary) ====
 +There is also a second Proxmox server with only one NIC. \\
 +It is running on a MacMini and will only be turned on if needed to safe power. \\
 +
 +Proxmox runs on Apple Hardware, but there are some driver issues. \\
 +Disable modules: \\
 +<code>
 +cat >> /etc/modprobe.d/pve-blacklist.conf <<EOF                               
 +blacklist pcspkr
 +blacklist b43
 +EOF
 +</code>
 +
 +don't enable readondriver (which breaks to console) \\
 +vi /etc/defaults/grub
 +<code>
 +GRUB_CMDLINE_LINUX_DEFAULT="nomodeset debug"
 +</code>
 +update-grub
 +
 +Management: \\
 +[[https://192.168.11.11:8006]]
  
 ==== Firewall ==== ==== Firewall ====
Line 271: Line 300:
 [[http://192.168.10.12:3000/]] [[http://192.168.10.12:3000/]]
  
-encryption: \\ +==== Reverse Proxy ==== 
-  * apt-get install nginx +There will be multiple services on the internal network with need certificates \\ 
-  * self-sigend cert +Lets encrypt should be used when possible \\ 
-    * mkdir /etc/ssl/private + 
-    * chmod 700 /etc/ssl/private +port 80 will point to one server \\ 
-    * openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt + 
-      Country Name (2 letter code) [AU]:DE +Fritzbox configuration: 
-      State or Province Name (full name) [Some-State]:BY +  * forward port 80 and 443 to 192.168.1.11 
-      Locality Name (eg, city) []:UZB + 
-      Organization Name (eg, company) [Internet Widgits Pty Ltd]:devbase +fw01.devbase.org configuration 
-      * Organizational Unit Name (eg, section) []:    +  * Firewall -> NAT 
-      * Common Name (e.g. server FQDN or YOUR name) []:chat.devbase.org +  * new 
-      Email Address []:admin@devbase.org +    * Interface: WAN 
-  * vi /etc/nginx/sites-enabled/default+    * Destination: WAN address 
 +    * Destination port rang: 3000 3000 
 +    * Redirect target IP192.168.10.12 
 +    Redirect target port3000 
 +  Firewall -> Rules -> WAN 
 +  move newly created NAT rule to the right place in the ruleset 
 +  don't forget to save and activate 
 + 
 +Server Configuration
 +  Install Ubuntu 16.04 LTS 
 +  * vi /etc/network/interfaces
 <code> <code>
 +iface ens18 inet static
 +  address 192.168.1.11/24
 +  gateway 192.168.1.1
 +  dns-nameservers 192.168.1.1
 +</code>
 +  * apt-get install software-properties-common
 +  * add-apt-repository ppa:certbot/certbot
 +  * apt-get update
 +  * apt-get install certbot nginx
 +  * mkdir /var/www/chat_devbase_org
 +  * vi /etc/nginx/sites-available/chat_devbase_org
 +<code>
 +server {
 +    listen 80;
 +    server_name chat.devbase.org;
 +    index index.html index.htm;
 +    location / {
 +        alias /var/www/chat_devbase.org/;
 +    }
 +}
 +
 server { server {
     #client_max_body_size 80M;     #client_max_body_size 80M;
Line 292: Line 352:
  
     ssl          on;     ssl          on;
-    ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt; +    ssl_certificate /etc/letsencrypt/live/chat.devbase.org/fullchain.pem; 
-    ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; +    ssl_certificate_key /etc/letsencrypt/live/chat.devbase.org/privkey.pem;
-    #ssl_certificate /etc/letsencrypt/live/chat.devbase.org/fullchain.pem; +
-    #ssl_certificate_key /etc/letsencrypt/live/chat.devbase.org/privkey.pem;+
  
     location / {     location / {
Line 302: Line 360:
         proxy_set_header Host $http_host;         proxy_set_header Host $http_host;
         proxy_set_header X-NginX-Proxy true;         proxy_set_header X-NginX-Proxy true;
-        proxy_pass http://127.0.0.1:3000;    +        proxy_pass http://192.168.1.10:3000;
         proxy_redirect off;         proxy_redirect off;
     }     }
 } }
 </code> </code>
-  * nginx -+  * ln -s /etc/nginx/sites-available/chat_devbase_org /etc/nginx/sites-enabled 
-  * systemctl start nginx +  * systemctl restart nginx.service 
-  * systemctl enable nginx +  * certbot certonly --webroot -/var/www/chat_devbase.org -d chat.devbase.org 
-  * lets encrypt cert +  * systemctl restart nginx.service 
-    * apt-get install letsencrypt +  * echo "3 * * * root certbot renew" >> /etc/crontab
-    * letsencrypt certonly --standalone -d chat.devbase.org +
-    * change ss-certificate and ssl_certificate_key lines in /etc/nginx/sites-enabled/default +
- +
-==== Chat (Matrix) ==== +
-  * Install Ubuntu 16.04 LTS +
-  * apt update && sudo apt upgrade +
-  * add-apt-repository https://matrix.org/packages/debian/ +
-  * wget https://matrix.org/packages/debian/repo-key.asc -O key +
-  * apt-key add - < key +
-  * apt update +
-  * apt install matrix-synapse python-matrix-synapse-ldap3 +
-    * Server Name: matrix.devbase.org +
-  * cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 34 | head -1 +
-  * vi /etc/matrix-synapse/homeserver.yaml +
-    * registration_shared_secret: your_random_string +
-  * systemctl enable matrix-synapse.service +
-  * systemctl start matrix-synapse.service +
-  register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost:8448 +
-    * name root +
-    * set password +
-    * make admin = yes +
- +
-Public DNS Recort for federation +
-SRV matrix_tcp.devbase.org +
  
 ==== VPN ==== ==== VPN ====
Line 398: Line 432:
  
 ==== nextcloud ==== ==== nextcloud ====
 +  * install Ubuntu 16.04 LTS
 +  * configure network
 +  * apt update && sudo apt upgrade
 +  * apt-get install apache2 mariadb-server libapache2-mod-php7.0
 +  * apt-get install php7.0-gd php7.0-json php7.0-mysql php7.0-curl php7.0-mbstring php7.0-intl php7.0-mcrypt php-imagick php7.0-xml php7.0-zip php7.0-ldap
 +  * mkdir /opt/install && cd /opt/install
 +  * wget "https://download.nextcloud.com/server/releases/nextcloud-12.0.2.tar.bz2"
 +  * tar xf nextcloud*.tar.bz2
 +  * mv nextcloud /var/www/
 +  * chown -R www-data.www-data /var/www/nextcloud/
 +  * cat >/etc/apache2/sites-available/nextcloud.conf <<EOF
 +<code>
 +Alias /nextcloud "/var/www/nextcloud/"
 + 
 +<Directory /var/www/nextcloud/>
 +   Options +FollowSymlinks
 +   AllowOverride All
 + 
 +   <IfModule mod_dav.c>
 +     Dav off
 +   </IfModule>
 + 
 +   SetEnv HOME /var/www/nextcloud
 +   SetEnv HTTP_HOME /var/www/nextcloud
 +</Directory>
 +EOF
 +  * ln -s /etc/apache2/sites-available/nextcloud.conf /etc/apache2/sites-enabled/
 +  * mysql_secure_installation
 +    * mysql -u root -p
 +      * CREATE USER 'nextcloud'@'localhost' IDENTIFIED BY 's3cret';
 +      * CREATE DATABASE nextcloud;
 +      * GRANT ALL PRIVILEGES ON nextcloud.* TO 'nextcloud'@'localhost';
 +  * cat >> /etc/php/7.0/apache2/php.ini <<EOF
 +<code>
 +opcache.enable=1
 +opcache.enable_cli=1
 +opcache.interned_strings_buffer=8
 +opcache.max_accelerated_files=10000
 +opcache.memory_consumption=128
 +opcache.save_comments=1
 +opcache.revalidate_freq=1
 +EOF
 +</code>
 +  * systemctl restart apache2.service
 +  * browse to http://192.168.10.14/nextcloud/
 +    * enter new user credentials for admin user
 +    * configure database
 +  * enable apps: 
 +    * calendar
 +    * contacts
 +    * deck
 +    * tasks
 +    * LDAP user and group backend
 +      * Admin -> LDAP
 +        * Advanced 
 +          * Turn off SSL certificate validation = checked
 +        * Server
 +          * Server: ldaps:/ /auth01.devbase.org:636 
 +          * User: cn=s-nextcloud,cn=users,dc=devbase,dc=org
 +          * Base DN: dc=devbase,dc=org
 +        * Users
 +          * persons
 +        * Login Attributes
 +          * LDAP user
 +        * Group
 +          * devbase_delegate_access_nextcloud
  
 +==== gitlab ====
 +  * install Ubuntu 16.04 LTS
 +  * configure network
 +  * apt update && sudo apt upgrade
 +  * sudo apt-get install -y curl openssh-server ca-certificates
 +  * curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh | sudo bash
 +  * sudo echo en_US.UTF-8 UTF-8 > /etc/locale.gen
 +  * sudo locale-gen en_US.UTF-8
 +  * LC_ALL="en_US.UTF-8"
 +  * LC_CTYPE="en_US.UTF-8"
 +  * sudo EXTERNAL_URL="http://gitlab.devbase.org" apt-get install gitlab-ee
 +  * browse to http://git.devbase.org -> set root password
 +  * vi /etc/gitlab/gitlab.rb
 +<code>
 +gitlab_rails['ldap_enabled'] = false
 +gitlab_rails['ldap_servers'] = YAML.load <<-'EOS'
 +  main: # 'main' is the GitLab 'provider ID' of this LDAP server
 +    label: 'LDAP'
 +    host: 'auth01.devbase.org'
 +    port: 389
 +    uid: 'sAMAccountName'
 +    bind_dn: 'CN=s-gitlab,CN=Users,DC=devbase,DC=org'
 +    password: 'mylittlepassword'
 +    encryption: 'start_tls' # "start_tls" or "simple_tls" or "plain"
 +    verify_certificates: false
 +    active_directory: true
 +    allow_username_or_email_login: false
 +    lowercase_usernames: false
 +    block_auto_created_users: false
 +    base: 'CN=Users,DC=devbase,DC=org'
 +#     user_filter: ''
 +EOS
 +</code>
 +  * gitlab-ctl reconfigure
 +  * gitlab-rake gitlab:ldap:check
 +
 +==== Music ====
 +Background music should not be interrupted by rebooting clients or phones leaving the building. \\
 +A dedicated client should play the music and controllable by all kind of devices.
 +
 +  * download runeaudio for RPi
 +  * flash to SD card and boot
 +  * browse http://ip
 +    * settings
 +      * hostname: music01
 +      * airplay: on
 +      * airplay name: music01
 +      * UPnP: on
 +      * UPnP name: music01
 +  * ssh root@ip
 +    * passwd
 +    * cat >> /etc/mpd.conf <<EOF
 +<code>
 +audio_output {
 +   type            "httpd"
 +   name            "My HTTP Stream"
 +   encoder         "flac"       # optional, vorbis or lame
 +   port            "8000"
 +   bind_to_address "0.0.0.0"    # optional, IPv4 or IPv6
 +   quality         "5.0"        # do not define if bitrate is defined
 +   # bitrate         "128"      # do not define if quality is defined
 +   format          "44100:16:1"
 +   max_clients     "0"          # optional 0=no limit
 +}
 +EOF
 +</code>
 +  * Library
 +    * Webradio
 +      * http://trance-high.rautemusik.fm
 +      * http://house-high.rautemusik.fm
 +      * http://stream03.uzic.ch:9010
 +
 +Management: \\
 +[[http://music01.devbase.org]]
 +
 +==== Ubiquiti Controller ====
 +To manage the Unfi WiFi access points a Controller is needed
 +  * Install Ubuntu 16.04 LTS
 +  * echo 'deb http://www.ubnt.com/downloads/unifi/debian stable ubiquiti' | sudo tee /etc/apt/sources.list.d/100-ubnt-unifi.list
 +  * apt-key adv --keyserver keyserver.ubuntu.com --recv 06E85760C0A52C50
 +  * apt-get install unifi
 +  * https://<ip>:8443
 ==== Wiki ==== ==== Wiki ====
  
 +==== Matrix - testing only ====
 +  * Install Ubuntu 16.04 LTS
 +  * apt update && sudo apt upgrade
 +  * add-apt-repository https://matrix.org/packages/debian/
 +  * wget https://matrix.org/packages/debian/repo-key.asc -O key
 +  * apt-key add - < key
 +  * apt update
 +  * apt install matrix-synapse python-matrix-synapse-ldap3
 +    * Server Name: matrix.devbase.org
 +  * cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 34 | head -1
 +  * vi /etc/matrix-synapse/homeserver.yaml
 +    * registration_shared_secret: your_random_string
 +  * systemctl enable matrix-synapse.service
 +  * systemctl start matrix-synapse.service
 +  * register_new_matrix_user -c /etc/matrix-synapse/homeserver.yaml https://localhost:8448
 +    * name root
 +    * set password
 +    * make admin = yes
 +
 +Public DNS Recort for federation
 +SRV matrix_tcp.devbase.org 
projects/base-infrastructure.1505223912.txt.gz · Last modified: 2017-09-12 13:45 by trinitor