website/content/post/2020-01-13-PaulBSD-Infrastructure.md

73 lines
3.2 KiB
Markdown
Raw Permalink Normal View History

2020-01-12 22:01:13 +01:00
+++
categories = ["Infra"]
2020-02-07 07:07:00 +01:00
date = "2020-01-13"
2020-01-12 22:01:13 +01:00
menu = ""
tags = ["paulbsd","infra"]
title = "PaulBSD Infrastructure"
+++
2020-01-13 20:45:09 +01:00
# PaulBSD Infrastructure
2020-01-12 22:01:13 +01:00
2020-01-12 22:31:17 +01:00
PaulBSD.com is a mixed infrastructure made of on-premise part based in Normandy, and cloud instances made upon scaleway.com in Amsterdam / object storage in Paris.
2020-01-12 22:01:13 +01:00
2020-01-12 22:31:17 +01:00
PaulBSD infrastructure main regions are :
- Caen (Legacy production)
- Saint-Lô (Backups)
- Amsterdam (Production)
- Paris (Object storage)
2020-01-12 22:01:13 +01:00
2020-02-08 18:05:13 +01:00
Up to 80% of the applications are managed by SaltStack, some in-house applications / scripts / tools written in Python 3 and Golang for back services, and HTML/CSS/JS on front services. All PaulBSD infrastructure is using open-source software
2020-01-12 22:01:13 +01:00
2020-01-13 20:45:09 +01:00
## Overview
2020-01-12 22:31:17 +01:00
2020-01-13 20:45:09 +01:00
Front machines are cloud instances with mixed architectures (amd64 and arm64). Back machines are mainly legacy. Front machines are Linux based instances based on Ubuntu 18.04 (20.04 migration is work-in-progress). NetBSD 8.1 is used for legacy, front and back services, and is going to be replaced.
2020-01-12 22:31:17 +01:00
Some services :
2020-01-13 20:45:09 +01:00
- Mail relaying is located on front instances, using postfix. Security and mail signing is based upon opendkim. IMAP is served using dovecot with a PostgreSQL instance for authentication and accounting.
2020-01-12 22:31:17 +01:00
- Nftables is used for firewall on Linux and NPF on NetBSD
- Monitoring tools are sensu-go (with mail and Telegram notifications), and metrics are centralized on a stack based on Telegraf, InfluxDB, Grafana
- Internal docs are based on dokuwiki
- Geographical data replication is made using Syncthing (locations are Caen and Saint-Lô. Amsterdam will replace Caen in the future). rsync is used for some scripts
- RDBMS are MariaDB and PostgreSQL
2020-02-08 18:05:13 +01:00
- Git repositories for in-house applications and a mirrored copy of GitHub useful projects are stored by Gitea (git.paulbsd.com)
2020-01-12 22:31:17 +01:00
- Burp / Bareos do the backups of code and data
2020-01-13 20:45:09 +01:00
- NextCloud serves the personal cloud storage / Caldav shares
2020-01-12 22:31:17 +01:00
- IPAM based on Netbox
- ...
Total data space (free + used) across all regions : <= 3,5 To
2020-01-13 20:45:09 +01:00
Clients are Ubuntu (managed by SaltStack) and Android devices
2020-01-12 22:31:17 +01:00
I will make a schema in the future to show how services are organized
2020-01-13 20:45:09 +01:00
## Internal applications
2020-02-08 18:05:13 +01:00
All projects described below are in-house applications / tools, open-licensed, for many usages
2020-01-13 20:45:09 +01:00
### qrz
qrz (qrz.paulbsd.com) is based on bootstrap and datatables for front application. Datatables is using the server-side processing, which is querying on a MariaDB database. A scrapper has been written in Golang that fetch data in an html page on the internet. This scrapper is available at git.paulbsd.com/paulbsd/qrz and is using an embedded MySQL connector
### g2g
2020-01-17 00:05:12 +01:00
g2g is a golang executable that fetchs starred project on github and POST mirror copies of these repositories on a gitea / gogs instance. It is mainly a script that call REST webservices of github and gitea
2020-01-13 20:45:09 +01:00
### fuelprices
2020-01-22 08:17:35 +01:00
fuelprices is a little golang script that fetch fuel prices on the opendata of the french governement, and insert it on an influxdb time-series
2020-01-13 20:45:09 +01:00
### dip / dipc
2020-01-17 06:00:39 +01:00
dip is a webservice based in golang that return public ip addresses like ipinfo.io or showmyip.com (instance available at ip.paulbsd.com). HTML or JSON output are possible
2020-01-13 20:45:09 +01:00
dipc is the client for dip (as a portable executable file)
2020-01-17 00:05:12 +01:00
Have fun !