using vector to send JSON-formatted nginx logs to ClickHouse
for some particular case i need to log and later analyze POST bodies of HTTP requests sent via nginx reverse proxy. ClickHouse will store the logs and be useful in analysis.
playing with min.io cluster
min.io is open source [ although AGPL-3 licensed ] object storage software providing s3-compatible interface and handling clustering for HA & data distribution. below – notes from setting it up on a tiny scale, without using Kubernetes. notes are based on RELEASE.2023-03-24T21-41-23Z i’ve fetched two binaries for the project: my cluster has 4 nodes, looks ... Read More
using hardware tokens to secure SSH, MS365 logins
i’ve done a bit of research, below – my understanding of the current state of affairs [ 2023-03 ]. i’m writing this while testing YubiKey 5 NFC, but consider different alternatives. SSH why: i’m considering an attack vector where malicious actor has remote control of my PC – can lift up arbitrary files [ including ... Read More
MySQL / MariaDB login audit
once in a while i need to review which database accounts are used, from what IPs connections arrive. MySQL / MariaDB does not have built-in mechanism [ unless you want to allow full query log ], but there’s quite easy way to get the data.
mysqldump headaches
over the years i’ve set up multiple backup workflows. one of them is described here. part of the cycles involves taking database backups. for MySQL i’m mostly using mysqldump. it comes with some headaches: it’s single-threaded by nature – both for backup, and restore [ unless you chop the dump file into pieces and try ... Read More
spooky HTTP requests arriving from the dark corners of the internet
i’ve spent most of the last week investigating spooky HTTP request that we’ve found in access logs of few production servers. very likely that traffic was replayed by mail content scanners used by messagelabs and mimecast.
UPC WiFree on Mikrotik and OpenWRT
UPC turns your cable modem into WiFi hot-spot available for others. outrageous! and i’ve been using it for quite a while – it generally works. below – how to connect OpenWRT or Mikrotik to such connection.
resizing btrfs on top of luks
recently we needed to expand storage space available on one of our servers. originally it was using RAID10 on 4 4TBSSD drives handled by Dell’s PERC h730p controller, we wanted to add 2 more 4TB drives and go from 8 to 12TB array. we’ve decided to be brave and use RAID10 -> RAID10 array expansion. ... Read More
fighting a false-positive flagging by multiple antivirus vendors
recently i woke up to this: “Hi, some of our employees are using your application. This morning they have received an upgrade notification (in yellow banner) to get the latest version of your app. Our anti-virus/malware has triggered on your module called “somefile.exe” detected at risk being a “Trojan.Gen.MBT “. below few resources that i’ve ... Read More
raspberry pi as a serial console terminal
i’ve bricked a cheap home router. i’ve installed a beta of dd-wrt firmware which did not work. raspberry pi gave me serial connection to it.
bridging lan segments across untrusted links
we’ve run out of the office space in one of the locations. in short term it was not possible to find a suitable and large enough place to rent so we had to split and relocate some of the staff to another building few kilometers away. it’s possible that we’ll shuffle people and servers between ... Read More
multi-master mysql replication with servers on 3 different continents
at work i’m using mysql replication quite extensively. first it was a straightforward one-way replication that has been rock-solid for us since 2009. in 2012, for another type of data, we’ve started using master-master setup. initially the servers were in different European countries, eventually the secondary site was moved to North America while primary one ... Read More
MySQL on BTRFS?
i’ve been running a set of production MySQL databases on BTRFS since April 2016. BTRFS is not exactly known for its stellar performance when hosting databases or images of virtual machines due to its COW nature. why would i do it then? to have data snapshots and be able to ‘go back in time’ quickly ... Read More
A wall of text on bad and good choices
In this line of work, you don’t just get to play with shiny toys having plenty of blinking lights. There’s plenty of choices to be done nearly every day. Choices or rather bets: some of the technologies, software stacks, products or services provided internally will eventually be a flop. Decisions made over the years are ... Read More
bridge on vlans on active-backup bonding under debian stretch
i use this setup for few lxc servers. bonding provides me layer2 failover based on arp probes [ so it’ll work even if switch link stays up yet forwarding fails the mechanism will kick in ]. this is continuation of an earlier post, this time under debian stretch
bridging two physical interfaces of esxi server
my colleagues got into an unpleasant situation where one of two dedicated servers, running vmware esxi 6.0, rented from a datacenter lost its network connectivity. the datacenter/internet-facing interface is down, hours later, during regular working day, the hosting provider did not react and resolve the problem. maybe the network card died, maybe switch port misbehaves ... Read More
ghettoVCB failing randomly on larger VMs
at work we’re using happily ghettoVCB.sh to back up and restore VMWare ESXi VMs. since a few weeks we’ve started to experience occasional failures of backups, only for one – larger VMs. in the logs produced in /tmp/ghettoVCB-2017-04-xxx.log we got: or after some head scratching, watching at iostat -x 1 and ifstat -b 1 -i ... Read More
simplistic gatekeeper limiting access to apache2-based proxy
i had to expose some web-based application hosted on a windows server to the internet. i don’t put too much trust in the developers of that particular application so i did not want to make it reachable from the public internet. while i could not use ip address based whitelist i could count on the ... Read More
2 hikes in Hong Kong
i was in HK again; just for a week, but there was enough time during the weekend to hike again in the wildness. High Junk Peak at first i did not enjoy this route at all – just after living the last buildings behind: there was ~25 min of walking up the narrow concrete stairs ... Read More
tuning spmassasin to treat more harshly mails with forged sender’s address claiming to come from us
once in a while we get e-mails with spoofed sender’s address claiming to come from @ourorg.com. this can fool some of our users; outlook displaying an image of sender solely based on the From: field does not help here. some of those messages have different Return-Path pointing to @someotherscammy.site, other have it also pointing to ... Read More