Uncategorized

Ideas: rsync-like tool parallelizing transfer of large files

by .pQd
i use rsync heavily for backups, batch jobs and one-off tasks. every day it moves around terabytes of data for me. while it’s great – saves time and bandwidth, it could make even better use of modern hardware – multicore, and often with plenty of unused storage IOps: one day, when i’ll have some spare ... Read More

Hong Kong, again

by .pQd
i got yet another chance to travel to Hong Kong. i’ve spent there 9 days in October this year. what did change since my previous visit in 2018? what did not change: few places that i’ve hiked around for the first time: Lamma Island – i got there with a ferry from Central. concrete path ... Read More

New supermicro server does not ‘see’ some of NVMe drives

by .pQd
i got 3 identical new servers from supermicro – SYS-121C-TN10R. all three have the same disk setup – 2+4 NVMe drives, but Debian 12 just see 5 of them instead of 6. when i run dmesg on newly installed system i can see those errors: to solve it i had to change PCIe bifurcation in ... Read More

playing with min.io cluster

by .pQd
min.io is open source [ although AGPL-3 licensed ] object storage software providing s3-compatible interface and handling clustering for HA & data distribution. below – notes from setting it up on a tiny scale, without using Kubernetes. notes are based on RELEASE.2023-03-24T21-41-23Z i’ve fetched two binaries for the project: my cluster has 4 nodes, looks ... Read More

using hardware tokens to secure SSH, MS365 logins

by .pQd
i’ve done a bit of research, below – my understanding of the current state of affairs [ 2023-03 ]. i’m writing this while testing YubiKey 5 NFC, but consider different alternatives. SSH why: i’m considering an attack vector where malicious actor has remote control of my PC – can lift up arbitrary files [ including ... Read More

MySQL / MariaDB login audit

by .pQd
once in a while i need to review which database accounts are used, from what IPs connections arrive. MySQL / MariaDB does not have built-in mechanism [ unless you want to allow full query log ], but there’s quite easy way to get the data.

mysqldump headaches

by .pQd
over the years i’ve set up multiple backup workflows. one of them is described here. part of the cycles involves taking database backups. for MySQL i’m mostly using mysqldump. it comes with some headaches: it’s single-threaded by nature – both for backup, and restore [ unless you chop the dump file into pieces and try ... Read More

xiaomi robot vaccuum disassembly

by .pQd
i have 1st gen xiaomi mi robot vacuum cleaner. it’s been serving very well for the past 3 years. this video was very helpful with disassembly when i had to replace the main sucking motor fan: https://www.youtube.com/watch?v=wdDSBgJ1kRM i had to replace a fan and bought https://www.aliexpress.com/item/4000804127893.html instead of https://www.aliexpress.com/item/4000259186760.html ; not an exact fit but ... Read More

clickhouse

by .pQd
clickhouse is a column oriented OLAP database. i’ve started using it about half a year ago. i’m impressed. earlier i’ve read about it on percona’s blog but did not fully grasp how performant it was. i’ve tried it when i wrestling with MariaDB’s query planner on table with ~100M rows got me tired and each ... Read More

letting dbeaver talk with manticoresearch

by .pQd
here’s what i did to get dbeaver 7.3.2 + oracle’s JDBC mysql-connector-java-8.0.17.jar work with manticoresearch 3.5.4: /etc/manticoresearch/manticore.conf, under searchd: in dbeaver – new connection using oracle’s JDBC driver as above, under data editor – set ResultFetchSize to 0 to prevent errors like this: when i initially tried to get it working with JDBC driver from ... Read More

UPC WiFree on Mikrotik and OpenWRT

by .pQd
UPC turns your cable modem into WiFi hot-spot available for others. outrageous! and i’ve been using it for quite a while – it generally works. below – how to connect OpenWRT or Mikrotik to such connection.

iostat -x 1 reporting 100% utilization of nearly-idle NVMe drives

by .pQd
after an upgrade to debian buster i’ve noticed that both iostat -x 1 and munin’s diskstats_utilization report that NVMe drives are busy most of the time. some empirical tests showed that disks are actually idle, performance did not drop. upgrade to 5.2 kernel resolved the miss-reporting issue.

let’s encrypt via proxy server under debian

by .pQd
i prefer to have strict DROP policy for the outgoing traffic from production servers. let’s encrypt API endpoint is behind Akamai’s CDN and IP address to which acme-v02.api.letsencrypt.org resolves changes frequently. i don’t like playing whack-a-mole every 3 months so i’ve: set up a squid-based proxy server that allows for filtering based on domain names: ... Read More

multi-master mysql replication with servers on 3 different continents

by .pQd
at work i’m using mysql replication quite extensively. first it was a straightforward one-way replication that has been rock-solid for us since 2009. in 2012, for another type of data, we’ve started using master-master setup. initially the servers were in different European countries, eventually the secondary site was moved to North America while primary one ... Read More