.pQd's log – Just another WordPress weblog

using host-managed SMR SATA HDD under linux

i’ve found refurbished host-managed SMR drives from Seagate sale; model ST25000NM000E-3L9103 coming from Exos Xz series. below – notes on what did i discover. those are not ordinary drives: e.g. they’ll not work under Windows, under linux only some file systems are supported directly or additional layer of dm-zoned is needed. i’ve also found out ... Read More

Thales CipherTrust Manager – setting up KMS access

techkmip, kmsLeave a Comment

this is reminder, mostly for myself, how to issue certificates that later can be used with PostgreSQL, MySQL to access Key Management Service [KMS] via Key Management Interoperability Protocol [KMIP] to store & retrieve encryption keys used in Table Data Encryption [TDE]. whenever i refer to PostgreSQL or MySQL in this post – i actually ... Read More

PyKMIP + PostgreSQL with pg_tde, Percona Server for MySQL

Uncategorizedmysql, postgresql, tdeLeave a Comment

i wanted to try applying transparent data encryption in PostgreSQL 17, MySQL 8.4 with key obtained via KMIP from KMS. i’ve used PyKMIP as Key Management System. i could not find any detailed step by step guide, even Percona’s own documents were pretty minimal. below – is what i’ve done step by step. my setup ... Read More

qcow2 snapshots leading to data corruption in KVM + libvirt coming from Debian Trixie?

techkvm, libvirtLeave a Comment

i don’t have any reproducible scenario, but i want to leave this one out in the open. maybe someone else experiences similar issue? after upgrading both OS on the physical servers and under VMs to Debian Trixie i’ve experienced 4 database corruptions – both under MySQL 5.8.4 and PostgreSQL 17, on different physical servers. it’s ... Read More

decoder for MySQL 8.4 row-level binlogs

techmysqlLeave a Comment

i’ve run into a problem – i had mysql 8.4 binlogs in row format generated with binlog_rows_query_log_events =OFF; i needed to see how exactly certain rows looked like before / after particular updates. problem was – those rows contained blobs and output from mysqlbinlog -v was a mess: i needed to find a way to ... Read More

using vector.dev to get haproxy, nginx logs into ClickHouse

techclickhouse, haproxy, nginx, vector.devLeave a Comment

at work we have few reverse proxies exposing some APIs. ClickHouse turned out to be great tool for analyzing logs, running ad-hoc research or gathering stats cyclically. below – configs that allow me to get HTTP requests passing via those proxies, including POST bodies, into ClickHouse. i’m using vector.dev to fetch, transform and ship logs. ... Read More

455GB data SIM from orange.pl, 2 years validity – 34 PLN

Uncategorizedcheap-data-simLeave a Comment

what i’ve done to pile up bonus offers to get a data SIM that has long validity and does not cost arm and a leg: my findings are based on: EDITS:

training for frontline drone operators

UncategorizedLeave a Comment

we ware on 3 day training led by a guy who spent 1 year on Ukrainian front lines operating reconnaissance and kamikaze drones. brain dump of some of the tidbits i’ve learned: interviews with the trainer: what we’ve heard about the reality of drone war is very close to what you can learn e.g. from ... Read More

manticore search available via ClickHouse

Uncategorizedclickhouse, manticoresearchLeave a Comment

i’m a big fan of ClickHouse; over time it became central hub for any sort of business intelligence analytics. at work we replicate data to it, use dictionaries loaded over http and mysql, recently – we’re also using MySQL table or database functions to be able to run queries with joins across data from various ... Read More

rsync without retransmitting moved files

techrsyncLeave a Comment

i’m using rsync a lot; both at work [ backups, replication of content to various servers, ad-hoc copying ] and privately. it’s smart enough to avoid re-sending the whole file if it has grown a bit [ like logs like to do ] or only few bytes changed in source or destination. out-of-the-box rsync is ... Read More

Ideas: rsync-like tool parallelizing transfer of large files

Uncategorizedproject ideas, rsyncLeave a Comment

i use rsync heavily for backups, batch jobs and one-off tasks. every day it moves around terabytes of data for me. while it’s great – saves time and bandwidth, it could make even better use of modern hardware – multicore, and often with plenty of unused storage IOps: one day, when i’ll have some spare ... Read More

Hong Kong, again

travel, Uncategorizedhong kongLeave a Comment

i got yet another chance to travel to Hong Kong. i’ve spent there 9 days in October this year. what did change since my previous visit in 2018? what did not change: few places that i’ve hiked around for the first time: Lamma Island – i got there with a ferry from Central. concrete path ... Read More

New supermicro server does not ‘see’ some of NVMe drives

UncategorizedLeave a Comment

i got 3 identical new servers from supermicro – SYS-121C-TN10R. all three have the same disk setup – 2+4 NVMe drives, but Debian 12 just see 5 of them instead of 6. when i run dmesg on newly installed system i can see those errors: to solve it i had to change PCIe bifurcation in ... Read More

getting code signing certificate on own Yubikey and then using it to sign windows binaries in unattended way, on KVM VM

UncategorizedLeave a Comment

starting from June 2023 code signing certificates for MS Windows cannot be delivered as a file anymore. files are easy to steal. now certs must reside on security modules which don’t allow private key extraction, at least not for mere mortals. we’re using such a cert to sign exe and msi files on a build ... Read More

using vector to send JSON-formatted nginx logs to ClickHouse

Uncategorizedclickhouse, nginxLeave a Comment

for some particular case i need to log and later analyze POST bodies of HTTP requests sent via nginx reverse proxy. ClickHouse will store the logs and be useful in analysis.

playing with min.io cluster

UncategorizedLeave a Comment

min.io is open source [ although AGPL-3 licensed ] object storage software providing s3-compatible interface and handling clustering for HA & data distribution. below – notes from setting it up on a tiny scale, without using Kubernetes. notes are based on RELEASE.2023-03-24T21-41-23Z i’ve fetched two binaries for the project: my cluster has 4 nodes, looks ... Read More

using hardware tokens to secure SSH, MS365 logins

UncategorizedLeave a Comment

i’ve done a bit of research, below – my understanding of the current state of affairs [ 2023-03 ]. i’m writing this while testing YubiKey 5 NFC, but consider different alternatives. SSH why: i’m considering an attack vector where malicious actor has remote control of my PC – can lift up arbitrary files [ including ... Read More

MySQL / MariaDB login audit

UncategorizedmysqlLeave a Comment

once in a while i need to review which database accounts are used, from what IPs connections arrive. MySQL / MariaDB does not have built-in mechanism [ unless you want to allow full query log ], but there’s quite easy way to get the data.

mysqldump headaches

UncategorizedLeave a Comment

over the years i’ve set up multiple backup workflows. one of them is described here. part of the cycles involves taking database backups. for MySQL i’m mostly using mysqldump. it comes with some headaches: it’s single-threaded by nature – both for backup, and restore [ unless you chop the dump file into pieces and try ... Read More

spooky HTTP requests arriving from the dark corners of the internet

UncategorizedLeave a Comment

i’ve spent most of the last week investigating spooky HTTP request that we’ve found in access logs of few production servers. very likely that traffic was replayed by mail content scanners used by messagelabs and mimecast.