.pQd

upgrading firmware on Intel’s SATA SSD drives behind Dell’s h730p RAID

by .pQd
fetch Intel_SSD_Data_Center_Tool from here. the zip archive contained, among other, .deb for 64 bit OS – that worked for me under debian stretch. to see drives run: to upgrade the firmware where 0 corresponds to the Index of drive returned by the first command the same likely works for other LSI/Avago cards – no matter ... Read More

iostat -x 1 reporting 100% utilization of nearly-idle NVMe drives

by .pQd
after an upgrade to debian buster i’ve noticed that both iostat -x 1 and munin’s diskstats_utilization report that NVMe drives are busy most of the time. some empirical tests showed that disks are actually idle, performance did not drop. upgrade to 5.2 kernel resolved the miss-reporting issue.

openvpn – “OpenSSL: error:1408518A:SSL routines:ssl3_ctx_ctrl:dh key too small” after upgrade to Debina Buster

by .pQd
another thing to adjust after upgrade to Buster. on openvpn endpoint that in it’s config – /etc/openvpn/whatever.conf – had: dh dh1024.pem the vpn service did not start. tail -f /var/log/syslog showed: apparently the new version of openssl no longer accepts 1024 Diffie Hellman group. solution: and change in /etc/openvpn/whatever.conf – from dh dh1024.pem to dh ... Read More

bind9 fails to start after an upgrade to Debian Buster

by .pQd
i’ve done a routine upgrade to Buster on some server and all went fine… except bind9 no longer started. every attempt to restart it ended up with long waiting and an error message: logs did not contain anything interesting – actually they showed that bind would start and then after ~ a minute was shut ... Read More

allowing custom file extensions to be uploaded to wordpress

by .pQd
WordPress restricts types of files you can upload. it not only checks the extension of what you’re adding to the media library but also its mime type. so you cannot fool it by uploading .exe renamed to .jpeg – if you try it, you’ll get an error saying: “Sorry, this file type is not permitted ... Read More

“error: internal error: unable to execute QEMU command ‘transaction’: Could not create file: Permission denied” when creating a snapshot under KVM

by .pQd
debian buster brings apparmor. apparmor brings problems – eg it’s too restrictive for libvirt KVM guests and does not allow KVM to create snapshot-related files in VM’s folders. so far i did not find a clean and generic way to address it so i had to disable apparmor for libvirt by adding security_driver = “none” ... Read More

let’s encrypt via proxy server under debian

by .pQd
i prefer to have strict DROP policy for the outgoing traffic from production servers. let’s encrypt API endpoint is behind Akamai’s CDN and IP address to which acme-v02.api.letsencrypt.org resolves changes frequently. i don’t like playing whack-a-mole every 3 months so i’ve: set up a squid-based proxy server that allows for filtering based on domain names: ... Read More

kvm: disabled by bios

by .pQd
does your kvm VM feel sluggish? mine did feel pretty slow.. as it turned out virt-install quietly overrode ‘hvm’ preferences and created a fully emulated qemu vm when it failed to make use of hardware-assisted virtualization.

bios upgrade on Dell PowerEdge T20 via AMT

by .pQd
prepare the floppy image file that later can be mounted via AMT, include in it just the bios update file – in my case it was PET20A18.exe from here. once the file is ready use the Manageability Commander Tool > Remote Control > Take control mount such img file as a virtual, oversize floppy. Using ... Read More

bridging lan segments across untrusted links

by .pQd
we’ve run out of the office space in one of the locations. in short term it was not possible to find a suitable and large enough place to rent so we had to split and relocate some of the staff to another building few kilometers away. it’s possible that we’ll shuffle people and servers between ... Read More

multi-master mysql replication with servers on 3 different continents

by .pQd
at work i’m using mysql replication quite extensively. first it was a straightforward one-way replication that has been rock-solid for us since 2009. in 2012, for another type of data, we’ve started using master-master setup. initially the servers were in different European countries, eventually the secondary site was moved to North America while primary one ... Read More

idrac upgrade failures

by .pQd
idrac gives you “RED006: Unable to download Update Package” and plenty of headaches during updates? it happened to me while jumping over few updates and going straight from 2.41.40.40 to 2.52.52.52. no amount of racadm racreset would help. upgrading first to 2.50.50.50 and only then to 2.52.52.52 did the trick for me.

MySQL on BTRFS?

by .pQd
i’ve been running a set of production MySQL databases on BTRFS since April 2016. BTRFS is not exactly known for its stellar performance when hosting databases or images of virtual machines due to its COW nature. why would i do it then? to have data snapshots and be able to ‘go back in time’ quickly ... Read More

A wall of text on bad and good choices

by .pQd
In this line of work, you don’t just get to play with shiny toys having plenty of blinking lights. There’s plenty of choices to be done nearly every day. Choices or rather bets: some of the technologies, software stacks, products or services provided internally will eventually be a flop. Decisions made over the years are ... Read More

E5-2643 in r620 stuck at a very low cpu frequency

by .pQd
few weeks ago i’ve done a general firmware upgrade of 2 identical Dell Poweredge r620 servers. a while later i’ve noticed that one of them had really sluggish performances. i’ve noticed that /proc/cpuinfo has shown CPU frequencies of hundreds of MHz rather than thousands. cat /proc/cpuinfo |grep MHz gave me 16 rows of values between ... Read More

lxc broken after upgrade from linux-image-4.12.0-0.bpo.1-amd64 to linux-image-4.13.0-0.bpo.1-amd64

by .pQd
lxc containers no longer start after i’ve upgraded debian stretch server from 4.12 kernel from stretch-backports repository to 4.13 also from backports. symptoms in /var/log/syslog: that was resolved by creating /etc/systemd/network/99-default.link with: via this comment. then there was nothing interesting in syslog but the lxc guest still would not start – lxc-start -F -n serverName ... Read More

btrfs and nfs dont play nicely together

by .pQd
i use ghettoVCB to take backups of vmware esxi 6.5 vms. data is copied to a nfs share hosted on Debian Linux. for over a year i used btrfs partition as an underlying datastore for the nfs share. and i just could not get it to work for backups which size larger than available RAM ... Read More