TIL
Today I learned

# openssl: use TLSv1.0 or TLSv1.1 with OpenSSL 3+  #openssl =2024-04-10

OpenSSL since version 3 disables TLS 1.0 and 1.1 by default,
you need to reduce the security level to 0 to still make it work.
This can be done by overriding the OpenSSL config file:

	% cat /tmp/openssl.conf
	openssl_conf = openssl_init
	
	[openssl_init]
	ssl_conf = ssl_configuration
	
	[ssl_configuration]
	system_default = tls_system_default
	
	[tls_system_default]
	MinProtocol = TLSv1.0
	CipherString = DEFAULT@SECLEVEL=0

	% OPENSSL_CONF=/tmp/openssl.conf curl https://tls-v1-1.badssl.com:1011/
	...

Source:
https://github.com/openssl/openssl/discussions/22752#discussioncomment-7617584

# git: push a subdirectory to a remote  #git =2023-09-13

Occasionally you need to push a subdirectory of the current repo to a
remote.  This can be done by creating an ad-hoc commit that contains
the directory (and no parents):

	% git push myremote $(git commit-tree HEAD:my/sub/dir </dev/null):refs/heads/main


# emacs: manage Sieve from Emacs  #emacs =2023-01-03

If you want to manage a remote Sieve configuration, you can use
sieveshell(1) from Cyrus IMAP, or use Emacs:

	M-x manage-sieve

which asks for hostname, username and password.

Press f to create a new rule.  Press C-c C-l to upload your changed rule.
Don't forget to activate the rule with m!


# pv: progress bar for running processes   #linux =2022-11-08

On Linux, pv(1) has the option -d PID[:FD], which will show a progress
bar for writing to the file descriptor FD of the process PID.

This is similar to progress(1).


# Read-only bind mounts  #linux =2022-01-22

Sometimes one needs a directory hierarchy that's read-only.  Read-only
bind mounts provide a way to create such a thing.

In theory, you can create them with

	% mount -o bind,ro /source /readonly

However, this doesn't work in busybox 1.33.1.  There you need two steps:

	% mount -o bind /source /readonly
	% mount -o bind,remount,ro /source /readonly

(You need to call mount(2) twice anyway, but util-linux mount
recognizes this usage.)


# Mapping major:minor ids to device names  #linux =2021-10-21

To map major:minor ids from a device node to a name,
look into /sys/dev/char/?:?/uevent or /sys/dev/block/?:?/uevent

	% stat -c %t:%T /dev/console
	5:1
	% cat /sys/dev/char/5:1/uevent 
	MAJOR=5
	MINOR=1
	DEVNAME=console

No udev needed! ("udevadm info -rq name /sys/dev/char/5:1" works too.)


# sh: comments in multiline statements  #shell =2021-09-10

Sometimes you want to comment out something in a line-wrapped statement,
but this does not work:

	foo \
		bar \
		# quux \
		bla

Compared to languages like Tcl, where the comment(!) will continue,
Unix shells will run bla as a separate command.

Instead, you can do this:

	foo \
		bar \
		`# quux` \
		bla

The shell will elide the empty output from the process substitution.
(Source: https://stackoverflow.com/a/12797512)

This trick also allows commenting part of a line, which can be useful.


# docker: check whether in privileged container  #docker #capabilities =2021-06-09

To check whether you are inside a docker container with --priviledged
you can try this heuristics:

	capsh --print | grep -Eo '!?cap_sys_time'

If it prints "cap_sys_time" it's a privileged container.
if it prints "!cap_sys_time" it's an unprivileged container.

If you don't have capsh, look at /proc/self/status, if it contains
something like CapEff: 0000003fffffffff you are privileged.


# ld: symbol redirection  #ld =2021-02-03

You can use --defsym to add additional symbol definitions to the symbol table,
e.g. to override malloc calls:

	gcc -o foo foo.c -Wl,--defsym=malloc=mymalloc -lmymalloc


# sed: replace tac  #sed #tac =2020-09-12

<kylie> tac isn't stricly POSIX, if that matters, and you can achieve
	the same with `sed '1!G;h;$!d'`


# fallocate: truncating a file at the beginning  #fallocate #linux =2020-04-18

It is common wisdom that Unix files can only be extended at the end,
and there is no way to insert or delete data inbetween without copying
data around.

Turns out, this is not true on Linux on ext4 and xfs file systems, due
to the FALLOC_FL_COLLAPSE_RANGE feature.

In our case, we had a big log file filling up the entire disk, and we
wanted to keep the second half of the file.  I later found out how to
do it correctly: fallocate -c can be used to run a
FALLOC_FL_COLLAPSE_RANGE request, but as a limitation, the ranges need
to align with the file system block size.  So let's find out that one
first:

	% stat -fc %s .                    
	4096

Now, we can create a big log file and determine where to cut:

	% seq 1 10000000 > big
	% wc -l big
	10000000 big
	% wc -c big
	78888897 big
	% size=$(wc -l big)                       
	% half=$((size/2 - (size/2)%4096))    

Finally, we can collapse the first half of the file:

	% fallocate -c -o 0 -l $half big
	% head -n 5 big
	5068937
	5068938
	5068939
	5068940
	5068941

(It's accidental that this aligns with the line structure of the file.)

This operation should be safe even if other processes append to the log
file during the collapse.


# rmdir: -p flag to remove parents  #rmdir =2020-04-17

Sometimes you extract an archive, and it creates multiple nested
directories before the actual contents come.  Then you move the
contents away, and now you need to clean up the nested empty
directories.

Of course, you could use "rm -rf", but it may be dangerous to do so.
For removing empty directories, "rmdir" is safe.  And rmdir has a "-p"
option that removes nested empty directories!

	% lr -F fox     
	fox/
	fox/baz/
	fox/baz/quux/
	% rmdir -p fox/baz/quux
	% lr -F fox
	lr: cannot stat 'fox': No such file or directory

I was close to writing a script to do this when I checked rmdir(1).
To my surprise, it's actually a POSIX feature.


# sqlite: operate on files with only read permissions  #sqlite =2020-04-02

Even if you only want to read from a sqlite database, sqlite wants to
lock it and it's "wal" to avoid concurrent writes.  Sometimes, you
don't have permissions to create a wal lock in the directory, however.

Instead of copying the file somewhere else, as a workaround, you can
open the sqlite with the immutable flag set.  This requires using the
URL syntax for filenames:

	sqlite3 file:///nix/var/nix/db/db.sqlite?immutable=1 .dump

Note that data may be corrupted if there are parallel writes to the
database.

This trick is not needed if the filesystem is mounted read-only.


# vim: underline current line  #vim =2019-06-25

yypVr= will make a new line equal in length to the previous consisting of =.

Thanks to osse of #vim.


# css: flow-root  #html #css =2019-01-18  +leah+ 

"display: flow-root" is the modern replacement for the clearfix hack.
(E.g. see https://alligator.io/css/no-more-clearfix-flow-root/)


# ruby: new regexp features   #ruby =2018-05-01

Ruby can intersect character classes, e.g. /[a-z&&[^g]]/
matches all characters between a to z that are not g.


# ruby: slice_* methods  #ruby =2018-05-01

Ruby has a few new slice variants for Enumerable:

>> [1,2,3,4,5,3,2,1].slice_after(3).to_a
=> [[1, 2, 3], [4, 5, 3], [2, 1]]

>> [1,2,3,4,5,3,2,1].slice_before(3).to_a
=> [[1, 2], [3, 4, 5], [3, 2, 1]]

>> [1,2,3,4,5,3,2,1].slice_when {|b,a| b==3 || a==3}.to_a
=> [[1, 2], [3], [4, 5], [3], [2, 1]]


# linux: list of mmap'ed files  #linux =2018-01-28

In /proc/$pid/map_files, you can find a list symlinks from adresses to
their mmap'ed files:

	% ls -l /proc/$$/map_files
	total 0
	lr-------- 1 leah leah 64 Jul 25 11:18 564fb0c78000-564fb0c8f000 -> /usr/bin/zsh*
	lr-------- 1 leah leah 64 Jul 25 11:18 564fb0c8f000-564fb0d22000 -> /usr/bin/zsh*
	lr-------- 1 leah leah 64 Jul 25 11:18 564fb0d22000-564fb0d43000 -> /usr/bin/zsh*
	lr-------- 1 leah leah 64 Jul 25 11:18 564fb0d43000-564fb0d45000 -> /usr/bin/zsh*
	lr-------- 1 leah leah 64 Jul 25 11:18 564fb0d45000-564fb0d4b000 -> /usr/bin/zsh*
	lr-------- 1 leah leah 64 Jul 25 11:18 7f3e31def000-7f3e31df2000 -> /usr/lib/zsh/5.7.1/zsh/computil.so*
	lr-------- 1 leah leah 64 Jul 25 11:18 7f3e31df2000-7f3e31dff000 -> /usr/lib/zsh/5.7.1/zsh/computil.so*
	...


# vim: where was a variable set?  #vim =2018-03-02

Sometimes, various filetype plugins modify vim settings, and it's not
clear where this happens.  Vim provides the command

	:verbose set VARIABLE

to show the value of VARIABLE and the file that least changed it, e.g.:

	:verbose set formatoptions
	  formatoptions=jtcql
	        Last set from ~/.vimrc

	:verbose set commentstring
	  commentstring=# %s
        	Last set from /usr/share/vim/vim80/ftplugin/gitcommit.vim


# ld: print the link map  #ld =2018-01-02

Often one wishes to see what ld(1) actually does, and passing `-M`
(or `-Wl,-M`) is a great way to gain more information.

E.g. use `musl-gcc -static -Wl,-M ~/prj/common/hello.c` to see which
symbols are pulled in and why.

Thanks to @RichFelker.


# GraphViz: utilities and syntax  #graphviz =2017-12-12

I use Graphviz mainly for dot(1) and neato(1), but it contains several
utilities that are useful on general graph structures:

	gc(1) - count graph components
		-n     Count nodes.
		-e     Count edges.
		-c     Count connected components.

	ccomps(1) - connected components filter for graphs
		-X node_name
			Prints only the component containing node_name, if any.

Also very interesting are:

	gvpr - graph pattern scanning and processing language
	acyclic - make directed graph acyclic
	sccmap - extract strongly connected components of directed graphs
	tred - transitive reduction filter for directed graphs

Finally, a syntactic shortcut:

	a -- b, c, d;  ===  a -- b; a -- c; a -- d;
	a, b -- c, d;  ===  a -- c, a -- d; b -- c; b -- d;

(https://www.reddit.com/r/adventofcode/comments/7j89tr/x/dr4fvrb/)


# last: show hostname in full  #last #linux #util-linux =2017-04-22

The venerable tool last(1) often truncates hostnames:

	xxxxxx   pts/1        ppp-777-777-777- Sat Apr  1 09:10 - 14:37  (05:26)

The flag -a can be used to put the hostname at the end and not truncate it:

	xxxxxx   pts/1        Sat Apr  1 09:10 - 14:37  (05:26)     ppp-777-777-777-777.dynamic.xxxx-xxxxxx.de

w(1) doesn't have such a nice option, but you can use

	PROCPS_FROMLEN=38 w

to increase the column at last to fit a full IPv6 address.
(Or use who(1), which I always forget about.)

# tmux: shortcuts  #tmux =2017-03-10

tmux commands can be shortened as long as they are unique.
In particular, "tmux a" is "tmux attach".	


# rsync: protect arguments  #rsync =2017-01-02

An annoying thing aboug rsync (and scp!) is that the arguments are
also parsed by the remote shell, so you need lots of escaping ala

	rsync -avP remote:/file/with\\\ a\\\ space .

Luckily, since rsync 3.0 there is an option -s (or --protect-args) to
do the escaping for you!  From a quick glimpse at the code, it
actually seems it sends the arguments over the rsync data channel,
so this works independently of the shell used.

	rsync -avP 'remote:/file/with a space' .

Also:
> You may also control this option via the RSYNC_PROTECT_ARGS
> environment variable.  If this variable has a non-zero value, this
> option will be enabled by default, otherwise it will be disabled by
> default.


# coreutils: BLAKE2 checksums  #unix =2017-01-01

coreutils 8.26 now contains b2sum(1), a tool to compute BLAKE2
checksums.  On my system, it is negligibly slower than md5sum (and
cryptographically stronger, of course), but up to 3x faster than
sha256sum.


# Git: autostash on pull --rebase  #git =2016-12-03

For a long time, git supported "git rebase --autostash" to stash the
current changes, rebase the branch, then re-apply the stash, but it
never worked with "git pull --rebase", which is my preferred way to
update repositories.

However, since Git 2.9.0 (i.e. June 2016):

> "git pull --rebase" learned "--[no-]autostash" option, so that
> the rebase.autostash configuration variable set to true can be
> overridden from the command line.

Very useful!


# Git: which patches did a merge pull in  #git =2016-12-02

Git 2.11 has a new syntax for excluding parents of a merge:

> "git log rev^..rev" is an often-used revision range specification
> to show what was done on a side branch merged at rev.  This has
> gained a short-hand "rev^-1".  In general "rev^-$n" is the same as
> "^rev^$n rev", i.e. what has happened on other branches while the
> history leading to nth parent was looking the other way.

Note that "rev^-1" also can abbreviated to "rev^-", so you can show
all commits that were merged in a particular merge using

	git log rev^-


# Randomly ordering lines  #shuf #unix =2016-09-25

It is well known GNU coreutils contains shuf(1) to shuffle lines (also
see =2015-07-23):

	% necho a b c | shuf
	b
	c
	a
	% necho a b c | shuf
	b
	a
	c

And I used to think `sort -R` could be used as halfway portable
replacement (FreeBSD and OpenBSD have it), but it isn't, because it still
sorts:

	% necho a a b b c c | sort -R
	c
	c
	a
	a
	b
	b

So this only works if you have unique lines.

So you may want to use perl instead:

	perl -MList::Util=shuffle -e 'print shuffle <>'

Or you'll have to implement Fisher-Yates in awk for a portable solution
(here we output directly and don't construct the array first):

	awk 'BEGIN{srand()}{l[NR]=$0}
	     END{for(i=NR;i>=1;i--){j=int(i*rand())+1;print l[j];l[j]=l[i]}}'


# Listing global variables in gdb  #gdb #debugging =2016-09-25

Figuring out where a global (or static) variable is declared can be
tricky in a C program heavily using macros.  In gdb, "info variables"
will print all global variables and the translation unit they are
from.  An additional optional argument is taken as a regex to filter
the names.

(Merely doing "info symbol &myvar" only tells you the ELF segment,
which is not helpful.)


# Debugging the Solaris kernel  #solaris #smartos #illumos #mdb #unix =2016-09-09

So, I happened to stumble on a serious bug in Illumos/SmartOS, details
of the actual issue are at https://www.illumos.org/issues/7366

Some quick notes on debugging Solaris: If you boot with the `-k` flag,
it will enter the kmdb kernel debugger as soon as possible.  Use `:c`
to continue.  You can enter kmdb again pressing F1 and A at the same
time on the VGA console.  (Or send a BREAK on a serial console, but
this didn't work for me in qemu.)

Alternatively, you can enter kmdb any time by running `mdb -k` (if
your box is not locked up).

In my case, the box locked up badly, but did not panic, so I needed to
do post-mortem debugging: on x86, sending a NMI will trigger a kernel
crash dump, which SmartOS dumps onto the `zones/dump` zvol, which I
needed to grow first to fit the dump.

You can send a NMI from qemu using

	echo '{ "execute": "qmp_capabilities" }{ "execute": "inject-nmi" }' |
	  nc -U /run/qemu/.../monitor.sock

On next boot, `zones/dump` is read into a file in `/var/crash/volatile`,
which you then can uncompress using savecore(1M), and then debug using
`mdb unix.0 vmcore.0`.

Some interesting mdb commands for kernel debugging:

::status  prints some general information such as kernel version number,
and reason for dumping core
::ps  lists all running processes at that time
::walk threads  can be used to see all addresses of kernel threads
ADDR::findstack  prints a stacktrace for the thread at ADDR
::stacks  aggregates the stacks of kernel threads and shows unique backtraces
::memstat  gives some general statistics of memory usage
::kmastat  gives detailed statistics of allocated memory
ADDR/W VAL  writes the 4-byte value VAL to address ADDR,
e.g. to enable snooping (a kind of watchdog): snooping/W 1

Note that all numbers are hexadecimal by default.

I poked around with mdb a bit but didn't really have a idea what's
going on.  I then looked at `vmstat 1` output while writing data over
NFS, and it was easy to see then that some kind of memory leak
happened---I thought I triggered some deadlock or disk I/O issue beforehand.


# tig blame  #git #tig =2016-08-19

I've known of the ncurses interface to git, called "tig", for a long
time, but never found it particularly convenient over using raw git on
the command line.

However, I recently found out a real feature: interactively browsing
"git blame".  Simply run

	tig blame FILE

and you see something very much like the output of "git blame", but
you can now press "," to get to the "previous" line in history.  And
press "<" to get back.  Use RET to look at the commit, and "q" to close
the overview.

(Set TERM=vt220 if the colors annoy you.)


# Word: allistic  #words =2016-08-05

Not a tech topic, but I first learned about the word "allistic" today:
It means "non-autistic".  (Via @lalobee.)

https://crackedmirrorinshalott.wordpress.com/2013/04/12/autistic-allistic-neurodiverse-and-neurotypical-say-what/


# WordStar-like move in Vim  #vim =2016-07-25

I learned about a nice feature in WordStar, where you select a region
using ^KB to mark the start and ^KK the end, then move the cursor
somewhere else, and type ^KV to move the region to the cursor.
So it's like cut and paste at once.

I think this is quite nice for restructuring documents, because you
never lose sight of the text.

Of course, its possible to simulate this in Vim.  We'll use the visual
selection to mark the region, so the range is kept in '<,'>.  Then we
just use the ed(1) command :m(ove) to shift the lines around.  Add a
bit of mark setting around to keep the cursor position at the
beginning of the block, and you end up with

	command! Mo k` | '<,'>m . | norm ``j


# Getting the default Git commit author/mail  #git =2016-07-21

Mark Dominus explains everything below, but in short, you can use
`git var GIT_COMMITTER_IDENT` to get the metadata that would
be used for a commit.

This resulted in the following new alias:

        whoami = !git var GIT_COMMITTER_IDENT | sed 's/>.*/>/'

http://blog.plover.com/prog/git-author-email.html


# Array literals in zsh  #zsh =2016-07-14

It's always nice when you notice you can combine two functions you
already knew, but never tought of putting them together.

In zsh, you can use ${=var} to do sh(1)-style word splitting:

	% s="78 54 21"
	% qecho $s
	»78 54 21« 
	% qecho ${=s}
	»78« »54« »21« 

Another zsh trick (4. below) is using ${VAR:-DEFAULT} expansion with
an empty VAR to do expansion on DEFAULT.

	% echo ${${:-string}:u}
	STRING

Combine them together and you have a syntax for an array literal in a
zsh expansion!

	% a=(78 54 21)
	% echo ${(n)a}
	21 54 78
	% echo ${(n)=:-78 54 21}
	21 54 78
	% echo ${(M)${=:-foo bar baz}:#*a*}
	bar baz
	% qecho ${(M)${=:-foo 'bar quux' baz}:#*a*} 
	»bar quux« »baz«

I shall call "=:-" the "punk operator" (notice the Mohican).

http://chneukirchen.org/blog/archive/2011/02/10-more-zsh-tricks-you-may-not-know.html
via http://stackoverflow.com/questions/37510168/joining-on-expanded-variables-in-zsh
via https://superuser.com/questions/1098551/append-string-to-array-elements-and-join-them-in-one-expression


# Remote VNC over SSH  #vnc #ssh #uds =2016-07-05

Sometimes you want to connect to a remote VNC session, for example, to
a host running qemu-kvm.  Instead of dealing with authentication and
autorisation on VNC level, it can be easier to tunnel with SSH instead.

The classic approach is to use

	ssh -L 5900:localhost:5900 user@remote
	vncviewer :0

But tigervnc provides the useful argument `-via HOST`, reducing this to

	vncviewer -via user@remote :0

Unfortunately, the syntax breaks down when you use VNC over Unix
Domain Sockets (which vncviewer doesn't support directly), even though
OpenSSH since 6.7 can forward them fine.  But you can override the via
command to fix this:

	VNC_VIA_CMD='ssh -f -L "$L:$H" "$G" sleep 20' \
		vncviewer -via user@remote /path/to/vnc.sock

Note that the flags in $VNC_VIA_CMD are misued, but it works.


# File lists in Vim quickfix  #vim #unix =2016-06-08

I wrote about the quickfix feature before.  With a simple tweak, Vim
can regard arbitary lists of files as quickfix lists:

	set errorformat+=%+A%f

Put this in your .vimrc, and e.g. use:

	find >files
	vim -q files

to open vim with the quickfix file.  Or use :cgetfile to select the file,
or use :cbuffer or :cgetbuffer to read the current buffer as a
quickfix list.

Plain "%f" would do, but with the "%+A%f" hack we put the filename into
the description, which is more useful when you have laststatus=1.


# Git log auto decoration  #git =2016-05-27

In a mail from Linus I learned about "git log --decorate=auto" and the
accompanying setting

	[log]
		decorate = auto

which will enable "decoration" (i.e. printing symbolic aliases of the
commit) only when "git log" is not piped.  Seems to be undocumented,
but very useful to me.

http://permalink.gmane.org/gmane.comp.version-control.git/295721


# Emacs indentation settings  #emacs #tabs =2016-05-12

Similar to my Vim indentation settings, I wrote a small elisp hook to
switch to tab-based indentation when the file contains a tab as the
first character of a line:

	(defun guess-tab-settings ()
	  (interactive)
	  (save-excursion
	    (goto-char (point-min))
	    (when (re-search-forward "^\t" 8192 t)
	      (setq indent-tabs-mode t)
	      (set (make-local-variable 'c-basic-offset) tab-width)
	      (set (make-local-variable 'sh-indentation) tab-width)
	      (set (make-local-variable 'sh-basic-offset) tab-width)
	      (set (make-local-variable 'standard-indent) tab-width)
	      (set (make-local-variable 'cperl-indent-level) tab-width)
	      (message "File uses tabs for indentation"))))
	(add-hook 'find-file-hook 'guess-tab-settings)

There seems to be no standard variable to set the indentation width,
so I defined it only for all file types I expect tabs in.


# Vim indentation settings  #vim #tabs =2016-05-12

My Vim indentation config boils down to:

	set ts=8 sts=2 sw=2 et
	autocmd BufRead * silent g/^\t/set sts=0 sw=8 noet|norm!``

It defaults to two spaces indentation, no tabs, but if there is a line
containing a tab, it sets indentation to 1 space (= 8 chars) with tabs.

Note the clever use of `g//` to set this in a BufRead hook.


# Using sed(1) to replace only the first occurrence  #unix #sed =2016-05-03

I had to replace only the first match of a string in a file, using
"sed -i".  This turned out not as easy as expected.  I found below
Stack Overflow answer the best solution:

	0,/Apple/{s/Apple/Banana/}

So you limit the replacement to the range of lines until the first match.

(Actually I had to replace the second match(!), and I still don't have
a good idea how to do it. (I guess with sufficient branching its
possible) I knew the first match was early, so I used

	50,/Apple/{s/Apple/Banana/}

If I ever implement a standalone "gres" tool, it will allow that more easily.)

http://stackoverflow.com/a/9453461


# Random MACs with wpa_supplicant  #wifi #privacy =2016-04-21

Recent versions of wpa_supplicant (2.5) can use MAC randomization for
better privacy, like Apple products do.

	mac_addr=1 # use random MAC address for each ESS connection
	mac_addr=2 # like 1, but maintain OUI (with local admin bit set)

	preassoc_mac_addr=1 # use random MAC address
	preassoc_mac_addr=2 # like 1, but maintain OUI (with local admin bit set)


# Tiny HTTP proxy with ncat  #nmap #ncat #proxy =2016-04-12

Sometimes one needs a proxy for various reasons.  SOCKS proxies can be
set up easily using "ssh -D", but when a HTTP proxy is needed, one is
tempted to use polipo or squid.

For a simple HTTP proxy, the "ncat" tool included with nmap is enough.
This will run a HTTP proxy on port 8888:

	ncat -l --proxy-type=http localhost 8888

Use "-vvv" to see the contents proxied as debugging output.
Likewise, --proxy-type can be socks4 or socks5.

Addendum: =2016-09-01: not true, the only supported listen mode is http.


# LaTeXML  #tex #latex #html =2016-04-01

LaTeXML is a fully-featured TeX to HTML converter written in Perl,
which actually parses/reimplements TeX so well even the most
obfuscated macros will work.  I ran it on a random paper of mine with
amazing results, translating formulas, tables and embedded graphics
very well.

To use with carton, you can use

	requires 'LaTeXML';
	requires 'Image::Magick';

Then run "carton install", "carton exec latexml"

http://dlmf.nist.gov/LaTeXML/


# Coping with Perl dependencies  #perl #cpan #carton =2016-04-01

When you want to play with a complex Perl program, it's likely your
distribution doesn't have all required dependencies from CPAN
packaged.

While you can use the included "cpan" tool to easily install them
yourself, there are multiple drawbacks (afaiu): it can't remove
installed things and only installs stuff globally.

The better "cpanm" tool can uninstall, install locally or into a
seperate directory.

An even better tool I didn't know before (thx @Grauwolf) is "carton",
which wraps cpanm in a bundler-style way, so you write a "cpanfile"
with "requires XXX;" lines, run "carton install" and it will install
the stuff into cwd, and provide "carton exec" for easy running.


# Normalize trailing newlines  #awk #unix =2016-03-18

If you want to do an operation that adds a trailing newline if there
is none, and does nothing else, the probably easiest way is to run

	awk 1

I.e, the awk program that accepts every line (and prints it by default
followed by a newlines).  (Found in @izabera's bin dotfiles.)


# Vim: shortcuts for the quickfix list  #vim =2016-03-17

Many useful vim functions use the quickfix list, which requires a bit
of typing to use.  I decided to bind cprev/cnext to _ and + so
they can be used easily from normal mode:

	nnoremap <silent> _ :cprev<CR>
	nnoremap <silent> + :cnext<CR>

I decided not to use - because it's actually useful in normal mode (as
opposed to + which is RET and _ which is like ^ without count), and note
that - and _ still keep their original meaning as a movement.
Also I can keep shift pressed when alternating between + and _.

:make, :vimgrep and :helpgrep are much more fun now.


# Close vim as pager on empty stdin  #vim =2016-03-15

I sometimes abuse vim as a pager ala "vim -".  This time, I wanted vim
not to run when the input was empty, to I did this hack:

	mycmd | vim - '+silent g/\%^\%$/cq!'

The regex `\%^\%$` will match when end-of-buffer comes directly after
beginning-of-buffer, i.e. input is empty.  Then `:cq` is run, which
exits vim with exit code 1.


# Linux 4.5: ext4 project quotas  #linux #ext4 #quota =2016-03-14

Linux 4.5 includes ext4 project quotas, a feature known from XFS,
which allows to define quotas on a set of directories (as opposed to
per-user or per-group).

Unclear to me how to actually enable it, the commit messages say it's
compatible to the xfs_quota ioctls.


# Git submodules  #git #submodules =2016-03-11

Git submodules are sometimes awkward to use.  The following alias

	sub = submodule update --init --recursive

can be run at any time and will check out the correct files.


# Git templates  #git =2016-03-11

By default, Git copies a lot of files into new repositories like hook
templates and so on.  I set init.templatedir to /var/empty, and now I
only get

	.git/
	.git/HEAD
	.git/config
	.git/objects/info/
	.git/objects/pack/
	.git/refs/heads/
	.git/refs/tags/

I did not see anything break due to this yet.


# Recursive hashing  #unix #sha256 #deephash =2016-03-07

I was in search of a tool to hash some archived data to check it later
if it has been silently corrupted.
There are various tools such as rhash, deephash or plain find + sha256sum.
I decided to use deephash since it can hash in parallel.

Usage:

	% hashdeep -c sha256 -r -e -W /etc/hashes /my/dir
	% hashdeep -k /etc/hashes -a -vvve -r /my/dir

There doesn't seem to be an option to print the filenames during hashing.


# Comments in Markdown  #markdown =2016-03-07

This weird hack is needed to do a comment in Markdown:
http://stackoverflow.com/questions/4823468/comments-in-markdown/32190021#32190021


# GNU patch support for git diffs  #patch #git =2016-03-04

Since version 2.7, GNU patch has support for "most features of the
"diff --git" format, including renames and copies, permission changes,
and symlink diffs.  Binary diffs are not supported yet; patch will
complain and skip them."


# Print terminal colors  #unix #msgcat =2016-03-01

Gettext ships a small utility which also can test your terminal colors:

% msgcat --color=test

Via Peter Fischer.

http://blog.sanctum.geek.nz/term-strings/


# POSIX regular expression intricacies  #unix #posix #regex =2016-03-01

POSIX specifies two regular expression syntaxes, Basic Regular
Expressions (BRE) and Extended Regular Expressions (ERE).
The main difference is whether you need to quote | ( { etc.

BRE allows to use *, ^ and $ in positions where they "don't make
sense" and then they match the literal character.

Learned due to a bug in musl 1.1.14 which rejected BRE starting with *.

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html


# OpenSSH 7.2: AddKeysToAgent  #unix #ssh =2016-02-29

OpenSSH 7.2 has been released and now features the new AddKeysToAgent function:

 * ssh(1): Add an AddKeysToAgent client option which can be set to
   'yes', 'no', 'ask', or 'confirm', and defaults to 'no'.  When
   enabled, a private key that is used during authentication will be
   added to ssh-agent if it is running (with confirmation enabled if
   set to 'confirm').

I used to maintain a patch that did something similar, it is now superfluous.

http://www.openssh.com/txt/release-7.2
http://chneukirchen.org/blog/archive/2014/02/the-road-to-openssh-bliss-ed25519-and-the-identitypersist-patch.html


# Copying permissions with chmod  #unix #chmod =2016-02-27

The symbolic mode of chmod not only understands the permissions
r w x s t but also u g o and X.

Here, u g o mean the permission bits for the user, group or other.

Thus, to reset the group permissions to the other permissions, you can
just run `chmod g=o file`.

X means executable if any executable bit it set.


# Chromium: download URL in extended attributes  #chromium #xattr =2016-02-25

Chromium saves the download URL of a file as an extended attribute
`user.xdg.origin.url` which you can display using `getfattr -d`.


# Make: implicit rules and intermediate files  #make #gmake =2016-02-24

Implicit make rules with patterns ala `%.o: %.c` are a SVR4 extension
and not part of POSIX make.  GNU make and Solaris make implement them.
(As well as a variant thereof in Plan 9 mk(1)).

GNU make deletes intermediate files which are generated by chains of
implicit rules, unless you add them to .PRECIOUS or .SECONDARY.


# Windows recent directories  #windows =2016-02-24

Seen on Windows 8 (also on 7): When you put Explorer into the Taskbar,
you can right click it and it will show the recently visited
directories.  You can also pin them and they will stick to the top of
the list.


# Aligning math with flexible spacing  #latex #amsmath =2016-02-23

The amsmath align environment generates columns like

  \begin{align}
  left&right&left&right&left&right\\
  left&right&left&right&left&right
  \end{align}

  left right     left right     left right
  left right     left right     left right

When you don't want additional space between the columns, you can use
alignat instead.  However, it requires an additional argument of how
many columns to provide (don't ask me why align simply works without
that...):

  \begin{alignat}{3}
  left&right&left&right&left&right\\
  left&right&left&right&left&right
  \end{alignat}

  left right left right left right
  left right left right left right

Then you can add appropriate spacing yourself.


# Vim packages  #vim =2016-02-22

As of today, vim has gained built-in support for packages, ala
pathogen or vundle.

https://github.com/vim/vim/commit/f6fee0e2d4341c0c2f5339c1268e5877fafd07cf


# Printing ASCII values in POSIX shell  #shell #printf =2016-02-21

There is a special syntax in printf(1) to print ASCII codes of single chars:

>       *  If  the  leading  character  is a single-quote or double-quote, the
>          value shall be the numeric value in the underlying codeset  of  the
>          character following the single-quote or double-quote.

% printf '%o %d %x\n' "'a" "'a" "'a"
141 97 61


# LaTeX operator spacing  #latex #tex =2016-02-18

Sometimes one likes to refer to a relation as a mathematical object,
e.g. a set `\rightarrow \subseteq \mathbb{N} \cross \mathbb{N}`.
In this case, the spacing will be off since `\rightarrow` is taken as
a operator.

If you instead write `{\rightarrow} \subseteq \mathbb{N} \cross \mathbb{N}`
the spacing will be a lot better.


# Reading Google Groups  #googlegroups =2016-02-17

Google Groups has a terrible interface, but it's possible to enter
a print view which makes it readable without JavaScript even.

Take a URL, e.g.
https://groups.google.com/forum/#!msg/stackage/4PLy1fYtsps/BBjYteTbAwAJ

If it doesn't refer to the full topic yet, change msg to topic and
drop the last segment:
https://groups.google.com/forum/#!topic/stackage/4PLy1fYtsps

Finally replace #!topic with print/

https://groups.google.com/forum/print/topic/stackage/4PLy1fYtsps

The result is a raw, readable HTML page.  (Thanks @nsz.)


# Vim: helpgrep  #vim =2016-02-15

Learned from Damian Conway: Vim's extensive help can easily be searched using

:helpg[rep] {pattern}[@xx]
                        Search all help text files and make a list of lines
                        in which {pattern} matches.  Jumps to the first match.

http://howivim.com/2016/damian-conway/


# Vim: repeating last colon command  #vim =2016-02-15

The last colon command can be repeated using @:, since it's kept in
register :.  Afterwards, you can repeat the command again with @@.


# Simple argument parsing in Ruby  #ruby #unix =2016-02-14

The 'optparse' library provides a very convenient shortcut for no-fuzz
argument parsing getopt(3) style:

>> require 'optparse'
>> ARGV.replace %w{-acd -o foo meh bar}
>> ARGV.getopts("abcdo:u:")
=> {"a"=>true, "b"=>false, "c"=>true, "d"=>true, "o"=>"foo", "u"=>nil}
>> ARGV
=> ["meh", "bar"]


# Revert to previous save in Vim  #vim #undo =2016-02-13

Vim (since 7.3) has the :earlier command to go back in time, not
only by seconds, minutes, hours, or days, but also by file writes:

:earlier {N}f           Go to older text state {N} file writes before.
                        When changes were made since the last write
                        ":earlier 1f" will revert the text to the state when
                        it was written.


# Font rendering in Firefox 44  #firefox #fonts =2016-02-12

When Firefox 44 shows you ugly font rendering, set
gfx.font_rendering.fontconfig.fontlist.enabled = false
and restart.

Should be fixed in FF 45.

https://bugzilla.mozilla.org/show_bug.cgi?id=1180560
https://bugzilla.mozilla.org/show_bug.cgi?id=1243226


# Clean TeX messages  #tex #latex #perl =2016-02-11

The Perl filter `texfot` can be used to run TeX and only show
significant output:

% texfot pdflatex -shell-escape foo   
/opt/texlive/2015/bin/x86_64-linux/texfot: invoking: pdflatex -shell-escape foo
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015) (preloaded format=pdflatex)
LaTeX Warning: Citation `z' on page 1 undefined on input line 11.
LaTeX Warning: There were undefined references.
Output written on foo.pdf (1 page, 45237 bytes).

Included in TeXLive 2015.


# Let's Encrypt DNS Challenges  #letsencrypt #ssl #dns =2016-02-11

Let's Encrypt enabled DNS challenges already in January.
letsencrypt.sh can now generate them, but needs a hook to update the
DNS server.
For nsd 4, this should be possible with `nsd-control reload`.

https://twitter.com/letsencrypt/status/689919523164721152
https://tools.ietf.org/html/draft-ietf-acme-acme-01#section-7.5


# lft  #unix #network #traceroute =2016-02-10

lft is a flexible layer 4 traceroute tool that has a good adaptive TCP
mode (-E) and may thus work when UDP or ICMP traceroute doesn't.

Packaged it for Void Linux.


# TIL files  #til =2016-02-10

The purpose of this file is to quickly note down things recently
learned.  #hashtags refer to topics, @mentions to persons,
and =timestamps to the date.  Entries below are backdated from
IRC logs.

https://github.com/jbranchaud/til


# Suspend to both  #linux #suspend #hibernate #pm =2016-02-05

Linux supports "suspend to both" since version 3.6, i.e.  write RAM to
disk (hibernate) and then suspend the system, such that when the
battery runs empty, it still can wake up.

% echo suspend > /sys/power/disk; echo disk > /sys/power/state

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=62c552ccc3eda1198632a4f344aa32623d226bab


# Look at git reflog with git log  #git =2016-02-03

git-log(1):

       -g, --walk-reflogs
           Instead of walking the commit ancestry chain, walk reflog entries
           from the most recent one to older ones.

Compared to `git reflog`, this allows custom formatting with --pretty.


# View HTML mail in browser from Gnus  #mail #gnus #html #web =2016-02-02

"K H" runs the command gnus-article-browse-html-article, which opens
the current HTML mail in a browser.


# Ruby prelude.rb  #ruby =2016-01-25

Ruby contains since 1.9.3 the file `prelude.rb` which is inlined into
the interpreter and always evaluated.


# Weechat bare mode  #weechat #irc =2016-01-21

Weechat can display the buffer in "bare" mode without highlighting,
formatting and wrapping the lines itself, so you can click on them.

This mode can not only be activated using M-l, but also using F1.


# pv line mode  #pv #unix =2016-01-03

The pipe viewer tool pv(1) can not only print statistics bytewise,
but also count lines, with the option `-l`.

Useful for example when piping file names to xargs or xe.
(`-0` exists too.)


# ISO week time formatting  #unix #strftime #time  =2016-01-02

strftime(3):
       %V     The ISO 8601 week number (see NOTES) of the current year as a
              decimal number, range 01 to 53, where week 1 is the first week
              that has at least 4 days in the new year.
       %G     The ISO 8601 week-based year (see NOTES) with century as a
              decimal number.  The 4-digit year corresponding to the ISO week
              number (see %V).

% date +"%V %G %Y" -d 2016-01-01 
53 2015 2016

Using %Y together with %V seems to be always wrong.


# Vim Patchmode  #vim =2015-10-15

'patchmode' 'pm'        string  (default "")
                        global
                        {not in Vi}
        When non-empty the oldest version of a file is kept.  This can be used
        to keep the original version of a file if you are changing files in a
        source distribution.  Only the first time that a file is written a
        copy of the original file will be kept.

Useful together with unpatch: http://chneukirchen.org/dotfiles/bin/unpatch


# envsubst  #unix #shell =2015-11-08

Utility contained in GNU gettext:

       envsubst - substitutes environment variables in shell format strings

Useful for safe expansion of strings.

% name=world envsubst <<<'Hello, ${name}!' 
Hello, world!


# git diff HEAD  #git =2015-09-28

git diff HEAD shows staged and unstaged changes at once.
git diff -c HEAD shows them as a combined diff:
the first +/- is working tree vs index,
the second +/- is index vs HEAD.


# Intersecting vim regex  #vim #regex =2015-09-14

Vim understands regexps ala `a\&b` which need to match `a` and `b` (in
any order, possibly overlapping).  The text of the last match is taken.

Can be combined with `\%#` (match cursor position) to match certain
chars under cursor, e.g. `\(\%#\&\s\)` matches when cursor is on whitespace.


# Calendar for the current year  #unix #shell #cal =2015-09-13

`cal -y` prints the 12-month calendar for the current year.


# Generating random permutations  #shuf #unix #shell =2015-07-23

GNU shuf can not only shuffle the lines of a file, but also
arguments (with `-e`) and numbers (with `-i MIN-MAX`).


# Fixing fuzzy postscript files  #tex #postscript #pdf =2015-07-05

Old postscript files (mostly TeX generated) with fuzzy bitmap fonts
are painful to read.  The tool `pkfix` tries to detect commonly used fonts
and replaces them with nice Type 1 fonts.


# Exiting vi  #vi #vim =2015-06-17

`ZZ`, `:wq`, `:x` are commonly used to exit vi(1), but not exactly the same:

`:wq` will always write the file, even if it was not modified, and exit.
`ZZ` and `:x` only write the file when it was modified, then exit.