gatillos.com - gun-toting kittens2024-03-07T16:43:47+00:00https://gatillos.com/yayosukawebmaster@gatillos.comBottlerocket and instance-store ephemeral SSD volumes in AWS2024-03-03T23:20:00+00:00https://gatillos.com/yay/2024/03/03/bottlerocket.html
<p>For a while now I've been using <strong>Bottlerocket</strong> as the base OS for instances in the cloud (i.e. the AMI), instead of using Amazon Linux or Ubuntu etc. In our workload we don't really have much use for local storage until recently so I finally invested some time in figuring out how to actually make use of the local SSDs where available (usually this type of storage is called <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html">instance-store</a> in AWS).</p>
<p>Some Bottlerocket concepts:</p>
<ul>
<li>Minimalist, container-centric</li>
<li>There is no ssh or even a login shell by default</li>
<li>It provides 'host containers' that are not orchestrated (not part of kubernetes)
<ul>
<li>Control container - this is the one that allows us to connect via SSM</li>
<li>Admin container - this is the one that allow us to interact with the host container as root</li>
<li>The <strong>root file system</strong> is stored in <code>/.bottlerocket/rootfs</code> and there you can find /dev etc</li>
</ul></li>
<li>You can launch bootstrap containers if you need to run tasks, instead of using user-data scripts</li>
</ul>
<p>Typically you don't connect to these instances, but if you need to you can enable the control and admin containers in the user-data settings, connect to the control container and then type <code>enter-admin-container</code>. To connect to the control container use SSM (easiest is to do this via the EC2 web console, but also with <code>aws ssm start-session --target <instanceid></code>))</p>
<p>You can learn more on <a href="https://github.com/bottlerocket-os/bottlerocket?tab=readme-ov-file">bottlerocket's github</a>, <a href="https://bottlerocket.dev/en/os/1.19.x/concepts/">their concept overview</a> and the FAQ (<a href="https://bottlerocket.dev/en/faq/">https://bottlerocket.dev/en/faq/</a>)</p>
<p>Bottlerocket is configured by providing <a href="https://bottlerocket.dev/en/os/1.19.x/api/settings/">settings</a> via user data in TOML format and what you provide will be <strong>merged</strong> with the defaults.</p>
<h2 id="configuring-bottlerocket-in-node-groups">Configuring Bottlerocket in node groups</h2>
<div class="highlight"><pre><code class="language-terraform" data-lang="terraform"><span class="k">module</span> <span class="s2">"my-nodegroup"</span> <span class="p">{</span>
<span class="nx">source</span> <span class="p">=</span> <span class="s2">"terraform-aws-modules/eks/aws//modules/eks-managed-node-group"</span>
<span class="p">...</span>
<span class="nx">ami_type</span> <span class="p">=</span> <span class="s2">"BOTTLEROCKET_ARM_64"</span>
<span class="nx">platform</span> <span class="p">=</span> <span class="s2">"bottlerocket"</span>
<span class="nx">bootstrap_extra_args</span> <span class="p">=</span> <span class="o"><<-</span><span class="no">EOT</span><span class="sh">
[settings.host-containers.admin]
enabled = true
[settings.host-containers.control]
enabled = true
[settings.xxx.xxx]
setting1 = "value"
setting2 = "value"
setting3 = true
</span><span class="no"> EOT
</span><span class="p">}</span>
</code></pre></div>
<h2 id="configuring-bottlerocket-in-karpenter">Configuring Bottlerocket in Karpenter</h2>
<p>See <a href="https://karpenter.sh/preview/concepts/nodeclasses/#bottlerocket-2">the default values that Karpenter sets</a>. </p>
<p>In Karpenter these days for AWS you define an <code>EC2NodeClass</code> that basically describes the image to be used and where it will be placed, and a <code>NodePool</code> that defines the hardware it will run in.</p>
<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">karpenter.k8s.aws/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">EC2NodeClass</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">myproject</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">amiFamily</span><span class="pi">:</span> <span class="s">Bottlerocket</span>
<span class="na">userData</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">[settings.host-containers.admin]</span>
<span class="s">enabled = true</span>
<span class="s">[settings.host-containers.control]</span>
<span class="s">enabled = true</span>
<span class="s">[settings.xxx.xxx]</span>
<span class="s">setting1 = "value"</span>
<span class="s">setting2 = "value"</span>
<span class="s">setting3 = true</span>
<span class="na">subnetSelectorTerms</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">tags</span><span class="pi">:</span>
<span class="na">Name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">us-east-1a"</span>
<span class="pi">-</span> <span class="na">tags</span><span class="pi">:</span>
<span class="na">Name</span><span class="pi">:</span> <span class="s2">"</span><span class="s">us-east-1b"</span>
<span class="na">securityGroupSelectorTerms</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">tags</span><span class="pi">:</span>
<span class="s">karpenter.sh/discovery</span><span class="pi">:</span> <span class="s2">"</span><span class="s">my-cluster"</span>
<span class="na">instanceProfile</span><span class="pi">:</span> <span class="s">eksctl-KarpenterNodeInstanceProfile-my-cluster</span>
<span class="nn">---</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">karpenter.sh/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">NodePool</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">myproject</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">limits</span><span class="pi">:</span>
<span class="c1"># limit how many instances this can actually create, we limit by cpu only</span>
<span class="na">cpu</span><span class="pi">:</span> <span class="m">100</span>
<span class="na">template</span><span class="pi">:</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="na">nodeClassRef</span><span class="pi">:</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">karpenter.k8s.aws/v1beta1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">EC2NodeClass</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">myproject</span>
<span class="na">requirements</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">karpenter.sh/capacity-type"</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">spot"</span><span class="pi">]</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">kubernetes.io/arch"</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">arm64"</span><span class="pi">]</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">karpenter.k8s.aws/instance-cpu"</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">4"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">8"</span><span class="pi">]</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">karpenter.k8s.aws/instance-generation"</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">Gt</span>
<span class="na">values</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">6"</span><span class="pi">]</span>
<span class="pi">-</span> <span class="na">key</span><span class="pi">:</span> <span class="s2">"</span><span class="s">karpenter.k8s.aws/instance-category"</span>
<span class="na">operator</span><span class="pi">:</span> <span class="s">In</span>
<span class="na">values</span><span class="pi">:</span> <span class="pi">[</span><span class="s2">"</span><span class="s">c"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">m"</span><span class="pi">,</span> <span class="s2">"</span><span class="s">r"</span><span class="pi">]</span>
<span class="c1"># alternatively</span>
<span class="c1">#- key: "node.kubernetes.io/instance-type"</span>
<span class="c1"># operator: In</span>
<span class="c1"># values:</span>
<span class="c1"># - "c6g.xlarge"</span>
<span class="c1"># - "m7g.xlarge"</span>
</code></pre></div>
<blockquote>
<p>This <a href="https://aws.amazon.com/blogs/containers/reduce-container-startup-time-on-amazon-eks-with-bottlerocket-data-volume/">article</a> discusses some options for pre-caching images that we won't do, but illustrates the architecture of Bottlerocket OS and how it can be combined with Karpenter.</p>
</blockquote>
<h2 id="storage-in-bottlerocket-permanent-ephemeral-and-local">Storage in Bottlerocket (permanent, ephemeral and local)</h2>
<p>Bottlerocket operates with two default storage volumes: root and data. The root is read only and the data is used as persistent storage (EBS that will survive reboots) for non-Kubernetes containers that run inside the instance. The data container is 20 GB EBS drive and the root device is around 4 GB.</p>
<p>Check the Karpenter <a href="https://karpenter.sh/preview/concepts/nodeclasses/#bottlerocket-1">default volume configuration</a></p>
<p>Now the whole point of this post is to show how you can use the local SSD disks that machines often have but that Amazon makes particularly cumbersome to use. Many instances have local SSD storage that will show up inside the host as an extra unmounted device (eg /dev/nvme2n1). How do you make this available to Kubernetes in an automated way?</p>
<h2 id="kubernetes-storage-local-static-provisioner">Kubernetes Storage local static provisioner</h2>
<p>Learn more <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner">on the storage-local-static-provisioner github page</a> and <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/docs/getting-started.md">the getting started guide</a>.</p>
<p>The way we expose local disks to kubernetes as resource is via a persistent storage class called 'fast-disks'. The local provisioner will allow creating of these resources of that type by finding local storage</p>
<ul>
<li>It expects the host to have a folder called /mnt/fast-disks (configurable)</li>
<li>It expects there to be links to the actual device files of drives we want exposed</li>
<li>It is installed by generating a configuration file using helm and applying it with kubectl</li>
</ul>
<p>Before going further we need to define a storage class that will be used to flag the new type of storage.</p>
<ul>
<li>Download <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/76b022f7dbb48757f19ee11f31f91ab19b938407/deployment/kubernetes/example/default_example_storageclass.yaml">the storage class file</a></li>
</ul>
<p>It looks like this and effectively does nothing (see the no-provisioner):</p>
<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">storage.k8s.io/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">StorageClass</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">fast-disks</span>
<span class="na">provisioner</span><span class="pi">:</span> <span class="s">kubernetes.io/no-provisioner</span>
<span class="na">volumeBindingMode</span><span class="pi">:</span> <span class="s">WaitForFirstConsumer</span>
<span class="c1"># Supported policies: Delete, Retain</span>
<span class="na">reclaimPolicy</span><span class="pi">:</span> <span class="s">Delete</span>
</code></pre></div><div class="highlight"><pre><code class="language-sh" data-lang="sh">kubectl apply <span class="nt">-f</span> ./default_example_storageclass.yaml
</code></pre></div>
<p>Then setup the local provisioner daemonset that will handle it</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="c"># Generate the configuration file</span>
helm repo add sig-storage-local-static-provisioner https://kubernetes-sigs.github.io/sig-storage-local-static-provisioner
helm template <span class="nt">--debug</span> sig-storage-local-static-provisioner/local-static-provisioner <span class="nt">--version</span> 2.0.0 <span class="nt">--namespace</span> myproject <span class="o">></span> local-volume-provisioner.generated.yaml
<span class="c"># edit local-volume-provisioner.generated.yaml if necessary</span>
<span class="c"># optional: kubectl diff -f local-volume-provisioner.generated.yaml</span>
kubectl apply <span class="nt">-f</span> local-volume-provisioner.generated.yaml
</code></pre></div>
<blockquote>
<p>See more <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/helm/README.md#install-local-volume-provisioner-with-helm">about the installation procedure</a></p>
</blockquote>
<p>This creates a <strong>daemonset</strong> that runs in all instances.</p>
<p>These are some excerpts of what it creates:</p>
<p>The storage class - note that by default it will use <code>shred.sh</code> to initialize the disk, which is rather slow. You can change it to other methods. This will be done only on first boot.</p>
<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="c1"># Source: local-static-provisioner/templates/configmap.yaml</span>
<span class="na">apiVersion</span><span class="pi">:</span> <span class="s">v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">ConfigMap</span>
<span class="na">metadata</span><span class="pi">:</span>
<span class="na">name</span><span class="pi">:</span> <span class="s">release-name-local-static-provisioner-config</span>
<span class="na">namespace</span><span class="pi">:</span> <span class="s">myproject</span>
<span class="na">labels</span><span class="pi">:</span>
<span class="s">helm.sh/chart</span><span class="pi">:</span> <span class="s">local-static-provisioner-2.0.0</span>
<span class="s">app.kubernetes.io/name</span><span class="pi">:</span> <span class="s">local-static-provisioner</span>
<span class="s">app.kubernetes.io/managed-by</span><span class="pi">:</span> <span class="s">Helm</span>
<span class="s">app.kubernetes.io/instance</span><span class="pi">:</span> <span class="s">release-name</span>
<span class="na">data</span><span class="pi">:</span>
<span class="na">storageClassMap</span><span class="pi">:</span> <span class="pi">|</span>
<span class="s">fast-disks:</span>
<span class="s">hostDir: /mnt/fast-disks</span>
<span class="s">mountDir: /mnt/fast-disks</span>
<span class="s">blockCleanerCommand:</span>
<span class="s">- "/scripts/shred.sh"</span>
<span class="s">- "2"</span>
<span class="s">volumeMode: Filesystem</span>
<span class="s">fsType: ext4</span>
<span class="s">namePattern: "*"</span>
</code></pre></div>
<h2 id="joining-local-static-provisioner-aws-instance-store-and-bottlerocket">Joining local static provisioner, AWS instance store and Bottlerocket</h2>
<p><a href="https://bottlerocket.dev/en/os/1.19.x/concepts/bootstrap-containers/#capabilities-and-permissions">Bootstrap containers</a> have root access and boot with the root file system mounted.</p>
<p>You can create one like this, and publish to your own repository:</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">FROM alpine:latest
RUN apk add <span class="nt">--no-cache</span> util-linux
FROM alpine:latest
RUN <span class="nb">echo</span> <span class="s1">'#!/bin/sh'</span> <span class="o">></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">'mkdir -p /.bottlerocket/rootfs/mnt/fast-disks'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">'cd /.bottlerocket/rootfs/mnt/fast-disks'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">'for device in $(ls /.bottlerocket/rootfs/dev/nvme*n1); do'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">' base_device=$(basename $device)'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">' if ! mount | grep -q "$base_device"; then'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">' [ ! -e "./$base_device" ] && ln -s "../../dev/$base_device" "./$base_device"'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">' fi'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">'done'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">echo</span> <span class="s1">'ls -l /.bottlerocket/rootfs/mnt/fast-disks'</span> <span class="o">>></span> /script.sh <span class="se">\</span>
<span class="o">&&</span> <span class="nb">chmod</span> +x /script.sh
CMD <span class="o">[</span><span class="s2">"/script.sh"</span><span class="o">]</span>
</code></pre></div>
<p>Add this to the Bottlerocket settings</p>
<div class="highlight"><pre><code class="language-toml" data-lang="toml"><span class="nn">[settings.bootstrap-containers.diskmounter]</span>
<span class="py">essential</span> <span class="p">=</span> <span class="kc">true</span>
<span class="py">mode</span> <span class="p">=</span> <span class="s">"always"</span>
<span class="py">source</span> <span class="p">=</span> <span class="s">"YOURACCOUNT.ecr.public.ecr.aws/YOURREPO:latest"</span>
</code></pre></div>
<h2 id="caveat-node-cleanup">Caveat: node cleanup</h2>
<p>By default, when you set this up if an instance dies (which in AWS can happen any minute) the persistent volume claim will remain.</p>
<p>Basically the life cycle is a bit like:</p>
<ul>
<li>Pod is created that has a persistent storage claim of a fast-disk</li>
<li>This is allocated in one of the instances</li>
<li>Once it is assigned, the pod will be assigned to that instance</li>
<li>If the instance dies, the pod is still assigned to that specific instance and deleting it won't fix it</li>
</ul>
<p>This will manifest as an error about the pod not being able to find <code>nodeinfo</code> for the now defunct node.</p>
<p>This cleaning process is achieved by the <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/docs/node-cleanup-controller.md">node-cleanup-controller</a></p>
<p>To set it up:</p>
<ul>
<li>Download <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/deployment/kubernetes/example/node-cleanup-controller/deployment.yaml">the deployment file</a></li>
<li>Download <a href="https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/deployment/kubernetes/example/node-cleanup-controller/rbac.yaml">the permissions file</a></li>
<li>Edit it to <strong>change the storage class</strong> (default is called nvme-ssd-block)</li>
</ul>
<div class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">apiVersion</span><span class="pi">:</span> <span class="s">apps/v1</span>
<span class="na">kind</span><span class="pi">:</span> <span class="s">Deployment</span>
<span class="nn">...</span>
<span class="na">spec</span><span class="pi">:</span>
<span class="nn">...</span>
<span class="na">containers</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">local-volume-node-cleanup-controller</span>
<span class="na">image</span><span class="pi">:</span> <span class="s">gcr.io/k8s-staging-sig-storage/local-volume-node-cleanup:canary</span>
<span class="na">args</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--storageclass-names=fast-disks"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--pvc-deletion-delay=60s"</span>
<span class="pi">-</span> <span class="s2">"</span><span class="s">--stale-pv-discovery-interval=10s"</span>
<span class="na">ports</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">metrics</span>
<span class="na">containerPort</span><span class="pi">:</span> <span class="m">8080</span>
</code></pre></div>
<p>Now apply both deployment and permissions</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">kubectl apply <span class="nt">-f</span> ./deployment.yaml
kubectl apply <span class="nt">-f</span> ./permissions.yaml
</code></pre></div>
<p>You will end up with</p>
<ul>
<li>A new Storage class called 'fast-disks'</li>
<li>The CleanupController looks for Local Persistent Volumes that have a NodeAffinity to a deleted Node. When it finds such a PV, it starts a timer to wait and see if the deleted Node comes back up again. If at the end of the timer, the Node is not back up, the PVC bound to that PV is deleted. The PV is deleted in the next step.</li>
<li>The Deleter looks for Local PVs with a NodeAffinity to deleted Nodes. When it finds such a PV it deletes the PV if (and only if) the PV's status is Available or if its status is Released and it has a Delete reclaim policy.</li>
</ul>
Paul the innovator (85): The goat2023-09-17T11:30:00+00:00https://gatillos.com/yay/2023/09/17/paul-the-innovator-85-the-goat.html
<img src="https://gatillos.com/yay/comics/paul/chap1_85_hi.png"/>
<br/>
<p><span class="multilingual">
<span class="en">A manouvre used since biblical times...
</span>
<span class="es">Maniobra usada desde tiempos bíblicos...
</span>
</span></p>
Diving into service account token authentication in Kubernetes2022-10-02T08:23:10+00:00https://gatillos.com/yay/2022/10/02/blog-how-do-tokens-work-in-Kubernetes.html
<p>I set up a local kubernetes cluster using microk8s
just as a development/home cluster and as part of it I ended up connecting to
the kubernetes dashboard application using a service account token, as <a href="https://microk8s.io/docs/addon-dashboard">recommended</a></p>
<p><a href="/yay/gfx/posts/superkube/001-superkube.png"><img class="" title="Superkube" src="/yay/gfx/posts/superkube/001-superkube.png" alt="A silly drawing of Superkube" style="width:10%"></a></p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>microk8s kubectl create token default
<span class="o">(</span>...token output...<span class="o">)</span>
</code></pre></div>
<p>This process lead me to a few questions:</p>
<ul>
<li>What is that token?</li>
<li>Where does it come from?</li>
<li>Where is it stored?</li>
<li>Does it expire?</li>
</ul>
<p>Essentially, "wait how does the whole token authentication thing
work?"</p>
<h2 id="token-types">Token types</h2>
<p>So far we have dealt with two token types:</p>
<ol>
<li>We have static tokens, like the one we added in the kube config
file. This token is equivalent to a user:password pair. If exposed
it could be used forever to act as that user.
<ul>
<li>These are either stored as secrets or via some other mechanism</li>
<li>In microk8s there are some credentials created during
installation, including the <code>admin</code> user we have been using.
This can be found in /var/snap/microk8s/current under
credentials/known_tokens.csv.
Changing the token can be done by editing the file.</li>
</ul></li>
<li>Then we have temporary tokens, like the one we used for the
dashboard access</li>
</ol>
<p>Now, let's think about those... what is a temporary token? where are
they stored? how do they expire? Those are the kind of questions I like
to dig in to learn about the internals, and I end up finding a lot of
this is implemented in a perfectly reasonable and standards-based way.</p>
<h2 id="inspecting-a-temporary-token">Inspecting a temporary token</h2>
<p>As usual:</p>
<ul>
<li>Kubernetes <a href="https://kubernetes.io/docs/reference/access-authn-authz/authentication/">authentication docs</a>
to the rescue</li>
</ul>
<p>There we see:</p>
<ul>
<li>The created token is a signed JSON Web Token (JWT)</li>
<li>The signed JWT can be used as a bearer token to authenticate as thegiven service account</li>
</ul>
<p>Now, let's take our token and inspect it - pasting it into
<a href="https://jwt.io">jwt.io</a> to see its content (this is safe as long as you
don't expose your cluster anywhere):</p>
<div class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span><span class="w">
</span><span class="nl">"alg"</span><span class="p">:</span><span class="w"> </span><span class="s2">"RS256"</span><span class="p">,</span><span class="w">
</span><span class="nl">"kid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"_h3i-pvtWSsSXXVyXxxxxxxxxxxxxxxx"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"aud"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="s2">"https://kubernetes.default.svc"</span><span class="w">
</span><span class="p">],</span><span class="w">
</span><span class="nl">"exp"</span><span class="p">:</span><span class="w"> </span><span class="mi">1664705605</span><span class="p">,</span><span class="w">
</span><span class="nl">"iat"</span><span class="p">:</span><span class="w"> </span><span class="mi">1664702005</span><span class="p">,</span><span class="w">
</span><span class="nl">"iss"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://kubernetes.default.svc"</span><span class="p">,</span><span class="w">
</span><span class="nl">"kubernetes.io"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"namespace"</span><span class="p">:</span><span class="w"> </span><span class="s2">"default"</span><span class="p">,</span><span class="w">
</span><span class="nl">"serviceaccount"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"default"</span><span class="p">,</span><span class="w">
</span><span class="nl">"uid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"755e1766-e817-xxxx-xxxx-xxxxxxxxxxxx"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="nl">"nbf"</span><span class="p">:</span><span class="w"> </span><span class="mi">1664702005</span><span class="p">,</span><span class="w">
</span><span class="nl">"sub"</span><span class="p">:</span><span class="w"> </span><span class="s2">"system:serviceaccount:default:default"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div>
<ul>
<li>The <code>aud</code> parameter indicates the audience, in this case kubernetes</li>
<li>The <code>sub</code> indicates the 'subject' (user, service account, etc).
When we created the token we didn't specify a service account,
which means it used the 'default' service account (not great
practice but hey we are starting)
<ul>
<li>This means the authentication request is coming from
system:serviceaccount:default:default</li>
<li>You can take a look at this account with kubectl get sa default -o yaml</li>
</ul></li>
<li>The date parameters define the validity period: <code>"nbf": 1664702005</code>
(not before), <code>"iat": 1664702005</code> (issued at) and
<code>"exp": 1664705605</code> (expiry) which you can quickly convert to a date
with:</li>
</ul>
<div class="highlight"><pre><code class="language-shell" data-lang="shell"><span class="nv">$ </span><span class="nb">date</span> <span class="nt">-d</span>@<span class="s2">"1664702005"</span>
Sun 2 Oct 10:13:25 BST 2022
<span class="nv">$ </span><span class="nb">date</span> <span class="nt">-d</span>@<span class="s2">"1664705605"</span>
Sun 2 Oct 11:13:25 BST 2022
</code></pre></div>
<h2 id="how-can-i-use-this-token">How can I use this token?</h2>
<p>Any client application we give the token to will be sending an HTTPS
request to the API server with a header set to
<code>Authentication: Bearer <token></code>. That's all is needed. Any requests
will then be approved/rejected based on the permissions of that account.</p>
<p>If you want to test a particular call, you can use the token with for
example:</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">kubectl <span class="nt">--token</span><span class="o">=</span><span class="s2">"xxxxxxxxxxxxxx"</span> get pods <span class="nt">--all-namespaces</span>
</code></pre></div>
<h2 id="and-finally-where-is-it-stored">And finally, where is it stored?</h2>
<ul>
<li>The token is <strong>not</strong> stored as a secret and it's not directly
linked from the service account object</li>
<li>In fact, the token is <strong>not stored anywhere</strong> in kubernetes, it's
validated from the signature</li>
<li>Secrets don't expire but these tokens do (an hour as seen above in
the JWT token itself)</li>
</ul>
<p>As usual, you will be able to dig into the Dashboard project itself, for
instance <a href="https://github.com/kubernetes/dashboard/blob/master/docs/user/accessing-dashboard/README.md">here you have alternative access methods</a></p>
<p>A final note on temporary tokens: they do provide advantages in limiting
the scope of attack as they will expire but you will still need to
associate them to limited users or service accounts with limited
restrictions, something I haven't gone into yet and we have been using
the default service account. For example, if the temporary account is
captured it can be used to create permanent accounts/secrets.</p>
Play with a local kubernetes cluster using microk8s2022-10-01T08:20:00+00:00https://gatillos.com/yay/2022/10/01/local-kubernetes-with-microk8s.html
<h2 id="but-why">But why?</h2>
<p>Whatever you think about Kubernetes, it is here to stay. At work it's
replaced all our custom orchestration and ECS usage. It solves
infrastructure problems in a common way that most Cloud providers have
agreed to, which is no small feat.</p>
<p>So why would you want to install it locally anyway? For work your
company will most likely provide you access to a cluster and give you a
set of credentials that somewhat limit what you can do. Depending on
your interests that won't be enough to properly learn about what is
going on under the hood. Getting familiar with how Kubernetes stores
containers, container logs, how networking is handled, DNS resolution,
load balancing etc will help remove a huge barrier.</p>
<p>The best thing I can say about Kubernetes is that when you dig into it,
everything makes sense and turns out to be relatively simple. You will see
that most tasks are actually performed by a container, one that has
parameters, logs and a github repo somewhere you can check. Your DNS
resolution is not working? Well then take a look at "kubectl -n
kube-system logs coredns-d489fb88-n7q9p", check the configuration in
"-n kube-system get deployments coredns -o yaml", proxy or
port-forward to it, check the metrics...</p>
<p>In my particular case I also always have a bunch of code I need to run,
scripts to aggregate and publish things, renew my Letsencrypt
certificates, update podcast and RSS services, installed tools like
Grocy and others. Over the years I've tried various solutions to handle
all of this. Currently I have a bunch of scripts running on my main
machine (Linux) that I usually have always running, some of those
spawning containers (to ease dependency management, sometimes it's not
completely under my control) and other tasks are done by my Synology DS.</p>
<p>For me, having kubernetes seems like a logical step even if it's just
to see if this solution will age better over time. I like the
declarative nature of the YAML configuration and the relative isolation
of having Dockerfiles with their own minimal needs defined.</p>
<p>But let's be clear, for most non developers this is a pretty silly
thing to do!</p>
<h2 id="lets-reuse-some-hardware">Let's reuse some hardware</h2>
<p>There are multiple ways to have a local kubernetes cluster. One of the
most popular is minikube but this time we are going to try the
<a href="https://microk8s.io/">Microk8s</a> offer from Canonical (Ubuntu) running
in a spare Linux box (rather than a VM).</p>
<p>Microk8s is a developer-orientated kubernetes distribution. You should
most certainly not use it as your day to day driver but it's a good
option to play with Kubernetes without getting too lost in the actual
setup</p>
<p>Do you have a spare computer somewhere? I have an old Mac Mini I was
using as my main OS X (for iOS builds) and as a secondary backup system.
Now it has been superseded by a much better Intel Mac Mini I kind friend
gave me, leaving me with a machine that is too nice to just shelve but
too cumbersome to maintain as another Mac OS X machine (it won't be
plugged in to a monitor most of the time).</p>
<ul>
<li> I installed a minimal Ubuntu 22.04.1 LTS on it (Server edition)</li>
<li> Cpu according to /proc/cpuinfo is i5-4278U CPU @ 2.60GHz</li>
<li> RAM according to /proc/raminfo is 16GB of RAM</li>
<li> And according to hdparm -I I have two SSDs since some time ago I
replaced the old HDD with a spare SDD: Samsung SSD 860 EVO 500GB and
the original APPLE SSD SM0128F
<ul>
<li> I decided to mount the smaller Apple SSD as ext4 in / and the
bigger 500GB one as zfs in a separate mountpoint called /main.
Mostly because I have tendency to reinstall the whole OS but
like to keep the ZFS partition between installs.</li>
</ul></li>
</ul>
<h2 id="setting-up-microk8s">Setting up microk8s</h2>
<p>Once the Mac Mini linux installation was setup (nothing extra needed to
do there), I enabled SSH and the remaining setup was done from inside
SSH sessions.</p>
<p>Installation didn't start very promising as it failed the first time,
but re-running did finish, following the official installation guide:</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span><span class="nb">sudo </span>snap <span class="nb">install </span>microk8s <span class="nt">--classic</span>
microk8s <span class="o">(</span>1.25/stable<span class="o">)</span> v1.25.2 from Canonical✓ installed
</code></pre></div>
<p>For ease of use I've given my own user access to it with:</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>usermod <span class="nt">-a</span> <span class="nt">-G</span> microk8s osuka
<span class="nb">sudo chown</span> <span class="nt">-f</span> <span class="nt">-R</span> osuka ~/.kube
<span class="c"># exiting the ssh session and logging in again is needed at this point</span>
</code></pre></div>
<p>That you have the correct permissions can be checked by running
<code>microk8s status --wait-ready</code> As per the guide, enable some services:</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>microk8s status
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
ha-cluster <span class="c"># (core) Configure high availability on the current node</span>
helm <span class="c"># (core) Helm - the package manager for Kubernetes</span>
helm3 <span class="c"># (core) Helm 3 - the package manager for Kubernetes</span>
metrics-server <span class="c"># (core) K8s Metrics Server for API access to service metrics</span>
disabled:
cert-manager <span class="c"># (core) Cloud native certificate management</span>
community <span class="c"># (core) The community addons repository</span>
dashboard <span class="c"># (core) The Kubernetes dashboard</span>
...
</code></pre></div>
<p>We activate the following services, by running <code>microk8s enable XXXX</code>
for each one:</p>
<ul>
<li>dashboard: Provides us with a web UI to check the status of the
luster. See <a href="https://github.com/kubernetes/dashboard">more on
github</a></li>
<li>dns: installs coredns which will allow us to have cluster-local DNS
resolution</li>
<li>registry: installs a private container registry where we can push
our images</li>
</ul>
<p>We enable also the "community" services with
<code>microk8s enable community</code> which gives us additional services we can
enable. From the community, we install</p>
<ul>
<li>istio: this creates a service mesh which provides load balancing,
service-to-service authentication and monitoring on top of existing
services, see <a href="https://istio.io/latest/about/service-mesh/">their
docs</a></li>
</ul>
<p>Make <code>microk8s kubectl</code> the default command to run:</p>
<ul>
<li>add <code>alias mkctl="microk8s kubectl"</code></li>
<li>this is just the standard kubectl, just provided via microk8s so we
on't need to install it separately - we can still use kubectl from
other machines</li>
</ul>
<h2 id="lets-use-it-remotely">Let's use it remotely</h2>
<h3 id="access-the-cluster-itself-with-kubectl">Access the cluster itself with kubectl</h3>
<p>The last thing I want is to have yet another computer I have to keep
connecting to a display so this installation will be used remotely. Also
I don't really want to be ssh'ing inside of it just to run kubectl
commands.</p>
<p>From inside the ssh session, check stuff is running:</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>mkctl get pods <span class="nt">--all-namespaces</span>
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system kubernetes-dashboard-74b66d7f9c-6ngts 1/1 Running 0 18m
kube-system dashboard-metrics-scraper-64bcc67c9c-pth8h 1/1 Running 0 18m
kube-system metrics-server-6b6844c455-5t85g 1/1 Running 0 19m
...
</code></pre></div>
<p>There's quite a bit of documentation on how to do most common tasks,
check the <a href="https://microk8s.io/docs/how-to">how to section</a> of the
Microk8s site. To configure access:</p>
<ul>
<li> Expose configuration</li>
</ul>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>microk8s config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRV... <span class="c"># (certificate)</span>
server: https://192.168.x.x:16443 <span class="c"># (ip in local network)</span>
name: microk8s-cluster
contexts:
- context:
cluster: microk8s-cluster
user: admin
name: microk8s
current-context: microk8s
kind: Config
preferences: <span class="o">{}</span>
<span class="nb">users</span>:
- name: admin
user:
token: WDBySG1....... <span class="c"># direct token authentication</span>
</code></pre></div>
<p>At some point we will create users but for initial development tasks all
we need is to concatenate the contents of this configuration into our
~/.kube/config file (or create it if it's the first time using
kubectl).</p>
<p>After doing this, all kubectl commands will work as usual. I've renamed
my context to 'minion':</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash">❯ kubectl config use-context minion
❯ kubectl get pods <span class="nt">--all-namespaces</span>
</code></pre></div>
<p>More details for this specific step <a href="https://microk8s.io/docs/working-with-kubectl">in the official
howto</a>.</p>
<h3 id="access-the-dashboard">Access the dashboard</h3>
<p>Since we haven't enabled any other mechanism, to access the dashboard
you will need a token. At this point you could use the same token you
have in .kube/config, but the recommended way is to create a temporary
token instead:</p>
<div class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>kubectl create token default
... a token is printed
<span class="nv">$ </span>kubectl port-forward <span class="nt">-n</span> kube-system service/kubernetes-dashboard 10443:443
</code></pre></div>
<p>Then connect to <a href="https://localhost:10443">https://localhost:10443</a> and paste the output from the
first command. See <a href="/yay/2022/10/02/blog-how-do-tokens-work-in-Kubernetes.html">the next post</a>
for an explanation of the token used there.</p>
Paul the innovator (84): Defensive programming2022-07-07T07:30:00+00:00https://gatillos.com/yay/2022/07/07/paul-the-innovator-84-defensive-programming.html
<img src="https://gatillos.com/yay/comics/paul/chap1_84_lo.png"/>
<br/>
<p><span class="multilingual">
<span class="en">I used python but this is arguably even worse in Javascript/Typescript. Sorry I mean JavaScript. It turned out the problem wasn't java it was the people (also: are katas a type of defensive programming?)
</span>
<span class="es">Aquí he usado python pero esto pasa igual o más en Javascript/Typescript. Quiero decir JavaScript. Al final resultó que el problema no era java si no la gente (por cierto: son las katas un tipo de programación defensiva?)
</span>
</span></p>
Paul the innovator (83): The A-Team2022-06-19T10:30:00+00:00https://gatillos.com/yay/2022/06/19/paul-the-innovator-83-the-a-team-copy.html
<img src="https://gatillos.com/yay/comics/paul/chap1_83_lo.png"/>
<br/>
<p><span class="multilingual">
<span class="en">Chewing tobacco in a work meeting with your clients, the ultimate power move.
</span>
<span class="es">Mascar tabaco en una reunión de trabajo frente a tus clients, el power move definitivo.
</span>
</span></p>
Paul the innovator (82): Mask mandate2021-06-20T10:30:00+00:00https://gatillos.com/yay/2021/06/20/paul-the-innovator-82-mask-mandate.html
<img src="https://gatillos.com/yay/comics/paul/chap1_82_lo.png"/>
<br/>
<p><span class="multilingual">
<span class="en">No matter what your pet may think of you, wear your mask don't be stupid.
</span>
<span class="es">Independientemente de lo que tu mascota pueda pensar de ti, ponte la máscara no seas estúpido.
</span>
</span></p>
Embedded software / architecture diagrams: Mermaid with Visual Studio Code and Jekyll2020-10-14T11:23:10+00:00https://gatillos.com/yay/2020/10/14/mermaid-diagrams.html
<p>When transmitting complex information I find having some kind of diagram extremely helpful. Sometimes they are there just to have something to look at and point, sometimes they are key to following a complex flow.</p>
<p>Over the years of trying lots of different tools, still seems to me that what I would call 'hand drawn' diagrams are best - but I don't mean actually hand drawn, I'm quite bad at whiteboard drawing but give me a tool like Omnigraffle and I'll make the nicest diagrams. I'll throw in some pastel colors even.</p>
<p>The issue is that integrating that with documentation when most of the documentation is in markdown is getting very annoying: you have to export the diagram as an image, save it somewhere and then insert a reference to it inside the markdown document.</p>
<p>It's pretty much impossible to do a decent version control on those, other than replacing a binary blow by another binary blob.</p>
<p><strong>WARNING:</strong> If you read this post in an RSS reader it's not going to look pretty - diagrams are rendered client-side, you'll need to go to the website.</p>
<h2 id="mermaid">Mermaid</h2>
<p>Many times all I want is to draw a small simple figure to help the narrative or to more effectively highlight some point, something that <em>belongs</em> to the document. Enter <a href="https://mermaid-js.github.io/mermaid/">mermaidjs</a>, that lets you embed diagrams by writing a description of them. Here is a simple example diagram, you will have to enclose it in <strong>three backticks and the word mermaid</strong> (I can't seem to be able to escape that in a way that works in jekyll):</p>
<div class="highlight"><pre><code class="language-text" data-lang="text">sequenceDiagram
Mobile Device->>Server: Request
Server->>Cache: get [x-something]
Cache-->>Server: null || contents
</code></pre></div>
<p>Renders:</p>
<div class="mermaid">
sequenceDiagram
Mobile Device->>Server: Request
Server->>Cache: get [x-something]
Cache-->>Server: null || contents
</div>
<p>The great thing about this is that now I can edit the source file for this post and open a preview in Visual Studio Code (by installing the extension <a href="https://marketplace.visualstudio.com/items?itemName=bierner.markdown-mermaidbierner.markdown-mermaid">Markdown Preview Mermaid Support</a>) that shows me what it would look like.</p>
<p>Here are some more examples, there is a ton of diagram types:</p>
<div class="mermaid">
pie title When to use a pie chart
"Never" : 235
"When lying about your data" : 85
"When hungry" : 15
</div>
<p>I can see myself writing a quick gantt diagram that I can store as part of my project's code, and then export as PDF when needed with the "Markdown PDF: Export (pdf)" command of the <a href="https://marketplace.visualstudio.com/items?itemName=yzane.markdown-pdf">Markdown PDF plugin</a>.</p>
<div class="mermaid">
gantt
title A Gantt Diagram: This is not how things work in reality but you need something
dateFormat YYYY-MM-DD
section Governing
Planning :done, planning, 2013-12-21, 5d
Budget allocation :done, budget, after planning, 5d
section Backend
API definition :active, api-interface, 2014-01-01, 5d
API implementation :api-implementation, after api-interface, 10d
E2E testing :e2e, after api-implementation, 3d
Announcing it's all done and leaving the company: after api-interface, 3d
section Frontend
Implementation :fe-implementation, after api-interface branding, 12d
Implementation extension : fe-again, after fe-implementation we-didnt-think-about-that, 12d
section Design
Branding :active, branding, after budget, 12d
We didn't think about that : we-didnt-think-about-that, after api-implementation, 24d
</div>
<p>You can do pretty interesting things in the gantt charts, multiple dependencies etc</p>
<div class="mermaid">
erDiagram
CUSTOMER ||--o{ ORDER : places
ORDER ||--|{ LINE-ITEM : contains
CUSTOMER }|..|{ DELIVERY-ADDRESS : uses
</div>
<p>You can have class diagrams</p>
<div class="highlight"><pre><code class="language-text" data-lang="text">classDiagram
Animal <|-- Duck
Animal <|-- Fish
Animal <|-- Zebra
class Duck{
+String beakColor
+swim()
+quack()
}
...
</code></pre></div>
<div class="mermaid">
classDiagram
Animal <|-- Duck
Animal <|-- Fish
Animal <|-- Zebra
Animal : +int age
Animal : +String gender
Animal: +isMammal()
Animal: +mate()
class Duck{
+String beakColor
+swim()
+quack()
}
class Fish{
-int sizeInFeet
-canEat()
}
class Zebra{
+bool is_wild
+run()
}
</div>
<p>You can have nice state diagrams</p>
<div class="highlight"><pre><code class="language-text" data-lang="text">stateDiagram
[*] --> State1
State3 --> State4
State1 --> State2
State3 --> [*]
State2 --> State4
State2 --> State3: Conked
</code></pre></div>
<p>or with the newer plugin:</p>
<div class="highlight"><pre><code class="language-text" data-lang="text">stateDiagram-v2
[*] --> State1
state fork_state <<fork>>
[*] --> fork_state
fork_state --> State1
fork_state --> State3
...
</code></pre></div>
<div class="mermaid">
stateDiagram-v2
[*] --> State1
state fork_state <<fork>>
[*] --> fork_state
fork_state --> State1
fork_state --> State3
State3 --> State4
State1: The state with a note
note right of State1
Important information! You can write
notes.
end note
State1 --> State2
note left of State2 : This is the note to the left.
State2 --> Active
state Active {
[*] --> NumLockOff
NumLockOff --> NumLockOn : EvNumLockPressed
NumLockOn --> NumLockOff : EvNumLockPressed
--
[*] --> ScrollLockOff
ScrollLockOff --> ScrollLockOn : EvCapsLockPressed
ScrollLockOn --> ScrollLockOff : EvCapsLockPressed
}
state join_state <<join>>
State4 --> join_state
Active --> join_state
join_state --> [*]
</div>
<div class="mermaid">
stateDiagram
[*] --> State1
State3 --> State4
State1 --> State2
State3 --> [*]
State2 --> State4
State2 --> State3: Conked
</div>
<p>Entity relationship diagrams:</p>
<div class="mermaid">
graph LR
A[Hard edge] -->|Link text| B(Round edge)
B --> C{Decision}
C -->|One| D[Result one]
C -->|Two| E[Result two]
</div>
<h2 id="caveats">Caveats</h2>
<p>One thing to note is the different diagram plugins are almost independent applications and have subtle syntax differences: for instance you must use quotes in pie charts, but not in gantt.</p>
<p>The "Markdown: Open preview to the side" command on visual studio code right now works but has a very annoying refresh effect on the document you are typing in making it mostly unusable. You will be better of just opening previews that refresh only when you focus them using the "Markdown: Open Preview" command.</p>
<h2 id="integration-with-jekyll">Integration with Jekyll</h2>
<p>There's no real integration with Jekyll, you just add the library to your layout and then embed a <code><div></code> which automatically gets expanded client-side by the library:</p>
<div class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><div</span> <span class="na">class=</span><span class="s">"mermaid"</span><span class="nt">></span>
sequenceDiagram
Mobile Device->>Server: Request
Server->>Cache: get [x-something]
Cache-->>Server: null || contents
<span class="nt"></div></span>
</code></pre></div>
<p>To add the library (quite a hefty library at 800K for the minified version, but for a site like this it's acceptable):</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nb">mkdir </span>lib
<span class="nb">cd </span>lib
curl <span class="nt">-O</span> https://raw.githubusercontent.com/mermaid-js/mermaid/develop/dist/mermaid.min.js
</code></pre></div>
<p>To add it to your layout, edit for instance <code>_layouts/default.html</code>:</p>
<div class="highlight"><pre><code class="language-html" data-lang="html">...
<span class="nt"><body></span>
...
<span class="nt"><script </span><span class="na">src=</span><span class="s">"/yay/lib/mermaid.min.js"</span><span class="nt">></script></span>
<span class="nt"></body></span>
</code></pre></div>
<h2 id="customization">Customization</h2>
<p>One of the greatest things about mermaid is that you can tweak most of it using just CSS <em>and</em> you can add event listeners to make the diagrams more interactive. Check the excellent <a href="https://mermaid-js.github.io/mermaid/diagrams-and-syntax-and-examples/">documentation</a></p>
Wacom on Linux: my 2020 setup with aspect ratio fix, pan to scroll, drag, multitouch and more2020-08-27T12:23:10+00:00https://gatillos.com/yay/2020/08/27/wacom-on-linux-setup.html
<p>I've been a very big fan of small Wacom tablets as a replacement for a mouse. I'll go through my current setup for Linux in case it can help anyone out there.</p>
<p>I started using a Wacom tablet for drawing but found it an excellent replacement for a mouse and moved to exclusive use when I started having symptons of RSI. The change from having the wrist on top of a mouse to having it gripping a pen made a big difference and my problems went away.</p>
<p>When I started using macbooks this became almost a non issue since the trackpad (on OS X) removes a lot of the stress points, in particular if you enable tap to click, tap with two fingers to right click and three finger drag.</p>
<p>Now I find myself on desktop PCs running Linux most of the time so I went back to using it as my main device, tweaking it to my liking. I have compiled my tweaks and hacks here, currently all is tested on a relatively old Wacom Intuos pen & touch small tablet (CTH-480).</p>
<p>Oh and kudos to Jason Gerecke the maintainer of the linuxwacom driver, such an essential but slightly thankless job (and Wacom for supporting him).</p>
<h2 id="the-basics">The basics</h2>
<p>Luckily most distros these days detect Wacom tablets without any hassle and provide even a tool that will help you do some simple mappings.</p>
<p>First of all you'll need to find the ID of the device you want to modify, and you'll need to have xsetwacom installed (part of xserver-xorg-input-wacom, normally installed by default).</p>
<p>In my case:</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">xsetwacom <span class="nt">--list</span> devices
Wacom Intuos PT S Finger <span class="nb">touch id</span>: 13 <span class="nb">type</span>: TOUCH
Wacom Intuos PT S Pad pad <span class="nb">id</span>: 14 <span class="nb">type</span>: PAD
Wacom Intuos PT S Pen eraser <span class="nb">id</span>: 17 <span class="nb">type</span>: ERASER
Wacom Intuos PT S Pen stylus <span class="nb">id</span>: 16 <span class="nb">type</span>: STYLUS
</code></pre></div>
<p>The following commands can use the name of the device or the id number, I'll use the name to make it easier to read.</p>
<h2 id="absolute-positioning">Absolute positioning</h2>
<p>This is a must, trying to use the pen as if it was a trackpad just doesn't function in an usable way. What you want is to map a point on the tablet to a point on the screen. This allows you to fly from one corner of the screen to the other faster than a trackpad could do.</p>
<p>For almost all distros you can switch this on via a UI, but if you don't have it, you can do it manually, for my case:</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen stylus"</span> Mode Absolute
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen eraser"</span> Mode Absolute
</code></pre></div>
<h2 id="aspect-ratio">Aspect ratio</h2>
<p>There is one subtle but very big problem I have noticed as the aspect ratio of my monitors have diverged from the aspect ratio of the tablet over time.</p>
<p>If your tablet proportions don't match the proportions of the screen it will be impossible to draw properly, this is painfully evident on an Ultrawide screen: you'll draw a circle but it will show as an oval. The brain is able to more or less adjust to that and you may be able to manage it but it's a very frustrating experience.</p>
<p>For my case I have written a small script that will calculate the aspect ratio of the screen and map the tablet to it.</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="c"># if you have more than one monitor you can specify which one to use by</span>
<span class="c"># adding a --display option to the xrandr call</span>
<span class="nv">s_w</span><span class="o">=</span><span class="sb">`</span>xrandr | <span class="nb">grep</span> <span class="nt">-w</span> connected | <span class="nb">awk</span> <span class="nt">-F</span><span class="s1">'[ \+]'</span> <span class="s1">'{print $4}'</span> | <span class="nb">sed</span> <span class="s1">'s/x/ /'</span> | <span class="nb">awk</span> <span class="s1">'{print $1}'</span><span class="sb">`</span>
<span class="nv">s_h</span><span class="o">=</span><span class="sb">`</span>xrandr | <span class="nb">grep</span> <span class="nt">-w</span> connected | <span class="nb">awk</span> <span class="nt">-F</span><span class="s1">'[ \+]'</span> <span class="s1">'{print $4}'</span> | <span class="nb">sed</span> <span class="s1">'s/x/ /'</span> | <span class="nb">awk</span> <span class="s1">'{print $2}'</span><span class="sb">`</span>
<span class="nv">ratio</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span> <span class="s2">"</span><span class="k">${</span><span class="nv">s_h</span><span class="k">}</span><span class="s2">/</span><span class="k">${</span><span class="nv">s_w</span><span class="k">}</span><span class="s2">"</span> | bc <span class="nt">-l</span><span class="sb">`</span>
<span class="nb">echo</span> <span class="s2">"Screen detected: </span><span class="k">${</span><span class="nv">s_w</span><span class="k">}</span><span class="s2">x</span><span class="k">${</span><span class="nv">s_h</span><span class="k">}</span><span class="s2">"</span>
<span class="k">for </span>device <span class="k">in</span> <span class="s1">'Wacom Intuos PT S Pen stylus'</span> <span class="s1">'Wacom Intuos PT S Pen eraser'</span> <span class="p">;</span> <span class="k">do
</span><span class="nb">echo</span> <span class="s2">""</span>
<span class="nb">echo</span> <span class="s2">"</span><span class="k">${</span><span class="nv">device</span><span class="k">}</span><span class="s2">:"</span>
xsetwacom <span class="nb">set</span> <span class="s2">"</span><span class="nv">$device</span><span class="s2">"</span> ResetArea
<span class="nv">area</span><span class="o">=</span><span class="sb">`</span>xsetwacom get <span class="s2">"</span><span class="nv">$device</span><span class="s2">"</span> area | <span class="nb">awk</span> <span class="s1">'{print $3,$4}'</span><span class="sb">`</span>
<span class="nv">w</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span> <span class="k">${</span><span class="nv">area</span><span class="k">}</span> | <span class="nb">awk</span> <span class="s1">'{print $1}'</span><span class="sb">`</span>
<span class="nv">h</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span> <span class="k">${</span><span class="nv">area</span><span class="k">}</span> | <span class="nb">awk</span> <span class="s1">'{print $2}'</span><span class="sb">`</span>
<span class="nv">hn</span><span class="o">=</span><span class="sb">`</span><span class="nb">echo</span> <span class="s2">"</span><span class="k">${</span><span class="nv">w</span><span class="k">}</span><span class="s2">*</span><span class="k">${</span><span class="nv">ratio</span><span class="k">}</span><span class="s2">"</span> | bc <span class="nt">-l</span><span class="sb">`</span>
<span class="nv">hn</span><span class="o">=</span><span class="sb">`</span><span class="nb">printf</span> %.0f <span class="s2">"</span><span class="k">${</span><span class="nv">hn</span><span class="k">}</span><span class="s2">"</span><span class="sb">`</span>
<span class="nb">echo</span> <span class="s2">" Area </span><span class="k">${</span><span class="nv">w</span><span class="k">}</span><span class="s2">x</span><span class="k">${</span><span class="nv">h</span><span class="k">}</span><span class="s2"> ==> </span><span class="k">${</span><span class="nv">w</span><span class="k">}</span><span class="s2">x</span><span class="k">${</span><span class="nv">hn</span><span class="k">}</span><span class="s2">"</span>
xsetwacom <span class="nb">set</span> <span class="s2">"</span><span class="nv">$device</span><span class="s2">"</span> area 0 0 <span class="k">${</span><span class="nv">w</span><span class="k">}</span> <span class="k">${</span><span class="nv">hn</span><span class="k">}</span>
<span class="k">done</span>
</code></pre></div>
<h2 id="fix-the-jitter">Fix the jitter</h2>
<p>My hand is not particularly steady and I find that if I try to keep it still I will get tiny movements (jitter). These two parameters make it less sensitive. You can play with the values. This seems to work for my Intuos S.</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen stylus"</span> Suppress 10
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen stylus"</span> RawSample 9
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen eraser"</span> RawSample 9
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen eraser"</span> Suppress 10
</code></pre></div>
<h2 id="mapping-of-the-tablet-buttons">Mapping of the tablet buttons</h2>
<p>There are some buttons on the tablet. I find the most useful thing to do is to map them to mouse buttons so that I can click more precisely. I find hovering over a selection and using the other hand to press the buttons makes me more precise. You can also map them to any key or combination of keys.</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="c"># the pad of my tablet has 4 buttons</span>
<span class="c"># the mapping is a bit weird because Xinput reserves some buttons</span>
<span class="c"># https://github.com/linuxwacom/xf86-input-wacom/wiki/Tablet-Configuration-1%3A-xsetwacom-and-xorg.conf</span>
<span class="c">#</span>
<span class="c"># [ button 3 ] [ button 9 ]</span>
<span class="c"># [ button 1 ] [ button 8 ]</span>
<span class="c">#</span>
xsetwacom <span class="nb">set</span> <span class="s2">"Wacom Intuos PT S Pad pad"</span> Button 9 <span class="s2">"key +ctrl z -ctrl"</span> <span class="c"># undo</span>
xsetwacom <span class="nb">set</span> <span class="s2">"Wacom Intuos PT S Pad pad"</span> Button 8 2 <span class="c"># middle button</span>
xsetwacom <span class="nb">set</span> <span class="s2">"Wacom Intuos PT S Pad pad"</span> button 1 1 <span class="c"># mouse click</span>
xsetwacom <span class="nb">set</span> <span class="s2">"Wacom Intuos PT S Pad pad"</span> button 3 2 <span class="c"># middle button</span>
</code></pre></div>
<p>You can use trial and error to find which buttons correspond to which numbers. Note the mapping for the 'undo' functionality, it's equivalent to saying "press control, while holding it press z, then release control"</p>
<h2 id="mapping-of-pen-buttons">Mapping of pen buttons</h2>
<p>You can map the buttons on the pen itself to various functions.</p>
<p>For years one feature I missed was the ability to scroll just using the pen. The way this works is I press on the button on the side of the pen (which is very ergonomic) and then press with the pen on the tablet and move up and down. Works like a charm as a relaxed way of scrolling. This is called "pan scroll", I have seen it called "drag to scroll" too and has worked in Linux for a couple of years now and it's implemented as a button mapping.</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="c"># pan scroll (press lower button + click and drag to scroll)</span>
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen stylus"</span> button 2 <span class="s2">"pan"</span>
xsetwacom <span class="nt">--set</span> <span class="s2">"Wacom Intuos PT S Pen stylus"</span> <span class="s2">"PanScrollThreshold"</span> 200
</code></pre></div>
<p>I leave the other buttons as they are, which is the tip of the button acts as a 'left click' and the second button on the side acts as a right click. This particular pen can also be used as an eraser - so you can change the eraser mapping too. Defaults work fine for me.</p>
<h2 id="touch-and-multi-touch-fusuma">Touch and multi-touch (fusuma)</h2>
<p>Without doing anything you get functional trackpad functionality on Wacom touch models. For mine you get standard movement, tap to click, tap with two fingers to right click and what I can best describe as "tap + release + tap again and move" to select or drag, plus scrolling by using two fingers up and down the tablet. Some distributions enable also going forwards and backwards in browsers by sliding both fingers left or right, while the most common case seems to enable scrolling left and right.</p>
<p>There's a few multi touch frameworks you can use to extend what's possible. I'm currently using <a href="https://github.com/iberianpig/fusuma">fusuma</a>, I simply followed the instructions to set it up and then launch it as part of my window manager startup. It can do a lot more than what I need so my configuration is very simple:</p>
<div class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nv">$ </span><span class="nb">cat</span> ~/.config/fusuma/config.yml
swipe:
3:
left:
sendkey: <span class="s2">"LEFTALT+RIGHT"</span> <span class="c"># History forward</span>
right:
sendkey: <span class="s2">"LEFTALT+LEFT"</span> <span class="c"># History back</span>
up:
sendkey: <span class="s2">"LEFTMETA"</span> <span class="c"># Activity</span>
4:
left:
sendkey: <span class="s2">"LEFTALT+LEFTMETA+LEFT"</span>
right:
sendkey: <span class="s2">"LEFTALT+LEFTMETA+RIGHT"</span>
pinch:
<span class="k">in</span>:
<span class="nb">command</span>: <span class="s2">"xdotool keydown ctrl click 4 keyup ctrl"</span> <span class="c"># Zoom in</span>
out:
<span class="nb">command</span>: <span class="s2">"xdotool keydown ctrl click 5 keyup ctrl"</span> <span class="c"># Zoom out</span>
</code></pre></div>
<p>Basically I use four fingers to move between workspaces (I have mapped those keys in my window manager to switch workspaces) and three fingers to go back/forwards in browsers. I have pinch configured but it's not very practical to be honest.</p>
<p>Here you can really play with whatever fits best with your preferences and window manager. Note that you can issue extended complicated commands via <a href="https://www.linux.org/threads/xdotool-keyboard.10528/">xdotool</a> in a similar way to what linuxwacom can already do by itself.</p>
<p>That's it. This is what works for me so far, feel free to ping me if you get stuck setting up something.</p>
Working from home: Pandemia Edition2020-04-02T18:00:00+00:00https://gatillos.com/yay/2020/04/02/paul-the-weekly-developers-call.html
<img src="https://gatillos.com/yay/comics/other/2020-04-02-paul-the-weekly-developers-call.png"/>
<br/>
<p>For some, things haven't changed that much.</p>