2026. What I Actually Do Now

It’s been over 20 years of on-and-off dumping into this blog.
27 and going years in tech, more than 10 years around AI, going back to Heili in 2014.

In 2026, my focus is different.

I work on:

• independent AI systems
• integrating LLMs into Dev and Ops workflows
• observability, operations, and “code factory” concepts

This blog will reflect that shift.

Some posts will be LLM-polished.
I care more about signal than stylistics.

solving error: Your current user or role does not have access to Kubernetes objects on this EKS cluster.

Trying to access EKS cluster with kubectl you might get an error similar to:

Your current user or role does not have access to Kubernetes objects on this EKS cluster

This may be due to the current user or role not having Kubernetes RBAC permissions to describe cluster resources or not having an entry in the cluster’s auth config map

it can happen on for example terraform created clusters or a new user joing organization.

so what happened was that EKS being amazon product by default relies on amazon security structure for RBAC and the role you currently use was not set to access it.

you can see the idenity mappings on your cluster with:

eksctl get iamidentitymapping --cluster YOUR_CLUSTER --region=YOUR_REGION

and you can add needed role using eksctl (no need for kubectl since those are rules beforehand)

eksctl create iamidentitymapping \

 --cluster YOUR_CLUSTER\

 ---region=YOUR_REGION\

 --arn arn:aws:iam::123456:role/YOUR_ROLE\

 --username admin \

 --group system:masters

and of you can delete the roles you no longer use with:

eksctl delete iamidentitymapping\

 --cluster YOUR_CLUSTER\

 --region=YOUR_REGION\

 --arn arn:aws:iam::123456:role/YOUR_ROLE

kubeflow Istio configuration for trustworthy JWTs on rancher 2.x

Introduction:

For some reason some of the default feature gates are not turned on in rancher. 

So deploying Kubeflow or any workload that uses Istio version 1.3.1 with SDS enabled you need to enable TokenRequest and TokenRequestProjection.

Issue symptoms:

1. istio-pilot and everything dependent will fail to start in Kubeflow deployment.
2. pod events / log similar to "MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: the API server does not have TokenRequest endpoints enabled"

How to prepare Rancher for Istio 1.3 and up (tested on 2.x)

Option 1, use server configuration file (yaml edit)

1. Login to your Rancher2.0 UI
2. Select relevant cluster
3. Click on options and edit
4. On cluster options choose "Cluster Options" and edit ad YAML
5. go to: "kube-api:"
   and add :
   extra_args:
          service-account-issuer: "kubernetes.default.svc"
          service-account-signing-key-file: "/etc/kubernetes/ssl/kube-service-account-token-key.pem"
6. Save the file / configuration

Cluster will reconfigure. 

Option 2, feature gates flags via Rancher API

Follow the instructions in this thread.

references:

Kubernetes documentation

Istio blog

Rancher git issue

mounting AWS (Amazon Web Services) EFS on Linux Ubuntu 18.04

Amazon Elastic File System (Amazon EFS) is a scalable file storage for EC2 and services that run on EC2 (for example Kubernetes clusters). The device is accessible on Linux via the NFS protocol and can be used my multiple instances and pods at the same time.
For more information on EFS visit AWS documentation.


Step one: Gather information
In our case ti is pretty straightforward. Ubuntu instance in the same VPC as the EFS and a DNS name of the file system we want to access. The format uses following convention:

http://file-system-id.efs.aws-region.amazonaws.com

And the exact URL is available on AWS console AWS home under filesystem's DNS name or via cli

Step two: Install the NFS Client for Linux

$ sudo apt-get update
$ sudo apt install nfs-kernel-server

Step three: Mount the file system on EC2 instance.
Create (if you don't have already) a mount point for the EFS

$ sudo mkdir -p /mnt/efs-mount-point

Mount the EFS share on the instance

$ sudo mount -t nfs -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport mount-target-DNS:/   /mnt/efs-mount-point

Now we have a mounted Amazon EFS file system on Ubuntu EC2 instance.
Keep in mind that command mounted doesn't persist across reboots. if you want it to be permanently accessible you have to add it to the fstab.


Common error:

efs mount.nfs: Connection timed out

This error can occur because either the Amazon EC2, mount target security groups or file system access are not configured properly.

For more troubleshooting tips you can visit:
https://docs.aws.amazon.com/efs/latest/ug/troubleshooting-efs-mounting.html

How to Install Terraform 0.12 on Ubuntu 18.04

To this day (8/8/2019) Terraform is not packaged in an official apt repository. There is an option to install it with Snap but be careful it will probably be an older version. When i checked it was v0.11.11
If you do want to install it with snap, run:

$  snap install terraform

To install the latest version follow this procedure.

You might want to update your system just in case:

$ sudo apt-get update

Now since you are getting a Terraform binary from official Hashicorp site, you will need both wget and unzip packages unless already installed:

$ sudo apt-get install wget unzip

Last step would be to download an unzip Terraform package (you can find latest here).

$ wget https://releases.hashicorp.com/terraform/0.12.6/terraform_0.12.6_linux_amd64.zip
$ sudo unzip ./terraform_0.12.6_linux_amd64.zip -d /usr/local/bin/

check that it is installed:

$ terraform -v

you are all done.

Provided by:Forthscale systems, cloud experts

Getting AWS EC2 instance id (instanceid) from within the ec2 instance

In general you can get a lot of instance metadata by accessing API on
http://169.254.169.254/latest/meta-data/
That includes instance id.

On generic Linux system, you can get the ID either using curl:
curl http://169.254.169.254/latest/meta-data/instance-id
or wget:
wget -q -O - http://169.254.169.254/latest/meta-data/instance-id

If you instance is based on Amazon Linux or have cloud-utils installed you can also run:
ec2-metadata -i
for instance id.

more documentation on metadata is a available here:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Provided by:Forthscale systems, cloud experts

HTTP 409 while provisioning Google Cloud SQL instance

While creating a new Google Cloud SQL be careful not to use instance name (master or replica) that was recently used. How recent? Up to two months.

errors you might encounter:
ERROR: (gcloud.sql.instances.create) Resource in project [Project name] is the subject of a conflict: The instance or operation is not in an appropriate state to handle the request.
HTTP 409

Provided by:Forthscale systems, cloud experts