Posts

kubeflow Istio configuration for trustworthy JWTs on rancher 2.x

Introduction: For some reason some of the default feature gates are not turned on in rancher.  So deploying Kubeflow or any workload that uses Istio version 1.3.1 with SDS enabled you need to enable  TokenRequest and  TokenRequestProjection. Issue symptoms: istio-pilot and everything dependent will fail to start in Kubeflow deployment. pod events / log similar to "MountVolume.SetUp failed for volume "istio-token" : failed to fetch token: the API server does not have TokenRequest endpoints enabled" How to prepare Rancher for Istio 1.3 and up (tested on 2.x) Option 1, use server configuration file (yaml edit) Login to your Rancher2.0 UI Select relevant cluster Click on options and edit On cluster options choose "Cluster Options" and edit ad YAML go to: "kube-api:" and add : extra_args:        service-account-issuer: "kubernetes.default.svc"        service-account-signing-key-file: "/etc/kubernetes/ssl/kube-service-account-token-key.p

mounting AWS (Amazon Web Services) EFS on Linux Ubuntu 18.04

Amazon Elastic File System (Amazon EFS) is a scalable file storage for EC2 and services that run on EC2 (for example Kubernetes clusters). The device is accessible on Linux via the NFS protocol and can be used my multiple instances and pods at the same time. For more information on EFS visit AWS documentation. Step one: Gather information In our case ti is pretty straightforward. Ubuntu instance in the same VPC as the EFS and a DNS name of the file system we want to access. The format uses following convention: http://file-system-id.efs.aws-region.amazonaws.com And the exact URL is available on AWS console AWS home under filesystem's DNS name or via cli Step two: Install the NFS Client for Linux $  sudo apt-get update $  sudo apt install nfs-kernel-server Step three: Mount the file system on EC2 instance. Create (if you don't have already) a mount point for the EFS $  sudo mkdir -p /mnt/efs-mount-point Mount the EFS share on the instance $  sudo mount -t

How to Install Terraform 0.12 on Ubuntu 18.04

To this day (8/8/2019) Terraform is not packaged in an official apt repository. There is an option to install it with Snap but be careful it will probably be an older version. When i checked it was v0.11.11 If you do want to install it with snap, run: $   snap install terraform To install the latest version follow this procedure. You might want to update your system just in case: $  sudo apt-get update Now since you are getting a Terraform binary from official Hashicorp site, you will need both wget and unzip packages unless already installed: $  sudo apt-get install wget unzip Last step would be to download an unzip Terraform package ( you can find latest here ). $  wget https://releases.hashicorp.com/terraform/0.12.6/terraform_0.12.6_linux_amd64.zip $  sudo unzip ./terraform_0.12.6_linux_amd64.zip -d /usr/local/bin/ check that it is installed: $ terraform -v you are all done. Provided by: Forthscale systems, cloud experts

Getting AWS EC2 instance id (instanceid) from within the ec2 instance

In general you can get a lot of instance metadata by accessing API on http://169.254.169.254/latest/meta-data/ That includes instance id. On generic Linux system, you can get the ID either using curl: curl http://169.254.169.254/latest/meta-data/instance-id or wget: wget -q -O - http://169.254.169.254/latest/meta-data/instance-id If you instance is based on Amazon Linux or have cloud-utils installed you can also run: ec2-metadata -i for instance id. more documentation on metadata is a available here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html Provided by: Forthscale systems, cloud experts

HTTP 409 while provisioning Google Cloud SQL instance

While creating a new Google Cloud SQL be careful not to use instance name (master or replica) that was recently used. How recent? Up to two months . errors you might encounter: ERROR: (gcloud.sql.instances.create) Resource in project [Project name] is the subject of a conflict: The instance or operation is not in an appropriate state to handle the request. HTTP 409 Provided by: Forthscale systems, cloud experts

DevOps and Site Reliability Engineering (SRE)

Image
As we all know, the Computer Age and the Internet Age have both profoundly impacted the world of commerce. As customer experience changes, led by internet giants, IT operations change accordingly to support new processes. Not so long ago, new product development could mostly be decoupled from operations. Of course, there were some connections, factories had to retool their machinery if changes were made. Yet the nature of physical products allowed for development operations to drift apart. With the explosion of cyber property in the last few decades, though, the product mix has changed. Digital products represent a large and growing part of global offerings. An expectation from such a product is to be always-reliable, accessible from anywhere by anyone at any time. Recent offerings from major cloud providers advertise simplicity in supporting this notion. In reality, everything is still technically grounded (servers need to physically be somewhere). To meet market expectations

Petya / NotPetya

Image
Petya / NotPetya Just last month the WannaCry ransom-ware spread to hundreds of thousands of machines and set off a global panic. The worm-style infection relied on a leaked NSA tool (EternalBlue) that allowed it to spread rapidly across the Internet. Microsoft released a patch shortly after the attack began, even supporting systems that had long been past their patch lifetimes (Windows XP, anyone?). A mere month later, the NotPetya malware burst onto the scene. Petya has been around since early 2016, and this outbreak is not actually Petya. However, it shares many similarities, hence the preliminary label as “Petya” and subsequently “NotPetya”. The attack bears resemblance to WannaCry in that it exploits EternalBlue, which, unfortunately, has not been patched on many systems because companies and individuals have decided uptime is more important. They effectively gambled with their data, and some of them have lost. This ransomware hasn’t spread like WannaCry, but it also us