k8s minikube rpc error
March 08, 2020
I started this udemy course on Kubernetes/Microservices to get some hands on experience on k8s and devops in general. My company uses k8s to orchestrate our miroservice containers in AWS - similar to what is being offered in this course. I installed minikube to begin local kubernetes deployment. I also have docker on my macOS so I thought I could get away without installing a specific hypervisor but needed to install Virtualbox anyway.
workloads.yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: position-simulator
spec:
selector:
matchLabels:
app: position-simulator
replicas: 1
template:
metadata:
labels:
app: position-simulator
spec:
containers:
- name: queue
image: richardchesterwood/k8s-fleetman-position-simulator:release1
env:
- name: SPRING_PROFILES_ACTIVE
value: production-microservice
I executed kubectl apply -f workloads.yaml
but I got an error
Failed to pull image "richardchesterwood/k8s-fleetman-position-simulator:release1":
rpc error: code = Unknown desc = Error response from daemon:
Get https://registry-1.docker.io/v2/:
dial tcp: lookup registry-1.docker.io on 192.168.64.1:53:
read udp 192.168.64.3:60508->192.168.64.1:53: read: connection refused
Turns out my minikube’s DNS resolver was not able to lookup docker’s registry. I thought at first that the solution is pretty straightforward. Add a nameserver for resolution in /etc/resolv.conf
Step 1.
minikube ssh
Step 2.
sudo vi /etc/resolv.conf
Step 3.
Add nameserver 8.8.8.8
Save & Exit.
However, there is a warning at the top of the file.
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
So what was my next option? I could ignore the warning and get the docker images in the same session.
However, the next time I run kubectl apply
my changes to resolve.conf would be reverted.
That’s when I had to retrace my steps. I realized this was probably the driver issue.
You see, when I deleted my local k8s instance to increase the RAM, I forgot to specify the vm-driver.
So I did minikube delete
followed by minikube start --vm-driver=virtualbox
.
After this, the docker images were downloaded correctly.