Issues while adding EKS Kubernetes Cluster

Harness Support
1

I am trying to setup Harness with our EKS cluster. I used the Kubernetes YAML for delegate setup. Delegate setup was successful with pod up and running but when I am trying to add our EKS cluster, I am seeing the following exception.

Invalid request: KubernetesClientException: Operation: [list] for kind: [Deployment] with name: [null] in namespace: [default] failed.

I see the following exception in delegate pod logs.

io.fabric8.kubernetes.client.KubernetesClientException: Operation: [list] for kind: [Deployment] with name: [null] in namespace: [default] failed.
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:71)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:588)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:49)
at software.wings.cloudprovider.gke.KubernetesContainerServiceImpl.listControllers(KubernetesContainerServiceImpl.java:517)
at software.wings.cloudprovider.gke.KubernetesContainerServiceImpl.validate(KubernetesContainerServiceImpl.java:373)
at software.wings.service.impl.ContainerServiceImpl.validate(ContainerServiceImpl.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.joor.Reflect.on(Reflect.java:677)
at org.joor.Reflect.call(Reflect.java:379)
at software.wings.delegatetasks.ServiceImplDelegateTask.run(ServiceImplDelegateTask.java:57)
at software.wings.delegatetasks.ServiceImplDelegateTask.run(ServiceImplDelegateTask.java:25)
at software.wings.delegatetasks.AbstractDelegateRunnableTask.Harness(AbstractDelegateRunnableTask.java:97)
at software.wings.delegatetasks.AbstractDelegateRunnableTask.run(AbstractDelegateRunnableTask.java:72)
at io.harness.manage.GlobalContextTaskWrapper.run(GlobalContextTaskWrapper.java:17)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:267)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:237)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:148)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:186)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at software.wings.service.impl.KubernetesHelperService$2.intercept(KubernetesHelperService.java:293)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:179)
at okhttp3.RealCall.execute(RealCall.java:63)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:328)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:584)
… 20 common frames omitted

I looked into the YAML and setup logs, I see the step clusterrolebinding.rbac.authorization.k8s.io/harness-delegate-cluster-admin was succesful.

DO I need to look into in any other configuration or add any additional privileges to delegate pod to resolve this issue ?

2

Anil,

Great meeting you in class yesterday! EKS can be a picky one, the Harness Delegate should have enough privileges to run KubeCTL.

For the class we were in we used EKS as our example cluster. Though to add additional ARNs to have access to the cluster, typically I followed this:

A cheat that we used [prob not the best for an enterprise] is leverage an Service Account Token along with the CA.

583Ă—832 49.5 KB

Quick way to get the token in EKS is how the K8s dashboard is deployed:

Hope that helps!

-Ravi

1 Like
3

Hi Ravi,

Thanks for the quick response. It was nice meeting you in the class.
After the adding cluster details with token manually it worked.

Thanks for the quick help !!

-Anil

4

@ammula

You shouldn’t need to add the certs manually from the cluster into Harness. If you add the delegate directly into the cluster, you should be able to select “Inherit Cluster Details from selected Delegate” and Harness will assume the CR and CRB from the delegate in the cluster. However, if the delegate does not live in the cluster, the cluster certs will be required.

5

I tried the “Inherit Cluster Details from selected Delegate” option first time with the delegator deployed as part of the cluster but I got the error which I mentioned in the above post.

6

Did you change any of the YAML for the Delegate before you installed it in the cluster?

7

I only added PROXY details, everything else unchanged.