docs: add tutorial for running minikube on AWS EC2#23017
Conversation
k8s-ci-robot
added
the
do-not-merge/invalid-commit-message
|
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: wasimat404 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Welcome @wasimat404! |
k8s-ci-robot
added
size/L
|
Hi @wasimat404. Thanks for your PR. I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Can one of the admins verify this patch? |
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
95a395d to
77f49af
Compare
k8s-ci-robot
removed
the
do-not-merge/invalid-commit-message
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
77f49af to
4363a18
Compare
k8s-ci-robot
added
cncf-cla: yes
|
/easycla |
nirs
left a comment
nirs
left a comment
There was a problem hiding this comment.
Added few comment of the first part, did not read the rest.
Not clear to me what is the use case for running minikube on an expensive remote vm when you can run it on any laptop these days.
It may help to mention typical use cases.
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
2 times, most recently
from
85f91e2 to
d9aad3e
Compare
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
d9aad3e to
b7d2cec
Compare
nirs
left a comment
nirs
left a comment
There was a problem hiding this comment.
Read the entire document now. Looks nice and useful!
It will help to add some images and screenshots to break the dry wall of text.
| - **Your local machine can't handle it.** Beginners on 4–8 GB laptops, or anyone already running Docker, an IDE, and browser tabs, often can't fit a minikube cluster on top. Offloading to a `t3.medium` keeps the laptop usable. | ||
| - **Corporate-managed laptops with restricted permissions.** Some work laptops are locked down by IT — you can't install Docker, KVM, or other minikube drivers. An EC2 sandbox sidesteps this entirely. | ||
| - **You don't want minikube draining battery and heating your laptop.** A local cluster runs 24/7 if you forget to stop it. Offloading the always-on workload to EC2 keeps your laptop cool and quiet. | ||
| - **You need a cluster that outlives your laptop session.** Local minikube is tied to your laptop lifecycle — close the lid, change networks, or hit sleep mode and things break ([known issue](https://github.com/kubernetes/minikube/issues/21593)). An EC2-hosted cluster keeps running while you're away, which matters for webhooks, long-running jobs, or async systems. |
There was a problem hiding this comment.
This bug was an issue with docker desktop:
#21593 (comment)
Minikube works fine with sleep on macOS - but cluster that outlive the laptop session is good reason.
There was a problem hiding this comment.
Added two screenshots: the colorful minikube start output and kubectl get nodes verifying the cluster. Fixed in 29ef3ef.
There was a problem hiding this comment.
Not sure the kubectl command is better as screenshot.
The Minikube output should look good as text - the emoji works well in a the browser.
| # http://192.168.49.2:30000 | ||
| ``` | ||
|
|
||
| This URL only works *from inside the EC2 instance*. Combine with SSH local port forwarding from Pattern 1 if you need it on your laptop. |
There was a problem hiding this comment.
Both kubectl port-forward and minikube service requires ssh tunnel to access from the local machine, right?
I think it will be simpler to show one solution - since this document is about running minikube remotely, I would pick the minikube service way and how how to use ssh tunnel to access it.
If using kubectl has some advantage we can show both but we should describe the advantage. The current content is confusing - which pattern should I use?
There was a problem hiding this comment.
Restructured around minikube service --url as the primary recommended path, with kubectl port-forward mentioned briefly as an alternative for non-NodePort use cases. Removed the confusing "two patterns" framing. Fixed in 29ef3ef.
|
|
||
| If you build a Docker image directly inside the EC2 instance and reference it in a pod with `imagePullPolicy: Never`, minikube can't see the image — minikube runs its own Docker daemon inside the kicbase container, separate from the host Docker. | ||
|
|
||
| **Fix:** load the image into minikube explicitly: |
There was a problem hiding this comment.
Or use a local registry, but loading the image to minikube is much easier.
There was a problem hiding this comment.
Added a brief note about pushing to a local registry as another option, while flagging that minikube image load is easier for most cases. Fixed in 29ef3ef.
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
b7d2cec to
29ef3ef
Compare
|
|
||
| ### `ErrImageNeverPull` after building an image locally | ||
|
|
||
| If you build a Docker image directly inside the EC2 instance and reference it in a pod with `imagePullPolicy: Never`, minikube can't see the image — minikube runs its own Docker daemon inside the kicbase container, separate from the host Docker. |
There was a problem hiding this comment.
using docker as the container runtime is deprecated, so better to use more neutral wording here
i.e. this will soon be containerd
There was a problem hiding this comment.
Swapped "Docker daemon" for "container runtime" since minikube v1.39+ defaults to containerd anyway. Fixed in 7e9a64e.
There was a problem hiding this comment.
minikube 1.39 will not change the runtime, it will in the next release.
There was a problem hiding this comment.
Some internal details might be subject to change, and better to keep as neutral.
https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/
The minikube docker-env is a compatibility layer, but minikube image is better...
|
You might want to add some (generic) links to working in a remote login environment, with the CLI? The benefit with the "ssh" driver (in theory) was that you could use your local GUI editor and browser... While there is some mention of port forwarding, there are some simple things (like editing) missing. |
| docker build -t my-app:dev . | ||
| ``` | ||
|
|
||
| You can also push to a local registry and pull from there. That's more involved to set up — `minikube image load` is easier for most cases. |
There was a problem hiding this comment.
Or use minikube image build, then you don't have to bother with saving or loading at all?
It will use the builder inside the cluster, instead of building the image outside of the cluster
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
29ef3ef to
7e9a64e
Compare
|
Does EC2 have some mirror for the kicbase image and the docker.io / ghcr.io registries, that can be used? I think that registry.k8s.io will find a local mirror (automatically through DNS), but not so sure about the others. https://kubernetes.io/blog/2022/11/28/registry-k8s-io-faster-cheaper-ga/ Same for the "preload", if using that feature (and it is enabled by default). |
This comment was marked as resolved.
This comment was marked as resolved.
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
7e9a64e to
3d8bc76
Compare
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
2 times, most recently
from
dc0db7e to
df68ec5
Compare
|
@wasimat404 I added a preview link to the pr message: |
| minikube is **not intended for production Kubernetes hosting** — for that, use EKS or self-managed clusters. This guide treats EC2 as an extension of your local development environment, like a remote dev box. | ||
| {{% /pageinfo %}} | ||
|
|
||
| {{% pageinfo color="warning" %}} |
There was a problem hiding this comment.
warning is too much, a note like the first one is enough.
For both putting the notes as the first thing in the page is not good. We want to put something interesting and fun at this place, and move notes to the end.
There was a problem hiding this comment.
I moved both pageinfo blocks down to a "Notes" section at the bottom of the page, and dropped the warning color on the third-party one so it matches the first. top of the page now leads straight into the use cases instead of two warning boxes in f9eab68.
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
df68ec5 to
f9eab68
Compare
| --- | ||
|
|
||
| This tutorial covers running minikube on an AWS EC2 instance, useful when you want to run your Kubernetes development cluster on a remote machine instead of locally. | ||
|
|
There was a problem hiding this comment.
It can nice to add a screenshot of running minikube remotely on a device that cannot run it (like an iPad).
Adds a tutorial covering minikube on AWS EC2 as a remote development environment. Discussed with afbjorklund on issue 22989 — framed as extending desktop resources, not production hosting. Covers: - Why use EC2 as a remote dev box - Instance sizing (t2.micro fails, t3.medium recommended) - Docker, minikube, and kubectl installation - Starting the cluster with the docker driver - Accessing services from outside the VM (SSH tunnel + port-forward) - Common troubleshooting (ErrImageNeverPull, CPU/RAM limits, disk fill) - Stopping the instance to avoid charges Signed-off-by: wasimat404 <huaweiwasim7@gmail.com>
wasimat404
force-pushed
the
docs/22989-running-minikube-on-aws-ec2
branch
from
f9eab68 to
350c805
Compare
|
@afbjorklund Can you review and ack? |
| {{% /pageinfo %}} | ||
|
|
||
| {{% pageinfo %}} | ||
| This page refers to third-party products and services (Amazon EC2, Ubuntu). The minikube project authors aren't responsible for those third-party products or services. See the [CNCF website guidelines](https://www.cncf.io/website-guidelines/) for more details. |
There was a problem hiding this comment.
This link is broken, and is replaced by the much more unwieldy https://github.com/cncf/foundation/blob/main/policies-guidance/website-guidelines.md
Compare with the text on other pages, such as "container runtimes" https://kubernetes.io/docs/setup/production-environment/container-runtimes/
| ## Notes | ||
|
|
||
| {{% pageinfo %}} | ||
| minikube is **not intended for production Kubernetes hosting** — for that, use EKS or self-managed clusters. This guide treats EC2 as an extension of your local development environment, like a remote dev box. |
There was a problem hiding this comment.
Might need "Amazon EKS" for context (and if so a mention below), or just something generic like "managed"
as in ... "use managed or self-managed clusters" (the production setup link is also missing from other minikube docs)
There was a problem hiding this comment.
It is more of a generic thing, it applies to all minikube setup: (i.e. "learning" vs" production")
There was a problem hiding this comment.
Made it generic- "managed or self-managed Kubernetes cluster" instead of naming EKS, and added a link to the upstream Kubernetes production-environment docs to tie the learning→production framing back to canonical k8s content.
| ## Next steps | ||
|
|
||
| - Deploy your first app: [Hello minikube]({{< ref "/docs/start" >}}) | ||
| - Try multi-node clusters on EC2: [multi_node tutorial]({{< ref "/docs/tutorials/multi_node" >}}) |
There was a problem hiding this comment.
"on EC2" is redundant (and can be removed), the documentation in that link is generic
|
|
||
| ## Install Docker | ||
|
|
||
| minikube needs a container engine on the host to run the cluster. We'll use the `docker` driver — the most common choice on Linux. `podman` is also supported as an alternative, though it has known issues on some setups; if you prefer it, expect to debug occasionally. Other drivers (`none`, `kvm`, `ssh`) exist for different scenarios — see the [drivers documentation](/docs/drivers/) for the full list. |
There was a problem hiding this comment.
The docker setup differs from the docker driver documentation
i.e. it is using the system docker instead of the vendor docker
There was a problem hiding this comment.
There was a problem hiding this comment.
Switched the install to follow the official Docker Engine guide (docker-ce) instead of the distro docker.io package, and linked the minikube docker-driver docs in the prose so the setup stays aligned with what the driver expects.
I can review the technical content, but a minikube maintainer still needs to decide if you want it or not. Most of the documentation is related to Amazon services, and as such it would be better on amazon.com There are lots of details that are very specific to a single market (US), and also will get outdated quickly... "amd64", "t3.medium", "$0.04", "us-east-1" |
- fix broken CNCF website-guidelines link - genericize production hosting note (managed/self-managed) + link k8s production-environment docs - remove redundant 'on EC2' from multi-node next-step - switch Docker install to vendor docker-ce via official guide; link docker driver docs
- replace hard-coded $0.04/hr us-east-1 with 'check current pricing for your region' - soften fixed $1/day estimate to be region-dependent
|
Appreciate the call, the $0.04 / us-east-1 hardcoding is out, replaced with a pointer to live AWS pricing. t3.medium stays as a suggested minimum but rest is upto @nirs whatever you think works: keep it lean, reshape it, or relocate it. |
5 participants
Closes #22989
Adds a tutorial for running minikube on AWS EC2 as a remote dev environment.
Framing discussed with @afbjorklund on #22989 — positioned as "extending desktop resources / remote dev box," not production hosting.
Preview: https://deploy-preview-23017--kubernetes-sigs-minikube.netlify.app/docs/tutorials/running_minikube_on_aws_ec2/
What's covered
t2.microfails (1 vCPU),t3.mediumrecommended minimum, with a comparison tabledockerdriver, with a callout thatnone/sshare alternativeskubectl port-forwardpatternErrImageNeverPull+minikube image loadworkaroundWhy this is useful
Lots of third-party blogs cover "minikube on EC2" (Medium, dev.to, etc.) and none are authoritative. This consolidates the actual workflow in official docs.
Tested
t3.mediumUbuntu EC2 over the past few monthssetup_minikube_in_github_actions.mdstructure)pageinfo,ref) matching other tutorials in the folderOpen to feedback on tone, depth, or any sections you'd want trimmed/expanded. Thanks!