Skip to content

terraform: add Lambda Labs cloud provider support

Luis Chamberlain kdevops 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest10 patches 1 replyIn review
  2. v210 patches 8 repliesSuperseded
  3. v11 patch 1 replySuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10gitignore: add entries for Lambda Labs dynamic configuration.gitignore+9 -00 replies
2/10scripts: add Lambda Labs Python API libraryscripts/lambdalabs_api.py+556 -00 replies
3/10scripts: add Lambda Labs testing and debugging utilitiesdocs/lambda-cli.1 scripts/cloud_list_all.sh scripts/lambda-cli+1036 -00 replies
4/10scripts: add Lambda Labs credentials managementscripts/lambdalabs_credentials.py+242 -00 replies
5/10scripts: add Lambda Labs SSH key management utilitiesscripts/lambdalabs_ssh_key_name.py scripts/lambdalabs_ssh_keys.py scripts/ssh_config_file_name.py +1 more+682 -00 replies
6/10kconfig: add dynamic cloud provider configuration infrastructuredocs/dynamic-cloud-kconfig.md scripts/dynamic-cloud-kconfig.Makefile scripts/generate_cloud_configs.py +2 more+741 -00 replies
7/10terraform/lambdalabs: add Kconfig structure for Lambda Labsterraform/lambdalabs/Kconfig terraform/lambdalabs/kconfigs/Kconfig.compute terraform/lambdalabs/kconfigs/Kconfig.identity +3 more+267 -00 replies
8/10terraform/lambdalabs: add terraform provider implementationterraform/lambdalabs/README.md terraform/lambdalabs/SET_API_KEY.sh terraform/lambdalabs/ansible_provision_cmd.tpl +6 more+700 -00 replies
9/10ansible/terraform: integrate Lambda Labs into build systemplaybooks/roles/gen_tfvars/defaults/main.yml playbooks/roles/gen_tfvars/templates/lambdalabs/terraform.tfvars.j2 playbooks/roles/terraform/tasks/main.yml +5 more+285 -120 replies
10/10kconfigs: enable Lambda Labs cloud provider in menusPROMPTS.md defconfigs/lambdalabs defconfigs/lambdalabs-gpu-1x-a10 +7 more+138 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 00/10] terraform: add Lambda Labs cloud provider support

This v3 takes up the idea shared by Chuck to implement a CLI tool and make the dynamic kconfig use it instead. We also add documentation about this. This should hopefully pave the way for other cloud provider support to leverage this more easily, AIs can just read the docs and go to town.

Luis Chamberlain (10):
gitignore: add entries for Lambda Labs dynamic configuration
scripts: add Lambda Labs Python API library
scripts: add Lambda Labs testing and debugging utilities
scripts: add Lambda Labs credentials management
scripts: add Lambda Labs SSH key management utilities
kconfig: add dynamic cloud provider configuration infrastructure terraform/lambdalabs: add Kconfig structure for Lambda Labs terraform/lambdalabs: add terraform provider implementation ansible/terraform: integrate Lambda Labs into build system
kconfigs: enable Lambda Labs cloud provider in menus

 .gitignore                                    |   9 +
 PROMPTS.md                                    |  56 ++
 defconfigs/lambdalabs                         |  15 +
 defconfigs/lambdalabs-gpu-1x-a10              |   9 +
 defconfigs/lambdalabs-gpu-1x-a100             |   8 +
 defconfigs/lambdalabs-gpu-1x-h100             |   8 +
 defconfigs/lambdalabs-gpu-8x-a100             |   8 +
 defconfigs/lambdalabs-gpu-8x-h100             |   8 +
 defconfigs/lambdalabs-shared-key              |  11 +
 defconfigs/lambdalabs-smart                   |  10 +
 docs/dynamic-cloud-kconfig.md                 | 461 +++++++++++++
 docs/lambda-cli.1                             | 245 +++++++
 kconfigs/Kconfig.bringup                      |   5 +
 playbooks/roles/gen_tfvars/defaults/main.yml  |  23 +
 .../templates/lambdalabs/terraform.tfvars.j2  |  18 +
 playbooks/roles/terraform/tasks/main.yml      |  85 +++
 scripts/cloud_list_all.sh                     | 152 +++++
 scripts/dynamic-cloud-kconfig.Makefile        |  44 ++
 scripts/dynamic-kconfig.Makefile              |   2 +
 scripts/generate_cloud_configs.py             | 113 ++++
 scripts/lambda-cli                            | 639 ++++++++++++++++++
 scripts/lambdalabs_api.py                     | 556 +++++++++++++++
 scripts/lambdalabs_credentials.py             | 242 +++++++
 scripts/lambdalabs_infer_region.py            |  61 ++
 scripts/lambdalabs_smart_inference.py         |  62 ++
 scripts/lambdalabs_ssh_key_name.py            | 135 ++++
 scripts/lambdalabs_ssh_keys.py                | 358 ++++++++++
 scripts/ssh_config_file_name.py               |  79 +++
 scripts/terraform.Makefile                    | 108 ++-
 scripts/update_ssh_config_lambdalabs.py       | 110 +++
 terraform/Kconfig.providers                   |  10 +
 terraform/Kconfig.ssh                         |  37 +-
 terraform/lambdalabs/Kconfig                  |  33 +
 terraform/lambdalabs/README.md                | 349 ++++++++++
 terraform/lambdalabs/SET_API_KEY.sh           |  20 +
 .../lambdalabs/ansible_provision_cmd.tpl      |   1 +
 terraform/lambdalabs/extract_api_key.py       |  40 ++
 terraform/lambdalabs/kconfigs/Kconfig.compute |  48 ++
 .../lambdalabs/kconfigs/Kconfig.identity      |  76 +++
 .../lambdalabs/kconfigs/Kconfig.location      |  73 ++
 terraform/lambdalabs/kconfigs/Kconfig.smart   |  25 +
 terraform/lambdalabs/kconfigs/Kconfig.storage |  12 +
 terraform/lambdalabs/main.tf                  | 154 +++++
 terraform/lambdalabs/output.tf                |  51 ++
 terraform/lambdalabs/provider.tf              |  19 +
 terraform/lambdalabs/shared.tf                |   1 +
 terraform/lambdalabs/vars.tf                  |  65 ++
 terraform/shared.tf                           |  14 +-
 48 files changed, 4656 insertions(+), 12 deletions(-)
 create mode 100644 defconfigs/lambdalabs
 create mode 100644 defconfigs/lambdalabs-gpu-1x-a10
 create mode 100644 defconfigs/lambdalabs-gpu-1x-a100
 create mode 100644 defconfigs/lambdalabs-gpu-1x-h100
 create mode 100644 defconfigs/lambdalabs-gpu-8x-a100
 create mode 100644 defconfigs/lambdalabs-gpu-8x-h100
 create mode 100644 defconfigs/lambdalabs-shared-key
 create mode 100644 defconfigs/lambdalabs-smart
 create mode 100644 docs/dynamic-cloud-kconfig.md
 create mode 100644 docs/lambda-cli.1
 create mode 100644 playbooks/roles/gen_tfvars/templates/lambdalabs/terraform.tfvars.j2
 create mode 100755 scripts/cloud_list_all.sh
 create mode 100644 scripts/dynamic-cloud-kconfig.Makefile
 create mode 100755 scripts/generate_cloud_configs.py
 create mode 100755 scripts/lambda-cli
 create mode 100755 scripts/lambdalabs_api.py
 create mode 100755 scripts/lambdalabs_credentials.py
 create mode 100755 scripts/lambdalabs_infer_region.py
 create mode 100755 scripts/lambdalabs_smart_inference.py
 create mode 100755 scripts/lambdalabs_ssh_key_name.py
 create mode 100755 scripts/lambdalabs_ssh_keys.py
 create mode 100755 scripts/ssh_config_file_name.py
 create mode 100755 scripts/update_ssh_config_lambdalabs.py
 create mode 100644 terraform/lambdalabs/Kconfig
 create mode 100644 terraform/lambdalabs/README.md
 create mode 100644 terraform/lambdalabs/SET_API_KEY.sh
 create mode 120000 terraform/lambdalabs/ansible_provision_cmd.tpl
 create mode 100755 terraform/lambdalabs/extract_api_key.py
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.compute
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.identity
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.location
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.smart
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.storage
 create mode 100644 terraform/lambdalabs/main.tf
 create mode 100644 terraform/lambdalabs/output.tf
 create mode 100644 terraform/lambdalabs/provider.tf
 create mode 120000 terraform/lambdalabs/shared.tf
 create mode 100644 terraform/lambdalabs/vars.tf
Signature · 1 line
2.50.1