daviddoji/Made-With-ML
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

updated cluser compute, environment and workloads

Browse Source
This commit is contained in:
GokuMohandas 2023-12-07 11:36:49 -08:00
parent 11ab35b251
commit be3d2732a8
4 changed files with 8 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -83,7 +83,7 @@ We'll start by setting up our cluster with the environment and compute configura
- Project: `madewithml` - Project: `madewithml`
- Cluster environment name: `madewithml-cluster-env` - Cluster environment name: `madewithml-cluster-env`
# Toggle `Select from saved configurations` # Toggle `Select from saved configurations`
- Compute config: `madewithml-cluster-compute` - Compute config: `madewithml-cluster-compute-g5.4xlarge`
``` ```
> Alternatively, we can use the [CLI](https://docs.anyscale.com/reference/anyscale-cli) to create the workspace via `anyscale workspace create ...` > Alternatively, we can use the [CLI](https://docs.anyscale.com/reference/anyscale-cli) to create the workspace via `anyscale workspace create ...`
@ -423,7 +423,7 @@ anyscale cluster-env build deploy/cluster_env.yaml --name $CLUSTER_ENV_NAME
The compute configuration determines **what** resources our workloads will be executes on. We've already created this [compute configuration](./deploy/cluster_compute.yaml) for us but this is how we can create it ourselves. The compute configuration determines **what** resources our workloads will be executes on. We've already created this [compute configuration](./deploy/cluster_compute.yaml) for us but this is how we can create it ourselves.
```bash ```bash
export CLUSTER_COMPUTE_NAME="madewithml-cluster-compute" export CLUSTER_COMPUTE_NAME="madewithml-cluster-compute-g5.4xlarge"
anyscale cluster-compute create deploy/cluster_compute.yaml --name $CLUSTER_COMPUTE_NAME anyscale cluster-compute create deploy/cluster_compute.yaml --name $CLUSTER_COMPUTE_NAME
``` ```

10
deploy/cluster_compute.yaml
View File

@ -1,12 +1,12 @@
cloud: madewithml-us-east-2 cloud: education-us-west-2
region: us-east2 region: us-west-2
head_node_type: head_node_type:
name: head_node_type name: head_node_type
instance_type: m5.2xlarge # 8 CPU, 0 GPU, 32 GB RAM instance_type: g5.4xlarge
worker_node_types: worker_node_types:
- name: gpu_worker - name: gpu_worker
instance_type: g4dn.xlarge # 4 CPU, 1 GPU, 16 GB RAM instance_type: g5.4xlarge
min_workers: 0 min_workers: 1
max_workers: 1 max_workers: 1
use_spot: False use_spot: False
aws: aws:

2
deploy/cluster_env.yaml
View File

@ -1,4 +1,4 @@
base_image: anyscale/ray:2.6.0-py310-cu118 base_image: anyscale/ray:2.7.0optimized-py310-cu118
env_vars: {} env_vars: {}
debian_packages: debian_packages:
- curl - curl

1
deploy/jobs/workloads.sh
View File

@ -1,6 +1,5 @@
#!/bin/bash #!/bin/bash
export PYTHONPATH=$PYTHONPATH:$PWD export PYTHONPATH=$PYTHONPATH:$PWD
export RAY_AIR_REENABLE_DEPRECATED_SYNC_TO_HEAD_NODE=1
mkdir results mkdir results
# Test data # Test data