1. 启动集群 (1)进入CloudFormation,在aws控制台搜索 CloudFormation,进入CloudFormation主页 (2)创建堆栈,选在现有模板->上传现有模板,模板下地址:https://chukonu.houdutech.cn/aws-download/cloudformation-chukonu-v1.0.yaml (3)设置参数,在指定堆栈详细信息中,InstanceCount 设置为3,InstanceType 选在ec2实例,推荐使用m6i.2xlarge,也可以根据自己实际情况调整 (4)其他堆栈选项按照默认参数即可,最后提交 (5) 北京地区和宁夏地区的镜像id不一样,北京地区:ami-0d5569dae64ab95a9,宁夏地区:ami-050748b8d4ee9f1d1,启动CloudFormation时注意替换对应的镜像id,目前默认北京地区 2. 运行测试程序 (1)下载测试脚本和程序。 测试脚本下载地址:https://chukonu.houdutech.cn/aws-download/submit-job.sh 测试程序下载地址:https://chukonu.houdutech.cn/aws-download/word_count.py (2)下载完成后将submit-job.sh、word_count.py 上传到chukonuMaster这台服务器用户目录下,然后再/mnt/data目录下创建word_count.txt,并写入一些文本,然后执行命令 sh submit-job.sh 进行测试 3. 安全说明 (1)启动集群需要用户有EC2,EFS,CloudFormation,EC2 Auto Scaling,Systems Manager操作权限,具体策略见附录权限策略json (2)连接服务器进入Systems Manager,Paramter store 可以查看密钥,通过该密钥连接服务器 (3)连接服务器时,安全组需要打开22端口 注意:订阅完成后按照上述操作进行,可以不用在网站或ec2控制台启动服务器 附录 (1) 权限策略json: { "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "ec2:AuthorizeSecurityGroupIngress", "ec2:DeleteSubnet", "ec2:DescribeInstances", "ec2:CreateKeyPair", "s3:CreateBucket", "ec2:AttachInternetGateway", "ssm:GetParameter", "ec2:DeleteRouteTable", "ec2:AssociateRouteTable", "ec2:DescribeInternetGateways", "ssm:DeleteParameter", "cloudformation:DescribeStackEvents", "ec2:GetLaunchTemplateData", "ssm:DescribeParameters", "autoscaling:DescribeAutoScalingGroups", "ec2:CreateRoute", "ec2:CreateInternetGateway", "ec2:DescribeVolumes", "autoscaling:UpdateAutoScalingGroup", "ec2:DeleteInternetGateway", "ec2:DescribeKeyPairs", "autoscaling:TerminateInstanceInAutoScalingGroup", "ec2:DescribeRouteTables", "cloudformation:ListStackResources", "ec2:ImportKeyPair", "ec2:DescribeLaunchTemplates", "ec2:CreateTags", "ec2:CreateRouteTable", "ec2:RunInstances", "ssm:GetParameters", "ec2:DetachInternetGateway", "ec2:DisassociateRouteTable", "ssm:DeleteParameters", "cloudformation:DescribeStacks", "ssm:PutParameter", "ec2:RevokeSecurityGroupIngress", "s3:PutObject", "s3:GetObject", "elasticfilesystem:CreateMountTarget", "cloudformation:DeleteStack", "autoscaling:DeleteAutoScalingGroup", "ec2:DeleteVpc", "ec2:CreateSubnet", "ec2:DescribeSubnets", "autoscaling:CreateAutoScalingGroup", "ec2:DeleteKeyPair", "elasticfilesystem:DeleteFileSystem", "cloudformation:CreateUploadBucket", "ec2:DescribeAddresses", "elasticfilesystem:CreateFileSystem", "ec2:DescribeInstanceAttribute", "ec2:CreateVpc", "ec2:ModifySubnetAttribute", "ec2:DescribeNetworkInterfaces", "autoscaling:DescribeScalingActivities", "ec2:DescribeAvailabilityZones", "ec2:CreateSecurityGroup", "ec2:ModifyVpcAttribute", "ec2:DeleteLaunchTemplateVersions", "elasticfilesystem:DescribeFileSystems", "elasticfilesystem:DeleteMountTarget", "ec2:DescribeInstanceStatus", "elasticfilesystem:DescribeMountTargets", "cloudformation:ListStacks", "ec2:DeleteLaunchTemplate", "ec2:TerminateInstances", "ec2:DeleteRoute", "ec2:DescribeLaunchTemplateVersions", "cloudformation:GetTemplateSummary", "ec2:DescribeSecurityGroups", "ec2:CreateLaunchTemplateVersion", "elasticfilesystem:TagResource", "ec2:DescribeImages", "ec2:CreateLaunchTemplate", "cloudformation:CreateStack", "ec2:DescribeVpcs", "ec2:DeleteSecurityGroup", "ec2:ModifyLaunchTemplate" ], "Resource": "*" } ] }