首页
学习
活动
专区
圈层
工具
发布
首页
学习
活动
专区
圈层
工具
MCP广场
社区首页 >问答首页 >通过亚马逊网络服务控制台为EC2执行根值交换

通过亚马逊网络服务控制台为EC2执行根值交换
EN

Stack Overflow用户
提问于 2017-10-27 23:49:54
回答 1查看 183关注 0票数 4

我最近一直在使用交换根卷方法来创建持久的Spot实例,如here所述(方法2)。通常,我的Spot实例完成和交换完成需要2-5分钟。然而,有些日子,这个过程永远不会结束(或者至少我在等待20分钟到一个小时后就不耐烦了!)。

需要明确的是,实例被创建了,但是交换永远不会发生:我可以通过ssh进入服务器,但是我的持久文件不在那里。我也可以通过转到我的AWS控制台并注意到"spotter“(我的持久存储)没有附件信息来看到这一点:

因为我正在使用的交换脚本从来没有给我任何错误,所以很难看出失败的是什么。因此,我想知道,根据我的屏幕截图,我是否可以只使用亚马逊网络服务EC2管理控制台来“手动”执行交换,如果是的话,我应该如何完成这一点。

而且,如果这对@Vorsprung有帮助,

我通过运行以下脚本启动该进程:

代码语言:javascript
运行
复制
    # The config file was created in ondemand_to_spot.sh
export config_file=my.conf
cd "$(dirname ${BASH_SOURCE[0]})"

. ../$config_file || exit -1

export request_id=`../ec2spotter-launch $config_file`
echo Spot request ID: $request_id

echo Waiting for spot request to be fulfilled...
aws ec2 wait spot-instance-request-fulfilled --spot-instance-request-ids $request_id

export instance_id=`aws ec2 describe-spot-instance-requests --spot-instance-request-ids $request_id --query="SpotInstanceRequests[*].InstanceId" --output="text"`

echo Waiting for spot instance to start up...
aws ec2 wait instance-running --instance-ids $instance_id

echo Spot instance ID: $instance_id

echo 'Please allow the root volume swap script a few minutes to finish.'
if [ "x$ec2spotter_elastic_ip" = "x" ]
then
        # Non elastic IP
        export ip=`aws ec2 describe-instances --instance-ids $instance_id --filter Name=instance-state-name,Values=running --query "Reservations[*].Instances[*].PublicIpAddress" --output=text`
else
        # Elastic IP
        export ip=`aws ec2 describe-addresses --allocation-ids $ec2spotter_elastic_ip --output text --query 'Addresses[0].PublicIp'`
fi

export name=fast-ai
if [ "$ec2spotter_key_name" = "aws-key-$name" ]
then    function aws-ssh-spot {
        ssh -i ~/.ssh/aws-key-$name.pem ubuntu@$ip
        }
        function aws-terminate-spot {
        aws ec2 terminate-instances --instance-ids $instance_id
        }
        echo  Jupyter Notebook -- $ip:8888
fi

其中,my.conf是:

代码语言:javascript
运行
复制
# Name of root volume.
ec2spotter_volume_name=spotter
# Location (zone) of root volume. If not the same as ec2spotter_launch_zone,
# a copy will be created in ec2spotter_launch_zone.
# Can be left blank, if the same as ec2spotter_launch_zone
ec2spotter_volume_zone=us-west-2b

ec2spotter_launch_zone=us-west-2b
ec2spotter_key_name=aws-key-fast-ai
ec2spotter_instance_type=p2.xlarge
# Some instance types require a subnet to be specified:
ec2spotter_subnet=subnet-c9cba8af

ec2spotter_bid_price=0.55

# uncomment and update the value if you want an Elastic IP
# ec2spotter_elastic_ip=eipalloc-64d5890a

# Security group
ec2spotter_security_group=sg-2be79356

# The AMI to be used as the pre-boot environment. This is NOT your target system installation.
# Do Not Modify this unless you have a need for a different Kernel version from what's supplied.
# ami-6edd3078 is ubuntu-xenial-16.04-amd64-server-20170113
ec2spotter_preboot_image_id=ami-bc508adc

Ec2potter-launch脚本是:

代码语言:javascript
运行
复制
    #!/bin/bash

    # "Phase 1" this is the user-facing script for launching a new spot istance

    if [ "$1" = "" ]; then echo "USER ERROR: please specify a configuration file"; exit -1; fi

    cd $(dirname $0)

    . $1 || exit -1

    # New instance:
    # Desired launch zone
    LAUNCH_ZONE=$ec2spotter_launch_zone
    # Region is LAUNCH_ZONE minus the last character
    LAUNCH_REGION=$(echo $LAUNCH_ZONE | sed -e 's/.$//')
    PUB_KEY=$ec2spotter_key_name

    # Existing Volume:
    # If no volume zone
    if [ "$ec2spotter_volume_zone" = "" ]
    then # Use instance zone
            ec2spotter_volume_zone=$LAUNCH_ZONE
    fi

    # Name of volume (find it by name later)
    ROOT_VOL_NAME=$ec2spotter_volume_name
    # zone of volume (needed if different than instance zone)
    ROOT_ZONE=$ec2spotter_volume_zone
    # Region is Zone minus the last character
    ROOT_REGION=$(echo $ROOT_ZONE | sed -e 's/.$//')


    #echo "ROOT_VOL_NAME=${ROOT_VOL_NAME}; ROOT_ZONE=${ROOT_ZONE}; ROOT_REGION=${ROOT_REGION}; "
    #echo "LAUNCH_ZONE=${LAUNCH_ZONE}; LAUNCH_REGION=${LAUNCH_REGION}; PUB_KEY=${PUB_KEY}"

    AWS_ACCESS_KEY=`aws configure get aws_access_key_id`
    AWS_SECRET_KEY=`aws configure get aws_secret_access_key`

    aws ec2 describe-volumes \
            --filters Name=tag-key,Values="Name" Name=tag-value,Values="$ROOT_VOL_NAME" \
            --region ${ROOT_REGION} --output=json > volumes.tmp || exit -1

    ROOT_VOL=$(jq -r '.Volumes[0].VolumeId' volumes.tmp)
    ROOT_TYPE=$(jq -r '.Volumes[0].VolumeType' volumes.tmp)

    #echo "ROOT_TYPE=$ROOT_TYPE; ROOT_VOL=$ROOT_VOL";
    if [ "$ROOT_VOL_NAME" = "" ]
then
  echo "root volume lacks a Name tag";
  exit -1;
fi

cat >user-data.tmp <<EOF
#!/bin/sh
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
echo AWSSecretKey=$AWS_SECRET_KEY >> /root/.aws.creds

apt-get update
apt-get install -y jq
apt-get install -y python-pip python-setuptools
apt-get install -y git

pip install awscli

cd /root
git clone --depth=1 https://github.com/slavivanov/ec2-spotter.git
echo Got spotter scripts from github.

cd ec2-spotter

echo Swapping root volume
./ec2spotter-remount-root  --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF

userData=$(base64 user-data.tmp | tr -d '\n');

cat >specs.tmp <<EOF
{
  "ImageId" : "$ec2spotter_preboot_image_id",
  "InstanceType": "$ec2spotter_instance_type",
  "KeyName" : "$PUB_KEY",
  "EbsOptimized": true,
  "Placement": {
     "AvailabilityZone": "$LAUNCH_ZONE"
  },
  "BlockDeviceMappings": [
    {
      "DeviceName": "/dev/sda1",
      "Ebs": {
        "DeleteOnTermination": true,
        "VolumeType": "gp2",
        "VolumeSize": 128
      }
    }
  ],
  "NetworkInterfaces": [
      {
        "DeviceIndex": 0,
        "SubnetId": "${ec2spotter_subnet}",
        "Groups": [ "${ec2spotter_security_group}" ],
        "AssociatePublicIpAddress": true
      }
  ],
  "UserData" : "${userData}"
}
EOF

SPOT_REQUEST_ID=$(aws ec2 request-spot-instances --launch-specification file://specs.tmp --spot-price $ec2spotter_bid_price --output="text" --query="SpotInstanceRequests[*].SpotInstanceRequestId" --region ${LAUNCH_REGION})
echo $SPOT_REQUEST_ID
# Clean up
rm user-data.tmp
rm specs.tmp
rm volumes.tmp
EN

回答 1

Stack Overflow用户

发布于 2017-11-05 23:36:39

这不是一个确切的答案,但它可能会帮助您找到调试问题的方法。据我所知,这是ec2spotter-launch脚本中负责卷交换的设置部分:

代码语言:javascript
运行
复制
...
cat >specs.tmp <<EOF
{
  "ImageId" : "$ec2spotter_preboot_image_id",
  ...
  "UserData" : "${userData}"
}
EOF

SPOT_REQUEST_ID=$(aws ec2 request-spot-instances --launch-specification file://specs.tmp --spot-price $ec2spotter_bid_price --output="text" --query="SpotInstanceRequests[*].SpotInstanceRequestId" --region ${LAUNCH_REGION})

实例启动规格:--launc-specification file:://specs.tmp,使用specs.tmp。

启动规范中的"UserData“是一个脚本,也是在es2spotter-launch中生成的

代码语言:javascript
运行
复制
cat >user-data.tmp <<EOF
#!/bin/sh
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
echo AWSSecretKey=$AWS_SECRET_KEY >> /root/.aws.creds

apt-get update
...

cd /root
git clone --depth=1 https://github.com/slavivanov/ec2-spotter.git
echo Got spotter scripts from github.

cd ec2-spotter

echo Swapping root volume
./ec2spotter-remount-root  --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF

交换根卷的实际工作由downloaded from github脚本ec2spotter-remount-root执行。

该脚本中有许多echo语句,所以我认为如果您找到输出的位置,就能理解哪里出了问题。因此,当您遇到问题时,您将通过ssh访问该实例并检查日志文件。问题是要检查哪个文件(以及脚本输出是否记录到某个文件中)。

以下是我建议尝试的:

  1. 检查实例启动时在/var/log下生成的标准日志(cloud-init.log、syslog等)查看是否可以找到ec2spotter-remount-root输出
  2. 尝试自己启用日志记录,请参阅here

我会尝试这样修改es2spotter-launch中的user-data.tmp部件:

代码语言:javascript
运行
复制
#!/bin/bash
set -x
exec > >(tee /var/log/user-data.log|logger -t user-data ) 2>&1
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
...
echo Swapping root volume
./ec2spotter-remount-root  --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF

在这里,我更改了前三行以启用登录到/var/log/user-data.log

  1. 如果1和2不起作用,我会尝试在github上询问脚本作者。由于脚本中有许多echo,因此作者应该知道在哪里查找该输出。

希望这对你有帮助,而且你也不需要等待问题出现来尝试这个方法,而是在成功运行时寻找脚本输出。或者,如果您能够进行很少的测试运行,那么这样做,并确保您可以找到带有脚本输出的日志。

票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/46978950

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档