首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >并行平扫

并行平扫
EN

Stack Overflow用户
提问于 2017-10-01 01:49:36
回答 3查看 1K关注 0票数 3

我正试图扫描一个IP块,总共有6.5万个地址。我们已经被指示使用带有bash的ICMP数据包,并找到一种并行化的方法。以下是我想出的:

代码语言:javascript
复制
#!/bin/bash
ping() {
  if ping -c 1 -W 5 131.212.$i.$j >/dev/null
  then
      ((++s))
      echo -n "*"
  else
      ((++f))
      echo -n "."
  fi
  ((++j))
  #if j has reached 255, set it to zero and increment i
  if [ $j -gt 255 ]; then
      j=0
      ((++i))
      echo "Pinging 131.212.$i.xx IP Block...\n"
  fi
}

s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2
curProcs=$(ps | wc -l)
maxProcs=$(getconf OPEN_MAX)
while [ $i -lt 256 ]; do
    curProcs=$(ps | wc -l)
    if [ $curProcs -lt $maxProcs ]; then
      ping &
    else
      sleep 10
    fi
done
echo "Found "$s" responses and "$f" timeouts."
echo /usr/bin/time -l
done

但是,我遇到了以下错误(在macOS上):

代码语言:javascript
复制
redirection error: cannot duplicate fd: Too many open files

我的理解是,我正在讨论一个资源限制,如果现有进程的数量少于指定的最大值,我尝试通过启动新的ping进程来纠正这个限制,但这不能解决问题。

谢谢您的时间和建议。

编辑:下面有很多很好的建议,可以用已有的工具来完成。由于我受到学术要求的限制,我最终将ping循环分割成一个不同的过程,用于每个12.34.x.x块,尽管这个过程很难看,但在不到5分钟的时间内就成功了。这段代码有很多问题,但对于将来的一些人来说,这可能是一个很好的起点:

代码语言:javascript
复制
#!/bin/bash

#############################
#      Ping Subfunction     #
#############################
# blocks with more responses will complete first since worst-case scenerio
# is O(n) if no IPs generate a response
pingSubnet() {
  for ((j = 0 ; j <= 255 ; j++)); do
    # send a single ping with a timeout of 5 sec, piping output to the bitbucket
    if ping -c 1 -W 1 131.212."$i"."$j" >/dev/null
    then
        ((++s))
    else
        ((++f))
    fi
  done
  #echo "Recieved $s responses with $f timeouts in block $i..."
  # output number of success results to the pipe opened in at the start
  echo "$s" >"$pipe"
  exit 0
}

#############################
#   Variable Declaration    #
#############################
start=$(date +%s) #start of execution time
startMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//');
startCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
s=0 #number of responses recieved
f=0 #number of failures recieved
i=0 #IP increment 1
j=0 #IP increment 2

#############################
#    Pipe Initialization    #
#############################
# create a pipe for child procs to write to
# child procs inherit runtime environment of parent proc, but cannot
# write back to it (like passing by value in C, but the whole env)
# hence, they need somewhere else to write back to that the parent
# proc can read back in
pipe=/tmp/pingpipe
trap 'rm -f $pipe' EXIT
if [[ ! -p $pipe ]]; then
    mkfifo $pipe
    exec 3<> $pipe
fi

#############################
#     IP Block Iteration    #
#############################
# adding an ampersand to the end forks the command to a separate, backgrounded
# child process. this allows for parellel computation but adds logistical
# challenges since children can't write the parent's variables
echo "Initiating scan processes..."
while [ $i -lt 256 ]; do
      #echo "Beginning 131.212.$i.x block scan..."
      #ping subnet asynchronously
      pingSubnet &
      ((++i))
done
echo "Waiting for scans to complete (this may take up to 5 minutes)..."
peakMem=$(vm_stat | awk '/Pages free/ {print $3}' | awk 'BEGIN { FS = "\." }; {print ($1*0.004092)}' | sed 's/\..*$//')
peakCPU=$(top -l 1 | grep "CPU usage" | awk '{print 100-$7;}' | sed 's/\..*$//')
wait
echo -e "done" >$pipe

#############################
#    Concat Pipe Outputs    #
#############################
# read each line from the pipe we created earlier, adding the number
# of successes up in a variable
success=0
echo "Tallying responses..."
while read -r line <$pipe; do
    if [[ "$line" == 'done' ]]; then
      break
    fi
    success=$((line+success))
done

#############################
#    Output Statistics      #
#############################
echo "Gathering Statistics..."
fail=$((65535-success))
#output program statistics
averageMem=$((peakMem-startMem))
averageCPU=$((peakCPU-startCPU))
end=$(date +%s) #end of execution time
runtime=$((end-start))
echo "Scan completed in $runtime seconds."
echo "Found $success active servers and $fail nonresponsive addresses with a timeout of 1."
echo "Estimated memory usage was $averageMem MB."
echo "Estimated CPU utilization was $averageCPU %"
EN

回答 3

Stack Overflow用户

回答已采纳

发布于 2017-10-01 09:52:55

这将为您提供一些关于GNU并行的想法。

代码语言:javascript
复制
parallel --dry-run -j 64 -k ping 131.212.{1}.{2} ::: $(seq 1 3) ::: $(seq 11 13)

ping 131.212.1.11
ping 131.212.1.12
ping 131.212.1.13
ping 131.212.2.11
ping 131.212.2.12
ping 131.212.2.13
ping 131.212.3.11
ping 131.212.3.12
ping 131.212.3.13
  • -j64一次并行执行64个ping。
  • -dry-run的意思是什么都不做,只是展示它会做什么。
  • -k的意思是保持输出有序-(只是为了让你理解它)

:::引入了这些参数,我用不同的数字(1到3,然后11到13)重复它们,这样您就可以区分这两个计数器,并看到所有的排列和组合都生成了。

票数 3
EN

Stack Overflow用户

发布于 2017-10-01 01:51:53

别干那事。

使用福平代替。它将比你的程序更有效地探测。

代码语言:javascript
复制
$ brew install fping

将使它可用,多亏了酿制的魔力。

票数 2
EN

Stack Overflow用户

发布于 2017-10-01 06:54:08

当然,它并不像上面试图构建的那样最优,但是您可以在后台启动最大允许数量的进程,等待它们结束并启动下一批进程,如下所示(但我使用的是sleep 1s):

代码语言:javascript
复制
for i in {1..20}             # iterate some
do 
    sleep 1 &                # start in the background
    if ! ((i % 5))           # after every 5th (using mod to detect)
    then 
        wait %1 %2 %3 %4 %5  # wait for all jobs to finish
    fi
done
票数 1
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/46508532

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档