« Back to Index

Shell: Multiple searches via abstraction and grep

View original Gist on GitHub

Tags: #bash #shell #grep #search #ag #silversearcher

Multiple searches via abstraction and grep.md

I have the following abstraction around the ag search tool:

function search {
  # search for files based on content pattern
  # uses 'silver searcher' `ag`

  local flags=${1:-}
  local pattern=$2
  local directory=${3:-.}
  local exclude=(
    '(js/d3/|jquery|prototype).*\.js'
    '\.eps'
    '\.gif'
    '\.git/'
    '\.html'
    '\.jpg'
    '\.json'
    '\.map'
    '\.mypy_cache'
    '\.psd'
    '\.sav'
    '\.so'
    '\.sql'
    'build/'
    'build\.js'
    'dist/'
    'fb\.js'
    'node_modules'
    'swagger'
    'tests/'
    'vendors-bundle\.js'
  )

  if [ -z "$1" ]; then
    printf "\\n\\tUsage:\\n\\t\\tsearch <flags:[--]> <pattern:['']> <directory:[./]>\\n"

    # shellcheck disable=SC1117
    # disabled because \\\\b for literal \b (with double quotes) is ridiculous
    printf '\n\tExample:\n\t\tsearch -- "def\\b" ~/python/app'
    printf '\n\t\tsearch "-G Dockerfile --context=5" "FROM" ./'
    return
  fi

  exclude_joined=$(join_by '|' ${exclude[@]})

  # for some reason I can't just execute the command, I needed to evaluate it?
  #
  cmd=$(echo time ag --ignore "'($exclude_joined)'" "$flags" "'$pattern'" "$directory" 2>/dev/null)
  eval $cmd

  # OLD IMPLEMENTATIONS...
  #
  # time sift -n -X json --err-skip-line-length --group --exclude-ipath "$exclude" "$flags" "$pattern" "$directory" 2>/dev/null
  # time grep --exclude-dir .git -irlno $pattern $directory
}

I needed a way to search all Dockerfiles and then from the results of that search I needed to search another file (service.yml) for another phrase (launch_type).

I couldn’t do this with the search abstraction because when dealing with xargs the command needs to exist in /bin/sh and my search abstraction only exists when spinning up a /bin/bash shell. So I had to resort to using grep to do the secondary search.

Note: I even tried doing xargs -I {} bash -c "search ..." but it still couldn’t locate the search abstraction I had defined.

search "-G Dockerfile -l" "FROM go" | cut -d '/' -f 1 | tee /tmp/goservices | xargs -I {} grep -r --include service.yml launch_type {}